Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(smokehouse): fix unintentional 404. remove max-len rule for expectations #9665

Merged
merged 5 commits into from
Sep 17, 2019
Merged

tests(smokehouse): fix unintentional 404. remove max-len rule for expectations #9665

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
135bf39
use array for console log expectations
connorjclark Sep 13, 2019
40c5a42
fix
connorjclark Sep 13, 2019
3b97c84
kill max-len for expectations
connorjclark Sep 13, 2019
25ce61e
narrow
connorjclark Sep 13, 2019
8f36c8f
rm
connorjclark Sep 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lighthouse-cli/test/fixtures/dobetterweb/dbw_tester.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
<!-- Note: these will only fail when using the static-server.js, which supports the ?delay=true param.
If you're using your own server, the resource will load instantly and the
stylesheets will be ignored for being below the threshold. -->
<link rel="stylesheet" href="./dobetterweb/dbw_tester.css?delay=100"> <!-- FAIL, when run under smokehouse -->
<link rel="stylesheet" href="./dbw_tester.css?delay=100"> <!-- FAIL, when run under smokehouse -->
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyone know what FAIL, when run under smokehouse is alluding to? when would this pass? Should I just replace with FAIL?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, I wish we put the audit id that these failed for.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's referring to the Note: above, this will fail but only when run under smokehouse and not a generic static file server

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personally I think all this caveat is nonsense at this point because like half of smokehouse would be different if not run in our custom server, so 🤷‍♂

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. if mentioned at all, it should be noted in a README describing how these smoke tests work. i'll leave as-is for a follow up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, I wish we put the audit id that these failed for.

is it possible to check this and add a note since it's being changed anyways? Presumably by the comment something currently failing should pass if there isn't a delay

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it actually didn't really fail anything (only failures were b/c the name changes) ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it actually didn't really fail anything (only failures were b/c the name changes) ...

make sense to remove the <!-- FAIL, when run under smokehouse --> then?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya done

<link rel="stylesheet" href="./unknown404.css?delay=200"> <!-- FAIL -->
<link rel="stylesheet" href="./dbw_tester.css?delay=2200"> <!-- FAIL -->
<link rel="stylesheet" href="./dbw_disabled.css?delay=200&isdisabled" disabled> <!-- PASS -->
Expand Down
39 changes: 34 additions & 5 deletions lighthouse-cli/test/smokehouse/dobetterweb/dbw-expectations.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ module.exports = [
{
tag: {
tagName: 'LINK',
url: 'http://localhost:10200/dobetterweb/dobetterweb/dbw_tester.css?delay=100',
url: 'http://localhost:10200/dobetterweb/dbw_tester.css?delay=100',
},
},
{
Expand Down Expand Up @@ -81,9 +81,38 @@ module.exports = [
'errors-in-console': {
score: 0,
details: {
items: {
length: 8,
},
items: [
{
source: 'network',
description: 'Failed to load resource: the server responded with a status of 404 (Not Found)',
url: 'http://localhost:10200/dobetterweb/unknown404.css?delay=200',
},
{
source: 'other',
description: 'Application Cache Error event: Manifest fetch failed (404) http://localhost:10200/dobetterweb/clock.appcache',
url: 'http://localhost:10200/dobetterweb/dbw_tester.html',
},
{
source: 'network',
description: 'Failed to load resource: the server responded with a status of 404 (Not Found)',
url: 'http://localhost:10200/dobetterweb/fcp-delayer.js?delay=5000',
},
{
source: 'network',
description: 'Failed to load resource: the server responded with a status of 404 (Not Found)',
url: 'http://localhost:10200/favicon.ico',
},
{
source: 'network',
description: 'Failed to load resource: the server responded with a status of 404 (Not Found)',
url: 'http://localhost:10200/dobetterweb/unknown404.css?delay=200',
},
{
source: 'Runtime.exception',
description: 'Error: An error\n at http://localhost:10200/dobetterweb/dbw_tester.html:57:38',
url: 'http://localhost:10200/dobetterweb/dbw_tester.html',
},
],
},
},
'is-on-https': {
Expand Down Expand Up @@ -143,7 +172,7 @@ module.exports = [
details: {
items: [
{
url: 'http://localhost:10200/dobetterweb/dobetterweb/dbw_tester.css?delay=100',
url: 'http://localhost:10200/dobetterweb/dbw_tester.css?delay=100',
},
{
url: 'http://localhost:10200/dobetterweb/unknown404.css?delay=200',
Expand Down