co19/LayoutTests/fast/css/max-device-aspect-ratio_t01 is flakky on Dartium

[DmitryOlshansky-google]

I think I've seen it at least once before but here is the latest failure log. It seems like an issue could be in a too strict timeout expectation.

FAILED: none-dartium-checked release_x64 co19/LayoutTests/fast/css/max-device-aspect-ratio_t01
Expected: Pass 
Actual: RuntimeError
CommandOutput[dartium]:

stdout:
events:
  [
        {
              type: debug
              value: test_controller.js started
              timestamp: 0
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = true, isStatusUpdate = true, isDone = false)
              timestamp: 0
        },
        {
              type: print
              value: dart-calling-main
              timestamp: 0.135
        },
        {
              type: print
              value: unittest-suite-wait-for-done
              timestamp: 0.136
        },
        {
              type: print
              value: dart-main-done
              timestamp: 0.371
        },
        {
              type: message_received
              value: dart-calling-main
              timestamp: 0.372
        },
        {
              type: message_received
              value: unittest-suite-wait-for-done
              timestamp: 0.372
        },
        {
              type: message_received
              value: dart-main-done
              timestamp: 0.373
        },
        {
              type: window_onerror
              value: window.onerror called: 
                    
                    http://127.0.0.1:49934/root_build/generated_tests/none-dartium-checked/short8953_ss_max-device-aspect-ratio_t01/test.html?crossOriginPort=49936:1:
                    Uncaught Unhandled exception:
                    Expect.fail('Stylesheet did not load in 1.0 second(s).')
                    #0      _fail (http://127.0.0.1:49934/root_dart/tests/co19/src/Utils/expect.dart:17:5)
                    #1      Expect.fail (http://127.0.0.1:49934/root_dart/tests/co19/src/Utils/expect_common.dart:79:5)
                    #2      testFailed (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/testcommon.dart:102:31)
                    #3      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:31:7)
                    #4      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #5      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #6      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #7      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #8      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #9      main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #10     main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #11     main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #12     main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #13     main.waitForStylesheetLoad (http://127.0.0.1:49934/root_dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart:35:37)
                    #14     Future.Future.delayed.<anonymous closure> (dart:async/future.dart:268)
                    #15     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18)
                    #16     _Timer._Timer.<anonymous closure> (dart:html:49221)
                    
                    
              timestamp: 0.583
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = false, isStatusUpdate = false, isDone = true)
              timestamp: 0.583
        },
        {
              type: dom
              value: <head>
                      <meta http-equiv="X-UA-Compatible" content="IE=edge">
                      <meta name="dart.unittest" content="full-stack-traces">
                      <title> Test /E:/b/build/slave/dartium-win-ia32-inc-be/build/src/dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart </title>
                      <style>
                         .unittest-table { font-family:monospace; border:1px; }
                         .unittest-pass { background: #6b3;}
                         .unittest-fail { background: #d55;}
                         .unittest-error { background: #a11;}
                      </style>
                    <link rel="stylesheet" media="screen and (max-device-aspect-ratio: 100/1)" href="/root_dart/tests/co19/src/LayoutTests/fast/css/resources/device-aspect-ratio.css"></head>
                    <body><p id="description"><span><p>This text is green if the max-device-aspect-ratio media query works properly.</p></span></p><div id="console"></div>
                      <h1> Running /E:/b/build/slave/dartium-win-ia32-inc-be/build/src/dart/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart </h1>
                      <script type="text/javascript" src="/root_dart/tools/testing/dart/test_controller.js">
                      </script>
                      <script type="application/dart" src="/root_build/generated_tests/none-dartium-checked/short8953_ss_max-device-aspect-ratio_t01/test.dart" onerror="scriptTagOnErrorCallback(null)" defer="">
                      </script>
                      <script type="text/javascript" src="/root_dart/pkg/browser/lib/dart.js"></script>
                    
                    </body>
              timestamp: 0.583
        },
  ]
  



stderr:


BrowserOutput while running the test (* EXPERIMENTAL *):
BrowserOutput.stdout:
  
BrowserOutput.stderr:
  



To retest, run:  E:\b\build\slave\dartium-win-ia32-inc-be\build\src\dart\tools\sdks\win\dart-sdk\bin\dart.exe E:\b\build\slave\dartium-win-ia32-inc-be\build\src\dart\tools\testing\dart\http_server.dart -p 49934 -c 49936 --build-directory=E:\b\build\slave\dartium-win-ia32-inc-be\build\src\out\Release\ --runtime=dartium --packages=E:\b\build\slave\dartium-win-ia32-inc-be\build\src\dart\.packages

Command[dartium]: E:\b\build\slave\dartium-win-ia32-inc-be\build\src\dart\tools\sdks\win\dart-sdk\bin\dart.exe tools/testing/dart/launch_browser.dart dartium http://127.0.0.1:49934/root_build/generated_tests/none-dartium-checked/short8953_ss_max-device-aspect-ratio_t01/test.html?crossOriginPort=49936
Took 0:00:00.601067

Short reproduction command (experimental):
    python tools/test.py -mrelease -rdartium --checked -t60 co19/LayoutTests/fast/css/max-device-aspect-ratio_t01

[DmitryOlshansky-google]

Another instance of that on Mac x64:

FAILED: none-dartium-checked release_x64 co19/LayoutTests/fast/css/device-aspect-ratio_t01
Expected: Pass 
Actual: RuntimeError
CommandOutput[dartium]:

stdout:
events:
  [
        {
              type: debug
              value: test_controller.js started
              timestamp: 0
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = true, isStatusUpdate = true, isDone = false)
              timestamp: 0.001
        },
        {
              type: print
              value: dart-calling-main
              timestamp: 0.781
        },
        {
              type: print
              value: unittest-suite-wait-for-done
              timestamp: 1.744
        },
        {
              type: print
              value: dart-main-done
              timestamp: 1.822
        },
        {
              type: message_received
              value: dart-calling-main
              timestamp: 1.827
        },
        {
              type: message_received
              value: unittest-suite-wait-for-done
              timestamp: 1.828
        },
        {
              type: message_received
              value: dart-main-done
              timestamp: 1.828
        },
        {
              type: window_onerror
              value: window.onerror called: 
                    
                    http://127.0.0.1:52750/root_build/generated_tests/none-dartium-checked/tests_co19_src_LayoutTests_fast_css_device-aspect-ratio_t01/test.html?crossOriginPort=52753:1:
                    Uncaught Unhandled exception:
                    Expect.fail('Stylesheet did not load in 1.0 second(s).')
                    #0      _fail (http://127.0.0.1:52750/root_dart/tests/co19/src/Utils/expect.dart:17:5)
                    #1      Expect.fail (http://127.0.0.1:52750/root_dart/tests/co19/src/Utils/expect_common.dart:79:5)
                    #2      testFailed (http://127.0.0.1:52750/root_dart/tests/co19/src/LayoutTests/testcommon.dart:102:31)
                    #3      main.waitForStylesheetLoad (http://127.0.0.1:52750/root_dart/tests/co19/src/LayoutTests/fast/css/device-aspect-ratio_t01.dart:34:7)
                    #4      Future.Future.delayed.<anonymous closure> (dart:async/future.dart:268)
                    #5      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18)
                    #6      _Timer._Timer.<anonymous closure> (dart:html:49221)
                    
                    
              timestamp: 7.181
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = false, isStatusUpdate = false, isDone = true)
              timestamp: 7.182
        },
        {
              type: dom
              value: <head id="myhead">
                      <meta http-equiv="X-UA-Compatible" content="IE=edge">
                      <meta name="dart.unittest" content="full-stack-traces">
                      <title> Test /b/build/slave/dartium-mac-x64-inc-be/build/src/dart/tests/co19/src/LayoutTests/fast/css/device-aspect-ratio_t01.dart </title>
                      <style>
                         .unittest-table { font-family:monospace; border:1px; }
                         .unittest-pass { background: #6b3;}
                         .unittest-fail { background: #d55;}
                         .unittest-error { background: #a11;}
                      </style>
                    <link rel="stylesheet" media="screen and (device-aspect-ratio: 1024/768" href="/root_dart/tests/co19/src/LayoutTests/fast/css/resources/device-aspect-ratio.css"></head>
                    <body><p id="description"><span><p>This text is green if the device-aspect-ratio media query works properly.</p></span></p><div id="console"></div>
                      <h1> Running /b/build/slave/dartium-mac-x64-inc-be/build/src/dart/tests/co19/src/LayoutTests/fast/css/device-aspect-ratio_t01.dart </h1>
                      <script type="text/javascript" src="/root_dart/tools/testing/dart/test_controller.js">
                      </script>
                      <script type="application/dart" src="/root_build/generated_tests/none-dartium-checked/tests_co19_src_LayoutTests_fast_css_device-aspect-ratio_t01/test.dart" onerror="scriptTagOnErrorCallback(null)" defer="">
                      </script>
                      <script type="text/javascript" src="/root_dart/pkg/browser/lib/dart.js"></script>
                    
                    </body>
              timestamp: 7.182
        },
  ]
  



stderr:


BrowserOutput while running the test (* EXPERIMENTAL *):
BrowserOutput.stdout:
  
BrowserOutput.stderr:
  



To retest, run:  /b/build/slave/dartium-mac-x64-inc-be/build/src/dart/tools/sdks/mac/dart-sdk/bin/dart /b/build/slave/dartium-mac-x64-inc-be/build/src/dart/tools/testing/dart/http_server.dart -p 52750 -c 52753 --build-directory=/b/build/slave/dartium-mac-x64-inc-be/build/src/out/Release/ --runtime=dartium --packages=/b/build/slave/dartium-mac-x64-inc-be/build/src/dart/.packages

Command[dartium]: /b/build/slave/dartium-mac-x64-inc-be/build/src/dart/tools/sdks/mac/dart-sdk/bin/dart tools/testing/dart/launch_browser.dart dartium http://127.0.0.1:52750/root_build/generated_tests/none-dartium-checked/tests_co19_src_LayoutTests_fast_css_device-aspect-ratio_t01/test.html?crossOriginPort=52753
Took 0:00:31.361253

Short reproduction command (experimental):
    python tools/test.py -mrelease -rdartium --checked -t60 co19/LayoutTests/fast/css/device-aspect-ratio_t01

[nex3]

@alan-knight I'm working on getting more gardening issues assigned. If you're not the right person to assign this to, please reassign to that person.

[floitschG]

Same test flaked again:

https://build.chromium.org/p/client.dart/builders/dart2js-win7-chrome-2-4-be/builds/1022/steps/dart2js-chrome-co19%20tests/logs/stdio

FAILED: dart2js-chrome release_x64 co19/LayoutTests/fast/css/max-device-aspect-ratio_t01
Expected: Pass 
Actual: RuntimeError
CommandOutput[dart2js]:

stdout:
Compiled 3,832,620 characters Dart to 282,622 characters JavaScript in 5.77 seconds
Dart file (E:\b\build\slave\me_2-be\build\sdk\tests\co19\src\LayoutTests\fast\css\max-device-aspect-ratio_t01.dart) compiled to JavaScript: out/ReleaseX64/generated_compilations/dart2js-sdk/short6825_ss_max-device-aspect-ratio_t01/test.js

CommandOutput[chrome]:

stdout:
events:
  [
        {
              type: debug
              value: test_controller.js started
              timestamp: 0
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = true, isStatusUpdate = true, isDone = false)
              timestamp: 0
        },
        {
              type: print
              value: dart-calling-main
              timestamp: 0.088
        },
        {
              type: print
              value: unittest-suite-wait-for-done
              timestamp: 0.094
        },
        {
              type: print
              value: dart-main-done
              timestamp: 0.119
        },
        {
              type: message_received
              value: dart-calling-main
              timestamp: 0.119
        },
        {
              type: message_received
              value: unittest-suite-wait-for-done
              timestamp: 0.119
        },
        {
              type: message_received
              value: dart-main-done
              timestamp: 0.12
        },
        {
              type: window_onerror
              value: window.onerror called: 
                    
                    http://127.0.0.1:60870/root_build/generated_compilations/dart2js-sdk/short6825_ss_max-device-aspect-ratio_t01/test.js:1411:
                    Uncaught 
                    
                    
              timestamp: 0.227
        },
        {
              type: debug
              value: Sending events to driver page (isFirstMessage = false, isStatusUpdate = false, isDone = true)
              timestamp: 0.227
        },
        {
              type: dom
              value: <head>
                      <meta http-equiv="X-UA-Compatible" content="IE=edge">
                      <meta name="dart.unittest" content="full-stack-traces">
                      <title> Test /E:/b/build/slave/me_2-be/build/sdk/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart </title>
                      <style>
                         .unittest-table { font-family:monospace; border:1px; }
                         .unittest-pass { background: #6b3;}
                         .unittest-fail { background: #d55;}
                         .unittest-error { background: #a11;}
                      </style>
                    <link rel="stylesheet" media="screen and (max-device-aspect-ratio: 100/1)" href="/root_dart/tests/co19/src/LayoutTests/fast/css/resources/device-aspect-ratio.css"></head>
                    <body><p id="description"><span><p>This text is green if the max-device-aspect-ratio media query works properly.</p></span></p><div id="console"></div>
                      <h1> Running /E:/b/build/slave/me_2-be/build/sdk/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart </h1>
                      <script type="text/javascript" src="/root_dart/tools/testing/dart/test_controller.js">
                      </script>
                      <script type="text/javascript" src="/root_build/generated_compilations/dart2js-sdk/short6825_ss_max-device-aspect-ratio_t01/test.js" onerror="scriptTagOnErrorCallback(null)" defer="">
                      </script>
                      <script type="text/javascript" src="/root_dart/pkg/browser/lib/dart.js"></script>
                    
                    </body>
              timestamp: 0.227
        },
  ]
  



stderr:


BrowserOutput while running the test (* EXPERIMENTAL *):
BrowserOutput.stdout:
  
BrowserOutput.stderr:
  



To retest, run:  E:\b\build\slave\me_2-be\build\sdk\tools\sdks\win\dart-sdk\bin\dart.exe E:\b\build\slave\me_2-be\build\sdk\tools\testing\dart\http_server.dart -p 60870 -c 60871 --build-directory=E:\b\build\slave\me_2-be\build\sdk\out\ReleaseX64\ --runtime=chrome --packages=E:\b\build\slave\me_2-be\build\sdk\.packages

Command[dart2js]: set DART_CONFIGURATION=ReleaseX64 & out\ReleaseX64\dart-sdk\bin\dart2js.bat --generate-code-with-compile-time-errors --test-mode --packages=E:\b\build\slave\me_2-be\build\sdk\.packages --out=E:/b/build/slave/me_2-be/build/sdk/out/ReleaseX64/generated_compilations/dart2js-sdk/short6825_ss_max-device-aspect-ratio_t01/test.js /E:/b/build/slave/me_2-be/build/sdk/tests/co19/src/LayoutTests/fast/css/max-device-aspect-ratio_t01.dart
Took 0:00:05.825000

Command[chrome]: E:\b\build\slave\me_2-be\build\sdk\tools\sdks\win\dart-sdk\bin\dart.exe tools/testing/dart/launch_browser.dart chrome http://127.0.0.1:60870/root_build/generated_tests/dart2js-chrome-sdk/short6824_ss_max-device-aspect-ratio_t01/test.html?crossOriginPort=60871
Took 0:00:00.258386

Short reproduction command (experimental):
    python tools/test.py -m release -c dart2js -r chrome --use-sdk --dart2js-batch --reset-browser-configuration --builder-tag win7 co19/LayoutTests/fast/css/max-device-aspect-ratio_t01

[floitschG]

Happened again.
Marking the test as flaky.

[terrylucas]

Created co19 issues see dart-lang/co19#116