Split cypress tests #6241

timifasubaa · 2018-10-31T08:32:32Z

This PR reduces the cypress runtime from ~18 minutes to ~11 minutes. It uses a few techniques listed below:

splitting the tests into several independent components run in parallel
using subprocesses to run non-dependent build up step in each test in parallel
removes the recording of video that we weren't using

It also times each subcomponent so we can track how the times are growing

@mistercrunch @michellethomas @john-bodley

codecov-io · 2018-10-31T09:12:16Z

Codecov Report

Merging #6241 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6241   +/-   ##
=======================================
  Coverage   77.31%   77.31%           
=======================================
  Files          67       67           
  Lines        9581     9581           
=======================================
  Hits         7408     7408           
  Misses       2173     2173

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb55668...741c06f. Read the comment docs.

michellethomas · 2018-11-13T01:54:33Z

superset/assets/cypress_build.sh

-npm run build
-npm run cypress run
+#run all the python steps in a background process
+(time /home/travis/build/apache/incubator-superset/superset/bin/superset db upgrade; time /home/travis/build/apache/incubator-superset/superset/bin/superset load_test_users; /home/travis/build/apache/incubator-superset/superset/bin/superset load_examples; time /home/travis/build/apache/incubator-superset/superset/bin/superset init; echo "[completed python build steps]"; flask run -p 8081 --with-threads --reload --debugger) &


did you mean to leave the time statements in here?

yeah, I left it there so we can see how long each subpart takes

michellethomas · 2018-11-13T02:05:42Z

🙏 thanks so much for doing this 👍

john-bodley · 2018-11-13T07:32:50Z

.travis.yml

@@ -4,7 +4,33 @@ jobs:
  include:
    - language: python
      python: 3.6
-      env: TOXENV=cypress
+      env: TOXENV=cypress_dashboard


Using a matrix here might help to reduce the repetition.

john-bodley · 2018-11-13T07:33:42Z

tox.ini

@@ -45,9 +45,31 @@ setenv =
 whitelist_externals =
    npm

-[testenv:cypress]
+[testenv:cypress_dashboard]


The use of factor conditions for the command could help reduce the repetition.

john-bodley · 2018-11-14T19:58:31Z

superset/assets/cypress_build.sh

-npm run build
-npm run cypress run
+#run all the python steps in a background process
+(time /home/travis/build/apache/incubator-superset/superset/bin/superset db upgrade; time /home/travis/build/apache/incubator-superset/superset/bin/superset load_test_users; /home/travis/build/apache/incubator-superset/superset/bin/superset load_examples; time /home/travis/build/apache/incubator-superset/superset/bin/superset init; echo "[completed python build steps]"; flask run -p 8081 --with-threads --reload --debugger) &


Should we be using && rather than ; here per this?

I think either way is fine. They will both lead to a failed build if something goes wrong in the middle

john-bodley · 2018-11-14T20:00:31Z

superset/assets/cypress_build.sh

+#run all the python steps in a background process
+(time /home/travis/build/apache/incubator-superset/superset/bin/superset db upgrade; time /home/travis/build/apache/incubator-superset/superset/bin/superset load_test_users; /home/travis/build/apache/incubator-superset/superset/bin/superset load_examples; time /home/travis/build/apache/incubator-superset/superset/bin/superset init; echo "[completed python build steps]"; flask run -p 8081 --with-threads --reload --debugger) &
+
+#block on the longer running javascript process


It seems like we should have flask run run after we've run npm run build as then there's no need to background the task above.

npm run build takes a really long time esp when it freezes on 92%. So I decided to put in as much as possible in the background while it is running so as to save the time it will take to run if it were to run after npm run build

john-bodley · 2018-11-14T20:00:40Z

superset/assets/cypress_build.sh

+(time /home/travis/build/apache/incubator-superset/superset/bin/superset db upgrade; time /home/travis/build/apache/incubator-superset/superset/bin/superset load_test_users; /home/travis/build/apache/incubator-superset/superset/bin/superset load_examples; time /home/travis/build/apache/incubator-superset/superset/bin/superset init; echo "[completed python build steps]"; flask run -p 8081 --with-threads --reload --debugger) &
+
+#block on the longer running javascript process
+(time yarn install --frozen-lockfile; time npm run build; echo "[completed js build steps]")


See note ^^.

* aplit cypress tests * split into three

timifasubaa force-pushed the split_cypress_tests branch 5 times, most recently from d0c83a3 to ec218d0 Compare October 31, 2018 09:01

timifasubaa force-pushed the split_cypress_tests branch 10 times, most recently from 22888d1 to f050685 Compare November 10, 2018 00:32

aplit cypress tests

d32f635

timifasubaa force-pushed the split_cypress_tests branch 13 times, most recently from c2d83c3 to a070e4a Compare November 11, 2018 12:09

timifasubaa force-pushed the split_cypress_tests branch 11 times, most recently from 8840356 to 34fb0d6 Compare November 13, 2018 01:04

timifasubaa changed the title ~~[WIP] Split cypress tests~~ Split cypress tests Nov 13, 2018

timifasubaa force-pushed the split_cypress_tests branch 2 times, most recently from 697fac5 to f5a9033 Compare November 13, 2018 01:35

michellethomas reviewed Nov 13, 2018

View reviewed changes

john-bodley reviewed Nov 13, 2018

View reviewed changes

split into three

741c06f

timifasubaa force-pushed the split_cypress_tests branch from f5a9033 to 741c06f Compare November 13, 2018 22:59

timifasubaa closed this Nov 13, 2018

timifasubaa reopened this Nov 13, 2018

john-bodley reviewed Nov 14, 2018

View reviewed changes

john-bodley approved these changes Nov 15, 2018

View reviewed changes

timifasubaa merged commit c17de39 into apache:master Nov 15, 2018

bipinsoniguavus pushed a commit to ThalesGroup/incubator-superset that referenced this pull request Dec 26, 2018

Split cypress tests (apache#6241)

eaab6ec

* aplit cypress tests * split into three

ktmud mentioned this pull request Apr 9, 2020

[Build] Add Github workflows #9490

Closed

7 tasks

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.34.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split cypress tests #6241

Split cypress tests #6241

timifasubaa commented Oct 31, 2018 •

edited

Loading

codecov-io commented Oct 31, 2018 •

edited

Loading

michellethomas Nov 13, 2018

timifasubaa Nov 13, 2018

michellethomas commented Nov 13, 2018

john-bodley Nov 13, 2018

john-bodley Nov 13, 2018

john-bodley Nov 14, 2018

timifasubaa Nov 14, 2018 •

edited

Loading

john-bodley Nov 14, 2018

timifasubaa Nov 14, 2018

john-bodley Nov 14, 2018

Split cypress tests #6241

Split cypress tests #6241

Conversation

timifasubaa commented Oct 31, 2018 • edited Loading

codecov-io commented Oct 31, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michellethomas commented Nov 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timifasubaa Nov 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timifasubaa commented Oct 31, 2018 •

edited

Loading

codecov-io commented Oct 31, 2018 •

edited

Loading

timifasubaa Nov 14, 2018 •

edited

Loading