pytest-split in CI

[janosh]

https://github.com/jerry-git/pytest-split is a pytest plugin that splits the test suite into equally sized splits based on test execution time.

[janosh]

Looks like this takes the runtime for the main text-linux workflow from ~18 min down to ~9 min. The number of parallel jobs went from 4 to 10. Setting up the repo and installing deps is a ~3.5 min constant time for every job. So probably not that useful to crank the parallel job count much higher.

@shyuep @mkhorton Wanted to get your opinion on this before merging since it's a bigger-than-usual change to the main CI workflow. We could add the same splitting to the mac and windows workflows. In fact, is there still a strong reason to not run the same test suite on all 3 platforms? Would make maintenance of the workflow files easier since we could get rid of .github/workflows/test-{max,win}.yml and simply add os to matrix strategy in .github/workflows/test-linux.yml.

[shyuep]

Sure this is fine.

[coveralls]

Coverage increased (+18.08%) to 81.711% when pulling 9af323a on pytest-split-in-ci into 302ca2c on master.

[janosh]

Coverage increased (+18.08%) to 81.711% when pulling 9af323a on pytest-split-in-ci into 302ca2c on master.

Just noticed the coverage increased by 18% in this PR. Is this because a fifth of the test suite wasn't running before with the old splitting setup?

include:
  - pkg: pymatgen/analysis/chemenv pymatgen/analysis/elasticity pymatgen/analysis/magnetism
    pkg_id: 1
  - pkg: pymatgen/analysis --ignore=pymatgen/analysis/chemenv --ignore=pymatgen/analysis/elasticity --ignore=pymatgen/analysis/magnetism
    pkg_id: 2
  - pkg: pymatgen/electronic_structure pymatgen/symmetry pymatgen/command_line pymatgen/ext
    pkg_id: 3
  - pkg: pymatgen --ignore=pymatgen/analysis --ignore=pymatgen/electronic_structure --ignore=pymatgen/symmetry --ignore=pymatgen/ext --ignore=pymatgen/command_line
    pkg_id: 4