Introduce caching of configured live accounts in each test process

[hpk42]

cache configured live account within a live test process, leading to a roughly 15% speed improvement on the overall test run as it removes an often multi-second overhead for the setup of live accounts from most tests. You can measure with "pytest --durations=10 tests/bench_test_setup.py".

This cache is only an in-process RAM dictionary for caching successfully configured live accounts from/to the filesystem.
#skip-changelog

[link2xt]

https://pypi.org/project/py/ description says to use https://docs.python.org/3/library/pathlib.html instead

[hpk42]

yes but usage is minimal, and easy enough to get ported if needed. While i co-authored py, i didn't co-author pathlib :) and as we are not using in the app or in the binbdings, it's fine to keep using py.

[hpk42]

(pytest has a dependency on py)

[link2xt]

Looks like pytest devs are constantly working on replacing py with pathlib everywhere: pytest-dev/pytest#7259
Once this work in finished, we'll have to port py usage to pathlib too.

[hpk42]

k, i ported it. it's more code and it took me a bit to learn the pathlib API. Requires a bit more calls than what i like for this 2-3 line code thing. Also i think moving away from tmpdir/py.path.local is going to take quite a while anyway. Many of our tests use it and i wouldn't start the porting until it's really needed.

[link2xt]

Looks like pytest devs are constantly working on replacing py with pathlib everywhere: pytest-dev/pytest#7259
Once this work in finished, we'll have to port py usage to pathlib too.

[link2xt]

Why not just store a single file with JSON or pickle? This is like 1 lines instead of going through files in a directory:

from pathlib import Path
with Path(target_dir).open('w') as fp: json.dump(dic, fp)

And why store to files at all if nothing is persisted between pytest runs? Each time temporary directory is new and all stored dictionaries are lost.

[hpk42]

Why not just store a single file with JSON or pickle? This is like 1 lines instead of going through files in a directory:

from pathlib import Path
with Path(target_dir).open('w') as fp: json.dump(dic, fp)

There is no persistence for the caching, on purpose, and to keep it simple. Each testprocess just maintains an in-process cache so we don't need to care for outdated accounts (CI email accounts are valid for one hour).
Note that the code transfers a directory of files into a dict, and from a dict to a directory. It's a cheap way to record and replay a DC database dir with blob files and "shm" and "wal" files etc.