Multi Agent

[anhhuyalex]

No description provided.

[cswinter]

Awesome! Some high level comments:

• Let's move all the _validate_x methods out of the MultiValidatingEnv into standalone helper methods and use them in ValidatingEnv so the code is not duplicated
• I wonder if some deduplication is possible for the ValidatingEnv as well? Since it already only expects a VecEnv, most of the code should just work the same way, except for any additional MultiEnvList specific metrics we might want to export.
• For testing, you can add a simple multiagent environment in env/examples and add a test case to tests/test_environment.py similar to test_env_list. Maybe a generic implementation of 2 player game with simultaneous actions and payoff matrix? That would also be useful for later checking that training is working correctly and we find the correct solution to classic game theory problems.
• If you want to go the extra mile, you could add some documentation for the multi agent environment (see the docs dir). You can build the docs by running poetry run make html and look at them at localhost:8000 by starting an http server poetry run python -m http.server 8000 in the docs/build/html dir.

[cswinter]

@anhhuyalex would love to get this merged! are you still interested in completing this?