Add standardised benchmarking capabilities, changes to lean_agent

[AntonOsika]

Benchmarking

See benchmark/types to understand how to implement a Benchmark.

New benchmarks are added as a new folder to gpt_engineer/benchmark/benchmarks and with a new entry to gpt_engineer/benchmark/benchmarks/load.py

To make it possible to benchmark an agent, just create a function default_config_agent that returns an instantiated Agent, and then pass the path to that function to the benchmark CLI tool.

How to run e.g. lean_agent on the gpteng benchmark:
python gpt_engineer/benchmark gpt_engineer/core/default/lean_agent gpteng

Results below:

Note that we add local caching to the LLM. Identical calls will reuse the cache stored in .langchain.db. Please delete this file if you want to check for how temperature affects the results.

Also – this is not meant to be merged to refactor. We should merge to main after refactor is merged!

[sweep-ai]

Apply Sweep Rules to your PR?

• Apply: Ensure all new functions and classes have very clear, concise and up-to-date docstrings. Take gpt_engineer/ai.py as a good example.