Fix the "improve" prompt to make sure that it generates diffs, and parse and apply those diffs to the existing codebase

[AntonOsika]

One way to do this is to write the prompt for gpt-engineer with -i flag to annotate each codeblock with one of:

1. NEW CODE
2. REPLACING ONE FUNCTION

If 1., the generated code can just be written to a new file (or appended to an existing file).

If it is replacing an existing function, we could make sure to find the name of the function that is being replaced using an AST parser (see how here)

Why this is necessary

As an example, I tried to use it on the project itself and got a codeblock that was just changing one of the function (so it should not be used to overwrite the entire file)

How to do it

We can take inspiration from Aider, that generates diffs, or sweep in how they prompt for "<copy_lines>" and parse the GPT4 output here

Should be quite straightforward!

[UmerHA]

Hey, I'd take this up. eta a few days

[UmerHA]

@AntonOsika I think replacing functions is not quite the right approach, because not every code is neatly made up of functions (eg in python we have code outside functions, classes, nested functions). What do you think about using git style diff files instead?

So we'd ask the LLM to produce something like this:

filename.py
second line before code change # used for locating changes
first line before code change
new code
new code
new code
first line after code change
second line after code change

I'd ask the LLM to add 2 lines before and after the code change to help us locate the change. The alternative would be asking for line numbers, but as LLMs are notoriously bad with numbers, I'd not go that way.

We would then parse the above output into a git diff file

--- a/filename.py
+++ b/filename.py
@@ -1,2 +1,3 @@
-original content
+changed content
unchanged content

What do you think?

[AntonOsika]

Awesome.

I looked at it took, I think we should first try using this prompt.

It should be possible to use it by doing something like loading EditBlockCoder here and then:

coder = EditBlockCoder()  # might need some args to instansiatie
coder.partial_response_content = "the GPT4 response with diff from the prompt above"   # (one could just paste prompt in chatgpt and get it from there

coder.apply_edits(coder.get_edits())

And then see what happens.

If we use code directly from Aider, or we use it for inspiration, we should give credits to Paul (the creator) in an inline comment 🙏