-r/--reuse mode

[simonw]

When using this on the command-line it would be useful if you could opt to get a previous response for the same prompt/system/model/options - if one exists:

cat file.py | llm "explain this file" --reuse

That way you could safely run loops against larger amounts of data while avoiding paying for the same prompt twice.

[simonw]

This could work by calculating a SHA1 hash of prompt + system + options_json + model and storing that on an indexed column in the responses table - then looking it up again later.

One tricky aspect of this is supporting conversations - the hash should be calculated against the actual prompt that is sent to the LLM (after adding in the previous conversation messages), but that's currently put together inside the model.execute() method so the implementation would need to be updated in every plugin.

One solution to that: only support --reuse for prompts outside of a conversation at first - then figure out a mechanism by which models could support it for conversation mode later on.

One option there: have models implement an optional .reuse_hash(prompt, conversation) -> str method which can be called to figure out the reuse hash for a prompt and conversation without calling .execute().