WIP: templates

[simonw]

Refs:

• Mechanism for storing prompt templates #23

TODO:

• Implement the llm -t name-of-template bit
• Implement additional named parameters with llm -t name-of-template -p name value
• Tests
• Implement the structure for the YAML - prompt, system and model (specifying default model) as a starting point
• If a YAML file is just a string, load it as prompt: that-string
• Better display for llm templates list
• Documentation
• Some example templates

[simonw]

Docs preview here: https://llm--30.org.readthedocs.build/en/30/templates.html

[simonw]

Well this is fun...

% llm templates show summary
prompt: 'Summarize this: $input'
system: You speak like an enthusiastic Victorian explorer ALL THE TIME, with lots
    of flourishes

And then:

curl -s 'https://simonwillison.net/2023/Jun/15/when-zeppelins-ruled-the-earth/' \
  | strip-tags -m | llm -t summary -m 4

Huzzah! Gather 'round, dear friends, for I shall regale thou with an enthralling tale of Mr. Simon Willison's Weblog! He hath unveiled a captivating presentation from a bygone era - a wondrous time when Zeppelins ruled the Earth! Thou shalt revel in the delight of this aural and visual melange, crafted through the ingenious combination of yon historical slides and an MP3 recording, dating back to the 28th of October in the year 2008. Thus, embark on a jaunt through recent chronicles, touching on topics as diverse as understanding GPT tokenizers, adventures with ChatGPT, and courtroom communique with a dash of artifice. Verily, let me assure you, my good sirs and madams, that thou shalt procure nothing short of sheer intellectual delight in the exploration of these venerable pages!

[sderev]

Templates are really useful, yes.

For now, I use scripts for this purpose, which reduce the amount of typing at its minimum via autocompletion, and make it even easier to call in vim with a shebang command (:.!command arg).

Some examples of what I use: a thesaurus and a dictionary. Very short template, but also very useful.

Thesaurus

% thesaurus an old car
Synonyms: vintage car, classic car, antique car, retro car, dated car, ancient car

Antonyms: new car, modern car, contemporary car, state-of-the-art car, cutting-edge car, advanced car

Script:

#! /usr/bin/bash

# Check if any arguments were passed to the script
if [[ -n "$@" ]]; then
    # Combine all arguments into a single string
    input="$*"
else
    # Read input from stdin
    read input
fi

# Call the `llm` command with the modified input
llm --system "You are a multilingual thesaurus. You give synonyms and antonyms for words in the language they are provided in." "$input" --stream

Dictionary

% define nimbleness
Definition: the quality of moving quickly and lightly; agility

Example sentence: The gymnast's nimbleness allowed her to perform the complicated routine with ease.

Script:

#! /usr/bin/bash

# Check if any arguments were passed to the script
if [[ -n "$@" ]]; then
    # Combine all arguments into a single string
    input="$*"
else
    # Read input from stdin
    read input
fi

# Call the `llm` command with the modified input
llm --system "You are a multilingual dictionary. Define the given word and create an example sentence using it, ensuring that both are in the language the word is most commonly used. If the language isn't clear, use English." "$input" --stream

Following the same idea, I also use translate word with another script. I'm gonna create other short commands like that from my prompts repertoire. I'm all ears for other practical ideas. The summary piping curl, strip-tag, and llm is a very good one :).

[simonw]

Those examples are great, thanks for that! Would you be OK with me including those prompts as examples in the documentation? Will credit you in the release notes.

[sderev]

Sure, go ahead! I appreciate being credited :)

Another one I use a lot is this one: proofread

% proofread "Who's book is this?"
This sentence should be written as "Whose book is this?" The contraction "who's" is a combination of "who" and "is" and does not fit in this sentence. "Whose" is the correct possessive form of "who", indicating that this book belongs to someone and asking for the owner's identity. The syntax and overall structure of the sentence are correct.

Script:

#! /usr/bin/bash

# Check if any arguments were passed to the script
if [[ -n "$@" ]]; then
    # Combine all arguments into a single string
    input="$*"
else
    # Read input from stdin
    read input
fi  

# Call the `llm` command with the modified input
llm --system "You are an expert English writer and grammarian. Review the following English sentence for grammatical correctness, syntax, and overall structure, and provide feedback as necessary." "$input" --stream

[sderev]

I created a repo to host some of my tools and the scripts I built from llm—fixing the stdin bug from my previous code above, which was reading only the first line, and not the complete file.

It can be found here: https://github.com/sderev/llm-toolbox

[simonw]

Here's how to detect the named variables in a string.Template:

>>> from string import Template
>>> t = Template("$foo and $bar")
>>> t.pattern
re.compile('\n            \\$(?:\n              (?P<escaped>\\$)  |   # Escape sequence of two delimiters\n              (?P<named>(?a:[_a-z][_a-z0-9]*))       |   # delimiter and a Python identifier\n          , re.IGNORECASE|re.VERBOSE)
>>> list(t.pattern.finditer(t.template))
[<re.Match object; span=(0, 4), match='$foo'>, <re.Match object; span=(9, 13), match='$bar'>]
>>> matches = list(t.pattern.finditer(t.template))
>>> matches
[<re.Match object; span=(0, 4), match='$foo'>, <re.Match object; span=(9, 13), match='$bar'>]
>>> matches[0]
<re.Match object; span=(0, 4), match='$foo'>
>>> matches[0][1]
>>> matches[0][0]
'$foo'
>>> matches[1][0]
'$bar'
>>> matches[0].groups()
(None, 'foo', None, None)
>>> matches[1].groups()
(None, 'bar', None, None)
>>> matches[1].groupdict()
{'escaped': None, 'named': 'bar', 'braced': None, 'invalid': None}

match.group('named') should do it.

[simonw]

Here's a problem: thanks to this code it isn't currently possible to pipe content to the tool AND specify extra params:

llm/llm/cli.py

Lines 71 to 73 in 0fc51f4

	if prompt is None and not param:
	# Read from stdin instead
	prompt = sys.stdin.read()

So this doesn't work:

cat file.txt | llm -t summarize -p voice GlaDOS

[simonw]

I can use sys.stdin.isatty() to try and detect that.

[simonw]

Documentation: https://llm--30.org.readthedocs.build/en/30/templates.html