feat(autoedit): Support image rendering for complex diffs (#6545)

<img width="542" alt="image"
src="https://github.com/user-attachments/assets/68601e33-8f0d-4ce1-a911-73d55abe8789"
/>

## Description

This PR adds support for generating diff-like images of code snippets,
that we can use to render auto-edit suggestions to the user.

We are using [CanvasKit](https://skia.org/docs/user/modules/canvaskit/),
which has good cross platform support (a WASM binary is provided) and
means we can efficiently draw and paint text to a canvas. We can then
convert this to a PNG for rendering as a decoration (via
`contentIconPath`).

We are also using [Shiki](https://shiki.matsu.io/), which provides
syntax highlighting for our code snippets. Unfortunately we cannot
access the users' selected theme colors programmatically ([see this
issue](microsoft/vscode#32813)).


### Design Considerations

Due to some limitations in the VS Code API, I have taken some design
considerations which I think work well - but I am happy to discuss
alternative solutions.

**Syntax Highlighting theme**: We cannot access the users' syntax
highlighting colors, so we use [Vitesse
Dark/Light](https://shiki.style/themes#:~:text=Vitesse%20Dark,vitesse%2Dlight).
From my testing, these colors are fairly neutral and work well, but
could possibly be an eye sore for users if it doesn't match their
preferred highlight colors.

**Image background**: I have made the background of the image
transparent, this is because we can actually provide our own background
through the VS Code decoration options. So we *can* theme the background
and border to match the users editor. This means that it many cases the
suggestions look great. There is a possibility that this could clash
with the highlighting theme we have chosen though. The alternative would
be to provide our own background color and accept it would likely not
match the users' theme. Note: We may have to do this for non-VSCode
clients anyway.

**Font**: We are using [DejaVu Sans
Mono](https://dejavu-fonts.github.io/). This is a publicly available
font that renders a wide range of code snippets clearly, from my
testing. It is also the font that the Menlo font (default on MacOS) is
based off, so many users are likely to be familiar with it. We do not
currently adjust the image to match the users' font size, but this is
something we should be able to do. Likewise with the users' line height.


### Screenshots

**Suggestion Dark**
<img width="514" alt="image"
src="https://github.com/user-attachments/assets/1ec2d178-7c75-4ae1-b5b9-8568c6022bec"
/>

**Suggestion Light**
<img width="512" alt="image"
src="https://github.com/user-attachments/assets/4ab1e670-1cf3-4bdf-a913-ffd8a9e7db73"
/>

**Suggestion Alternative Theme A**
<img width="497" alt="image"
src="https://github.com/user-attachments/assets/046dbc1d-d32b-4d6f-a326-8ac66ec94674"
/>

**Suggestion Alternative Theme B**
<img width="529" alt="image"
src="https://github.com/user-attachments/assets/460b9b85-3a93-4e37-aae1-32e5603d920e"
/>

**Raw Image**

![image](https://github.com/user-attachments/assets/47e3245c-fbc9-4bfc-b567-91714168f8d0)

## How to manually test

1. Checkout this branch and run it locally in VS Code
2. Ensure that the following settings are set, to enable auto-edit and
the image generation
```
    "cody.suggestions.mode": "auto-edit (Experimental)",
    "cody.experimental.autoedit.renderer": "image",
```
3. Try triggering some auto-edits!


# Test plan

1. Enable auto edits
4. Generate suggestions where lines are expected to be added/modified
5. Observe results

Also tested with E2E and unit visual tests

vscode/.gitignore

@@ -11,3 +11,6 @@ tsconfig.typehacks.json
 
 # these are snapshots that haven't been accepted
 *.new.json
+
+# jest-image-snapshot debugging files
+__diff_output__

vscode/.vscodeignore

@@ -13,6 +13,7 @@
 !dist/*.exe
 !dist/*.wasm
 !dist/*.node
+!dist/*.ttf
 
 !resources/*
 !walkthroughs/*.md

vscode/package.json

@@ -10,7 +10,7 @@
   "description": "Sourcegraph’s AI code assistant goes beyond individual dev productivity, helping enterprises achieve consistency and quality at scale with AI. & codebase context to help you write code faster. Cody brings you autocomplete, chat, and commands, so you can generate code, write unit tests, create docs, and explain complex code using AI. Choose from the best LLMs, including GPT-4o and Claude 3.5 Sonnet.",
   "scripts": {
     "build:root": "pnpm -C .. run -s build",
-    "postinstall": "pnpm download-wasm && pnpm copy-win-ca-roots",
+    "postinstall": "pnpm download-wasm && pnpm download-fonts && pnpm copy-win-ca-roots",
     "dev": "pnpm run -s dev:desktop",
     "dev:insiders": "pnpm run -s dev:desktop:insiders",
     "start:dev:desktop": "NODE_ENV=development code --extensionDevelopmentPath=$PWD --disable-extension=sourcegraph.cody-ai --disable-extension=github.copilot --inspect-extensions=9333 --new-window . --goto ./src/completions/inline-completion-item-provider.ts:16:5",
@@ -30,16 +30,17 @@
     "build:prod:webviews": "pnpm run -s _build:webviews --mode production",
     "watch:build:dev:web": "concurrently \"pnpm run -s _build:esbuild:web --watch\" \"pnpm run -s _build:webviews --mode development --watch\"",
     "watch:build:dev:desktop": "concurrently \"pnpm run -s _build:esbuild:desktop --watch\" \"pnpm run -s _build:webviews --mode development --watch\"",
-    "_build:esbuild:desktop": "pnpm download-wasm && pnpm run -s _build:esbuild:uninstall && pnpm run -s _build:esbuild:node",
+    "_build:esbuild:desktop": "pnpm download-wasm && pnpm download-fonts && pnpm run -s _build:esbuild:uninstall && pnpm run -s _build:esbuild:node",
     "_build:esbuild:node": "esbuild ./src/extension.node.ts --bundle --outfile=dist/extension.node.js --loader:.node=copy --external:vscode --external:typescript --alias:@sourcegraph/cody-shared=@sourcegraph/cody-shared/src/index --alias:@sourcegraph/cody-shared/src=@sourcegraph/cody-shared/src --alias:lexical=./build/lexical-package-fix --format=cjs --platform=node --sourcemap --target=es2022",
     "_build:esbuild:web": "esbuild ./src/extension.web.ts --platform=browser --bundle --outfile=dist/extension.web.js --alias:@sourcegraph/cody-shared=@sourcegraph/cody-shared/src/index --alias:@sourcegraph/cody-shared/src=@sourcegraph/cody-shared/src --alias:path=path-browserify --external:typescript --alias:node:path=path-browserify --alias:node:os=os-browserify --alias:os=os-browserify --external:vscode  --external:node:child_process  --external:node:util --external:node:fs --external:node:fs/promises --external:node:process --define:process='{\"env\":{}}' --define:window=self --format=cjs --sourcemap",
     "_build:esbuild:uninstall": "node ./uninstall/esbuild.mjs",
     "_build:webviews": "vite -c webviews/vite.config.mts build",
     "_build:vsix_for_test": "vsce package --no-dependencies --out dist/cody.e2e.vsix",
     "release": "ts-node-transpile-only ./scripts/release.ts",
     "download-wasm": "ts-node-transpile-only ./scripts/download-wasm-modules.ts",
+    "download-fonts": "ts-node-transpile-only ./scripts/download-fonts.ts",
     "copy-win-ca-roots": "ts-node-transpile-only ./scripts/copy-win-ca-roots.ts",
-    "release:dry-run": "pnpm run download-wasm && CODY_RELEASE_DRY_RUN=1 ts-node ./scripts/release.ts",
+    "release:dry-run": "pnpm run download-wasm && pnpm run download-fonts && CODY_RELEASE_DRY_RUN=1 ts-node ./scripts/release.ts",
     "storybook": "storybook dev -p 6007 --no-open --no-version-updates",
     "test:e2e": "playwright install && tsc --build && node dist/tsc/test/e2e/install-deps.js && pnpm run -s _build:vsix_for_test && pnpm run -s build:dev:desktop && pnpm run -s test:e2e:run",
     "test:e2e:run": "playwright test",
@@ -1479,6 +1480,7 @@
     "agent-base": "^7.1.1",
     "async-mutex": "^0.4.0",
     "axios": "^1.3.6",
+    "canvaskit-wasm": "^0.39.1",
     "class-variance-authority": "^0.7.0",
     "classnames": "^2.5.1",
     "clsx": "^2.1.1",
@@ -1515,6 +1517,7 @@
     "remark-gfm": "^4.0.0",
     "safe-stable-stringify": "^2.5.0",
     "semver": "^7.5.4",
+    "shiki": "^1.26.1",
     "signal-exit": "^4.1.0",
     "socks-proxy-agent": "^8.0.1",
     "tailwind-merge": "^2.3.0",
@@ -1552,6 +1555,7 @@
     "@types/glob": "^8.0.0",
     "@types/graceful-fs": "^4.1.9",
     "@types/ini": "^4.1.0",
+    "@types/jest-image-snapshot": "^6.4.0",
     "@types/js-levenshtein": "^1.1.1",
     "@types/lodash": "^4.14.195",
     "@types/marked": "^5.0.0",
@@ -1593,6 +1597,7 @@
     "htmlnano": "^2.1.1",
     "http-proxy-middleware": "^3.0.0",
     "immer": "^10.1.1",
+    "jest-image-snapshot": "^6.4.0",
     "keytar": "^7.9.0",
     "kill-sync": "^1.0.3",
     "mocha": "^10.2.0",

vscode/scripts/download-fonts.ts

@@ -0,0 +1,27 @@
+import { copyFileSync, existsSync, mkdirSync } from 'node:fs'
+import path from 'node:path'
+
+const DIST_DIRECTORY = path.join(__dirname, '../dist')
+const FONT_PATH = path.join(__dirname, '../resources/DejaVuSansMono.ttf')
+
+export async function main(): Promise<void> {
+    try {
+        copyFonts()
+        console.log('Fonts were successfully copied to dist directory')
+    } catch (error) {
+        console.error('Error copying fonts:', error)
+        process.exit(1)
+    }
+}
+
+void main()
+
+function copyFonts(): void {
+    const hasDistDir = existsSync(DIST_DIRECTORY)
+
+    if (!hasDistDir) {
+        mkdirSync(DIST_DIRECTORY)
+    }
+
+    copyFileSync(FONT_PATH, path.join(DIST_DIRECTORY, 'DejaVuSansMono.ttf'))
+}

vscode/scripts/download-wasm-modules.ts

@@ -14,6 +14,8 @@ const TREE_SITTER_WASM_FILE = 'tree-sitter.wasm'
 const TREE_SITTER_WASM_PATH = require.resolve(`web-tree-sitter/${TREE_SITTER_WASM_FILE}`)
 const JS_GRAMMAR_PATH = require.resolve('@sourcegraph/tree-sitter-wasms/out/tree-sitter-javascript.wasm')
 const GRAMMARS_PATH = path.dirname(JS_GRAMMAR_PATH)
+const CANVASKIT_WASM_FILE = 'canvaskit.wasm'
+const CANVASKIT_WASM_PATH = require.resolve(`canvaskit-wasm/bin/${CANVASKIT_WASM_FILE}`)
 
 export async function main(): Promise<void> {
     const hasStoreDir = existsSync(WASM_DIRECTORY)
@@ -50,4 +52,5 @@ function copyFilesToDistDir(): void {
     }
 
     copyFileSync(TREE_SITTER_WASM_PATH, path.join(DIST_DIRECTORY, TREE_SITTER_WASM_FILE))
+    copyFileSync(CANVASKIT_WASM_PATH, path.join(DIST_DIRECTORY, CANVASKIT_WASM_FILE))
 }

vscode/scripts/measure-bundle-size.ts

@@ -2,8 +2,8 @@ import { promises as fs } from 'node:fs'
 import { appendFileSync } from 'node:fs'
 
 const SIZE_LIMITS = {
-    extension: 15 * 1024 * 1024, // 15MB
-    webview: 10 * 1024 * 1024, // 10MB
+    extension: 20 * 1024 * 1024, // 20MB
+    webview: 15 * 1024 * 1024, // 15MB
 }
 
 function prettyPrintMB(bytes: number): string {

vscode/src/autoedits/autoedits-provider.ts

@@ -35,6 +35,7 @@ import type { DecorationInfo } from './renderer/decorators/base'
 import { DefaultDecorator } from './renderer/decorators/default-decorator'
 import { InlineDiffDecorator } from './renderer/decorators/inline-diff-decorator'
 import { getDecorationInfo } from './renderer/diff-utils'
+import { initImageSuggestionService } from './renderer/image-gen'
 import { AutoEditsInlineRendererManager } from './renderer/inline-manager'
 import { AutoEditsDefaultRendererManager, type AutoEditsRendererManager } from './renderer/manager'
 import {
@@ -81,9 +82,15 @@ export class AutoeditsProvider implements vscode.InlineCompletionItemProvider, v
     private readonly onSelectionChangeDebounced: DebouncedFunc<typeof this.onSelectionChange>
     public readonly rendererManager: AutoEditsRendererManager
     private readonly modelAdapter: AutoeditsModelAdapter
+
+    /**
+     * Default: Current supported renderer
+     * Inline: Experimental renderer that uses inline decorations to show additions
+     * Image: Experimental renderer that uses images to show additions.
+     */
     private readonly enabledRenderer = vscode.workspace
         .getConfiguration()
-        .get<'default' | 'inline'>('cody.experimental.autoedit.renderer', 'default')
+        .get<'default' | 'inline' | 'image'>('cody.experimental.autoedit.renderer', 'default')
 
     private readonly promptStrategy = new ShortTermPromptStrategy()
     public readonly filterPrediction = new FilterPredictionBasedOnRecentEdits()
@@ -101,14 +108,23 @@ export class AutoeditsProvider implements vscode.InlineCompletionItemProvider, v
             chatClient: chatClient,
         })
 
+        if (this.enabledRenderer === 'image') {
+            // Initialise the canvas renderer for image generation.
+            // TODO: Consider moving this if we decide to enable this by default.
+            initImageSuggestionService()
+        }
+
         this.rendererManager =
             this.enabledRenderer === 'inline'
                 ? new AutoEditsInlineRendererManager(
                       editor => new InlineDiffDecorator(editor),
                       fixupController
                   )
                 : new AutoEditsDefaultRendererManager(
-                      (editor: vscode.TextEditor) => new DefaultDecorator(editor),
+                      (editor: vscode.TextEditor) =>
+                          new DefaultDecorator(editor, {
+                              shouldRenderImage: this.enabledRenderer === 'image',
+                          }),
                       fixupController
                   )
 

vscode/src/autoedits/renderer/decorators/blockify.test.ts

@@ -21,7 +21,9 @@ function blockifyAndExtractForTest(
         expect(highlightedText).toBe(expectedText)
 
         addedLines.push({
-            ranges: [[range.start.character, range.end.character]],
+            highlightedRanges: [
+                { type: 'diff-added', range: [range.start.character, range.end.character] },
+            ],
             afterLine: range.start.line,
             lineText: document.lineAt(range.start.line).text,
         })
@@ -30,7 +32,9 @@ function blockifyAndExtractForTest(
     const blockified = blockify(document, addedLines)
     return {
         code: blockified.map(({ lineText }) => lineText).join('\n'),
-        ranges: blockified.flatMap(({ ranges }) => ranges),
+        ranges: blockified.flatMap(({ highlightedRanges }) =>
+            highlightedRanges.map(({ range }) => range)
+        ),
     }
 }
 

vscode/src/autoedits/renderer/decorators/blockify.ts

@@ -55,22 +55,21 @@ export function convertToSpaceIndentation(
         // this as we are converting this text to use spaces.
         // 1. Account for the fact that each tab is being replaced with tabSize spaces
         // 2. Adjust the position based on how many tabs appear before the range
-        const newRanges = line.ranges.map(([start, end]) => {
+        const newRanges = line.highlightedRanges.map(({ range: [start, end], ...rest }) => {
             // Count tabs before the start and end positions
             const tabsBeforeStart = (line.lineText.slice(0, start).match(/\t/g) || []).length
             const tabsBeforeEnd = (line.lineText.slice(0, end).match(/\t/g) || []).length
 
             // Each tab expands to tabSize spaces, so we need to add (tabSize - 1) for each tab
             const adjustedStart = start + tabsBeforeStart * (tabSize - 1)
             const adjustedEnd = end + tabsBeforeEnd * (tabSize - 1)
-
-            return [adjustedStart, adjustedEnd] as [number, number]
+            return { ...rest, range: [adjustedStart, adjustedEnd] as [number, number] }
         })
 
         return {
             ...line,
             lineText: newLineText,
-            ranges: newRanges,
+            highlightedRanges: newRanges,
         }
     })
 }
@@ -110,10 +109,13 @@ function removeLeadingWhitespaceBlock(
     return addedLines.map(line => ({
         ...line,
         lineText: line.lineText.replace(leastCommonWhitespacePrefix, ''),
-        ranges: line.ranges.map(([start, end]) => [
-            start - leastCommonWhitespacePrefix.length,
-            end - leastCommonWhitespacePrefix.length,
-        ]),
+        highlightedRanges: line.highlightedRanges.map(({ range: [start, end], ...rest }) => ({
+            ...rest,
+            range: [
+                Math.max(0, start - leastCommonWhitespacePrefix.length),
+                Math.max(0, end - leastCommonWhitespacePrefix.length),
+            ],
+        })),
     }))
 }