Umbrella Ticket: Improve Code Completion #201
Description
Below you can find several suggestions for improvements. Please create a dedicated subticket when working on them.
Description
The current implementation of the code completion agent sends either the full file or a fixed number of surrounding lines to the LLM without any preprocessing or contextual enhancements.
By incorporating additional information and smarter prompt engineering, the completion results can become more accurate.
- Complexity:
- Easy: 2h
- Medium: 4h
- High: 4+
It is important to strike a balance between speed and quality, especially for code completion, where response time is crucial. We should aim to send as little as possible while maximizing the impact of the information, since more input tokens increase processing time.
LLM Models
-
Use of Specialized Code LLMs
Leverage models fine-tuned for code completions, such as: -
Or a LLM good in coding
Prompt Engineering (Medium)
Improve the prompt format with model-appropriate patterns and inline examples if beneficial.
Depending on the selected model, adjust prompting strategy accordingly:
- Include model-specific prompt formatting (e.g., Codestral/Qwen expects certain structure)
- Example-driven prompts may benefit some models (e.g., Claude) but not others
- https://github.com/continuedev/continue/blob/main/core/autocomplete/templating/AutocompleteTemplate.ts
Context Enhancements
Include the following information when constructing the prompt:
IDE
-
Project File Context (Easy)
Add context from the project structure (see Enhance project info file and add prompt templates for the Theia Code base #208).Handled in Improve Code Completion > Context Enhancements > Project File Context #225
-
Caching and Throttling (Medium-High)
Introduce a caching mechanism to avoid redundant LLM requests:Skip requests if completion is triggered again at the same line/positionDo not re-trigger completion if the user is typing the already returned suggestionhttps://github.com/continuedev/continue/blob/main/core/autocomplete/CompletionProvider.ts#L196https://github.com/continuedev/continue/blob/main/core/autocomplete/util/AutocompleteLruCache.ts#L33
Handled in Improve Code Completion > IDE > Caching and Throttling #226
Code
-
Language Server Information (High)
Based on the cursor position, query the language server for available parameters and type definitions to enhance context. -
Imports (High)
Depending on the location, including information about the imports (not all!) may be helpful.
For example, what does the imported function to? What are the concrete parameters? Where was it imported from?- https://github.com/continuedev/continue/blob/main/core/autocomplete/CompletionProvider.ts#L177
- https://github.com/continuedev/continue/blob/main/core/autocomplete/snippets/getAllSnippets.ts#L164-L165
- https://github.com/continuedev/continue/blob/main/core/autocomplete/context/ContextRetrievalService.ts#L12
User Specific Snippets
-
Recently Snippets (Easy-Medium)
Include information about the recently visited widgets, recently edited, active file, etc. -
Git Diff Snippet (Easy-Medium)
Include recent changes in edited files viagit diffto reflect ongoing development activity.- https://github.com/continuedev/continue/blob/main/core/autocomplete/CompletionProvider.ts#L173
- https://github.com/continuedev/continue/blob/main/core/autocomplete/snippets/getAllSnippets.ts#L143
- https://github.com/continuedev/continue/blob/main/core/autocomplete/snippets/getAllSnippets.ts#L113
- https://github.com/continuedev/continue/blob/main/extensions/vscode/src/util/ideUtils.ts#L603
Look at other OS tools such as continue.dev to get the latest state-of-the-artRepair "immediate inline completion" (currently triggers to eager and often)