Open
Description
Before submitting your bug report
- I believe this is a bug. I'll try to join the Continue Discord for questions
- I'm not able to find an open issue that reports the same bug
- I've seen the troubleshooting guide on the Continue Docs
Relevant environment info
- OS: macOS
- Continue version: v0.8.61
- IDE version: VSCode 1.94.2
- Model: Clause Sonnet 3.5 via Bedrock
- config.json:
{
"allowAnonymousTelemetry": false,
"models": [
{
"title": "Bedrock: Claude 3.5 Sonnet",
"provider": "bedrock",
"model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
"region": "us-west-2",
"profile": "continue",
"systemMessage": "You are an expert software developer. You give helpful and concise responses. Please be as concise as reasonably possible, and assume that I have a good background in the concepts you are detailing - if I need clarification, I will ask follow up questions. If ideas need examples, please lean towards providing the most minimal and concise examples possible.",
// Enable prompt caching: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
"cacheBehavior": {
"cacheSystemMessage": true,
"cacheConversation": true
},
"contextLength": 200000
},
{
"apiBase": "http://localhost:11434/",
"title": "Starcoder 3b",
"provider": "ollama",
"model": "starcoder2:3b"
},
{
"apiBase": "http://localhost:11434/",
"title": "Codestral 22b",
"provider": "ollama",
"model": "codestral:22b"
}
],
"customCommands": [
{
"name": "test",
"prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
"description": "Write unit tests for highlighted code"
}
],
"contextProviders": [
{
"name": "code",
"params": {}
},
{
"name": "docs",
"params": {}
},
{
"name": "diff",
"params": {}
},
{
"name": "terminal",
"params": {}
},
{
"name": "problems",
"params": {}
},
{
"name": "folder",
"params": {}
},
{
"name": "codebase",
"params": {}
}
],
"slashCommands": [
{
"name": "edit",
"description": "Edit selected code"
},
{
"name": "comment",
"description": "Write comments for the selected code"
},
{
"name": "share",
"description": "Export the current chat session to markdown"
},
{
"name": "cmd",
"description": "Generate a shell command"
},
{
"name": "commit",
"description": "Generate a git commit message"
}
],
"embeddingsProvider": {
"provider": "transformers.js"
},
"reranker": {
"name": "free-trial"
}
}Description
When I send a simple message, even "test", to the Bedrock model, I get the following popup:
A relevant comment here describes bumping this in the Converse API to fix the issue, but I am not sure how to do that from Continue's config, or if it is even exposed at all?
To reproduce
No response
Log output
webview message: {
"msg": {
"messageId": "SCRUBBED",
"messageType": "llm/streamChat",
"data": {
"messages": [
{
"role": "system",
"content": "When generating new code:\n\n1. Always produce a single code block.\n2. Never separate the code into multiple code blocks.\n3. Only include the code that is being added.\n4. Replace existing code with a \"lazy\" comment like this: \"// ... existing code ...\"\n5. The \"lazy\" comment must always be a valid comment in the current context (e.g. \"<!-- ... existing code ... -->\" for HTML, \"// ... existing code ...\" for JavaScript, \"{/* ... existing code */}\" for TSX, etc.)\n6. You must always provide 1-2 lines of context above and below a \"lazy\" comment\n7. If the user submits a code block that contains a filename in the language specifier, always include the filename in any code block you generate based on that file. The filename should be on the same line as the language specifier in your code block.\n\nExample 1:\nInput:\n.js\nimport addition from \"addition\"\n\nclass Calculator {\n constructor() {\n this.result = 0;\n }\n \n add(number) {\n this.result += number;\n return this;\n }\n}\n```\nUser request: Add a subtract method\n\nOutput:\n```javascript test.js\n// ... existing code ...\nimport subtraction from \"subtraction\"\n\nclass Calculator {\n // ... existing code ...\n \n subtract(number) {\n this.result -= number;\n return this;\n }\n}\n```\n\nExample 2:\nInput:\n```javascript test.js (6-9)\nfunction helloWorld() {}\n```\n\nOutput:\n```javascript test.js\nfunction helloWorld() {\n // New code here\n}\n```\n\nAlways follow these guidelines when generating code responses.\n\nWhen using tools, follow the following guidelines:\n- Avoid calling tools unless they are absolutely necessary. For example, if you are asked a simple programming question you do not need web search. As another example, if the user asks you to explain something about code, do not create a new file."
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "test"
}
]
},
{
"role": "assistant",
"content": ""
}
],
"title": "Bedrock: Claude 3.5 Sonnet",
"completionOptions": {}
}
}
}
ValidationException: The maximum tokens you requested exceeds the model limit of 4096. Try again with a maximum tokens value that is lower than 4096.