Autocompletion uses dummy template with Qwen Coder + Ollama

[AnnoyingTechnology]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: MacOS
- Continue version: main
- IDE version: VS Code
- Model: N/A
- config.json:
  "tabAutocompleteModel": {
    "title": "Qwen 2.5 Coder 3B",
    "provider": "ollama",
    "model": "qwen2.5-coder:3b",
    "contextLength": 2048,
    "completionOptions": {
      "temperature": 0.15,
      "topP": 0.6,
      "topK": 10,
      "maxTokens": 256,
      "keepAlive": 14400
    }
  },

Description

When using qwen coder via ollama, a dummy prompt format (madeupFimPromptTemplate) instead of Qwen's proper FIM prompt template.

To reproduce

This doesn't happen with LM Studio as a backend. I was not able to provide a PR.

Log output

No response

[AnnoyingTechnology]

This this especially severe, as Qwen Coder is the commended local model, as per the docs of continue.
And this makes it try to talk instead of complete code, or provide absurd suggestions.

[tomasz-stefaniak]

@AnnoyingTechnology this is confusing but we only use madeUpFimPrompt for logging purposes. @sestinj is opening a PR now to make it more obvious.

That said, let us know If you see this template in Ollama logs specifically (not in Continue logs). That would indicate that there's an issue after all.

[AnnoyingTechnology]

The bug may have got fixed in the meantime, I'm quite confused by the situation.

Using a fork from last week, ollama logs :
time=2024-12-14T11:27:46.024+01:00 level=DEBUG source=routes.go:270 msg="generate request" images=0 prompt="<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n<|im_start|>user\n\n// Untitled-1\n<|im_end|>\n<|im_start|>assistant\n"

It's not the <FIM> from the fake prompt that I assumed to be the culrit, but it's a wrong prompt and causes no suggestions are absurd ones.

Using the latest main branch, ollama logs :
time=2024-12-14T11:06:20.140+01:00 level=DEBUG source=routes.go:270 msg="generate request" images=0 prompt="<|fim_prefix|>\n// scratchpad/lambda.js\nconst <|fim_suffix|>\n<|fim_middle|>"

So clearly there was a bug somewhere.

Also, depending on the model loaded, i saw <FIM> being logged by continue or an actual correct prompt syntax.

[Swipe4057]

I have a similar issue with VLLM. #3372

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.