Support for Mini CPM models

[micoraweb]

Is there a way to add support or somehow be able to load MiniCPM model like this one here?
https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf

Thank you in advance!

[martindevans]

Do you know if llama.cpp can load MiniCPM models?

[micoraweb]

Yes, they do. Supported models are:
LLaVA 1.5 models,
LLaVA 1.6 models
BakLLaVA
Obsidian
ShareGPT4V
MobileVLM 1.7B/3B models
Yi-VL
Mini CPM - the one we need ;)
Moondream
Bunny

Inference with llama.cpp

MiniCPM-V 2.6 can run with llama.cpp now! See our fork of llama.cpp for more detail. This implementation supports smooth inference of 16~18 token/s on iPad (test environment：iPad Pro + M4).