Discussion: Add pylate (python) reranking API to pylate_rs?

[Jacqkues]

I would like to know why there is no rerank function in the Rust version of pylate. Is sorting the array returned by
let similarities = model.similarity(&query_embeddings, &document_embeddings)?; sufficient to obtain an effective reranking?

[raphaelsty]

Hi @Jacqkues the purpose of pylate-rs is to compute embeddings. Did not work on scoring.
If you want to perform retrieval: https://github.com/lightonai/fast-plaid is amazing. For re-ranking you can use the re-ranking function of PyLate: https://lightonai.github.io/pylate/api/rank/rerank/

or even https://github.com/mixedbread-ai/maxsim-cpu if you want to re-rank on cpu

Sorry, I have been late to answer

[Jacqkues]

Thanks for your reply! Do you think it would make sense to add this reranking feature to pylate_rs? I’d like to start contributing to open source and improve my Rust skills. If you think it could be useful, I’d be happy to start working on it.

[raphaelsty]

If you would be interested in contributing to PyLate, something great would be to support ONNX with Rust within pylate-rs.

ONNX is a format used to make inference with neural networks. It's very efficient. It's distinct from pytorch or candle. It's very powerful with cpu inference.

I think ONNX if properly implemented might yield great results. Onnx support is available within candle.

So here it would consist of first converting an existing ColBERT model to ONNX, then creating maybe an ONNX Colbert Class in Rust, benchmarking it, and then making it available to both rust and python useers, not straightforward but very impactful :)

[Jacqkues]

Okay, thank you! I’ll look into that.

Do you think it would be better to use candle-onnx or the ort crates?

[raphaelsty]

I would say candle-onnx because pylate-rs is based on candle.

Something really important is having a test that current pylate-rs / PyLate model yield the same output as onnx model and a Python script that allow to convert a PyLate model to ONNX would be great :)

[Jacqkues]

Alright, I’ll do that. I’m new to Candle and ONNX, so it’s a good opportunity to learn something interesting :)