examples

May 6, 2025

85eb85a · May 6, 2025

Name	Name	Last commit message	Last commit date
parent directory ..
api	api	Update examples to accomodate changes	Apr 30, 2025
inference	inference	Update chat completions example to print dict object	Mar 20, 2025
logits	logits	Ruff format, fix incorrect assetion expected format for test_load_def…	Feb 1, 2025
slurm_dependency	slurm_dependency	slurm dependency example added	May 6, 2025
README.md	README.md	slurm dependency example added	May 6, 2025

README.md

inference: Examples for sending inference requests
- llm/chat_completions.py: Python example of sending chat completion requests to OpenAI compatible server
- llm/completions.py: Python example of sending completion requests to OpenAI compatible server
- llm/completions.sh: Bash example of sending completion requests to OpenAI compatible server, supports JSON mode
- text_embedding/embeddings.py: Python example of sending text embedding requests to OpenAI compatible server
- vlm/vision_completions.py: Python example of sending chat completion requests with image attached to prompt to OpenAI compatible server for vision language models
logits: Example for logits generation
- logits.py: Python example of getting logits from hosted model.
api: Examples for using the Python API
- basic_usage.py: Basic Python example demonstrating the Vector Inference API
slurm_dependency: Example of launching a model with vec-inf and running a downstream SLURM job that waits for the server to be ready before sending a request.