@@ -42,11 +42,24 @@ SELECT ai_log_info(1);
4242
4343** Description:**
4444Loads a GGUF model from the specified file path with optional comma separated key=value configuration.
45+ If no options are provided the following default value is used: ` gpu_layers=99 `
46+
47+ The following keys are available:
48+ ```
49+ gpu_layers=N (N is the number of layers to store in VRAM)
50+ main_gpu=K (K is the GPU that is used for the entire model when split_mode is 0)
51+ split_mode=N (how to split the model across multiple GPUs, 0 means none, 1 means layer, 2 means rows)
52+ vocab_only=1/0 (only load the vocabulary, no weights)
53+ use_mmap=1/0 (use mmap if possible)
54+ use_mlock=1/0 (force system to keep model in RAM)
55+ check_tensors=1/0 (validate model tensor data)
56+ log_info=1/0 (enable/disable the logging of info)
57+ ```
4558
4659** Example:**
4760
4861``` sql
49- SELECT llm_model_load(' ./models/llama.gguf' , ' n_predict=512,n_gpu_layers =99' );
62+ SELECT llm_model_load(' ./models/llama.gguf' , ' gpu_layers =99' );
5063```
5164
5265---
@@ -71,8 +84,13 @@ SELECT llm_model_free();
7184** Returns:** ` NULL `
7285
7386** Description:**
74- Creates a new inference context with optional with optional comma separated key=value configuration.
75- If no context is explicitly created, one will be created automatically when needed.
87+ Creates a new inference context with comma separated key=value configuration.
88+
89+ Context must explicitly created before performing any AI operation!
90+
91+ The following keys are available:
92+ ```
93+ ```
7694
7795** Example:**
7896
@@ -82,6 +100,63 @@ SELECT llm_context_create('n_ctx=2048');
82100
83101---
84102
103+ ## ` llm_context_create_embedding() `
104+
105+ ** Returns:** ` NULL `
106+
107+ ** Description:**
108+ Creates a new inference context specifically set for embedding generation.
109+
110+ It is equivalent to ` SELECT llm_context_create('generate_embedding=1,normalize_embedding=1,pooling_type=last'); `
111+
112+ Context must explicitly created before performing any AI operation!
113+
114+ ** Example:**
115+
116+ ``` sql
117+ SELECT llm_context_create_embedding();
118+ ```
119+
120+ ---
121+
122+ ## ` llm_context_create_chat() `
123+
124+ ** Returns:** ` NULL `
125+
126+ ** Description:**
127+ Creates a new inference context specifically set for chat conversation.
128+
129+ It is equivalent to ` SELECT llm_context_create('context_size=4096'); `
130+
131+ Context must explicitly created before performing any AI operation!
132+
133+ ** Example:**
134+
135+ ``` sql
136+ SELECT llm_context_create_chat();
137+ ```
138+
139+ ---
140+
141+ ## ` llm_context_create_textgen() `
142+
143+ ** Returns:** ` NULL `
144+
145+ ** Description:**
146+ Creates a new inference context specifically set for text generation.
147+
148+ It is equivalent to ` SELECT llm_context_create('context_size=4096'); `
149+
150+ Context must explicitly created before performing any AI operation!
151+
152+ ** Example:**
153+
154+ ``` sql
155+ SELECT llm_context_create_textgen();
156+ ```
157+
158+ ---
159+
85160## ` llm_context_free() `
86161
87162** Returns:** ` NULL `
0 commit comments