Skip to content

Commit 9db1d83

Browse files
Fix opensearch query parser (qdrant#172)
* Fix opensearch query parser Also specify schema for id in arxiv-titles-384-angular filters dataset. This is needed as default mapping created by opensearch takes the type as float, but in the dataset we also have string. This leads to 99% precision on the test dataset with opensearch. ``` { "params": { "dataset": "arxiv-titles-384-angular-filters", "experiment": "opensearch-default", "engine": "opensearch", "parallel": 10, "config": { "knn.algo_param.ef_search": 128 } }, "results": { "total_time": 391.3466711850051, "mean_time": 0.37053632343088827, "mean_precisions": 0.98962, "std_time": 0.3485163122349799, "min_time": 0.04950378900684882, "max_time": 3.548553360000369, "rps": 25.552791773390613, "p95_time": 1.0473700279486362, "p99_time": 1.4396829416653787 } } ``` Fixes qdrant#171 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent dd9a4c0 commit 9db1d83

File tree

2 files changed

+8
-2
lines changed

2 files changed

+8
-2
lines changed

datasets/datasets.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,8 @@
138138
"schema": {
139139
"update_date_ts": "int",
140140
"labels": "keyword",
141-
"submitter": "keyword"
141+
"submitter": "keyword",
142+
"id": "keyword"
142143
}
143144
},
144145
{

engine/clients/opensearch/parser.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,12 @@ def build_range_filter(
2525
lte: Optional[FieldValue],
2626
gte: Optional[FieldValue],
2727
) -> Any:
28-
return {"range": {field_name: {"lt": lt, "gt": gt, "lte": lte, "gte": gte}}}
28+
field_filters = {
29+
k: v
30+
for k, v in {"lt": lt, "gt": gt, "lte": lte, "gte": gte}.items()
31+
if v is not None
32+
}
33+
return {"range": {field_name: field_filters}}
2934

3035
def build_geo_filter(
3136
self, field_name: str, lat: float, lon: float, radius: float

0 commit comments

Comments
 (0)