Skip to content

Commit dd9a4c0

Browse files
authored
Use efficient filtering for opensearch (qdrant#167)
Current filtering query does post-filtering which is not ideal and resulted in low precision. Results with post filtering: ``` { "params": { "dataset": "arxiv-titles-384-angular-filters", "experiment": "opensearch-default", "engine": "opensearch", "parallel": 1, "config": { "knn.algo_param.ef_search": 128 } }, "results": { "total_time": 708.6168032020068, "mean_time": 0.07045893384491791, "mean_precisions": 0.11399200000000001, "std_time": 0.06840096039381999, "min_time": 0.008397486002650112, "max_time": 3.3753458530118223, "rps": 14.111999538838594, "p95_time": 0.18164870390755816, "p99_time": 0.20864198897208555 } } ``` Results with new efficient filtering: ``` { "params": { "dataset": "arxiv-titles-384-angular-filters", "experiment": "opensearch-default", "engine": "opensearch", "parallel": 1, "config": { "knn.algo_param.ef_search": 128 } }, "results": { "total_time": 394.4290532110026, "mean_time": 0.03913764159695711, "mean_precisions": 0.610144, "std_time": 0.05352479065894972, "min_time": 0.0009066620114026591, "max_time": 2.1307434440095676, "rps": 25.35310195481576, "p95_time": 0.1274049270534305, "p99_time": 0.2078282342318563 } } ```
1 parent fd6bea4 commit dd9a4c0

File tree

1 file changed

+1
-6
lines changed

1 file changed

+1
-6
lines changed

engine/clients/opensearch/search.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -59,12 +59,7 @@ def search_one(cls, query: Query, top: int) -> List[Tuple[int, float]]:
5959

6060
meta_conditions = cls.parser.parse(query.meta_conditions)
6161
if meta_conditions:
62-
opensearch_query = {
63-
"bool": {
64-
"must": [opensearch_query],
65-
"filter": meta_conditions,
66-
}
67-
}
62+
opensearch_query["knn"]["vector"]["filter"] = meta_conditions
6863

6964
res = cls.client.search(
7065
index=OPENSEARCH_INDEX,

0 commit comments

Comments
 (0)