Skip to content

Commit 51308c4

Browse files
valkirilovclaude
andcommitted
feat(redis-query-engine): add spec-compliant skill for RQE indexing and search
Introduces skills/redis-query-engine/ covering the six rqe-* rules from skills/redis-development/rules/ in agentskills.io spec layout: rqe-dialect → references/dialect.md rqe-field-types → references/field-types.md rqe-index-creation → references/index-creation.md rqe-index-management → references/index-management.md rqe-query-optimization → references/query-optimization.md rqe-skip-initial-scan → references/skip-initial-scan.md SKILL.md carries decision-oriented summaries (field-type table, DIALECT 2 default, schema/prefix rule, alias-based zero-downtime updates, SKIPINITIALSCAN guidance, query-optimization levers) with pointers into references/ for full code samples. Additive only: the source rqe-*.md rules under skills/redis-development/ remain in place so the legacy compiled AGENTS.md continues to serve existing plugin consumers unchanged. They are removed in the final cleanup PR alongside the rest of the rules/ tree. Validation: - skill-validator check skills/redis-query-engine → passed (0 warnings) - npm run validate → rules + plugin validators green Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent d3d89fa commit 51308c4

7 files changed

Lines changed: 455 additions & 0 deletions

File tree

skills/redis-query-engine/SKILL.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
name: redis-query-engine
3+
description: Redis Query Engine (RQE) guidance covering FT.CREATE schema design, field type selection (TEXT, TAG, NUMERIC, GEO, GEOSHAPE, VECTOR), DIALECT 2 query syntax, efficient FT.SEARCH and FT.AGGREGATE queries, zero-downtime index updates via aliases, and the SKIPINITIALSCAN option. Use when defining a search index on Hash or JSON documents, picking between TEXT and TAG for filtering, writing FT.SEARCH queries with filters and SORTBY, managing or swapping indexes in production, or troubleshooting slow searches with FT.PROFILE.
4+
license: MIT
5+
metadata:
6+
author: Redis, Inc.
7+
version: "0.1.0"
8+
---
9+
10+
# Redis Query Engine
11+
12+
Guidance for using the Redis Query Engine (RQE) to index and search Hash or JSON documents. Covers schema design with `FT.CREATE`, field-type choices, query syntax, index lifecycle management, and the most common performance pitfalls.
13+
14+
## When to apply
15+
16+
- Creating, modifying, or reviewing an RQE index (`FT.CREATE`, `FT.ALTER`).
17+
- Writing or optimizing `FT.SEARCH` / `FT.AGGREGATE` queries.
18+
- Deciding between `TEXT`, `TAG`, `NUMERIC`, `GEO`, `GEOSHAPE`, or `VECTOR` for a field.
19+
- Rolling out a new index schema without downtime.
20+
- Spinning up an index that should only cover newly written keys.
21+
22+
## 1. Use DIALECT 2 (the modern default)
23+
24+
`DIALECT 2` is the baseline. Other dialects (1, 3, 4) are deprecated as of Redis 8. Most modern client libraries already default to it — but specify it explicitly in raw commands for portability.
25+
26+
```
27+
FT.SEARCH idx:products "@name:laptop" DIALECT 2
28+
```
29+
30+
`DIALECT 2` is **required** for vector search queries. It also handles special characters and NULLs predictably.
31+
32+
See [references/dialect.md](references/dialect.md).
33+
34+
## 2. Pick the right field type
35+
36+
The field type decides both what you can query and how fast that query is. Use the narrowest type that supports your access pattern.
37+
38+
| Field type | Use when | Notes |
39+
|---|---|---|
40+
| `TEXT` | Full-text search needed | Tokenized + stemmed; **not** for exact match |
41+
| `TAG` | Exact match / filtering | Add `SORTABLE UNF` for fastest tag queries |
42+
| `NUMERIC` | Range queries, sorting | Prices, counts, timestamps |
43+
| `GEO` | Lat/long point queries | Single points (stores, users) |
44+
| `GEOSHAPE` | Polygon / area queries | Delivery zones, regions |
45+
| `VECTOR` | Similarity search | HNSW or FLAT; see redis-vector-search |
46+
47+
The classic mistake is using `TEXT` for a category or status field because "it's a string." `TAG` is 10× faster for those.
48+
49+
See [references/field-types.md](references/field-types.md).
50+
51+
## 3. Index only what you query — and always set a prefix
52+
53+
`FT.CREATE` without a `PREFIX` indexes **every** matching key in the database; with a wide schema it can blow up index size and write latency.
54+
55+
```
56+
FT.CREATE idx:products ON HASH PREFIX 1 product:
57+
SCHEMA
58+
name TEXT WEIGHT 2.0
59+
category TAG SORTABLE
60+
price NUMERIC SORTABLE
61+
location GEO
62+
```
63+
64+
Rules of thumb:
65+
66+
- Start with the minimum schema. Add fields as new query patterns emerge.
67+
- Always set `PREFIX` (or filter via `FILTER` expression).
68+
- Use `FT.INFO idx:<name>` to monitor index size after adding fields.
69+
- Use `SORTABLE` only on fields you actually sort by; it has a memory cost.
70+
71+
See [references/index-creation.md](references/index-creation.md).
72+
73+
## 4. Zero-downtime index updates — use aliases
74+
75+
For schema changes in production, keep application queries pointed at an alias and swap the underlying index.
76+
77+
```
78+
FT.CREATE idx:products_v2 ON HASH PREFIX 1 product: SCHEMA ...
79+
FT.ALIASUPDATE products idx:products_v2
80+
81+
# App queries are stable:
82+
FT.SEARCH products "@category:{electronics}"
83+
```
84+
85+
Useful management commands: `FT.INFO`, `FT.DROPINDEX`, `FT._LIST`, `FT.ALIASADD/UPDATE/DEL`.
86+
87+
See [references/index-management.md](references/index-management.md).
88+
89+
## 5. SKIPINITIALSCAN — only when historical data is irrelevant
90+
91+
By default `FT.CREATE` walks all existing keys that match the prefix and indexes them. Use `SKIPINITIALSCAN` only when:
92+
93+
- You're standing up the index for a *new* feature and existing data shouldn't be queryable.
94+
- Existing data is too large to scan synchronously.
95+
- You're indexing event streams where only future events matter.
96+
97+
For most schema migrations, the default (scan everything) is what you want.
98+
99+
See [references/skip-initial-scan.md](references/skip-initial-scan.md).
100+
101+
## 6. Write specific queries, not `*`
102+
103+
Narrow the result set with filters before paging or aggregating.
104+
105+
```
106+
# Good — specific filter, limited fields returned
107+
FT.SEARCH idx:products "@category:{electronics} @price:[100 500]"
108+
LIMIT 0 20
109+
RETURN 3 name price category
110+
```
111+
112+
```
113+
# Bad — full scan plus unbounded LIMIT
114+
FT.SEARCH idx:products "*" LIMIT 0 10000
115+
```
116+
117+
Other levers:
118+
119+
- `SORTBY` requires `SORTABLE` on the sort field. Without it, sort is slow.
120+
- `LIMIT` early; the engine still processes everything above the limit if you don't.
121+
- `RETURN` specific fields — don't fetch the whole document if you only need a few.
122+
- Profile with `FT.PROFILE idx:<name> SEARCH QUERY "<query>"` when a query is slow.
123+
124+
See [references/query-optimization.md](references/query-optimization.md).
125+
126+
## References
127+
128+
- [Redis: Query Engine — Indexing](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/)
129+
- [Redis: Query syntax](https://redis.io/docs/latest/develop/interact/search-and-query/query/)
130+
- [Redis: Query dialects](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/dialects/)
131+
- [Redis: Administration (aliases, dropindex)](https://redis.io/docs/latest/develop/interact/search-and-query/administration/)
132+
- [FT.CREATE](https://redis.io/docs/latest/commands/ft.create/)
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Use DIALECT 2 for Query Syntax
2+
3+
Use DIALECT 2 for consistent query behavior. Many Redis client libraries now default to DIALECT 2, and other dialects (1, 3, 4) are deprecated as of Redis 8.
4+
5+
**Correct:** Use DIALECT 2 explicitly or rely on modern client defaults.
6+
7+
```python
8+
from redis import Redis
9+
10+
r = Redis()
11+
12+
# Modern redis-py (6.0+) defaults to DIALECT 2
13+
# You can also set it explicitly
14+
results = r.ft("idx:products").search(
15+
"@name:laptop",
16+
dialect=2
17+
)
18+
```
19+
20+
```
21+
# In raw commands, specify DIALECT 2
22+
FT.SEARCH idx:products "@name:laptop" DIALECT 2
23+
24+
FT.AGGREGATE idx:products "@category:{electronics}"
25+
GROUPBY 1 @category
26+
REDUCE COUNT 0 AS count
27+
DIALECT 2
28+
```
29+
30+
**Note:** DIALECT 2 is required for vector search queries. Most modern client libraries (redis-py 6.0+, go-redis, Lettuce) now use DIALECT 2 by default.
31+
32+
**Why DIALECT 2:**
33+
- Consistent handling of special characters
34+
- Better NULL value handling
35+
- More predictable query parsing
36+
- Required for vector search
37+
38+
Reference: [Query Dialects](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/dialects/)
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Choose the Correct Field Type
2+
3+
Each field type has different capabilities and performance characteristics.
4+
5+
| Field Type | Use When | Notes |
6+
|------------|----------|-------|
7+
| TEXT | Full-text search needed | Tokenized, stemmed |
8+
| TAG | Exact match, filtering | Faster than TEXT for filtering |
9+
| NUMERIC | Range queries, sorting | Use for prices, counts, timestamps |
10+
| GEO | Point location queries | Lat/long coordinates (single points) |
11+
| GEOSHAPE | Area/region queries | Polygons, circles, rectangles |
12+
| VECTOR | Similarity search | HNSW or FLAT algorithm |
13+
14+
**Correct:** Use TAG for exact matching.
15+
16+
```
17+
# Good: TAG for exact category matching
18+
FT.CREATE idx:products ON HASH PREFIX 1 product:
19+
SCHEMA
20+
category TAG SORTABLE
21+
status TAG
22+
```
23+
24+
**Java** (Jedis):
25+
```java
26+
import redis.clients.jedis.search.*;
27+
28+
Schema schema = new Schema()
29+
.addTextField("name", 1)
30+
.addTagField("categories"); // TAG for exact matching
31+
32+
IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH);
33+
34+
jedis.ftCreate("idx", IndexOptions.defaultOptions().setDefinition(def), schema);
35+
36+
// Query with TAG syntax
37+
SearchResult result = jedis.ftSearch("idx", "@categories:{chef|runner}");
38+
```
39+
40+
**Incorrect:** Using TEXT when you don't need full-text features.
41+
42+
```
43+
# Overkill: TEXT for category adds unnecessary tokenization
44+
FT.CREATE idx:products ON HASH PREFIX 1 product:
45+
SCHEMA
46+
category TEXT
47+
status TEXT
48+
```
49+
50+
**Java** (Jedis):
51+
```java
52+
// Bad: TEXT for categories adds unnecessary overhead
53+
Schema schema = new Schema()
54+
.addTextField("name", 1)
55+
.addTextField("categories", 1); // Overkill for exact matching
56+
```
57+
58+
**Correct:** Use GEO for points, GEOSHAPE for areas.
59+
60+
```
61+
# GEO for point locations (stores, users)
62+
FT.CREATE idx:stores ON HASH PREFIX 1 store:
63+
SCHEMA
64+
location GEO
65+
66+
# GEOSHAPE for areas (delivery zones, boundaries)
67+
FT.CREATE idx:zones ON JSON PREFIX 1 zone:
68+
SCHEMA
69+
$.boundary AS boundary GEOSHAPE
70+
```
71+
72+
Reference: [Redis Search Field Types](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/geoindex/)
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Index Only Fields You Query
2+
3+
Create indexes with only the fields you need to search, filter, or sort on.
4+
5+
**Correct:** Index specific fields and use prefixes.
6+
7+
```
8+
FT.CREATE idx:products ON HASH PREFIX 1 product:
9+
SCHEMA
10+
name TEXT WEIGHT 2.0
11+
description TEXT
12+
category TAG SORTABLE
13+
price NUMERIC SORTABLE
14+
location GEO
15+
```
16+
17+
**Java** (Jedis):
18+
```java
19+
import redis.clients.jedis.search.*;
20+
21+
Schema schema = new Schema()
22+
.addTextField("name", 1)
23+
.addTagField("categories");
24+
25+
// Good: Specify prefix to index only matching keys
26+
IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH)
27+
.setPrefixes("person:");
28+
29+
jedis.ftCreate("idx", IndexOptions.defaultOptions().setDefinition(def), schema);
30+
```
31+
32+
**Incorrect:** Over-indexing or indexing unused fields.
33+
34+
```
35+
# Bad: Indexing every field "just in case"
36+
FT.CREATE idx:products ON HASH PREFIX 1 product:
37+
SCHEMA
38+
name TEXT
39+
description TEXT
40+
category TEXT
41+
subcategory TEXT
42+
brand TEXT
43+
sku TEXT
44+
price NUMERIC
45+
cost NUMERIC
46+
margin NUMERIC
47+
...
48+
```
49+
50+
**Java** (Jedis):
51+
```java
52+
// Bad: No prefix means all hashes get indexed
53+
IndexDefinition def = new IndexDefinition(IndexDefinition.Type.HASH);
54+
// This will index every hash in the database!
55+
```
56+
57+
**Tips:**
58+
- Start with the minimum required fields
59+
- Add fields as query patterns emerge
60+
- Use `FT.INFO` to monitor index size
61+
- Always specify a prefix to avoid indexing unrelated keys
62+
63+
Reference: [Redis Search Indexing](https://redis.io/docs/latest/develop/interact/search-and-query/indexing/)
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Manage Indexes for Zero-Downtime Updates
2+
3+
Use aliases to swap indexes without application changes.
4+
5+
**Correct:** Use aliases for production indexes.
6+
7+
```
8+
# Create versioned index
9+
FT.CREATE idx:products_v2 ON HASH PREFIX 1 product:
10+
SCHEMA
11+
name TEXT
12+
category TAG SORTABLE
13+
price NUMERIC SORTABLE
14+
15+
# Point alias to new index
16+
FT.ALIASADD products idx:products_v2
17+
18+
# Application queries use alias
19+
FT.SEARCH products "@category:{electronics}"
20+
21+
# Later, swap to new version
22+
FT.ALIASUPDATE products idx:products_v3
23+
```
24+
25+
**Useful management commands:**
26+
27+
```
28+
# Check index info
29+
FT.INFO idx:products
30+
31+
# Drop and recreate (non-blocking)
32+
FT.DROPINDEX idx:products
33+
FT.CREATE idx:products ...
34+
35+
# List all indexes
36+
FT._LIST
37+
```
38+
39+
Reference: [Redis Search Index Management](https://redis.io/docs/latest/develop/interact/search-and-query/administration/)
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Write Efficient Queries
2+
3+
Be specific and use filters to reduce the result set early.
4+
5+
**Correct:** Use specific filters and limit results.
6+
7+
```
8+
# Good: Specific query with filters
9+
FT.SEARCH idx:products "@category:{electronics} @price:[100 500]"
10+
LIMIT 0 20
11+
RETURN 3 name price category
12+
13+
# Good: Use SORTBY and LIMIT
14+
FT.SEARCH idx:products "@name:laptop"
15+
SORTBY price ASC
16+
LIMIT 0 10
17+
```
18+
19+
**Incorrect:** Broad queries returning large result sets.
20+
21+
```
22+
# Bad: Wildcard prefix scans entire index
23+
FT.SEARCH idx:products "*" LIMIT 0 10000
24+
25+
# Bad: Loading all fields from source document
26+
FT.AGGREGATE idx:products "*" LOAD *
27+
```
28+
29+
**Performance tips:**
30+
- Add `SORTABLE` to fields used in `SORTBY`
31+
- Use `TAG SORTABLE UNF` for best performance on tag fields
32+
- Use `NOSTEM` if you don't need stemming
33+
- Profile queries with `FT.PROFILE`
34+
35+
```
36+
FT.PROFILE idx:products SEARCH QUERY "@category:{electronics}"
37+
```
38+
39+
Reference: [Redis Search Query Syntax](https://redis.io/docs/latest/develop/interact/search-and-query/query/)

0 commit comments

Comments
 (0)