Skip to content

Commit 9425512

Browse files
matthewmcneelyCopilot
matthewmcneely
and
Copilot
authored
chore: add load example and query CLI (#91)
**Description** This PR adds a load example and a query cli **Checklist** - [x] Code compiles correctly and linting passes locally --------- Co-authored-by: Copilot <[email protected]>
1 parent daa4dc3 commit 9425512

File tree

9 files changed

+510
-12
lines changed

9 files changed

+510
-12
lines changed

‎README.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,38 @@ modusGraph has a few limitations to be aware of:
105105
- **Schema evolution**: While modusGraph supports schema inference through tags, evolving an
106106
existing schema with new fields requires careful consideration to avoid data inconsistencies.
107107

108+
## CLI Commands and Examples
109+
110+
modusGraph provides several command-line tools and example applications to help you interact with
111+
and explore the package. These are organized in the `cmd` and `examples` folders:
112+
113+
### Commands (`cmd` folder)
114+
115+
- **`cmd/query`**: A flexible CLI tool for running arbitrary DQL (Dgraph Query Language) queries
116+
against a modusGraph database.
117+
- Reads a query from standard input and prints JSON results.
118+
- Supports file-based modusGraph storage.
119+
- Flags: `--dir`, `--pretty`, `--timeout`, `-v` (verbosity).
120+
- See [`cmd/query/README.md`](./cmd/query/README.md) for usage and examples.
121+
122+
### Examples (`examples` folder)
123+
124+
- **`examples/basic`**: Demonstrates CRUD operations for a simple `Thread` entity.
125+
126+
- Flags: `--dir`, `--addr`, `--cmd`, `--author`, `--name`, `--uid`, `--workspace`.
127+
- Supports create, update, delete, get, and list commands.
128+
- See [`examples/basic/README.md`](./examples/basic/README.md) for details.
129+
130+
- **`examples/load`**: Shows how to load the standard 1million RDF dataset into modusGraph for
131+
benchmarking.
132+
133+
- Downloads, initializes, and loads the dataset into a specified directory.
134+
- Flags: `--dir`, `--verbosity`.
135+
- See [`examples/load/README.md`](./examples/load/README.md) for instructions.
136+
137+
You can use these tools as starting points for your own applications or as references for
138+
integrating modusGraph into your workflow.
139+
108140
## Open Source
109141

110142
The modus framework, including modusGraph, is developed by [Hypermode](https://hypermode.com/) as an

‎client.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -344,6 +344,15 @@ func (c client) DropData(ctx context.Context) error {
344344

345345
// QueryRaw implements raw querying (DQL syntax).
346346
func (c client) QueryRaw(ctx context.Context, q string) ([]byte, error) {
347+
if c.engine != nil {
348+
ns := c.engine.GetDefaultNamespace()
349+
resp, err := ns.Query(ctx, q)
350+
if err != nil {
351+
return nil, err
352+
}
353+
return resp.GetJson(), nil
354+
}
355+
347356
client, err := c.pool.get()
348357
if err != nil {
349358
c.logger.Error(err, "Failed to get client from pool")

‎cmd/query/README.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# modusGraph Query CLI
2+
3+
This command-line tool allows you to run arbitrary DQL (Dgraph Query Language) queries against a
4+
modusGraph database, either in local file-based mode or (optionally) against a remote
5+
Dgraph-compatible endpoint.
6+
7+
## Requirements
8+
9+
- Go 1.24 or higher
10+
- Access to a directory containing a modusGraph database (created by modusGraph)
11+
12+
## Installation
13+
14+
```bash
15+
# Navigate to the cmd/query directory
16+
cd cmd/query
17+
18+
# Run directly
19+
go run main.go --dir /path/to/modusgraph [options]
20+
21+
# Or build and then run
22+
go build -o modusgraph-query
23+
./modusgraph-query --dir /path/to/modusgraph [options]
24+
```
25+
26+
## Usage
27+
28+
The tool reads a DQL query from standard input and prints the JSON response to standard output.
29+
30+
```sh
31+
Usage of ./main:
32+
--dir string Directory where the modusGraph database is stored (required)
33+
--pretty Pretty-print the JSON output (default true)
34+
--timeout Query timeout duration (default 30s)
35+
-v int Verbosity level for logging (e.g., -v=1, -v=2)
36+
```
37+
38+
### Example: Querying the Graph
39+
40+
```bash
41+
echo '{ q(func: has(name@en), first: 10) { id: uid name@en } }' | go run main.go --dir /tmp/modusgraph
42+
```
43+
44+
### Example: With Verbose Logging
45+
46+
```bash
47+
echo '{ q(func: has(name@en), first: 10) { id: uid name@en } }' | go run main.go --dir /tmp/modusgraph -v 1
48+
```
49+
50+
### Example: Build and Run
51+
52+
```bash
53+
go build -o modusgraph-query
54+
cat query.dql | ./modusgraph-query --dir /tmp/modusgraph
55+
```
56+
57+
## Notes
58+
59+
- The `--dir` flag is required and must point to a directory initialized by modusGraph.
60+
- The query must be provided via standard input.
61+
- Use the `-v` flag to control logging verbosity (higher values show more log output).
62+
- Use the `--pretty=false` flag to disable pretty-printing of the JSON response.
63+
- The tool logs query timing and errors to standard error.
64+
65+
## Example Output
66+
67+
```json
68+
{
69+
"q": [
70+
{ "id": "0x2", "name@en": "Ivan Sen" },
71+
{ "id": "0x3", "name@en": "Peter Lord" }
72+
]
73+
}
74+
```
75+
76+
---
77+
78+
For more advanced usage and integration, see the main [modusGraph documentation](../../README.md).

‎cmd/query/main.go

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
/*
2+
* SPDX-FileCopyrightText: © Hypermode Inc. <[email protected]>
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package main
7+
8+
import (
9+
"bufio"
10+
"context"
11+
"encoding/json"
12+
"flag"
13+
"fmt"
14+
"io"
15+
"log"
16+
"os"
17+
"path/filepath"
18+
"strconv"
19+
"strings"
20+
"time"
21+
22+
"github.com/go-logr/stdr"
23+
"github.com/hypermodeinc/modusgraph"
24+
)
25+
26+
func main() {
27+
// Define flags
28+
dirFlag := flag.String("dir", "", "Directory where the modusGraph database is stored")
29+
prettyFlag := flag.Bool("pretty", true, "Pretty-print the JSON output")
30+
timeoutFlag := flag.Duration("timeout", 30*time.Second, "Query timeout duration")
31+
flag.Parse()
32+
33+
// Initialize the stdr logger with the verbosity from -v
34+
stdLogger := log.New(os.Stdout, "", log.LstdFlags)
35+
logger := stdr.NewWithOptions(stdLogger, stdr.Options{LogCaller: stdr.All}).WithName("mg")
36+
vFlag := flag.Lookup("v")
37+
if vFlag != nil {
38+
val, err := strconv.Atoi(vFlag.Value.String())
39+
if err != nil {
40+
log.Fatalf("Error: Invalid verbosity level: %s", vFlag.Value.String())
41+
}
42+
stdr.SetVerbosity(val)
43+
}
44+
45+
// Validate required flags
46+
if *dirFlag == "" {
47+
log.Println("Error: --dir parameter is required")
48+
flag.Usage()
49+
os.Exit(1)
50+
}
51+
52+
// Create clean directory path
53+
dirPath := filepath.Clean(*dirFlag)
54+
if _, err := os.Stat(dirPath); os.IsNotExist(err) {
55+
log.Fatalf("Error: Directory %s does not exist", dirPath)
56+
}
57+
58+
// Initialize modusGraph client with the directory where data is stored
59+
logger.V(1).Info("Initializing modusGraph client", "directory", dirPath)
60+
client, err := modusgraph.NewClient(fmt.Sprintf("file://%s", dirPath),
61+
modusgraph.WithLogger(logger))
62+
if err != nil {
63+
logger.Error(err, "Failed to initialize modusGraph client")
64+
os.Exit(1)
65+
}
66+
defer client.Close()
67+
68+
// Read query from stdin
69+
reader := bufio.NewReader(os.Stdin)
70+
query := ""
71+
for {
72+
line, err := reader.ReadString('\n')
73+
if err != nil && err != io.EOF {
74+
logger.Error(err, "Error reading from stdin")
75+
os.Exit(1)
76+
}
77+
78+
query += line
79+
80+
if err == io.EOF {
81+
break
82+
}
83+
}
84+
85+
query = strings.TrimSpace(query)
86+
if query == "" {
87+
logger.Error(nil, "Empty query provided")
88+
os.Exit(1)
89+
}
90+
91+
logger.V(1).Info("Executing query", "query", query)
92+
93+
// Set up context with timeout
94+
ctx, cancel := context.WithTimeout(context.Background(), *timeoutFlag)
95+
defer cancel()
96+
97+
start := time.Now()
98+
99+
// Execute the query
100+
resp, err := client.QueryRaw(ctx, query)
101+
if err != nil {
102+
logger.Error(err, "Query execution failed")
103+
os.Exit(1)
104+
}
105+
106+
elapsed := time.Since(start)
107+
elapsedMs := float64(elapsed.Nanoseconds()) / 1e6
108+
logger.V(1).Info("Query completed", "elapsed_ms", elapsedMs)
109+
110+
// Format and print the response
111+
if *prettyFlag {
112+
var data any
113+
if err := json.Unmarshal(resp, &data); err != nil {
114+
logger.Error(err, "Failed to parse JSON response")
115+
os.Exit(1)
116+
}
117+
118+
prettyJSON, err := json.MarshalIndent(data, "", " ")
119+
if err != nil {
120+
logger.Error(err, "Failed to format JSON response")
121+
os.Exit(1)
122+
}
123+
124+
fmt.Println(string(prettyJSON))
125+
} else {
126+
fmt.Println(string(resp))
127+
}
128+
}

‎examples/basic/main.go

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import (
77
"log"
88
"os"
99
"path/filepath"
10+
"strconv"
1011
"strings"
1112

1213
"github.com/go-logr/logr"
@@ -84,11 +85,14 @@ func main() {
8485
// Initialize standard logger with stdr
8586
stdLogger := log.New(os.Stdout, "", log.LstdFlags)
8687
logger := stdr.NewWithOptions(stdLogger, stdr.Options{LogCaller: stdr.All}).WithName("mg")
87-
88-
// Set verbosity level
89-
stdr.SetVerbosity(1)
90-
91-
logger.Info("Logger initialized")
88+
vFlag := flag.Lookup("v")
89+
if vFlag != nil {
90+
val, err := strconv.Atoi(vFlag.Value.String())
91+
if err != nil {
92+
log.Fatalf("Error: Invalid verbosity level: %s", vFlag.Value.String())
93+
}
94+
stdr.SetVerbosity(val)
95+
}
9296

9397
// Initialize modusGraph client with logger
9498
client, err := mg.NewClient(endpoint,

‎examples/load/README.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# modusGraph 1Million Dataset Loader
2+
3+
This command-line application demonstrates how to load the 1million dataset into modusGraph. The
4+
1million dataset consists of approximately one million RDF triples representing relationships
5+
between various entities and is commonly used for benchmarking graph database performance.
6+
7+
## Requirements
8+
9+
- Go 1.24 or higher
10+
- Approximately 500MB of disk space for the downloaded dataset
11+
- Internet connection (to download the dataset files)
12+
13+
## Usage
14+
15+
```sh
16+
# Navigate to the examples/load directory
17+
cd examples/load
18+
19+
# Run directly
20+
go run main.go --dir /path/to/data/directory
21+
22+
# Or build and then run
23+
go build -o modusgraph-loader
24+
./modusgraph-loader --dir /path/to/data/directory
25+
```
26+
27+
### Command Line Options
28+
29+
```sh
30+
Usage of ./modusgraph-loader:
31+
--dir string Directory where modusGraph will initialize and store the 1million dataset (required)
32+
--verbosity int Verbosity level (0-2) (default 1)
33+
```
34+
35+
## How It Works
36+
37+
1. The application creates the specified directory if it doesn't exist
38+
2. It initializes a modusGraph engine in that directory
39+
3. Downloads the 1million schema and RDF data files from the Dgraph benchmarks repository
40+
4. Drops any existing data in the modusGraph instance
41+
5. Loads the schema and RDF data into the database
42+
6. Provides progress and timing information
43+
44+
## Performance Considerations
45+
46+
- Loading the 1million dataset may take several minutes depending on your hardware
47+
- The application sets a 30-minute timeout for the loading process
48+
- Memory usage will peak during the loading process
49+
50+
## Using the Loaded Dataset
51+
52+
After loading is complete, you can use the database in other applications by initializing modusGraph
53+
with the same directory:
54+
55+
```go
56+
// Initialize modusGraph client with the same directory
57+
client, err := mg.NewClient("file:///path/to/data/directory")
58+
if err != nil {
59+
// handle error
60+
}
61+
defer client.Close()
62+
63+
// Now you can run queries against the 1million dataset
64+
```
65+
66+
## Dataset Details
67+
68+
The 1million dataset represents:
69+
70+
- Films, directors, and actors
71+
- Relationships between these entities
72+
- Various properties like names, dates, and film details
73+
74+
This is a great dataset for learning and testing graph query capabilities.

0 commit comments

Comments
 (0)