Skip to content

Commit ee289a8

Browse files
committed
Revert "GitBook: [master] 18 pages and 5 assets modified"
This reverts commit 4265a7a.
1 parent 4265a7a commit ee289a8

17 files changed

+193
-211
lines changed
-142 KB
Binary file not shown.
-125 KB
Binary file not shown.
-125 KB
Binary file not shown.
-125 KB
Binary file not shown.
-142 KB
Binary file not shown.

README.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
1-
# Introduction
1+
# gitbase [![GitHub version](https://badge.fury.io/gh/src-d%2Fgitbase.svg)](https://github.com/src-d/gitbase/releases) [![Build Status](https://travis-ci.org/src-d/gitbase.svg?branch=master)](https://travis-ci.org/src-d/gitbase) [![codecov](https://codecov.io/gh/src-d/gitbase/branch/master/graph/badge.svg)](https://codecov.io/gh/src-d/gitbase) [![GoDoc](https://godoc.org/gopkg.in/src-d/gitbase.v0?status.svg)](https://godoc.org/gopkg.in/src-d/gitbase.v0) [![Go Report Card](https://goreportcard.com/badge/github.com/src-d/gitbase)](https://goreportcard.com/report/github.com/src-d/gitbase)
22

33
**gitbase**, is a SQL database interface to Git repositories.
44

5-
It can be used to perform SQL queries about the Git history and about the [Universal AST](https://doc.bblf.sh/) of the code itself. gitbase is being built to work on top of any number of git repositories.
5+
It can be used to perform SQL queries about the Git history and
6+
about the [Universal AST](https://doc.bblf.sh/) of the code itself. gitbase is being built to work on top of any number of git repositories.
67

7-
gitbase implements the _MySQL_ wire protocol, it can be accessed using any MySQL client or library from any language.
8+
gitbase implements the *MySQL* wire protocol, it can be accessed using any MySQL
9+
client or library from any language.
810

911
[src-d/go-mysql-server](https://github.com/src-d/go-mysql-server) is the SQL engine implementation used by `gitbase`.
1012

@@ -14,21 +16,20 @@ The project is currently in **alpha** stage, meaning it's still lacking performa
1416

1517
## Examples
1618

17-
You can see some [query examples](using-gitbase/examples.md) in [gitbase documentation](https://github.com/src-d/gitbase/tree/f68dd7f644c24e3acba490a20c6c735b7770c54b/docs/README.md).
19+
You can see some [query examples](/docs/using-gitbase/examples.md) in [gitbase documentation](/docs).
1820

1921
## Motivation and scope
2022

2123
gitbase was born to ease the analysis of git repositories and their source code.
2224

2325
Also, making it MySQL compatible, we provide the maximum compatibility between languages and existing tools.
2426

25-
It comes as a single self-contained binary and it can be used as a standalone service. The service is able to process local repositories or integrate with existing tools and frameworks \(e.g. Apache Spark\) to make source code analysis on a large scale.
27+
It comes as a single self-contained binary and it can be used as a standalone service. The service is able to process local repositories or integrate with existing tools and frameworks (e.g. Apache Spark) to make source code analysis on a large scale.
2628

2729
## Further reading
2830

29-
From here, you can directly go to [getting started](using-gitbase/getting-started.md).
31+
From here, you can directly go to [getting started](/docs/using-gitbase/getting-started.md).
3032

3133
## License
3234

33-
Apache License Version 2.0, see [LICENSE](https://github.com/src-d/gitbase/tree/f68dd7f644c24e3acba490a20c6c735b7770c54b/LICENSE/README.md)
34-
35+
Apache License Version 2.0, see [LICENSE](LICENSE)

docs/README.md

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,13 @@
1-
# Table of contents
1+
# gitbase
22

3-
* [Introduction](../README.md)
4-
* [Join the community](../join-the-community.md)
3+
* [Join the community](join-the-community.md)
54

6-
## Using gitbase
7-
8-
* [Getting started](../using-gitbase/getting-started.md)
9-
* [Configuration](../using-gitbase/configuration.md)
10-
* [Schema](../using-gitbase/schema.md)
11-
* [Supported syntax](../using-gitbase/supported-syntax.md)
12-
* [Functions](../using-gitbase/functions.md)
13-
* [Indexes](../using-gitbase/indexes.md)
14-
* [Examples](../using-gitbase/examples.md)
5+
### Using gitbase
156

7+
* [Getting started](using-gitbase/getting-started.md)
8+
* [Configuration](using-gitbase/configuration.md)
9+
* [Schema](using-gitbase/schema.md)
10+
* [Supported syntax](using-gitbase/supported-syntax.md)
11+
* [Functions](using-gitbase/functions.md)
12+
* [Indexes](using-gitbase/indexes.md)
13+
* [Examples](using-gitbase/examples.md)

join-the-community.md renamed to docs/join-the-community.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,17 @@
22

33
## Chat
44

5-
If you need support, want to contribute or just want to say hi, join us at the [source{d} community Slack](https://join.slack.com/t/sourced-community/shared_invite/enQtMjc4Njk5MzEyNzM2LTFjNzY4NjEwZGEwMzRiNTM4MzRlMzQ4MmIzZjkwZmZlM2NjODUxZmJjNDI1OTcxNDAyMmZlNmFjODZlNTg0YWM). We hang out in the \#general channel.
5+
If you need support, want to contribute or just want to say hi, join us at the [source{d} community Slack](https://join.slack.com/t/sourced-community/shared_invite/enQtMjc4Njk5MzEyNzM2LTFjNzY4NjEwZGEwMzRiNTM4MzRlMzQ4MmIzZjkwZmZlM2NjODUxZmJjNDI1OTcxNDAyMmZlNmFjODZlNTg0YWM). We hang out in the #general channel.
66

77
## Contributing
88

99
You can start contributing in many ways:
1010

11-
* [Report bugs](join-the-community.md#reporting-bugs)
11+
* [Report bugs](/docs/join-the-community.md#reporting-bugs)
1212
* [Request a feature](https://github.com/src-d/gitbase/issues)
1313
* Improve the [documentation](https://github.com/src-d/gitbase/docs)
1414
* Contribute code to [gitbase](https://github.com/src-d/gitbase)
1515

1616
## Reporting bugs
1717

1818
Bugs should be reported through [GitHub Issues](https://github.com/src-d/gitbase/issues).
19-

docs/using-gitbase/configuration.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Configuration
2+
3+
## Environment variables
4+
5+
| Name | Description |
6+
|:-----------------------------|:-----------------------------------------------------------------------------------|
7+
| `BBLFSH_ENDPOINT` | bblfshd endpoint, default "127.0.0.1:9432" |
8+
| `PILOSA_ENDPOINT` | pilosa endpoint, default "http://localhost:10101" |
9+
| `GITBASE_BLOBS_MAX_SIZE` | maximum blob size to return in MiB, default 5 MiB |
10+
| `GITBASE_BLOBS_ALLOW_BINARY` | enable retrieval of binary blobs, default `false` |
11+
| `GITBASE_SKIP_GIT_ERRORS` | do not stop queries on git errors, default disabled |
12+
| `GITBASE_INDEX_DIR` | directory to save indexes, default `/var/lib/gitbase/index` |
13+
| `GITBASE_TRACE` | enable jaeger tracing, default disabled |
14+
| `GITBASE_READONLY` | allow read queries only, disabling creating and deleting indexes, default disabled |
15+
16+
### Jaeger tracing variables
17+
18+
*Extracted from https://github.com/jaegertracing/jaeger-client-go/blob/master/README.md*
19+
20+
Property| Description
21+
--- | ---
22+
JAEGER_SERVICE_NAME | The service name
23+
JAEGER_AGENT_HOST | The hostname for communicating with agent via UDP
24+
JAEGER_AGENT_PORT | The port for communicating with agent via UDP
25+
JAEGER_REPORTER_LOG_SPANS | Whether the reporter should also log the spans
26+
JAEGER_REPORTER_MAX_QUEUE_SIZE | The reporter's maximum queue size
27+
JAEGER_REPORTER_FLUSH_INTERVAL | The reporter's flush interval (ms)
28+
JAEGER_SAMPLER_TYPE | The sampler type
29+
JAEGER_SAMPLER_PARAM | The sampler parameter (number)
30+
JAEGER_SAMPLER_MANAGER_HOST_PORT | The host name and port when using the remote controlled sampler
31+
JAEGER_SAMPLER_MAX_OPERATIONS | The maximum number of operations that the sampler will keep track of
32+
JAEGER_SAMPLER_REFRESH_INTERVAL | How often the remotely controlled sampler will poll jaeger-agent for the appropriate sampling strategy
33+
JAEGER_TAGS | A comma separated list of `name = value` tracer level tags, which get added to all reported spans. The value can also refer to an environment variable using the format `${envVarName:default}`, where the `:default` is optional, and identifies a value to be used if the environment variable cannot be found
34+
JAEGER_DISABLED | Whether the tracer is disabled or not. If true, the default `opentracing.NoopTracer` is used.
35+
JAEGER_RPC_METRICS | Whether to store RPC metrics
36+
37+
## Command line arguments
38+
39+
```
40+
Please specify one command of: server or version
41+
Usage:
42+
gitbase [OPTIONS] <server | version>
43+
44+
Help Options:
45+
-h, --help Show this help message
46+
47+
Available commands:
48+
server Starts a gitbase server instance
49+
version Show the version information
50+
```
51+
52+
`server` command contains the following options:
53+
54+
```
55+
Usage:
56+
gitbase [OPTIONS] server [server-OPTIONS]
57+
58+
Starts a gitbase server instance
59+
60+
By default when gitbase encounters an error in a repository it
61+
stops the query. With GITBASE_SKIP_GIT_ERRORS variable it won't
62+
complain and just skip those rows or repositories.
63+
64+
Help Options:
65+
-h, --help Show this help message
66+
67+
[server command options]
68+
-d, --directories= Path where the git repositories are located (standard and siva), multiple directories can be defined. Accepts globs.
69+
--depth= load repositories looking at less than <depth> nested subdirectories. (default: 1000)
70+
--host= Host where the server is going to listen (default: localhost)
71+
-p, --port= Port where the server is going to listen (default: 3306)
72+
-u, --user= User name used for connection (default: root)
73+
-P, --password= Password used for connection
74+
--pilosa= URL to your pilosa server (default: http://localhost:10101) [$PILOSA_ENDPOINT]
75+
-i, --index= Directory where the gitbase indexes information will be persisted. (default: /var/lib/gitbase/index) [$GITBASE_INDEX_DIR]
76+
--no-squash Disables the table squashing.
77+
--trace Enables jaeger tracing [$GITBASE_TRACE]
78+
-r, --readonly Only allow read queries. This disables creating and deleting indexes as well. [$GITBASE_READONLY]
79+
--no-git disable the load of git standard repositories.
80+
--no-siva disable the load of siva files.
81+
-v Activates the verbose mode
82+
83+
```

using-gitbase/examples.md renamed to docs/using-gitbase/examples.md

Lines changed: 34 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
11
# Examples
22

3-
## Examples
4-
5-
### Get all the repositories where a specific user contributes on HEAD reference
3+
## Get all the repositories where a specific user contributes on HEAD reference
64

75
```sql
86
SELECT refs.repository_id
@@ -11,28 +9,28 @@ NATURAL JOIN commits
119
WHERE commits.commit_author_name = 'Javi Fontan' AND refs.ref_name='HEAD';
1210
```
1311

14-
### Get all the HEAD references from all the repositories
12+
## Get all the HEAD references from all the repositories
1513

1614
```sql
1715
SELECT * FROM refs WHERE ref_name = 'HEAD';
1816
```
1917

20-
### First commit on HEAD history for all repositories
18+
## First commit on HEAD history for all repositories
2119

2220
```sql
2321
SELECT
24-
file_path,
25-
ref_commits.repository_id
22+
file_path,
23+
ref_commits.repository_id
2624
FROM
27-
commit_files
25+
commit_files
2826
NATURAL JOIN
29-
ref_commits
27+
ref_commits
3028
WHERE
31-
ref_commits.ref_name = 'HEAD'
32-
AND ref_commits.history_index = 0;
29+
ref_commits.ref_name = 'HEAD'
30+
AND ref_commits.history_index = 0;
3331
```
3432

35-
### Commits that appear in more than one reference
33+
## Commits that appear in more than one reference
3634

3735
```sql
3836
SELECT * FROM (
@@ -44,7 +42,7 @@ SELECT * FROM (
4442
) t WHERE num > 1;
4543
```
4644

47-
### Get the number of blobs per HEAD commit
45+
## Get the number of blobs per HEAD commit
4846

4947
```sql
5048
SELECT COUNT(c.commit_hash), c.commit_hash
@@ -56,7 +54,7 @@ INNER JOIN commit_blobs cb
5654
GROUP BY c.commit_hash;
5755
```
5856

59-
### Get commits per committer, per month in 2015
57+
## Get commits per committer, per month in 2015
6058

6159
```sql
6260
SELECT COUNT(*) as num_commits, month, repo_id, committer_email
@@ -73,25 +71,25 @@ FROM (
7371
GROUP BY committer_email, month, repo_id;
7472
```
7573

76-
### Files from first 6 commits from HEAD references that contains some key and are not in vendor directory
74+
## Files from first 6 commits from HEAD references that contains some key and are not in vendor directory
7775

7876
```sql
7977
select
80-
files.file_path,
81-
ref_commits.repository_id,
78+
files.file_path,
79+
ref_commits.repository_id,
8280
files.blob_content
8381
FROM
84-
files
82+
files
8583
NATURAL JOIN
86-
commit_files
84+
commit_files
8785
NATURAL JOIN
88-
ref_commits
86+
ref_commits
8987
WHERE
90-
ref_commits.ref_name = 'HEAD'
91-
AND ref_commits.history_index BETWEEN 0 AND 5
92-
AND is_binary(blob_content) = false
88+
ref_commits.ref_name = 'HEAD'
89+
AND ref_commits.history_index BETWEEN 0 AND 5
90+
AND is_binary(blob_content) = false
9391
AND files.file_path NOT REGEXP '^vendor.*'
94-
AND (
92+
AND (
9593
blob_content REGEXP '(?i)facebook.*[\'\\"][0-9a-f]{32}[\'\\"]'
9694
OR blob_content REGEXP '(?i)twitter.*[\'\\"][0-9a-zA-Z]{35,44}[\'\\"]'
9795
OR blob_content REGEXP '(?i)github.*[\'\\"][0-9a-zA-Z]{35,40}[\'\\"]'
@@ -105,7 +103,7 @@ WHERE
105103
);
106104
```
107105

108-
### Create an index for columns on a table
106+
## Create an index for columns on a table
109107

110108
You can create an index either on a specific column or on several columns:
111109

@@ -115,27 +113,27 @@ CREATE INDEX commits_hash_idx ON commits USING pilosa (commit_hash);
115113
CREATE INDEX files_commit_path_blob_idx ON commit_files USING pilosa (commit_hash, file_path, blob_hash);
116114
```
117115

118-
### Create an index for an expression on a table
116+
## Create an index for an expression on a table
119117

120118
Note that just one expression at a time is allowed to be indexed.
121119

122120
```sql
123121
CREATE INDEX files_lang_idx ON files USING pilosa (language(file_path, blob_content));
124122
```
125123

126-
### Drop a table's index
124+
## Drop a table's index
127125

128126
```sql
129127
DROP INDEX files_lang_idx ON files;
130128
```
131129

132-
## UAST UDFs Examples
130+
# UAST UDFs Examples
133131

134132
First of all, you should check out the [bblfsh documentation](https://docs.sourced.tech/babelfish) to get yourself familiar with UAST concepts.
135133

136-
Also, you can take a look to all the UDFs and their signatures in the [functions section](functions.md)
134+
Also, you can take a look to all the UDFs and their signatures in the [functions section](/docs/using-gitbase/functions.md)
137135

138-
### Retrieving UASTs with the UDF `uast`
136+
## Retrieving UASTs with the UDF `uast`
139137

140138
```sql
141139
SELECT file_path, uast(blob_content, language(file_path)) FROM files;
@@ -147,9 +145,9 @@ This function allows you to directly filter the retrieved UAST by performing a X
147145
SELECT file_path, uast(blob_content, language(file_path), "//FuncLit") FROM files;
148146
```
149147

150-
This UDF will give you `semantic` UASTs by default. To get some other type see the UDF [`uast_mode`](examples.md#retrieving-different-kinds-of-uasts-using-uast_mode).
148+
This UDF will give you `semantic` UASTs by default. To get some other type see the UDF [`uast_mode`](#retrieving-different-kinds-of-uasts-using-uast_mode).
151149

152-
### Retrieving different kinds of UASTs using `uast_mode`
150+
## Retrieving different kinds of UASTs using `uast_mode`
153151

154152
[bblfsh](https://docs.sourced.tech/babelfish) UAST modes: `semantic`, `annotated`, `native`
155153

@@ -161,15 +159,15 @@ SELECT file_path, uast_mode("annotated", blob_content, language(file_path)) FROM
161159
SELECT file_path, uast_mode("native", blob_content, language(file_path)) FROM files;
162160
```
163161

164-
### Filtering UASTs by XPath queries
162+
## Filtering UASTs by XPath queries
165163

166164
```sql
167165
SELECT file_path, uast_xpath(uast(blob_content, language(file_path)), "//FieldList") FROM files;
168166

169167
SELECT file_path, uast_xpath(uast_mode("annotated", blob_content, language(file_path)), "//*[@roleFunction]") FROM files;
170168
```
171169

172-
### Extracting information from UAST nodes
170+
## Extracting information from UAST nodes
173171

174172
You can retrieve information from the UAST nodes either through the special selectors `@type`, `@token`, `@role`, `@startpos`, `@endpos`:
175173

@@ -185,19 +183,18 @@ SELECT file_path, uast_extract(uast(blob_content, language(file_path), "//FuncLi
185183

186184
As result, you will get an array of arrays showing a list of the retrieved information for each of the given nodes:
187185

188-
```bash
186+
```sh
189187
+-------------------+------------------------------------------------------------------------------------------------+
190188
| file_path | uast_extract(uast(files.blob_content, language(files.file_path), "//FuncLit"), "internalRole") |
191189
+-------------------+------------------------------------------------------------------------------------------------+
192190
| benchmark_test.go | [["Args"],["Args"],["Args"],["Args"],["Args"],["Args"],["Args"]] |
193191
+-------------------+------------------------------------------------------------------------------------------------+
194192
```
195193

196-
### Getting the children of a list of nodes
194+
## Getting the children of a list of nodes
197195

198196
The UDF `uast_children` will return a flattened array of the children nodes from all the nodes in the given array.
199197

200198
```sql
201199
SELECT file_path, uast_children(uast(blob_content, language(file_path), "//FuncLit")) FROM files;
202200
```
203-

0 commit comments

Comments
 (0)