CONTRIBUTING.md

Contributing to databao-context-engine

databao-context-engine is open source software. We welcome and encourage everyone to contribute code, documentation, issues or just raise any questions you might have.

Table of Contents

• Setting up an environment
  • Pre-commit
  • Docker
  • Make
• Running dce locally
  • Using the dce command directly
    • Installing dce locally
    • Create dce alias
    • Create alias using nix shell
• Testing
  • Running test commands
  • Running linters
  • Running Mypy
• Generating JSON Schemas for our plugin's config files

Setting up an environment

These are the tools used in databao-context-engine development and testing:

• uv is our package and project manager. It's used to build, manage environments, run tests and linters. This is the only mandatory tool to run project locally (except docker which is used only for testing), for installation follow the instructions here
• pytest is our test framework
• ruff for code linting and formatting
• mypy for static type checking
• ruff-pre-commit is a pre-commit hook for ruff

Pre-commit

If you are going to push to the repository, please make sure to install git pre-commit hooks by running.

  uv run pre-commit install

Docker

Some tests rely on testcontainers thus docker installation is mandatory to run them. Specific instructions for your OS can be found here.

Make

make is used just for running high level aggregated commands. It's not mandatory, but it's usually installed by default on many distributions. Running make without target by default will execute every step required before pushing to repository:

• check code formatting
• check static type checks
• run tests

Running dce locally

You can run dce with this command:

  uv run dce info

Not providing the info subcommand or using the --help flag will show the help screen for the command.

Using the dce command directly

To be able to use the dce command directly (without using uv run or python) there are two options.

Installing dce locally

For development purposes or just to try the library, one could install dce locally.

Using uv

For that one needs to:

1. Build the project by running:

  uv build

2. Installing the project on local machine by running:

  uv tool install -e .

This second step will install the dce script on your machine and add it into your path.

Using pipx

Alternatively, one could run pipx install instead.

Create dce alias

To create a dce alias in your terminal one needs to run:

alias dce='uv --project ${projectDir} run dce'

After that, you can then directly use from any folder in your file system:

  dce --help

Create alias using nix shell

This method will simply create a new shell environment with dce alias for you automatically. For that one needs to install nix package manager (https://nixos.org/download/). After that one could simply run:

$ nix-shell {path_to_dce_repository}

This will make sure dce command is available in the current terminal session.

Testing

For testing we use uv and pytest framework. uv manages environments and dependencies automatically.

Running test commands

To run all tests:

  uv run --all-extras pytest

or a use a shortcut - make test

Running linters

To make sure there are no linting errors you can run:

uv run ruff check

You can also run linters to make sure that code is correctly formatted:

uv run ruff format --check

You can run both these commands with make lint

Running Mypy

You can run mypy to staticaly check types with the following command:

  uv run mypy . --exclude dist --exclude docs

or just with make mypy so you don't have to remember all the arguments.

Generating JSON Schemas for our plugin's config files

To be able to build a datasource, each plugin requires a YAML config file that describes how to connect to the datasource, as well as other information needed to customize the plugin.

To document what each config file should look like, we can generate a JSON schema describing the fields allowed in that file.

You can generate all JSON schemas for all plugins by running:

  uv run generate_configs_schemas

Some options can be provided to the command to choose which plugins to include or exclude from the generation. To see the options available, you can refer to the help:

  uv run generate_configs_schemas --help