jetstream

Automated experiment analysis.

Jetstream automatically calculates metrics and applies statistical treatments to collected experiment data for different analysis windows.

For more information, see the documentation.

Local installation

# Create and activate a python virtual environment.
python3 -m venv venv/
source venv/bin/activate
# --no-deps to workaround https://github.com/pypa/pip/issues/9644
pip install --no-deps -r requirements.txt
pip install -e .

Running tests

• Install tox (included in requirements.txt)

• Run tox from wherever you cloned this repository. (You don't need to install jetstream first.)

  • To run integration tests, run tox -e py310-integration.

Linting / Formatting

We use ruff for linting and formatting:

• lint: ruff check jetstream
• format: ruff format --check jetstream

ruff can also fix (some) issues:

• lint: ruff check jetstream --fix
• format: ruff format jetstream

We also use mypy:

• mypy -p jetstream

You can also run tox py310-format to run all the ruff and mypy checks.

Dependencies

Jetstream uses pip-tools to manage dependencies, along with a script that runs the pip-tools commands. There are two requirements files auto-generated by script/update_deps:

• requirements.in: Listing of dependencies and versions generated from the dependencies list in pyproject.toml.
• requirements.txt: Auto-generated by pip-tools (pip-compile) from the requirements.in file. Also contains the hashes of each package for verification by pip during installation, and comments showing lineage for each dependency.

Add a new dependency

1. Add to pyproject.toml list of dependencies.
2. Generate new requirements files with ./script/update_deps
3. Install with python -m pip install -r requirements.txt

Update all dependencies

./script/update_deps

Be sure to run pip install --no-deps -r requirements.txt and reinstall jetstream (pip install -e .) afterwards, and test functionality!

Update a single dependency

0. Prefer to update all dependencies (see above)!

1. Edit requirements.in

• mypy==1.8.0 --> mypy==1.9.0

2. Regenerate requirements.txt

• pip-compile --generate-hashes -o requirements.txt requirements.in
  • (Note: this is the last line of script/update_deps)

3. Install dependencies

• pip install --no-deps -r requirements.txt
• pip install -e .

4. Test!