feat: Investigation Snapshot Export and Import (fn-39)#107
Merged
Conversation
Define canonical snapshot archive structure and metadata.json schema: - SNAPSHOT_SCHEMA_VERSION constant (1.0) - SnapshotMetadata Pydantic model with all required fields - ArchivePaths helper with well-known archive paths - validate_metadata() for import validation - json_schema() for external tooling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement SnapshotBuilder class that creates tar.gz archives: - Evidence items serialized as numbered JSON files - Query results stored as Parquet when pyarrow is available - Graceful JSON fallback when pyarrow not installed - metadata.json with complete file inventory and schema_version - SnapshotSizeExceededError when archive exceeds max_size_bytes - pyarrow added as optional dependency under [snapshot] extra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add API endpoints for snapshot archive export and import:
- GET /investigations/{id}/snapshot returns tar.gz with correct headers
- POST /investigations/import accepts tar.gz upload
- Validates metadata.json schema version on import
- Imported investigations marked with is_replay: true
- 404 for non-existent investigations
- 413 for files exceeding max_size_bytes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement SnapshotImporter class for importing snapshot archives: - validate_archive() validates schema version and path traversal - extract_evidence() extracts all evidence items - extract_prompts() extracts prompt templates - import_investigation() returns ImportResult with replay status - Supports Parquet reading when pyarrow available - Added InvalidSnapshotError, UnsupportedSchemaVersionError - Added SUPPORTED_SCHEMA_VERSIONS set for forward compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add CLI commands for snapshot export and import: - `dataing run snapshot <id>` downloads tar.gz archive - `dataing run import <file>` uploads and imports archive - --output flag for custom output path - --max-size flag for size limit in MB - Rich-formatted progress and results display SDK methods added: - download_snapshot(investigation_id) -> bytes - import_snapshot(file_data) -> dict Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add unit tests for CLI snapshot commands: - TestRunSnapshotCommand with 4 tests - TestRunImportCommand with 3 tests - Tests cover success, error handling, max-size limits, not found Note: Core module tests were already added in previous tasks (test_snapshot_schema.py, test_snapshot_builder.py, test_snapshot_importer.py) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
bordumb
pushed a commit
that referenced
this pull request
Feb 1, 2026
# [1.18.0](v1.17.0...v1.18.0) (2026-01-31) ### Features * Investigation Snapshot Export and Import (fn-39) ([#107](#107)) ([6bf5bab](6bf5bab))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.tar.gzsnapshot archives for investigation export/importChanges
Core
snapshot_schema.py- Schema definitions, metadata validation, archive pathssnapshot_builder.py- Builds tar.gz archives with evidence, lineage, metadatasnapshot_importer.py- Validates and imports archives as replay investigationsAPI
GET /investigations/{id}/snapshot- Stream download tar.gz archivePOST /investigations/import- Upload and import archiveCLI
dataing run snapshot <id>- Export investigation to local filedataing run import <file>- Import archive as replay investigationSDK
download_snapshot()/import_snapshot()methodsTest plan