Skip to content

Add integration tests for MCP batch operations #40

@jayscambler

Description

@jayscambler

Context

During Phase 3.2 implementation of batch operations, we created unit tests for the BatchOperationHandler base class but were unable to complete integration tests for the actual batch tools due to several technical challenges.

Current State

  • ✅ Unit tests for BatchOperationHandler - 6 tests, all passing, 87% coverage
  • ❌ Integration tests for BatchTools - Scaffolded but not functional

Technical Challenges Encountered

  1. Transport Issues
    • StdioAdapter requires real stdin/stdout file descriptors
    • Doesn't work in pytest environment (raises UnsupportedOperation: redirected stdin is pseudofile)
    • Mock transport adapter works for unit tests but doesn't test real integration
  2. Schema Validation
    • FrameRecord has strict metadata validation
    • custom_metadata fields must have x_ prefix and string values
    • Test fixtures need to match exact schema requirements
  3. Async Testing
    • Batch operations return coroutines that need proper await handling
    • Need to ensure all async operations are properly tested

What Needs to Be Done

  1. Create proper test fixtures
    • Build test datasets that comply with FrameRecord schema
    • Use proper field names and types (e.g., metadata vs custom_metadata)
  2. Implement integration tests for all 8 batch tools
    • batch_search - Test parallel search execution
    • batch_add - Test atomic and non-atomic additions
    • batch_update - Test filter-based and ID-based updates
    • batch_delete - Test dry-run and confirmation flow
    • batch_enhance - Test with mock LLM provider
    • batch_extract - Test with sample files
    • batch_export - Test all export formats
    • batch_import - Test all import formats
  3. Consider alternative testing approaches
    • Use a test-specific transport that doesn't require real file descriptors
    • Create integration tests that run the full MCP server in a subprocess
    • Mock only the transport layer while testing real tool execution

Acceptance Criteria

  • All 8 batch tools have working integration tests
  • Tests cover both success and error paths
  • Tests verify atomic transactions and rollback
  • Tests verify progress tracking
  • Tests run in CI without special setup
  • At least 80% code coverage for batch tools

Technical Details

The test file is already created at contextframe/tests/test_mcp/test_batch_tools.py but needs to be fixed to work properly with the schema and transport requirements.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions