feat: add OGC Connected Systems API (CSAPI) support#136
feat: add OGC Connected Systems API (CSAPI) support#136Sam-Bolling wants to merge 17 commits intocamptocamp:mainfrom
Conversation
|
Greetings, @jahow I hope all is going well! I opened this up as a draft because I have a friend that will review it within the next week. Respectfully, |
|
Impressive work @Sam-Bolling, thank you for the contribution. It is indeed a very large amount of code and doc :) I'm willing to take you at your word and consider this as a separate module which you, @Sam-Bolling, will be in charge to maintain (at least in the near future, maybe other parties will join in later). I have neither the time nor expertise to review all of it. This being said, I would request one major thing: that all things related to the CS API not be part of the main This means that:
(unless we find a better way to handle tree-shaking). I'm going to review the changes to the existing code and give you a more thorough feedback. |
|
@Sam-Bolling also could you please give me a rough time frame for when this would be ready? I'd really like to do a 2.0 release for the library soon, and I'd like to know if this will be in it or in a subsequent release. Thank you again! |
Greetings, @jahow , thank you for the positive reception and for the clear architectural guidance and acceptance criteria. I fully agree with the separate entry point approach and will accept responsibility to maintain the CSAPI component of the library. To make sure I understand the requirements, here's my understanding of what needs to change:
Regarding timeline, I estimate 1-2 weeks to complete the refactoring, re-verify all tests, and get the CI checks passing. I noticed the formatting check failed because I hadn't run Prettier against your config, which I'll fix as part of this. What does your target timeline for 2.0 look like? I'd like to calibrate against that so we can figure out whether this lands in 2.0 or a follow-on release. Either way works for me, I want to get the architecture right instead of rush it. Respectfully, |
…ernal research plans - Archive v1 research strategy to docs/research/phase-6/archive/ - Add Plans 04-05: TypeScript Sub-Module API Design Patterns (industry case studies) and Module Decoupling Patterns (architectural patterns) - Add formal Boundary Conditions section with jahow's four non-negotiable rules from PR #136 review - Scope all research questions to respect constraints (excluded patterns section, boundary verification checklists) - Renumber former Plans 05-06 to 07-08 - Add notes document capturing analysis rationale - Total plans: 6 -> 8, estimated time: 16-24 hours
Add scope-alignment-review-notes.md documenting the review of all 8 research plans against jahow's actual PR #136 acceptance requirements. The review confirms research scope is appropriate (broad research informs good design choices) while adding implementation scope gates to prevent research findings from expanding work beyond jahow's requirements. Changes: - New: scope-alignment-review-notes.md (two-part assessment) - Plan 06: Added 'Implementation Scope Gate' section and success criterion - Plan 08: Added 'Implementation Scope Gate' section and success criterion - research-strategy.md: Added 'Implementation Scope Principle' section Key principle: Research broadly, implement minimally.
…attern Analysis Execute Research Plan 02 and produce structured findings report answering all 35 detailed questions across 5 sub-topics: 1. EDR Integration Inventory (Q1-8): 1 import, 3 public members, 0 info.ts imports, 0 index.ts exports, ~45 lines EDR types in shared model.ts 2. CSAPI Integration Inventory (Q9-16): 2 imports, 4 public members + 1 private helper, 0 info.ts imports, 183 lines index.ts exports, 0 types in shared model.ts 3. Scale Comparison (Q17-23): CSAPI is 18x larger (11,767 vs 656 lines), 9x more files (27 vs 3), 4 sub-directories vs 0 4. Info.ts and Model.ts Analysis (Q24-29): info.ts has zero imports from either module; EDR types in shared model are architecturally appropriate; CSAPI types correctly self-contained 5. Architectural Boundary Analysis (Q30-35): jahow's exact requirements extracted from PR #136; boundary between embed and separate has 3 dimensions (public API surface, import direction, module self-containment) Key finding: The integration pattern is identical for both EDR and CSAPI. The pattern itself is not the problem — the scale is. CSAPI's 183 lines of root exports and 2 reverse imports violate jahow's two explicit constraints. Feeds into: Plan 06 (Endpoint Decoupling Architecture), Plan 08 (Changelist)
Defines Phase 6 scope, acceptance criteria, and contribution definition based on jahow's PR #136 review feedback and the 8-plan research arc. Traces all architectural decisions to research findings and jahow's explicit requirements.
Evaluates all 5 open deferred issues (#98, #100, #102, #110, #111) against Phase 6 acceptance criteria (jahow's PR #136 requirements). All 5 operate in the CSAPI internal business logic layer while Phase 6 operates at the module boundary/entry point layer — zero overlap. Recommendation: Continue deferring all 5. None serve jahow's requirements, and inclusion would expand the PR diff, mix structural with behavioral changes, and increase regression risk for zero acceptance gain.
bf5c90c to
6e759a6
Compare
Greetings, @jahow, the refactoring is complete. Here's how it maps to your requirements:
This is ready now and available to land in 2.0 if the timing works, or a subsequent release — whatever suits your plan. You mentioned you'd review the changes to existing code more thoroughly I welcome that whenever you're ready. The modified upstream files are: endpoint.ts, endpoint.spec.ts, info.ts, mime-type.ts, mime-type.spec.ts, package.json, .gitignore, and README.md. Respectfully, |
Documents the discovery that compiled OpenSensorHub server JARs use different JSON property names than the OGC spec and newer source code: - `paramsSchema` vs `parametersSchema` in control stream schema responses - `params` vs `parameters` in command payloads Includes bytecode decompilation evidence, cross-server comparison, impact assessment on the upstream PR (camptocamp#136), and the local fix already applied in ogc-csapi-explorer.
|
Thanks, this looks good so far. @Sam-Bolling is it OK if I add a commit or two to adjust a few things related to the integration with the core library and the documentation? |
By all means, @jahow please do! Respectfully, |
eb0fcfc to
c94ce3c
Compare
|
Greetings, @jahow, I have a quick update: My senior dev friend came through with a code review over the weekend, and since it came after the refactoring, all the findings were internal to the CSAPI module — nothing touching the integration surface you are working on. I went ahead and knocked them all out in a single commit (Commit 17). QA workflow passes all 5 gates. Respectfully, |
Add core TypeScript interfaces and types for OGC API - Connected Systems (CSAPI) Part 1 and Part 2 resources: - System, Deployment, Procedure, SamplingFeature (Part 1) - DataStream, Observation, ControlStream, Command, CommandStatus (Part 2) - Query option types for all resource collections - CSAPIResourceTypes enum mapping resource names to URL segments - ResourceLink and navigation types for @link relations
Implement CSAPIQueryBuilder class that constructs endpoint URLs for all Connected Systems resource types with full query parameter support: - 50+ methods covering systems, deployments, procedures, sampling features, datastreams, observations, control streams, commands, and command status - GET (list/byId), POST (create), PUT (update), DELETE operations - Typed query options: bbox, datetime, limit, statusCode, controlledProperty - Nested resource navigation (e.g., system -> datastreams -> observations) - Automatic URL encoding and query string assembly
Add shared utility functions and command dispatch logic: - encodeResourceId(): safe URL encoding for resource identifiers - scanCsapiLinks(): extract CSAPI link relations from API root documents - Resource-type validation and date/time parameter formatting - Command routing: dispatches command requests through the correct control-stream -> command path for Part 2 command operations
Extract typed resources from GeoJSON feature collections per the Connected Systems Part 1 standard: - constants.ts: resource-type link relation URIs, media type strings - property.ts: extract typed properties from GeoJSON feature objects - classification.ts: parse classifier/taxonomy term arrays - geojson.ts: parseResourceRef() for @link navigation, parseValidTime() for temporal extent extraction, GeoJSON-to-typed-resource mapping
Implement parsers for the OGC SWE Common Data Model JSON encoding, used by CSAPI Part 2 for observation and command result schemas: - types.ts: SWE Common interfaces (DataRecord, DataArray, scalar/range types) - components.ts: parse Quantity, Count, Boolean, Text, Category, Time components - data-record.ts: parse DataRecord structures with nested fields - data-array.ts: parse DataArray with encoding support (JSON, text, CSV) - parser.ts: top-level dispatch that routes to the correct component parser - Full test coverage for all component types and encoding formats
Implement parsers for SensorML JSON encoding (application/sml+json), used by CSAPI to describe sensor and actuator procedures: - types.ts: SensorML interfaces (PhysicalSystem, SimpleProcess, AggregateProcess) - physical-system.ts: parse physical sensor systems with position/components - simple-process.ts: parse single-step processing descriptions - aggregate-process.ts: parse multi-component process chains - parser.ts: top-level dispatch routing to the correct process type parser - errors.ts: SensorML-specific error classes - Full test coverage for all three process types and edge cases
Implement parsers for Connected Systems Part 2 dynamic data resources: - Observation parser: extract result, phenomenonTime, resultTime, datastream ref - Command parser: extract parameters, issueTime, control-stream ref - CommandStatus parser: extract statusCode, percentComplete, timestamps - Shared parseValidTime() reuse from GeoJSON module for temporal extents - Comprehensive tests covering all Part 2 resource types and edge cases
Add the format orchestration layer that ties together all individual format parsers into a unified pipeline: - response.ts: Response wrapper that normalizes parsed CSAPI responses into a consistent structure with metadata and typed content - schema-response.ts: Schema-response parser for SWE Common observation schemas, bridging schema definitions to typed format handling - formats/index.ts: Barrel module re-exporting all format parsers (GeoJSON Part 1, SWE Common, SensorML, Part 2) with pipeline orchestration for content-type-based format selection Includes comprehensive test suites for response handling, schema-response parsing, and format module exports/pipeline behavior.
Add mock API response fixtures for a sample CSAPI-enabled OGC API endpoint (sample-data-hub): - sample-data-hub.json: Root landing page with CSAPI conformance links - sample-data-hub/conformance.json: Conformance declaration including CSAPI conformance classes - sample-data-hub/collections.json: Collections listing with CSAPI-enabled collection metadata - sample-data-hub/collections/iot-sensors.json: Individual collection detail with connected systems properties These fixtures support integration tests for endpoint discovery, conformance checking, and collection-level CSAPI capability detection.
Add end-to-end integration tests that exercise the full CSAPI module stack against mock fixtures: - discovery.spec.ts: Tests for CSAPI endpoint discovery, conformance class detection, and collection-level capability enumeration - navigation.spec.ts: Tests for hierarchical system/subsystem navigation and parent-child relationship traversal - command.spec.ts: Tests for command (tasking) operations including CRUD lifecycle and status tracking - observation.spec.ts: Tests for observation retrieval with temporal filtering, format negotiation, and result parsing - pipeline.spec.ts: Tests for the full request/response pipeline including URL construction, format selection, and response parsing These tests validate cross-cutting behavior across the URL builder, format parsers, helpers, and command routing layers.
Wire the CSAPI module into the existing ogc-client infrastructure by extending 5 upstream files with CSAPI awareness: - endpoint.ts: Add CSAPI collection detection, hasConnectedSystems flag, and CSAPI query method to OgcApiEndpoint class (+136 lines) - endpoint.spec.ts: Add tests for CSAPI endpoint discovery and collection capability detection (+53 lines) - info.ts: Extend OGC API info types with CSAPI conformance classes and collection-level connected systems metadata (+31 lines) - mime-type.ts: Register CSAPI-specific MIME types (SWE Common, SensorML, SOS observation formats) in the shared registry (+64 lines) - mime-type.spec.ts: Add tests for CSAPI MIME type registration and content-type negotiation (+111 lines) This commit represents the integration point where the standalone CSAPI module (commits 1-10) connects to the existing ogc-client library architecture.
Extend the library's public API surface in src/index.ts to re-export all CSAPI types, interfaces, and enums for consumer use: - CSAPI model types (system, observation, command, collection info) - URL builder types (query parameters, filter options, pagination) - Format parser types (SWE Common, SensorML, GeoJSON Part 1, Part 2) - Response wrapper types and schema-response interfaces - Helper utility types and command routing enums This makes the full CSAPI type system available to downstream consumers via the main ogc-client package entry point.
Add editor-specific and test artifact patterns to .gitignore: - .vscode: VS Code workspace settings (editor-specific) - test-output*.txt: Temporary test output capture files
…port
Phase 6 architecture refactoring — moves CSAPI from tightly-coupled
endpoint method to independent sub-path module.
Created:
- src/ogc-api/csapi/index.ts: barrel file re-exporting 171 public symbols
- src/ogc-api/csapi/factory.ts: createCSAPIBuilder() async factory function
- src/ogc-api/csapi/factory.spec.ts: 2 migrated tests from endpoint.spec.ts
Modified:
- src/ogc-api/endpoint.ts: removed csapi() method, CSAPI imports, cache;
made root and getCollectionDocument public
- src/ogc-api/endpoint.spec.ts: removed 3 migrated/obsolete CSAPI tests
- src/index.ts: removed ~186 CSAPI export lines (zero CSAPI references)
- package.json: added ./csapi sub-path export, sideEffects: false
Consumer API: import { ... } from '@camptocamp/ogc-client/csapi'
One-way dependency: core compiles without CSAPI (litmus-tested).
All 12 verification gates green (V1-V4, C1-C5, A4, B1-B3).
Closes camptocamp#120, Closes camptocamp#121, Closes camptocamp#122, Closes camptocamp#123, Closes camptocamp#124
Adds Connected Systems (CSAPI) to the supported standards list and a new section documenting the sub-path export consumer API: - What CSAPI is (OGC API Parts 1 & 2) - How to opt in via '@camptocamp/ogc-client/csapi' import - Usage example with createCSAPIBuilder() - Why a separate import path (bundle-size isolation)
- Replace unsafe double cast with runtime type guard in factory.ts - Remove stale `as any` cast and outdated comment in factory.ts - Rename SystemTypeUris to SYSTEM_TYPE_RECOGNITION_VALUES (avoids shadowing the type export of the same name) - Extract shared SensorML parsing helpers to _helpers.ts (reduces duplication across aggregate-process, physical-system, physical-component) - Consolidate duplicate isRecord() type guard into shared _parse-utils.ts - Remove unused imports in SensorML parsers - Reword @see tag to use plain URL (avoids false grep matches) - Apply Prettier formatting to all source files - Expand factory tests from 2 to 6 cases - Expand endpoint CSAPI tests from 3 to 7 cases - Add live-validated Property resource fixtures and test cases
Port all 20 execution passes of the Phase 7 code review cleanup plan: Phase A: camptocamp#98 (@see link), #148 (redundant Record casts) Phase B: #149 (requireObject helper), #146 (parseBaseStream), #140 (paramsSchema) Phase C: #154/#155 (parseItem callback), #143 (null guard), #144 (SensorML spread) Phase D: #142 (subPath constraint), #139 (deploymentSystems), #156/#157 (remove assertResourceAvailable), camptocamp#102 (nested parent IDs), #158/#159/#160 (build() wrapper, resolves camptocamp#111), #150 (createCommands delegates) Phase E: #147 (URL scheme validation) Phase F: #151 (shared test fixtures) Also includes post-ST#24 fixes: #162 (..' sentinel), #163 (toArray defensive), #164 (202 JSDoc), #165 (label fallback) Source: OS4CSAPI/ogc-client-CSAPI_2 branch phase-7, commit b11f893 Validation: tsc 0 errors, 30 CSAPI suites / 1,349 tests all passing
|
hey @Sam-Bolling , sorry for not answering sooner, other stuff got in the way. I still want to do a 2.0 release with this merged but I need some time to properly work on it. Thank you for your understanding, I hope you can use a fork in the meantime |
Summary
Resolves #118
Adds client-side support for the OGC API — Connected Systems standard (Parts 1 & 2) to ogc-client. This enables discovery, navigation, and querying of IoT sensor systems, observations, datastreams, commands, and related resources through Connected Systems-enabled OGC API endpoints.
Motivation
The OGC API — Connected Systems standard (OGC 23-001, OGC 23-002) defines a REST API for managing sensor systems, observations, and tasking — extending the OGC API family into the IoT/sensor domain. There is currently no JavaScript/TypeScript client library with CSAPI support. This contribution fills that gap within ogc-client's existing architecture.
Architecture (Phase 6 update)
CSAPI is implemented as a fully decoupled sub-path module — it can be imported independently without pulling in the core library, and the core library compiles and runs without CSAPI present.
Key design points:
@camptocamp/ogc-client/csapisub-path export with 171 public symbolscreateCSAPIBuilder()async factory replaces the formerendpoint.csapi()methodhasConnectedSystemsandcsapiCollectionsremain onOgcApiEndpoint(zero CSAPI imports)package.jsondeclares"sideEffects": falsefor bundler tree-shakingConsumer guide
CSAPI is available as an opt-in sub-path export. Users install one package (
npm install @camptocamp/ogc-client) — same as before. CSAPI code is only included in production bundles if explicitly imported. No configuration required; modern bundlers (Webpack 5+, Vite, Rollup, esbuild) handle this automatically.If you don't need Connected Systems — nothing changes:
CSAPI code will not be included in your bundle.
If you need Connected Systems — add a second import:
The
createCSAPIBuilder()factory connects to the endpoint and discovers available resources. The returnedCSAPIQueryBuilderprovides methods for all 9 resource types: systems, deployments, sampling features, procedures, properties, datastreams, observations, control streams, and commands.Why a separate import path? The OGC Connected Systems standard spans 9 resource types across 2 specification parts with multiple response formats (GeoJSON, SWE Common, SensorML). By isolating CSAPI behind a sub-path export, users who only need WMS/WFS/WMTS/etc. don't pay the bundle-size cost for functionality they're not using.
A note on PR size and maintainability
This contribution is ~32k lines across 86 files, and I want to address that directly. This is primarily because OGC API — Connected Systems Parts 1 & 2 represents a complex standard that spans both feature-style metadata resources and dynamic data interactions (observations, commands, events), and covers multiple resource families and response models. As a result, a complete client implementation requires substantial type coverage, URL/query builders, and format parsing logic, along with support for resource lifecycle operations, advanced filtering, and the standard's optional pagination patterns.
A significant portion of the line count is test coverage and inline documentation/JSDoc added to keep the module understandable and maintainable, rather than implementation logic alone. The integration impact on existing ogc-client behavior is intentionally small: CSAPI is implemented as a self-contained module, with only a small set of upstream touchpoints (endpoint discovery, conformance typing, and MIME type registration), no new dependencies, and no breaking API changes. I'm committed to maintaining the CSAPI module and responding to follow-up issues, and I'm also happy to refactor, split, or defer portions of the contribution if that would make review and upstream acceptance easier.
Quality assurance
This contribution has been incrementally code reviewed and smoke tested throughout development. All reports are maintained in our development archive repo https://github.com/OS4CSAPI/ogc-client-CSAPI_2
32 code review reports - incremental reviews conducted after each implementation phase, covering type safety, standards compliance, error handling, and test coverage. Reports are located in https://github.com/OS4CSAPI/ogc-client-CSAPI_2/tree/main/docs/implementation; the latest is https://github.com/OS4CSAPI/ogc-client-CSAPI_2/blob/phase-7/docs/implementation/phase-7.1-code-review.md.
25 live server smoke tests - tested against two live CSAPI servers: OpenSensorHub (http://45.55.99.236:8080/sensorhub/api) and 52°North (https://csa.demo.52north.org/), validating endpoint discovery, resource navigation, format parsing, and content negotiation against real-world implementations. Reports are also in https://github.com/OS4CSAPI/ogc-client-CSAPI_2/tree/main/docs/implementation; the latest is https://github.com/OS4CSAPI/ogc-client-CSAPI_2/blob/phase-7/docs/implementation/live-server-smoke-test-post-phase-7.md.
Usability - A separate Webapp project was created solely to evaluate the usability of the full range of functionality and compliance of the CSAPI client library. It has been assessed extensively against the two live CSAPI server implementations. The CSAPI explorer demo webapp with setup instructions is available here https://github.com/OS4CSAPI/ogc-csapi-explorer.
Phase 6 verification (12 gates):
src/ogc-api/csapi/hasConnectedSystems,csapiCollections, all non-CSAPI functionality unchangedWhat's included
86 files across 17 commits (13 feature + 1 architecture refactoring + 1 docs + 1 quality hardening + 1 code review cleanup):
.gitignore,package.json,README.md)Modified upstream files (9 files)
.gitignore.vscodeand test-output entriesREADME.mdpackage.json./csapisub-path export,sideEffects: falsesrc/index.tssrc/ogc-api/endpoint.tshasConnectedSystems,csapiCollections;csapi()method removed (decoupled to factory in Phase 6);root/getCollectionDocumentmade publicsrc/ogc-api/endpoint.spec.tssrc/ogc-api/info.tssrc/shared/mime-type.tssrc/shared/mime-type.spec.tsCommits 1–13: CSAPI implementation
Commit 14: Architecture refactoring (Phase 6)
Decouples CSAPI from
OgcApiEndpointinto an independent sub-path module:src/ogc-api/csapi/index.tssrc/ogc-api/csapi/factory.tscreateCSAPIBuilder()async factory functionsrc/ogc-api/csapi/factory.spec.tssrc/ogc-api/endpoint.tscsapi()method, CSAPI imports, cache; maderoot/getCollectionDocumentpublicsrc/ogc-api/endpoint.spec.tssrc/index.tspackage.json./csapisub-path export,sideEffects: falseCommit 15: Consumer documentation
README.mdCommit 16: Code quality hardening
Post-merge code audit improvements (no new features, no API changes):
as anycast and outdated comment in factory.tsSystemTypeUristoSYSTEM_TYPE_RECOGNITION_VALUES(avoids shadowing the type export of the same name)_helpers.ts(reduces duplication across aggregate-process, physical-system, physical-component)isRecord()type guard into shared_parse-utils.ts@seetag to use plain URL (avoids false grep matches)Commit 17: Code review cleanup (Phase 7)
Comprehensive cleanup from a senior-developer-level code review — 17 issues resolved across 20 execution passes (no new features, no API changes):
requireObjecthelper,parseBaseStreamshared fields,build()URL construction wrapper (rewrites 87 methods)parseItemcallback forparseCollectionResponse, null guard inextractCSAPIFeature, explicit SensorML field extraction,subPathunion type constraintparamsSchemafallback for older servers,parseValidTime..sentinel,observedPropertieslabel fallbackscanCsapiLinks,assertResourceAvailableremoval (33+39 methods), nested parent ID parametersas Recordcast removal (27 instances),createCommandsdelegation, shared test fixture extraction,@seelink precisionWhat's NOT included
How to test
All test files can run independently. Integration tests use the included fixtures.
Standards references