Telemetry deamon 😈

CHANGELOG.md

@@ -5,14 +5,12 @@
 ### Added
 
 * Documented environment variables used by the config server and client in the `README.md`. #336
+* Added `panoptes-utils telemetry` `run`, `stop`, and `current` commands, a Python client, and a dedicated telemetry server documentation page with append-only NDJSON logging and `/current` snapshot support. #338
 
 ### Fixed
 
 * Unified CLI logging behavior: `DEBUG` level output is now silenced by default in all CLI tools (`panoptes-config-server` and `panoptes-utils`) and only enabled when the `--verbose` flag is provided. #336
-* Config server CLI `run` command no longer exits immediately when started with `--host 0.0.0.0`; the wildcard bind address is now normalized to `localhost` for client-side connectivity checks. #336
 * Config server CLI `run` command now waits up to 30 seconds (configurable via `--startup-timeout`) for the server socket to be ready before entering the monitoring loop. #336
-* Startup "Bad connection" log message is now emitted at `TRACE` level (invisible by default) instead of `DEBUG` during the expected server-ready polling phase. #336
-* Added null guard in `set_config_entry` before saving to disk when `_pocs_cut` is uninitialized. #336
 
 ### Changed
 

README.md

@@ -28,7 +28,7 @@ pip install panoptes-utils
 Full options for install:
 
 ```bash
-pip install "panoptes-utils[config,docs,images,testing,social]"
+pip install "panoptes-utils[config,docs,images,telemetry]"
 ```
 
 See the full documentation at: https://panoptes-utils.readthedocs.io
@@ -61,7 +61,10 @@ subcommand is `image`, which includes commands for converting `cr2` files into
 The `panoptes-utils image watch <path>` command will watch the given path for
 new files and convert them to `jpg` and/or `fits` files as they are added.
 
-See `panoptes-utils --help` and `panoptes-utils image --help` for details.
+The telemetry server is also available under the main CLI as `panoptes-utils telemetry`.
+
+See `panoptes-utils --help`, `panoptes-utils image --help`, and `panoptes-utils telemetry --help`
+for details.
 
 
 Config Server
@@ -86,6 +89,51 @@ The config server and client use the following environment variables:
 | `PANOPTES_CONFIG_FILE` | The YAML configuration file to load (used by CLI). | |
 | `PANOPTES_DEBUG` | Enables verbose logging if set. | `False` |
 
+Telemetry Server
+----------------
+
+After installing with the `telemetry` option as above, start the server:
+
+```bash
+panoptes-utils telemetry run
+```
+
+The public telemetry model is intentionally simple: there is one telemetry feed,
+and `start_run()` optionally activates a run context. When a run is active,
+subsequent events are automatically associated with that run and stamped with
+`meta["run_id"]`.
+
+Example local workflow with Python:
+
+```python
+from panoptes.utils.telemetry import TelemetryClient
+
+client = TelemetryClient()
+
+print(client.ready())
+
+# Before start_run(), events are recorded without any active run context.
+client.post_event("weather", {"sky": "clear", "wind_mps": 2.1}, meta={"source": "demo"})
+
+# start_run() activates the run context for subsequent events.
+client.start_run(run_id="001")
+event = client.post_event("status", {"state": "running"})
+print(event["meta"]["run_id"])
+client.stop_run()
+
+# Or let the server create the next run automatically.
+next_run = client.start_run()
+print(next_run["run_id"], next_run["run_dir"])
+
+print(client.current()["current"])
+
+client.stop_run()
+client.shutdown()
+```
+
+For server internals, HTTP API examples, and environment variables, see the
+[Telemetry Server documentation](https://panoptes-utils.readthedocs.io/en/latest/telemetry.html).
+
 ### Development with UV
 
 This project uses UV for fast Python package and environment management with modern PEP 735 dependency groups.

docs/index.rst

@@ -14,6 +14,7 @@ Contents
 
     CLI <cli>
     Config Server <config-server>
+    Telemetry Server <telemetry>
     Module Reference <api/modules>
     Changelog <changelog>
     Authors <authors>

docs/telemetry.rst

@@ -0,0 +1,187 @@
+================
+Telemetry Server
+================
+
+The telemetry server provides a lightweight local HTTP API plus a Python client
+for recording observatory telemetry.
+
+Public telemetry model
+----------------------
+
+The public model is intentionally simple:
+
+* There is one telemetry feed.
+* ``start_run()`` activates an optional run context.
+* While a run is active, new events are automatically associated with that run.
+* Run-scoped events are stamped with ``meta.run_id``.
+
+Internally, the server stores always-on site telemetry separately from
+run-specific telemetry, but callers normally do not need to choose between
+those storage targets explicitly.
+
+CLI usage
+---------
+
+Start the telemetry server:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry run
+
+Enable verbose server logging, including one debug log line per incoming event:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry run --verbose
+
+Stop it cleanly:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry stop
+
+Inspect the current reading:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry current
+
+Follow live updates by polling for changes:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry current --follow
+
+This uses a live Rich display that refreshes in place when the current reading
+changes.
+
+Fetch one event type only:
+
+.. code-block:: bash
+
+    panoptes-utils telemetry current status
+
+Python client example
+---------------------
+
+.. code-block:: python
+
+    from panoptes.utils.telemetry import TelemetryClient
+
+    client = TelemetryClient()
+
+    client.post_event("weather", {"sky": "clear"})
+    client.start_run(run_id="001")
+    client.post_event("status", {"state": "running"})
+    print(client.current()["current"])
+    client.stop_run()
+
+HTTP API
+--------
+
+The server listens on ``localhost:6562`` by default.
+
+Endpoints
+~~~~~~~~~
+
+* ``GET /health`` - liveness check
+* ``GET /ready`` - readiness plus run-active status
+* ``GET /run`` - active run metadata
+* ``POST /run/start`` - start a run context
+* ``POST /run/stop`` - stop the active run context
+* ``POST /event`` - record an event in the current context
+* ``GET /current`` - latest telemetry values keyed by event type
+* ``GET /current/{event_type}`` - latest telemetry value for one type
+* ``POST /shutdown`` - ask the server to exit
+
+``httpie`` examples
+~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: bash
+
+    # Check readiness.
+    http :6562/ready
+
+    # Record telemetry before a run is active.
+    http POST :6562/event type=weather data:='{"sky":"clear","wind_mps":2.1}'
+
+    # Start a run explicitly.
+    http POST :6562/run/start run_id=001
+
+    # Or let the server derive the next numeric run automatically.
+    http POST :6562/run/start
+
+    # Events are now associated with the active run and stamped with meta.run_id.
+    http POST :6562/event type=status data:='{"state":"running"}'
+
+    # Inspect the materialized current view.
+    http :6562/current
+
+``curl`` examples
+~~~~~~~~~~~~~~~~~
+
+.. code-block:: bash
+
+    curl -s http://localhost:6562/ready
+
+    curl -s \
+      -X POST http://localhost:6562/event \
+      -H 'Content-Type: application/json' \
+      -d '{"type":"weather","data":{"sky":"clear","wind_mps":2.1}}'
+
+    curl -s \
+      -X POST http://localhost:6562/run/start \
+      -H 'Content-Type: application/json' \
+      -d '{"run_id":"001"}'
+
+    curl -s \
+      -X POST http://localhost:6562/event \
+      -H 'Content-Type: application/json' \
+      -d '{"type":"status","data":{"state":"running"}}'
+
+Response shape
+--------------
+
+Successful event responses include:
+
+* ``seq`` - monotonically increasing sequence number within the storage target
+* ``ts`` - UTC timestamp
+* ``type`` - event type
+* ``data`` - event payload
+* ``meta`` - caller metadata plus ``run_id`` when a run is active
+
+Example response:
+
+.. code-block:: json
+
+    {
+      "seq": 1,
+      "ts": "2026-03-18T00:05:48.955Z",
+      "type": "status",
+      "data": {"state": "running"},
+      "meta": {"run_id": "001"}
+    }
+
+Run handling
+------------
+
+``start_run()`` and ``POST /run/start`` accept optional ``run_dir`` and
+``run_id`` values.
+
+* If ``run_dir`` is relative, it is resolved under the configured site
+  telemetry directory.
+* If ``run_dir`` is omitted, the server uses ``site_dir/run_id``.
+* If both ``run_dir`` and ``run_id`` are omitted, the server derives the next
+  numeric run directory under the site telemetry directory (for example
+  ``001``, ``002``, ``003``).
+
+Environment variables
+---------------------
+
+=============================== ====================================== ============
+Variable                        Description                            Default
+=============================== ====================================== ============
+``PANOPTES_TELEMETRY_HOST``     Host address for the telemetry server. ``localhost``
+``PANOPTES_TELEMETRY_PORT``     Port number for the telemetry server.  ``6562``
+``PANOPTES_TELEMETRY_SITE_DIR`` Directory for telemetry storage.       ``telemetry/``
+=============================== ====================================== ============

pyproject.toml

@@ -38,6 +38,7 @@ dependencies = [
     "pyserial>=3.4",
     "pytest-loguru",
     "python-dateutil",
+    "rich",
     "requests",
     "ruamel.yaml",
     "typer",
@@ -78,6 +79,10 @@ images = [
     "sep",
     "watchfiles",
 ]
+telemetry = [
+    "fastapi>=0.115",
+    "uvicorn>=0.34",
+]
 
 [project.scripts]
 panoptes-config-server = "panoptes.utils.config.cli:config_server_cli"
@@ -86,6 +91,7 @@ panoptes-utils = "panoptes.utils.cli.main:app"
 [dependency-groups]
 testing = [
     "coverage",
+    "httpx",
     "pycodestyle",
     "pytest",
     "pytest-cov",