A Python library that interfaces with a wide range of cameras to flexibly record visual stream data as video files.
This library abstracts all necessary steps for acquiring and saving video data. During each runtime, it interfaces with one or more cameras to grab the raw frames and encodes them as video files stored in the non-volatile memory. The library is specifically designed for working with multiple cameras at the same time and supports fine-tuning the acquisition and saving parameters to precisely balance the resultant video quality and real-time throughput for a wide range of applications.
- Supports Windows, Linux, and macOS.
- Uses OpenCV or GeniCam (Harvesters) to interface with a wide range of consumer, industrial, and scientific cameras.
- Uses FFMPEG to efficiently encode acquired data as videos in real time using CPU or GPU.
- Highly customizable and can be extensively fine-tuned for quality or throughput.
- GPL 3 License.
- FFMPEG version n8.0. The installed FFMPEG must be available on the system’s path and callable from Python processes.
- A GenTL Producer interface compatible with the Harvesters library if the target camera requires the 'harvesters' camera interface. It is recommended to use the CTI interface supplied by the camera’s vendor, if possible, as this typically ensures that the camera performs as advertised. If the camera-specific CTI file is not available, it is possible to instead use a general interface, such as MvImpactAcquire. This library has been tested using MvImpactAcquire version 2.9.2.
For users, all other library dependencies are installed automatically by all supported installation methods (see the Installation section).
Note! Developers should see the Developers section for information on installing additional development dependencies.
Note, installation from source is highly discouraged for anyone who is not an active project developer.
- Download this repository to the local machine using the preferred method, such as git-cloning. Use one of the stable releases that include precompiled binary and source code distribution (sdist) wheels.
- If the downloaded distribution is stored as a compressed archive, unpack it using the appropriate decompression tool.
cdto the root directory of the prepared project distribution.- Run
python -m pip install .to install the project. Alternatively, if using a distribution with precompiled binaries, usepython -m pip install WHEEL_PATH, replacing 'WHEEL_PATH' with the path to the wheel file.
Use the following command to install the library using pip: pip install ataraxis-video-system
While this library works on all major operating systems, it is largely up to the maintainers of the low-level library components (OpenCV, Harvesters, FFMPEG) to ensure that the operation is smooth on each supported OS. That, in turn, is not always possible for many nuanced reasons. This section summarizes the current state of the library for the three explicitly supported operating systems: macOS, Windows, and Linux.
This library was primarily written on and for Linux systems. It is extensively tested on Linux and performs well under all test conditions. It is very likely that Linux users will not experience any issues specific to this library.
The library is mostly stable on Windows systems, but requires additional setup to ensure smooth operation. First, the FFMPEG has to be updated to the latest stable version, as older versions may have a drastically reduced encoding speed even with hardware acceleration. Additionally, some of the advanced OpenCV’s features, such as the MSMF HW transformations, have to be disabled to support smooth runtimes on the Windows platform. Typically, the information of which features to disable is readily available from the OpenCV’s Windows community.
macOS mostly works as expected except for live frame displaying, which does not work for modern macOS devices. The issue is due to the OS restriction on drawing certain GUI elements outside the main thread of the application. The restriction interferes with the library, as it displays the acquired frames from the same process that interfaces with the camera to minimize the visual lag between grabbing and displaying the frame. This is a persistent issue that is unlikely to be fixed any time soon.
This is a minimal example of how to use this library. It is also available as a script. This example is intentionally kept minimal, consult the API documentation for all available VideoSystem configuration parameters.
Most library functionality is accessible through the VideoSystem class:
from pathlib import Path
import numpy as np
import tempfile
from ataraxis_time import PrecisionTimer
from ataraxis_data_structures import DataLogger, assemble_log_archives
from ataraxis_base_utilities import console, LogLevel
from ataraxis_video_system import VideoSystem, VideoEncoders, CameraInterfaces, extract_logged_camera_timestamps
# Since the VideoSystem and DataLogger classes use multiprocessing under-the-hood, the runtime must be protected by the
# __main__ guard.
if __name__ == "__main__":
# Enables the console module to communicate the example's runtime progress via the terminal.
console.enable()
# Specifies the directory where to save the acquired video frames and timestamps.
tempdir = tempfile.TemporaryDirectory() # Creates a temporary directory for illustration purposes
output_directory = Path(tempdir.name)
# The DataLogger is used to save frame acquisition timestamps to disk as uncompressed .npy files.
logger = DataLogger(output_directory=output_directory, instance_name="webcam")
# The DataLogger has to be started before it can save any log entries.
logger.start()
# The VideoSystem minimally requires an ID and a DataLogger instance. The ID is critical, as it is used to identify
# the log entries generated by the VideoSystem. For VideoSystems that will be saving frames, output_directory is
# also required
vs = VideoSystem(
system_id=np.uint8(101),
data_logger=logger,
output_directory=output_directory,
camera_interface=CameraInterfaces.OPENCV, # OpenCV interface for webcameras
display_frame_rate=15, # Displays the acquired data at a rete of 15 frames per second
color=False, # Acquires images in MONOCHROME mode
video_encoder=VideoEncoders.H264, # Uses H264 CPU video encoder.
quantization_parameter=25, # Increments the default qp parameter to reflect using the H264 encoder.
)
# Calling this method arms the video system and starts frame acquisition. However, the frames are not initially
# saved to disk.
vs.start()
console.echo(f"VideoSystem: Started", level=LogLevel.SUCCESS)
console.echo(f"Acquiring frames without saving...")
timer = PrecisionTimer("s")
timer.delay(delay=5, block=False) # During this delay, camera frames are displayed to the user but are not saved
# Begins saving frames to disk as an MP4 video file
console.echo(f"Saving the acquired frames to disk...")
vs.start_frame_saving()
timer.delay(delay=5, block=False) # Records frames for 5 seconds, generating ~150 frames
vs.stop_frame_saving()
# Frame acquisition can be started and stopped as needed, although all frames are written to the same output
# video file.
# Stops the VideoSystem runtime and releases all resources
vs.stop()
console.echo(f"VideoSystem: Stopped", level=LogLevel.SUCCESS)
# Stops the DataLogger and assembles all logged data into a single .npz archive file. This step is required to be
# able to extract the timestamps for further analysis.
logger.stop()
console.echo(f"Assembling the frame timestamp log archive...")
assemble_log_archives(remove_sources=True, log_directory=logger.output_directory, verbose=True)
# Extracts the list of frame timestamps from the assembled log archive generated above. This returns a list of
# timestamps. Each is given in microseconds elapsed since the UTC epoch onset.
console.echo(f"Extracting frame acquisition timestamps from the assembled log archive...")
timestamps = extract_logged_camera_timestamps(log_path=logger.output_directory.joinpath(f"101_log.npz"))
# Computes and prints the frame rate of the camera based on the extracted frame timestamp data.
timestamp_array = np.array(timestamps, dtype=np.uint64)
time_diffs = np.diff(timestamp_array)
fps = 1 / (np.mean(time_diffs) / 1e6)
console.echo(
message=(
f"According to the extracted timestamps, the interfaced camera had an acquisition frame rate of "
f"approximately {fps:.2f} frames / second."
),
level=LogLevel.SUCCESS,
)
# Cleans up the temporary directory before shutting the runtime down.
tempdir.cleanup()
This library relies on the DataLogger class to save frame acquisition timestamps to disk during runtime. Each saved frame’s acquisition timestamp is serialized and saved as an uncompressed .npy file.
The same DataLogger instance as used by the VideoSystem instances may be shared by multiple other Ataraxis assets that generate log entries, such as MicroControllerInterface instances. To support using the same logger instance for multiple concurrently active sources, each source has to use a unique identifier value (system id) when sending data to the logger instance.
Each frame’s acquisition timestamp is logged as a one-dimensional numpy uint8 array, saved as an .npy file. Inside the array, the data is organized in the following order:
- The uint8 id of the data source (video system instance). The ID occupies the first byte of each log entry.
- The uint64 timestamp that specifies the number of microseconds elapsed since the acquisition of the onset timestamp (see below). The timestamp occupies 8 bytes following the ID byte. This value communicates when each saved camera frame has been acquired.
Note: Timestamps are generated at frame acquisition but are only submitted to the logger when the corresponding frame is saved to disk. Therefore, the timestamps always match the order that the saved frames appear in the video file.
Each VideoSystem generates an onset timestamp as part of its start() method runtime. This log entry uses a modified
data order and stores the current UTC time, accurate to microseconds, as the total number of microseconds elapsed since
the UTC epoch onset. All further log entries for the same source use the timestamp section of their payloads to
communicate the number of microseconds elapsed since the onset timestamp acquisition.
The onset log entry uses the following data organization order:
- The uint8 id of the data source (video system instance).
- The uint64 value 0 that occupies 8 bytes following the source id. A 'timestamp' value of 0 universally indicates that the log entry stores the onset timestamp.
- The uint64 value that stores the number of microseconds elapsed since the UTC epoch onset. This value specifies the current time when the onset timestamp was generated.
See the quickstart example above for a demonstration on how to assemble and parse the frame acquisition log archives generated by the VideoSystem instance at runtime.
Note! The parsed frame acquisition timestamps are returned as a tuple of values that match the order in which the frames were saved to disk as an .np4 file. Each timestamp is given as the number of microseconds elapsed since the UTC epoch onset.
This library exposes the axvs Command-Line Interface (CLI) as part of its installation into a Python environment. To
see the list of available CLI commands, call the axvs --help command from the environment that has the library
installed or see the API documentation below.
This library supports all cameras compatible with the GeniCam standard, which includes most GigE+ scientific and machine vision cameras.
Note! Before using the library with a GeniCam camera, it must be provided with the path to the .cti GenTL Producer
Interface file. Without an interface, the library is not able to interface with the GeniCam cameras. Use the
axtl cti CLI command to configure the library to use the .cti file provided by the camera vendor (preferred) or a
general .cti file, such as mvImpactAcquire. This command only needs to be called once, as the library
remembers and reuses the provided .cti file for all future runtimes.
See the API documentation for the detailed description of the methods and classes exposed by components of this library.
Note! The API documentation also includes the details about the axvs CLI interface exposed by this library.
This section provides installation, dependency, and build-system instructions for project developers.
Note! This installation method requires mamba version 2.3.2 or above. Currently, all Sun lab automation pipelines require that mamba is installed through the miniforge3 installer.
- Download this repository to the local machine using the preferred method, such as git-cloning.
- If the downloaded distribution is stored as a compressed archive, unpack it using the appropriate decompression tool.
cdto the root directory of the prepared project distribution.- Install the core Sun lab development dependencies into the base mamba environment via the
mamba install tox uv tox-uvcommand. - Use the
tox -e createcommand to create the project-specific development environment followed bytox -e installcommand to install the project into that environment as a library.
In addition to installing the project and all user dependencies, install the following dependencies:
- Python distributions, one for each version supported by the developed project. Currently, this library supports the three latest stable versions. It is recommended to use a tool like pyenv to install and manage the required versions.
This project comes with a fully configured set of automation pipelines implemented using
tox. Check the tox.ini file for details about the
available pipelines and their implementation. Alternatively, call tox list from the root directory of the project
to see the list of available tasks.
Note! All pull requests for this project have to successfully complete the tox task before being merged.
To expedite the task’s runtime, use the tox --parallel command to run some tasks in-parallel.
Many packages used in 'tox' automation pipelines (uv, mypy, ruff) and 'tox' itself may experience runtime failures. In most cases, this is related to their caching behavior. If an unintelligible error is encountered with any of the automation components, deleting the corresponding .cache (.tox, .ruff_cache, .mypy_cache, etc.) manually or via a CLI command typically solves the issue.
This project uses semantic versioning. See the tags on this repository for the available project releases.
This project is licensed under the GPL3 License: see the LICENSE file for details.
- All Sun lab members for providing the inspiration and comments during the development of this library.
- The creators of all other dependencies and projects listed in the pyproject.toml file.