Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,5 @@ git_override(
commit = "3c65b223e9f516f95935bb4cd2e83d6088ca016f",
remote = "https://github.com/eclipse-score/baselibs.git",
)

bazel_dep(name = "score_itf", version = "0.1.0")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended

[Maintainability] score_itf is only used by integration tests (the BUILD file sets default_testonly = True), but the bazel_dep is declared as a non-dev dependency. This forces all downstream consumers of score_logging to resolve score_itf, even though they don't need it.

Other test-only dependencies in this file (e.g., score_process, score_platform) use dev_dependency = True. Apply the same here:

Suggested change
bazel_dep(name = "score_itf", version = "0.1.0")
bazel_dep(name = "score_itf", version = "0.1.0", dev_dependency = True)
Context for Agents
`score_itf` is only used by integration tests (the BUILD file sets `default_testonly = True`), but the `bazel_dep` is declared as a non-dev dependency. This forces all downstream consumers of `score_logging` to resolve `score_itf`, even though they don't need it.

Other test-only dependencies in this file (e.g., `score_process`, `score_platform`) use `dev_dependency = True`. Apply the same here:
```suggestion
bazel_dep(name = "score_itf", version = "0.1.0", dev_dependency = True)
```

File: MODULE.bazel
Line: 124
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.95
  • Likelihood Score: 5/5
  • Likelihood Reason: The issue is inevitable: any downstream consumer of score_logging will be forced to resolve score_itf even though it's only used by tests. This is a direct consequence of the missing dev_dependency = True flag.
  • Suggested Action: keep
  • Tool Calls Used: 9

Reasoning

The comment is accurate and actionable:

  1. New code introduced by this PR: The bazel_dep(name = "score_itf", version = "0.1.0") on line 124 is entirely new — the base MODULE.bazel ends at line 122. This is not legacy debt.

  2. score_itf is only used in test code: The only reference to score_itf is in tests/integration/BUILD, which:

    • Has package(default_testonly = True) on line 16
    • Uses load("@score_itf//:defs.bzl", "py_itf_test") and @score_itf//config
    • No other BUILD file in the repository references score_itf
  3. Inconsistency with existing patterns: Other test-only dependencies in MODULE.bazel consistently use dev_dependency = True:

    • score_process (line 34): dev_dependency = True
    • score_platform (line 35): dev_dependency = True
    • score_toolchains_gcc (line 38): dev_dependency = True
    • score_toolchains_qnx (line 39): dev_dependency = True
    • etc.
  4. Real downstream impact: In Bazel's module system, non-dev dependencies propagate to downstream consumers. Missing dev_dependency = True means any project depending on score_logging will need to resolve score_itf, even though they have no use for it.

  5. The fix is concrete and correct: Adding dev_dependency = True is the right solution, consistent with the existing codebase conventions.

Evidence Summary

1. File Snapshot (HEAD)

  • File: tests/integration/BUILD
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

2. File Snapshot (HEAD)

  • File: MODULE.bazel
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

3. File Snapshot (BASE)

  • File: MODULE.bazel
  • Lines: 115-125
115: )
116: 
117: bazel_dep(name = "score_baselibs", version = "0.2.0")
118: git_override(
119:     module_name = "score_baselibs",
120:     commit = "3c65b223e9f516f95935bb4cd2e83d6088ca016f",
121:     remote = "https://github.com/eclipse-score/baselibs.git",
122: )
123:

4. File Lookup

  • Filename: BUILD
  • Max Results: 10
Found 10 file(s) matching 'BUILD':
- BUILD
- examples/BUILD
- score/datarouter/BUILD
- score/datarouter/build_configuration_flags/BUILD
- score/datarouter/daemon_communication/BUILD
- score/datarouter/dlt_filetransfer_trigger_lib/BUILD
- score/datarouter/error/BUILD
- score/datarouter/lib/synchronized/BUILD
- score/datarouter/network/BUILD
...

5. File Lookup

  • Filename: BUILD.bazel
  • Max Results: 10
No files found matching 'BUILD.bazel'

6. File Snapshot (HEAD)

  • File: examples/BUILD
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

7. File Snapshot (HEAD)

  • File: score/datarouter/BUILD
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

8. File Snapshot (HEAD)

  • File: BUILD
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

Fix in Cursor

36 changes: 36 additions & 0 deletions tests/integration/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

load("@score_itf//:defs.bzl", "py_itf_test")

package(default_testonly = True)

py_itf_test(
name = "test_datarouter_dlt",
srcs = [
"__init__.py",
"conftest.py",
"test_datarouter_dlt.py",
],
args = [
"--ecu=s_core_ecu_qemu_bridge_network",
"--target_config=$(location @score_itf//config)",
"--hw", # Required even if we run the test against a QEMU target
],
data = [
"@score_itf//config",
],
plugins = [
"itf.plugins.base.base_plugin",
],
)
69 changes: 69 additions & 0 deletions tests/integration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

# Integration Tests

Integration tests for the `score_logging` component that verify DLT (Diagnostic Log and Trace) message capture on a QNX 8.0 QEMU target.

## Prerequisites

Complete the QNX 8.0 QEMU environment setup as described in the [QNX QEMU documentation](score_reference_integration/qnx_qemu/README.md).

### Network Bridge Setup (virbr0)

The integration tests use QEMU with bridge networking, which requires the `virbr0` virtual bridge interface. Instructions: reference_integration/qnx_qemu/README.md#Configure QEMU networking

This creates:
- Bridge interface `virbr0` with IP `192.168.122.1/24` on the host. Reference: /usr/share/libvirt/networks/default.xml
- DHCP server assigning IPs `192.168.122.2-254` to VMs
- NAT rules for outbound internet access

## Usage

### 1. Start the QNX QEMU Target

Build and launch the QEMU virtual machine from the `score_reference_integration/qnx_qemu` directory:

```bash
bazel build --config=x86_64-qnx \
--credential_helper=*.qnx.com=$(pwd)/../toolchains_qnx/tools/qnx_credential_helper.py \
//build:init

bazel run --config=x86_64-qnx //:run_qemu
```

Note the assigned IP address once the QEMU VM boots (e.g., `192.168.122.76`) since the actual IP address of the target (QEMU VM) is assigned dynamically via DHCP hence changes every reboot. We use the S_CORE_ECU_QEMU_BRIDGE_NETWORK_PP target config for the tests.

### 2. Execute the Tests

Run the integration tests from the `score_logging` directory, specifying the target IP:

```bash
bazel test //tests/integration:test_datarouter_dlt \
--test_arg=--target-ip=<QEMU_VM_IP> \
--test_output=streamed \
--nocache_test_results
```

## Test Details

### test_datarouter_dlt

Validates DLT message capture from the datarouter by:

1. Connecting to the QNX target via SSH
2. Starting the datarouter process if not already running
3. Capturing DLT messages over UDP for approximately 10 seconds
4. Verifying receipt of messages with `APP_ID=DR` and `CTX_ID=STAT`

The datarouter emits statistics messages every 5 seconds. The test expects at least 1 message during the capture window for reliable test results

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended

[Documentation] The documentation states the test expects "at least 1 message", but the implementation in test_datarouter_dlt.py asserts message_count > 1 (i.e., at least 2 messages). The documentation should be updated to match the code to avoid confusion.

Suggested change
The datarouter emits statistics messages every 5 seconds. The test expects at least 1 message during the capture window for reliable test results
The datarouter emits statistics messages every 5 seconds. The test expects more than 1 message during the capture window for reliable test results
Context for Agents
The documentation states the test expects "at least 1 message", but the implementation in `test_datarouter_dlt.py` asserts `message_count > 1` (i.e., at least 2 messages). The documentation should be updated to match the code to avoid confusion.

```suggestion
The datarouter emits statistics messages every 5 seconds. The test expects more than 1 message during the capture window for reliable test results
```

File: tests/integration/README.md
Line: 69
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.97
  • Likelihood Score: 5/5
  • Likelihood Reason: The mismatch between documentation and code is definitively present. Anyone reading the README will get a wrong understanding of the test's assertion threshold.
  • Suggested Action: keep
  • Tool Calls Used: 1

Reasoning

The comment correctly identifies a documentation-code mismatch introduced in this PR:

  • README line 69 states: "The test expects at least 1 message during the capture window"
  • test_datarouter_dlt.py line 65 asserts: message_count &gt; 1 (i.e., at least 2 messages)

"At least 1" corresponds to &gt;= 1, but the code uses &gt; 1 which means "more than 1" or "at least 2". The suggested fix ("The test expects more than 1 message during the capture window") accurately matches the code's assertion. Both files are newly added in this PR, so this is not legacy debt. The comment is specific, actionable, and correct.

Evidence Summary

1. File Snapshot (HEAD)

  • File: tests/integration/test_datarouter_dlt.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

Fix in Cursor

12 changes: 12 additions & 0 deletions tests/integration/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
86 changes: 86 additions & 0 deletions tests/integration/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

import logging
import time
import pytest

from itf.plugins.com.ssh import execute_command, execute_command_output

logger = logging.getLogger(__name__)


def pytest_addoption(parser):
parser.addoption(
"--target-ip",
action="store",
required=True,
help="Target IP address for SSH connection to target (QEMU VM)",
)


@pytest.hookimpl(trylast=True)
def pytest_sessionstart(session):
"""Override test target config IP with the current target (QEMU VM)

We use the S_CORE_ECU_QEMU_BRIDGE_NETWORK_PP target config
for the tests, but the actual IP address of the target (QEMU VM) is
assigned dynamically via DHCP. This hook updates the target
configuration for the test session.
"""
from itf.plugins.base.base_plugin import TARGET_CONFIG_KEY

if TARGET_CONFIG_KEY in session.stash:
target_ip_address = session.config.getoption("--target-ip")
target_config = session.stash[TARGET_CONFIG_KEY]
target_config._BaseProcessor__ip_address = target_ip_address

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended

[Maintainability] The hook rewrites target_config._BaseProcessor__ip_address. That field is name-mangled specifically to keep it private to the ITF BaseProcessor implementation, so mutating it from the test suite creates a hard dependency on score_itf internals. Any future change to the plugin (renaming the attribute, validating writes, or making the config immutable) will break these tests even though the public API remains compatible, making upgrades or rollbacks of score_itf high risk. Please override the target IP through the supported configuration surface—e.g., update the target config file passed via --target_config, or use a public setter/fixture provided by score_itf—so the tests depend only on the published contract.

Context for Agents
The hook rewrites `target_config._BaseProcessor__ip_address`. That field is name-mangled specifically to keep it private to the ITF BaseProcessor implementation, so mutating it from the test suite creates a hard dependency on score_itf internals. Any future change to the plugin (renaming the attribute, validating writes, or making the config immutable) will break these tests even though the public API remains compatible, making upgrades or rollbacks of score_itf high risk. Please override the target IP through the supported configuration surface—e.g., update the target config file passed via `--target_config`, or use a public setter/fixture provided by score_itf—so the tests depend only on the published contract.

File: tests/integration/conftest.py
Line: 46
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.82
  • Likelihood Score: 4/5
  • Likelihood Reason: Accessing a name-mangled private attribute of an external library creates a hard coupling to internal implementation details. Since this is an external dependency (score_itf/ITF plugin), any update to the library that changes the BaseProcessor internals will break these tests. The double-underscore naming convention was specifically chosen by the library author to signal this attribute should not be accessed externally.
  • Suggested Action: keep
  • Tool Calls Used: 11

Reasoning

The comment correctly identifies a real anti-pattern introduced by this PR:

  1. New code, not legacy: The file tests/integration/conftest.py is entirely new in this PR, so this is not pre-existing debt.

  2. Name-mangling is correctly identified: _BaseProcessor__ip_address is indeed a Python name-mangled attribute, resulting from __ip_address being defined in class BaseProcessor. This naming convention is Python's mechanism for making attributes private to the class.

  3. External dependency concern is valid: The itf.plugins.base.base_plugin module is not part of this repository (confirmed by find_file returning no results for base_plugin.py). Accessing private internals of an external library is particularly risky since the consuming code has no control over when those internals change.

  4. Actionability: The comment suggests concrete alternatives - updating the target config file via --target_config or using a public setter/fixture from score_itf. While we can't verify the exact API of the external library, these are reasonable and specific suggestions for avoiding the anti-pattern.

  5. The risk is real: When score_itf updates, if BaseProcessor changes its __ip_address attribute name, adds validation, or makes configs immutable, line 46 will fail with an AttributeError or unexpected behavior, even though the public API may remain compatible.

The only limitation is that without access to the score_itf library source, we can't verify exactly what public alternatives exist. However, the fundamental concern about accessing name-mangled private attributes of an external dependency is well-founded and actionable.

Evidence Summary

1. File Lookup

  • Filename: base_plugin.py
No files found matching 'base_plugin.py'

2. File Snapshot (HEAD)

  • File: tests/integration/test_datarouter_dlt.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

3. File Snapshot (HEAD)

  • File: tests/integration/conftest.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

4. File Lookup

  • Filename: MODULE.bazel
Found 1 file(s) matching 'MODULE.bazel':
- MODULE.bazel

Fix in Cursor

logger.info("Connecting to target IP: %s", target_ip_address)


_DATAROUTER_CHECK_CMD = "/proc/boot/pidin | /proc/boot/grep datarouter"
# pathspace ability provides the datarouter access to the `procnto` pathname prefix space
# required for mw/com message passing with mw::log frontend
_DATAROUTER_START_CMD = (
"cd /usr/bin && on -A nonroot,allow,pathspace -u 1051:1091 "
"./datarouter --no_adaptive_runtime &"
)
_DATAROUTER_STARTUP_TIMEOUT_SEC = 2


@pytest.fixture(scope="module")
def datarouter_running(target_fixture):
with target_fixture.sut.ssh() as ssh:
exit_status, stdout_lines, _ = execute_command_output(ssh, _DATAROUTER_CHECK_CMD)
output = "\n".join(stdout_lines)

if "datarouter" not in output:
logger.info("Datarouter not running. Starting Datarouter..")
execute_command(ssh, _DATAROUTER_START_CMD)
time.sleep(_DATAROUTER_STARTUP_TIMEOUT_SEC)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommended

[Testing] Using a fixed time.sleep(2) to wait for the datarouter process to start can lead to flaky tests. On a heavily loaded system or a slow VM, the process might take longer to initialize, causing the test to fail unnecessarily. A more robust approach is to poll for the process status in a loop with a timeout.

Suggested change
time.sleep(_DATAROUTER_STARTUP_TIMEOUT_SEC)
timeout_seconds = 10
poll_interval = 0.5
end_time = time.time() + timeout_seconds
datarouter_started = False
while time.time() < end_time:
_, stdout_lines, _ = execute_command_output(ssh, _DATAROUTER_CHECK_CMD)
if "datarouter" in "\n".join(stdout_lines):
datarouter_started = True
break
time.sleep(poll_interval)
if not datarouter_started:
pytest.fail(f"Datarouter did not start within {timeout_seconds} seconds")
Context for Agents
Using a fixed `time.sleep(2)` to wait for the `datarouter` process to start can lead to flaky tests. On a heavily loaded system or a slow VM, the process might take longer to initialize, causing the test to fail unnecessarily. A more robust approach is to poll for the process status in a loop with a timeout.

```suggestion
            timeout_seconds = 10
            poll_interval = 0.5
            end_time = time.time() + timeout_seconds
            datarouter_started = False
            while time.time() < end_time:
                _, stdout_lines, _ = execute_command_output(ssh, _DATAROUTER_CHECK_CMD)
                if "datarouter" in "\n".join(stdout_lines):
                    datarouter_started = True
                    break
                time.sleep(poll_interval)

            if not datarouter_started:
                pytest.fail(f"Datarouter did not start within {timeout_seconds} seconds")
```

File: tests/integration/conftest.py
Line: 69
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.85
  • Likelihood Score: 3/5
  • Likelihood Reason: The test runs against a QEMU VM (mentioned in the code comments). On a heavily loaded CI system or slow VM, 2 seconds may not be enough for the datarouter process to initialize, causing flaky test failures. The concern is plausible but not inevitable since the author likely tested with this timeout.
  • Suggested Action: keep
  • Tool Calls Used: 2

Reasoning

The comment targets line 69 in a brand-new file introduced by this PR (tests/integration/conftest.py). The code uses time.sleep(_DATAROUTER_STARTUP_TIMEOUT_SEC) where the constant is 2 seconds, to wait for a datarouter process to start on a QEMU VM before checking if it's running.

Accuracy: The comment correctly identifies the fixed sleep pattern and its potential for flakiness. The current code does have a verification check after the sleep (lines 71-73) that will fail the test if the process hasn't started, but this means if 2 seconds isn't long enough, the test will fail unnecessarily rather than retrying.

Actionability: The suggestion provides a complete, concrete code replacement using a polling loop with configurable timeout (10 seconds) and poll interval (0.5 seconds). This is highly actionable.

Diff Responsibility: This is entirely new code introduced by this PR, so the issue is squarely introduced by this change.

Technical merit: The polling approach is strictly better because:

  1. It can return faster if the process starts quickly (no need to always wait the full 2 seconds)
  2. It tolerates slower environments by allowing up to 10 seconds
  3. It's a well-established pattern for avoiding flaky integration tests

The context that this runs against a QEMU VM (as noted in the code's own comments) makes variable startup times more likely, strengthening the comment's concern.

Evidence Summary

1. File Snapshot (HEAD)

  • File: tests/integration/conftest.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

Fix in Cursor


_, stdout_lines, _ = execute_command_output(ssh, _DATAROUTER_CHECK_CMD)
if "datarouter" not in "\n".join(stdout_lines):
pytest.fail("Failed to start datarouter on target")
logger.info("Datarouter started successfully..")
else:
logger.info("Datarouter already running!")
yield


@pytest.fixture(scope="module")
def dlt_receiver_config(target_config_fixture):
data_router_config = target_config_fixture.data_router_config
return {
"vlan_address": data_router_config["vlan_address"],
"multicast_addresses": data_router_config["multicast_addresses"],
}
65 changes: 65 additions & 0 deletions tests/integration/test_datarouter_dlt.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

import logging
import os
import time

from itf.plugins.dlt.dlt_receive import DltReceive, Protocol


logger = logging.getLogger(__name__)

# Datarouter messages with Context ID "STAT" sent every ~5s hence 10s to reliably capture and verify
CAPTURE_DURATION_SECONDS = 10

# DLT message identifiers for datarouter statistics
APP_ID = "DR"
CTX_ID = "STAT"


def test_dlt_capture(datarouter_running, test_config_fixture, dlt_receiver_config):
"""Verify DLT messages can be captured from the datarouter.
"""
dlt_file = "/tmp/test_dlt_capture.dlt"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

[Testing] The test uses a hardcoded file path /tmp/test_dlt_capture.dlt. This can cause conflicts if tests are run in parallel and is not clean. Use the pytest tmp_path fixture to create a unique temporary file for the test run. This requires adding tmp_path to the test function's arguments, using it to construct the file path, and passing str(dlt_file) to DltReceive.

Example:

# In test function signature:
def test_dlt_capture(..., tmp_path):

# In test body:
dlt_file = tmp_path / "test_dlt_capture.dlt"

# In DltReceive call:
with DltReceive(
    ...,
    file_name=str(dlt_file),
    ...
):
Context for Agents
The test uses a hardcoded file path `/tmp/test_dlt_capture.dlt`. This can cause conflicts if tests are run in parallel and is not clean. Use the pytest `tmp_path` fixture to create a unique temporary file for the test run. This requires adding `tmp_path` to the test function's arguments, using it to construct the file path, and passing `str(dlt_file)` to `DltReceive`.

Example:
```python
# In test function signature:
def test_dlt_capture(..., tmp_path):

# In test body:
dlt_file = tmp_path / "test_dlt_capture.dlt"

# In DltReceive call:
with DltReceive(
    ...,
    file_name=str(dlt_file),
    ...
):
```

File: tests/integration/test_datarouter_dlt.py
Line: 34
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.82
  • Likelihood Score: 3/5
  • Likelihood Reason: This is an integration test that connects to a specific QEMU VM target and takes 10+ seconds, so parallel execution of the same test is unlikely in practice. However, the hardcoded path has no cleanup, and collisions with other tests or stale files from previous runs are plausible. The cleanliness concern is real even if parallel execution is rare.
  • Suggested Action: keep
  • Tool Calls Used: 3

Reasoning

The comment correctly identifies that the newly added test file uses a hardcoded path /tmp/test_dlt_capture.dlt at line 34. This is introduced by this PR (the entire file is new).

Accuracy: The comment accurately describes the issue. The hardcoded /tmp path:

  1. Could cause conflicts if tests run in parallel (though unlikely for this specific integration test)
  2. Leaves stale files without cleanup
  3. Is not idiomatic pytest — the tmp_path fixture exists precisely for this purpose

Actionability: The comment is highly actionable, providing:

  1. The specific problem (hardcoded path at line 34)
  2. A concrete solution (use tmp_path fixture)
  3. Exact code examples showing what to change in the function signature and body

Considerations: This is a Bazel-based integration test (py_itf_test) that connects to a QEMU VM. While the parallel test conflict risk is somewhat low for this specific test (it requires a running QEMU target), the suggestion to use tmp_path is still a good practice improvement. The DltReceive accepts a file_name parameter as a string, so passing str(dlt_file) from tmp_path should work identically. pytest's tmp_path also provides automatic cleanup, which the hardcoded path lacks.

Diff Responsibility: This is newly introduced code in this PR, so the issue is squarely within the PR's responsibility.

Evidence Summary

1. File Snapshot (HEAD)

  • File: tests/integration/BUILD
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

2. File Snapshot (HEAD)

  • File: tests/integration/conftest.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

Fix in Cursor


vlan_address = dlt_receiver_config["vlan_address"]
multicast_addresses = dlt_receiver_config["multicast_addresses"]

# TODO: Replace with DltWindow when fixed in ITF.
with DltReceive(
target_ip=vlan_address,
protocol=Protocol.UDP,
file_name=dlt_file,
binary_path=test_config_fixture.dlt_receive_path,
data_router_config={
"vlan_address": vlan_address,
"multicast_addresses": multicast_addresses,
},
):
time.sleep(CAPTURE_DURATION_SECONDS)

assert os.path.exists(dlt_file), f"DLT file not created: {dlt_file}"

with open(dlt_file, "rb") as f:
dlt_data = f.read()

logger.info("DLT file size: %d bytes", len(dlt_data))

# DLT extended header: APP-ID (4 bytes) followed by CTX-ID (4 bytes)
pattern = f"{APP_ID}\x00\x00{CTX_ID}".encode()
message_count = dlt_data.count(pattern)

logger.info("Found %d messages with app_id=%s, context_id=%s", message_count, APP_ID, CTX_ID)

assert message_count > 1, f"Expected more than 1 message, but got {message_count}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

[Logic] The README and docstring state that the test only needs to observe at least one APP_ID=DR / CTX_ID=STAT message during the capture window, but the assertion requires message_count > 1. Because the datarouter emits roughly every 5 s and the capture runs for only 10 s, any clock skew or jitter that produces a single emission will cause this test to fail even though the system behaved as intended, making the test flaky. Please align the assertion with the documented requirement:

-    assert message_count > 1, f"Expected more than 1 message, but got {message_count}"
+    assert message_count >= 1, f"Expected at least 1 message, but got {message_count}"

(If a higher threshold is really required, update the documentation and fixture comments accordingly.)

Context for Agents
The README and docstring state that the test only needs to observe *at least one* `APP_ID=DR` / `CTX_ID=STAT` message during the capture window, but the assertion requires `message_count > 1`. Because the datarouter emits roughly every 5 s and the capture runs for only 10 s, any clock skew or jitter that produces a single emission will cause this test to fail even though the system behaved as intended, making the test flaky. Please align the assertion with the documented requirement:

```diff
-    assert message_count > 1, f"Expected more than 1 message, but got {message_count}"
+    assert message_count >= 1, f"Expected at least 1 message, but got {message_count}"
```

(If a higher threshold is really required, update the documentation and fixture comments accordingly.)

File: tests/integration/test_datarouter_dlt.py
Line: 65
Chain of Thought
  • Validation Decision: accepted
  • Confidence: 0.97
  • Likelihood Score: 4/5
  • Likelihood Reason: With messages emitted every ~5s and a 10s capture window, capturing exactly 1 message is a realistic scenario depending on timing. The assertion &gt; 1 would cause the test to fail in that case, making it flaky. The documentation explicitly states "at least 1 message" but the code requires more than 1.
  • Suggested Action: keep
  • Tool Calls Used: 2

Reasoning

The comment accurately identifies a real inconsistency between the documentation and the code:

  1. README.md line 69: "The test expects at least 1 message during the capture window for reliable test results" — clearly documents &gt;= 1.
  2. Test line 65: assert message_count &gt; 1 — requires strictly more than 1, i.e., at least 2 messages.
  3. Test line 23-24: Comment says messages sent every ~5s, capture window is 10s.

With a 5-second emission interval and a 10-second capture window, the test could realistically capture 1 or 2 messages depending on when the capture starts relative to the emission cycle. If capture starts just after an emission, the next one arrives at ~5s and the one after at ~10s — but the capture may end before the second one. Getting exactly 1 message is plausible and would cause a test failure despite the system working correctly.

The comment is:

  • Accurate: The mismatch between documentation and assertion is real and verified.
  • Actionable: It provides a concrete fix (change &gt; 1 to &gt;= 1) with a clear diff.
  • Introduced by this PR: This is entirely new code added in this PR.
  • Addresses flakiness: A legitimate concern for integration tests with timing dependencies.

Evidence Summary

1. File Snapshot (HEAD)

  • File: tests/integration/conftest.py
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

2. File Snapshot (HEAD)

  • File: tests/integration/README.md
1: # *******************************************************************************
2: # Copyright (c) 2025 Contributors to the Eclipse Foundation
3: #
4: # See the NOTICE file(s) distributed with this work for additional
5: # information regarding copyright ownership.
6: #
7: # This program and the accompanying materials are made available under the
8: # terms of the Apache License Version 2.0 which is available at
9: # https://www.apache.org/licenses/LICENSE-2.0
10: #
...

Fix in Cursor