UDP Multicast DLT Tests Failing Locally - Docker/Network Configuration Issue

[clanghans]

Context

The DLT UDP multicast tests are failing locally, but they pass in the CI pipeline and on colleagues' machines.

Failing tests:

• test_dlt_multicast_udp
• test_dlt_window_stdout
• test_dlt_window_with_filter
• test_dlt_window_with_record

The TCP-based test (test_dlt_direct_tcp) passes without issues.

Environment:

• Tests run Docker containers with the default bridge network
• Issue is isolated to the local machine only
• CI pipeline and colleagues' machines run the tests successfully

Questions about Your Docker/Network Setup

Docker daemon configuration:

1. Do you have any custom Docker daemon configurations (/etc/docker/daemon.json) that might affect network behavior?
2. Are there any custom Docker networks or network drivers configured on your machine?
3. Have you modified Docker's default networking settings?

For Consistency

1. Should we update the tests to document the network requirements for running locally?
2. Would it be helpful to add a setup guide or troubleshooting section for multicast networking issues?
3. Should we consider alternative Docker network configurations that are more portable across different local setups?

[PiotrKorkus]

I dont remember setting anything extra in docker to make it run.

❯ cat /etc/docker/daemon.json
{
  "features": {
    "containerd-snapshotter": true
  }
}

What is your config that you observe issues?
What is your setup linux/wsl/vm?
What errors do you get?

[clanghans]

I have a native Ubuntu 22-04 running behind corporate proxy.

/etc/docker/daemon.json is empty on my machine.

>cat /etc/default/docker

export http_proxy="http://172.17.0.1:3128/"
export https_proxy="http://172.17.0.1:3128/"
export no_proxy="bosch.com"

and the docker service has env vars set to use the proxy:

[Service]
Environment="HTTP_PROXY=http://127.0.0.1:3128/"
Environment="HTTPS_PROXY=http://127.0.0.1:3128/"

error message is as follows.

=================================== FAILURES ===================================
____________________________ test_dlt_multicast_udp ____________________________

target = <score.itf.plugins.docker.DockerTarget object at 0x7fc4e8b83f20>
dlt_config = {'host_ip': '127.0.0.1', 'target_ip': '127.0.0.1', 'multicast_ips': [], 'dlt_receive_path': '../+_repo_rules+dlt_daemon/dlt-receive'}
caplog = <_pytest.logging.LogCaptureFixture object at 0x7fc4e8b81040>

    def test_dlt_multicast_udp(target, dlt_config, caplog):
        ipaddress = get_container_ip(target)
        gateway = get_docker_network_gateway(target)
        target.execute(f"/usr/bin/dlt-daemon -d")

        with DltReceive(
            protocol=Protocol.UDP,
            host_ip=gateway,
            multicast_ips=["224.0.0.1"],
            print_to_stdout=True,
            logger_name="fixed_dlt_receive",
            binary_path=dlt_config.dlt_receive_path,
        ):
            send_secret_dlt_message(target)

        captured_logs = []
        for record in caplog.records:
            if record.name == "fixed_dlt_receive":
                if "This is a secret message" in record.getMessage():
                    break
        else:
>           pytest.fail("Expected DLT message was not received")
E           Failed: Expected DLT message was not received

test/test_dlt.py:110: Failed
____________________________ test_dlt_window_stdout ____________________________

target = <score.itf.plugins.docker.DockerTarget object at 0x7fc4e8eeae40>
dlt_config = {'host_ip': '127.0.0.1', 'target_ip': '127.0.0.1', 'multicast_ips': [], 'dlt_receive_path': '../+_repo_rules+dlt_daemon/dlt-receive'}

    def test_dlt_window_stdout(target, dlt_config):
        ipaddress = get_container_ip(target)
        gateway = get_docker_network_gateway(target)
        target.execute(f"/usr/bin/dlt-daemon -d")

        with DltWindow(
            protocol=Protocol.UDP,
            host_ip=gateway,
            multicast_ips=["224.0.0.1"],
            print_to_stdout=True,
            binary_path=dlt_config.dlt_receive_path,
        ) as window:
            send_secret_dlt_message(target)
>           assert 0 != len(window.get_captured_logs())
E           assert 0 != 0
E            +  where 0 = len([])
E            +    where [] = get_captured_logs()
E            +      where get_captured_logs = <score.itf.plugins.dlt.dlt_window.DltWindow object at 0x7fc4e9dcb080>.get_captured_logs

test/test_dlt.py:143: AssertionError
_________________________ test_dlt_window_with_filter __________________________

target = <score.itf.plugins.docker.DockerTarget object at 0x7fc4e8b7f710>
dlt_config = {'host_ip': '127.0.0.1', 'target_ip': '127.0.0.1', 'multicast_ips': [], 'dlt_receive_path': '../+_repo_rules+dlt_daemon/dlt-receive'}

    def test_dlt_window_with_filter(target, dlt_config):
        ipaddress = get_container_ip(target)
        gateway = get_docker_network_gateway(target)
        target.execute(f"/usr/bin/dlt-daemon -d")

        with DltWindow(
            protocol=Protocol.UDP,
            host_ip=gateway,
            multicast_ips=["224.0.0.1"],
            print_to_stdout=True,
            dlt_filter="SINA SINC",
            binary_path=dlt_config.dlt_receive_path,
        ) as window:
            send_secret_dlt_message(target)
>           assert 0 != len(window.get_captured_logs())
E           assert 0 != 0
E            +  where 0 = len([])
E            +    where [] = get_captured_logs()
E            +      where get_captured_logs = <score.itf.plugins.dlt.dlt_window.DltWindow object at 0x7fc4e8bd5b80>.get_captured_logs

test/test_dlt.py:162: AssertionError
_________________________ test_dlt_window_with_record __________________________

target = <score.itf.plugins.docker.DockerTarget object at 0x7fc4e8eea480>
dlt_config = {'host_ip': '127.0.0.1', 'target_ip': '127.0.0.1', 'multicast_ips': [], 'dlt_receive_path': '../+_repo_rules+dlt_daemon/dlt-receive'}

    def test_dlt_window_with_record(target, dlt_config):
        ipaddress = get_container_ip(target)
        gateway = get_docker_network_gateway(target)
        target.execute(f"/usr/bin/dlt-daemon -d")

        with DltWindow(
            protocol=Protocol.UDP,
            host_ip=gateway,
            multicast_ips=["224.0.0.1"],
            print_to_stdout=False,
            binary_path=dlt_config.dlt_receive_path,
        ) as window:
            send_secret_dlt_message(target)
            record = window.record()

>           for frame in record.find():
                         ^^^^^^^^^^^^^

test/test_dlt.py:182:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
score/itf/plugins/dlt/dlt_window.py:193: in find
    for msg in self._dlt_content:
               ^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <DLTFile object filename=b'/tmp/tmpt09ejk7j' with 0 messages>

    def __iter__(self):  # pylint: disable=too-many-branches
        """Iterate over messages in the file"""
        logger.debug("Starting File Read")
        logger.debug(
            "File Position: %d File Counter: %d File Name: %s", self.file_position, self.counter, self.filename
        )
        cached_mtime = 0
        cached_file_pos = 0
        corruption_check_try = True

        self._open_file()

        found_data = False
        while not self._is_stop_reading_set() or corruption_check_try:  # pylint: disable=too-many-nested-blocks
            os_stat = os.stat(self.filename)
            mtime = os_stat.st_mtime

            if mtime != cached_mtime and os_stat.st_size or corruption_check_try:
                cached_mtime = mtime
                corruption_check_try = False

                while not self._is_stop_reading_set() and (
                    dltlib.dlt_file_read(ctypes.byref(self), self.verbose) >= DLT_RETURN_OK
                ):
                    found_data = True
                    if (
                        self.filter
                        and dltlib.dlt_message_filter_check(ctypes.byref(self.msg), self.filter, 0) != DLT_RETURN_TRUE
                    ):
                        continue

                    index = self.position
                    msg = self[index]
                    if not index % 100000:
                        self._log_message_progress()
                    yield msg

                if cached_file_pos != self.file_position:
                    # We were able to read messages, don't do a corrupt message check yet.
                    corruption_check_try = True
                    cached_file_pos = self.file_position
                else:
                    next_header_position = self._find_next_header()
                    if next_header_position:
                        if self.file_position == next_header_position:
                            if not self.live_run:
                                logger.warning("Incomplete message while parsing DLT file at %s", self.file_position)
                                break
                        else:
                            logger.warning("Found a corrupt message at %s, skipping it", self.file_position)
                            self.file_position = next_header_position
                            self.corrupt_msg_count += 1
                            corruption_check_try = True
                    # Wait for further messages to determine if corrupt, else just end of file
                    else:
                        if not self.live_run:
                            logger.info("End of file reached at %s", self.file_position)
                            break

            time.sleep(0.1)

        if not found_data:
>           raise IOError(DLT_EMPTY_FILE_ERROR)
E           OSError: DLT TRACE FILE IS EMPTY

../+_repo_rules+python_dlt/dlt/dlt.py:888: OSError
=========================== short test summary info ============================
FAILED test/test_dlt.py::test_dlt_multicast_udp - Failed: Expected DLT messag...
FAILED test/test_dlt.py::test_dlt_window_stdout - assert 0 != 0
FAILED test/test_dlt.py::test_dlt_window_with_filter - assert 0 != 0
FAILED test/test_dlt.py::test_dlt_window_with_record - OSError: DLT TRACE FIL...
========================= 4 failed, 4 passed in 13.08s =========================
================================================================================
INFO: Found 1 test target...
Target //test:test_dlt up-to-date:
  bazel-bin/test/test_dlt
INFO: Elapsed time: 14.102s, Critical Path: 13.92s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed, 1 test FAILED, 2 total actions
//test:test_dlt                                                          FAILED in 13.8s
  /home/lcn2abt/.cache/bazel/_bazel_lcn2abt/fd8508121e91b4cb4af79b5fbd1fdc7b/execroot/_main/bazel-out/k8-fastbuild/testlogs/test/test_dlt/test.log

Executed 1 out of 1 test: 1 fails locally.