Some tests are unstable

[CharileBrown]

Unstable tests mean they sometimes pass and some other times fail,and some tests may not be run all the time.

1 packetdrill/gtests/net/tcp/blocking_blocking-accept_ipv4. is unstable.
2 packetdrill/gtests/net/tcp/blocking_blocking-write_ipv4-mapped-v6. is unstable.
3 packetdrill/gtests/net/tcp/cwnd_moderation_cwnd-moderation-disorder-no-moderation_ipv4-mapped-v6. is unstable.
4 packetdrill/gtests/net/tcp/shutdown_shutdown-rd-wr-close_ipv6. is unstable.
5 packetdrill/gtests/net/tcp/blocking_blocking-accept_ipv4-mapped-v6. is unstable.
6 packetdrill/gtests/net/tcp/blocking_blocking-connect_ipv6. is unstable.
7 packetdrill/gtests/net/tcp/epoll_epoll_in_edge_ipv6. is unstable.
8 packetdrill/gtests/net/tcp/shutdown_shutdown-rdwr-write-queue-close_ipv4. is unstable.
9 packetdrill/gtests/net/tcp/blocking_blocking-connect_ipv4. is unstable.
10 packetdrill/gtests/net/tcp/tcp_info_tcp-info-sndbuf-limited_ipv4-mapped-v6. is unstable.
11 packetdrill/gtests/net/tcp/timestamping_partial_ipv4-mapped-v6. is unstable.
12 packetdrill/gtests/net/tcp/sack_sack-shift-sacked-7-5-6-8-9-fack_ipv4. is unstable.
13 packetdrill/gtests/net/tcp/ts_recent_reset_tsval_ipv4-mapped-v6. is unstable.
14 packetdrill/gtests/net/tcp/user_timeout_user-timeout-probe_ipv6. is unstable.
15 packetdrill/gtests/net/tcp/timestamping_server_ipv4-mapped-v6. is unstable.
16 packetdrill/gtests/net/tcp/zerocopy_basic_ipv6. is unstable.
17 packetdrill/gtests/net/tcp/zerocopy_closed_ipv6. is unstable.
18 packetdrill/gtests/net/tcp/zerocopy_epoll_oneshot_ipv6. is unstable.
19 packetdrill/gtests/net/tcp/zerocopy_fastopen-server_ipv6. is unstable.
20 packetdrill/gtests/net/tcp/zerocopy_maxfrags_ipv6. is unstable.
21 packetdrill/gtests/net/tcp/zerocopy_small_ipv6. is unstable.
22 packetdrill/gtests/net/tcp/zerocopy_batch_ipv6. is unstable.
23 packetdrill/gtests/net/tcp/zerocopy_epoll_edge_ipv6. is unstable.
24 packetdrill/gtests/net/tcp/zerocopy_fastopen-client_ipv6. is unstable.
25 packetdrill/gtests/net/tcp/zerocopy_epoll_exclusive_ipv6. is unstable.
26 packetdrill/gtests/net/tcp/blocking_blocking-connect_ipv4-mapped-v6. is unstable.
27 packetdrill/gtests/net/tcp/cwnd_moderation_cwnd-moderation-ecn-enter-cwr-no-moderation-700_ipv6. is unstable.
28 packetdrill/gtests/net/tcp/nagle_sockopt_cork_nodelay_ipv6. is unstable.
29 packetdrill/gtests/net/tcp/sendfile_sendfile-simple_ipv6. is unstable.
30 packetdrill/gtests/net/tcp/shutdown_shutdown-rdwr-write-queue-close_ipv4-mapped-v6. is unstable.
31 packetdrill/gtests/net/tcp/blocking_blocking-write_ipv4. is unstable.
32 packetdrill/gtests/net/tcp/shutdown_shutdown-rdwr-close_ipv6. is unstable.
33 packetdrill/gtests/net/tcp/sack_sack-shift-sacked-2-6-8-3-9-nofack_ipv6. is unstable.
34 packetdrill/gtests/net/tcp/shutdown_shutdown-rdwr-write-queue-close_ipv6. is unstable.
35 packetdrill/gtests/net/tcp/syscall_bad_arg_fastopen-invalid-buf-ptr_ipv6. is unstable.
36 packetdrill/gtests/net/tcp/syscall_bad_arg_syscall-invalid-buf-ptr_ipv6. is unstable.
37 packetdrill/gtests/net/tcp/tcp_info_tcp-info-sndbuf-limited_ipv4. is unstable.
38 packetdrill/gtests/net/tcp/timestamping_server_ipv6. is unstable.
39 packetdrill/gtests/net/tcp/ts_recent_reset_tsval_ipv6. is unstable.
40 packetdrill/gtests/net/tcp/shutdown_shutdown-rd-close_ipv6. is unstable.
41 packetdrill/gtests/net/tcp/shutdown_shutdown-wr-close_ipv6. is unstable.
42 packetdrill/gtests/net/tcp/splice_tcp_splice_loop_test_ipv6. sometimes may not run.
43 packetdrill/gtests/net/tcp/timestamping_server_ipv4. is unstable.
44 packetdrill/gtests/net/tcp/timestamping_partial_ipv6. is unstable.
45 packetdrill/gtests/net/tcp/validate_validate-established-no-flags_ipv6. is unstable.
46 packetdrill/gtests/net/tcp/user_timeout_user_timeout_ipv6. is unstable.
47 packetdrill/gtests/net/tcp/zerocopy_client_ipv6. is unstable.
48 packetdrill/gtests/net/tcp/user_timeout_user_timeout_ipv4-mapped-v6. is unstable.
49 packetdrill/gtests/net/tcp/blocking_blocking-read_ipv6. is unstable.
50 packetdrill/gtests/net/tcp/sack_sack-shift-sacked-7-3-4-8-9-fack_ipv6. is unstable.
51 packetdrill/gtests/net/tcp/sack_sack-shift-sacked-7-5-6-8-9-fack_ipv6. is unstable.
52 packetdrill/gtests/net/tcp/inq_server_ipv6. is unstable.
53 packetdrill/gtests/net/tcp/tcp_info_tcp-info-last_data_recv_ipv4-mapped-v6. is unstable.
54 packetdrill/gtests/net/tcp/blocking_blocking-read_ipv4-mapped-v6. is unstable.
55 packetdrill/gtests/net/tcp/md5_md5-only-on-client-ack_ipv6. is unstable.
56 packetdrill/gtests/net/tcp/tcp_info_tcp-info-rwnd-limited_ipv4. is unstable.
57 packetdrill/gtests/net/tcp/ts_recent_invalid_ack_ipv6. is unstable.
58 packetdrill/gtests/net/tcp/tcp_info_tcp-info-last_data_recv_ipv6. is unstable.
59 packetdrill/gtests/net/tcp/tcp_info_tcp-info-sndbuf-limited_ipv6. is unstable.
60 packetdrill/gtests/net/tcp/timestamping_partial_ipv4. is unstable.
61 packetdrill/gtests/net/tcp/mss_mss-getsockopt-tcp_maxseg-server_ipv6. is unstable.
62 packetdrill/gtests/net/tcp/syscall_bad_arg_sendmsg-empty-iov_ipv6. is unstable.
63 packetdrill/gtests/net/tcp/ts_recent_fin_tsval_ipv6. is unstable.
64 packetdrill/gtests/net/tcp/tcp_info_tcp-info-last_data_recv_ipv4. is unstable.
65 packetdrill/gtests/net/tcp/user_timeout_user_timeout_ipv4. is unstable.
66 packetdrill/gtests/net/tcp/blocking_blocking-read_ipv4. is unstable.
67 packetdrill/gtests/net/tcp/sack_sack-shift-sacked-7-3-4-8-9-fack_ipv4-mapped-v6. is unstable.
68 packetdrill/gtests/net/tcp/ts_recent_fin_tsval_ipv4-mapped-v6. is unstable.
69 packetdrill/gtests/net/tcp/nagle_sendmsg_msg_more_ipv4-mapped-v6. is unstable.
70 packetdrill/gtests/net/tcp/blocking_blocking-write_ipv6. is unstable.
71 packetdrill/gtests/net/tcp/close_close-remote-fin-then-close_ipv4-mapped-v6. is unstable.
72 packetdrill/gtests/net/tcp/blocking_blocking-accept_ipv6. is unstable.
73 packetdrill/gtests/net/tcp/gro_gro-mss-option_ipv6. is unstable.
74 packetdrill/gtests/net/tcp/user_timeout_user-timeout-probe_ipv4-mapped-v6. is unstable.
75 packetdrill/gtests/net/tcp/user_timeout_user-timeout-probe_ipv4. is unstable.
76 packetdrill/gtests/net/tcp/tcp_info_tcp-info-rwnd-limited_ipv6. is unstable.

[ligurio]

@CharileBrown, could you provide details about your test environment (operating system version, applied network settings etc) and packetdrill version?

[ligurio]

See also a note in 51a1a06:

Note that some tests may be flaky due to timing constraints. One way of addressing that is with --tolerance_usecs. We may refine tests after initial commit.

[nealcardwell]

As noted above, more details (kernel version, test output for failed tests) would be helpful.

Could you please use the run_all.py script in the packetdrill distribution, and report the output? You can run it with something like:

cd ./packetdrill/gtests/net/
./packetdrill/run_all.py -S -l -v tcp

This will run all tests in a serialized fashion, and log the error messages for failed tests.

For documentation on flags see:
./packetdrill/run_all.py --help

Thanks!