Skip to content

[RUST] Added XDS module #2088

Open
vatsalkeshav wants to merge 34 commits intoCCExtractor:masterfrom
vatsalkeshav:rusty-xds-decoder
Open

[RUST] Added XDS module #2088
vatsalkeshav wants to merge 34 commits intoCCExtractor:masterfrom
vatsalkeshav:rusty-xds-decoder

Conversation

@vatsalkeshav
Copy link
Copy Markdown
Contributor

@vatsalkeshav vatsalkeshav commented Feb 3, 2026

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.
  • I have mentioned this change in the changelog.

My familiarity with the project is as follows (check one):

  • I have never used CCExtractor.
  • I have used CCExtractor just a couple of times.
  • I absolutely love CCExtractor, but have not contributed previously.
  • I am an active contributor to CCExtractor.

(This PR Supersedes #1890; rebased on latest master with fixes from review.)

This pull request migrates ccx_decoders_xds.c and ccx_decoders_xds.h to /src/rust/xds rust module.

  • mod.rs : exposes the Rust implementation to the C codebase
  • types.rs : has types and structures for decoding extended data
  • handlers.rs : has handler functions for processing extended data packets

Tested against stream : https://sampleplatform.ccextractor.org/sample/b22260d065ab537899baaf34e78a5184671f4bcb2df0414d05e6345adfd7812f

@vatsalkeshav vatsalkeshav mentioned this pull request Feb 3, 2026
10 tasks
@vatsalkeshav
Copy link
Copy Markdown
Contributor Author

vatsalkeshav commented Feb 4, 2026

Hi @cfsmp3
I’ve rebased on latest master and addressed the feedback from #1890.
Please let me know if any changes are needed.
Thanks

@vatsalkeshav
Copy link
Copy Markdown
Contributor Author

Force-pushed after rebasing onto latest master
No functional changes

@vatsalkeshav vatsalkeshav mentioned this pull request Feb 18, 2026
10 tasks
Copy link
Copy Markdown
Member

@steel-bucket steel-bucket left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test it in a setup which tests for the Console output being same as in C, test across 20-30 samples, make a simple test script which tests both for Console Output and subtitle output. I'll give you one you can salvage something from
https://gist.github.com/steel-bucket/11cfe6fa76318256d974f286b8e6cd30

Comment on lines +7 to +8
// declare rust implementation of process_xds_bytes (function)
extern void ccxr_process_xds_bytes(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +256 to +257
#ifndef DISABLE_RUST
ccxr_process_xds_bytes(ctx, hi, lo); // use the rust implementation
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove DISABLE_RUST, we deprecated that

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +49 to +52
use libc::free;
use libc::malloc;

use crate::xds::types::*;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use libc

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +60 to +63
pub enum XDSError {
Err,
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was introduced to remove clippy warning :

warning: this returns a `Result<_, ()>`
   --> src/xds/handlers.rs:100:1
    |
100 | / pub unsafe fn write_xds_string(
101 | |     sub: &mut cc_subtitle,
102 | |     ctx: &mut CcxDecodersXdsContext,
103 | |     p: String,
104 | |     ts_start_of_xds: i64,
105 | | ) -> Result<(), ()> {
    | |___________________^
    |
    = help: use a custom `Error` type instead
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#result_unit_err
    = note: `#[warn(clippy::result_unit_err)]` on by default

warning: this returns a `Result<_, ()>`
   --> src/xds/handlers.rs:149:1
    |
149 | / pub unsafe fn xdsprint(
150 | |     sub: &mut cc_subtitle,
151 | |     ctx: &mut CcxDecodersXdsContext,
152 | |     message: String,
153 | | ) -> Result<(), ()> {
    | |___________________^
    |
    = help: use a custom `Error` type instead
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#result_unit_err

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Result<(), Err> ?

Copy link
Copy Markdown
Contributor Author

@vatsalkeshav vatsalkeshav Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed c0f44dc

Comment on lines +167 to +168
log::debug!(
"XDS Start: {}.{} Is new: {} | Class: {} ({}), Used buffers: {}",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import debug standalong instead of log::

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +423 to +459
const US_TV_AGE_TEXT: [&str; 8] = [
"None",
"TV-Y (All Children)",
"TV-Y7 (Older Children)",
"TV-G (General Audience)",
"TV-PG (Parental Guidance Suggested)",
"TV-14 (Parents Strongly Cautioned)",
"TV-MA (Mature Audience Only)",
"None",
];

/// MPA rating text
const MPA_RATING_TEXT: [&str; 8] = ["N/A", "G", "PG", "PG-13", "R", "NC-17", "X", "Not Rated"];

/// Canadian English Language Rating
const CANADIAN_ENGLISH_RATING_TEXT: [&str; 8] = [
"Exempt",
"Children",
"Children eight years and older",
"General programming suitable for all audiences",
"Parental Guidance",
"Viewers 14 years and older",
"Adult Programming",
"[undefined]",
];

/// Canadian French Language Rating
const CANADIAN_FRENCH_RATING_TEXT: [&str; 8] = [
"Exempt?es",
"G?n?ral",
"G?n?ral - D?conseill? aux jeunes enfants",
"Cette ?mission peut ne pas convenir aux enfants de moins de 13 ans",
"Cette ?mission ne convient pas aux moins de 16 ans",
"Cette ?mission est r?serv?e aux adultes",
"[invalid]",
"[invalid]",
];
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move above

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +634 to +635
// XDS_TYPE_PIN_START_TIME = 1
1 => {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an enum

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +741 to +743
debug!(
msg_type = DebugMessageFlag::DECODER_XDS;
":{:02}", el_sec
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct type of debug, use this in other impls

Comment on lines +59 to +62
pub static XDS_PROGRAM_TYPES: [&str; 96] = [
"Education",
"Entertainment",
"Movie",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move all constants, enums, and statics in a different file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +381 to +392
impl CType<xds_buffer> for XdsBuffer {
unsafe fn to_ctype(&self) -> xds_buffer {
xds_buffer {
in_use: self.in_use,
xds_class: self.xds_class.map(|c| c.to_c_int()).unwrap_or(-1),
xds_type: self.xds_type.map(|t| t.to_c_int()).unwrap_or(-1),
bytes: self.bytes,
used_bytes: self.used_bytes,
}
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test for all of these

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@vatsalkeshav
Copy link
Copy Markdown
Contributor Author

I've comitted the required changes from the review
@steel-bucket Thanks for the testing script
Will share the results soon

@vatsalkeshav
Copy link
Copy Markdown
Contributor Author

Hi,
@steel-bucket the testing script is really helpful - pushed some fixes.
Please review whenever possible
see results here : https://github.com/vatsalkeshav/ccx-regression-testing

Regression ID Sample Command Runtime (ms) Exit code Result

98 725a49f871... --autoprogram --out=ttxt --latin1 --ucla --xds 674 0 Pass
99 d037c7509e... --autoprogram --out=ttxt --xds --latin1 --ucla 58621 0 Pass
100 d037c7509e... --autoprogram --out=srt --latin1 --ucla 46315 0 Pass
101 e274a73653... --autoprogram --out=smptett --latin1 --ucla 53170 0 Pass
102 e274a73653... --autoprogram --out=ttxt --xds --latin1 --ucla 47493 0 Pass
103 85058ad37e... --autoprogram --out=ttxt --xds --latin1 --ucla 877 0 Fail
104 b22260d065... --autoprogram --out=ttxt --latin1 --ucla --xds 37597 0 Pass
105 b22260d065... --autoprogram --out=srt --latin1 --ucla 33084 0 Fail
106 c813e713a0... --autoprogram --out=ttxt --latin1 --xds --ucla 64918 0 Pass
107 c813e713a0... --autoprogram --out=srt --latin1 --ucla 54805 0 Pass
108 27fab4dbb6... --autoprogram --out=ttxt --latin1 --ucla --xds 66355 0 Pass
110 27fab4dbb6... --autoprogram --out=srt --latin1 --ucla 57529 0 Pass
111 bbd5bb52fc... --autoprogram --out=ttxt --latin1 --ucla --xds 54610 0 Pass
112 bbd5bb52fc... --autoprogram --out=srt --latin1 --ucla 47706 0 Pass
113 b992e0cccb... --autoprogram --out=ttxt --latin1 --ucla --xds 2404 0 Pass
114 d0291cdcf6... --autoprogram --out=ttxt --latin1 --ucla --xds 336 0 Pass
115 7d2730d38e... --autoprogram --out=ttxt --latin1 --ucla 55007 0 Pass
116 7d2730d38e... --autoprogram --out=srt --latin1 --ucla 47725 0 Pass
117 c8dc039a88... --autoprogram --out=ttxt --latin1 --ucla --xds 100098 0 Pass
118 c8dc039a88... --autoprogram --out=srt --latin1 --ucla 96697 0 Pass
119 53339f3455... --autoprogram --out=ttxt --latin1 --ucla --xds 8193 0 Pass
120 53339f3455... --autoprogram --out=srt --latin1 --ucla 6005 0 Pass
121 83b03036a2... --autoprogram --out=ttxt --latin1 --ucla --xds 33551 0 Pass
122 83b03036a2... --autoprogram --out=srt --latin1 --ucla 29904 0 Pass
123 7d3f25c32c... --autoprogram --out=ttxt --latin1 --ucla --xds 33890 0 Pass
124 7d3f25c32c... --autoprogram --out=srt --latin1 --ucla 29817 0 Pass
125 f41d4c29a1... --autoprogram --out=ttxt --latin1 --ucla --xds 1862 0 Pass
126 f41d4c29a1... --autoprogram --out=srt --latin1 --ucla 1427 0 Pass
127 88cd42b89a... --autoprogram --out=ttxt --latin1 --ucla --xds 77404 0 Pass
128 88cd42b89a... --autoprogram --out=srt --latin1 --ucla 76172 0 Pass
129 88cd42b89a... --autoprogram --out=srt --latin1 --output-field 2 --ucla 70642 0 Pass
130 7f41299cc7... --autoprogram --out=ttxt --latin1 --ucla --xds 8563 0 Fail
131 7f41299cc7... --autoprogram --out=srt --latin1 --ucla 6881 0 Pass
132 0069dffd21... --autoprogram --out=ttxt --latin1 --ucla --xds 1537 0 Pass

Comment on lines 28 to 29
libc = "0.2.178"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove libc

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines +106 to +108
pub extern "C" fn ccxr_set_ts_start_of_xds(value: i64) {
TS_START_OF_XDS.store(value, Ordering::SeqCst);
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verify for correctness

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed to sync TS_START_OF_XDS between C and Rust

@steel-bucket
Copy link
Copy Markdown
Member

Other than these and the format failure, PR looks good if there are no regressions.

@vatsalkeshav
Copy link
Copy Markdown
Contributor Author

Other than these and the format failure, PR looks good if there are no regressions.

i;ve pushed the required fixes

@ccextractor-bot
Copy link
Copy Markdown
Collaborator

CCExtractor CI platform finished running the test files on linux. Below is a summary of the test results, when compared to test for commit d56a6be...:
Report Name Tests Passed
Broken 9/13
CEA-708 1/14
DVB 3/7
DVD 3/3
DVR-MS 2/2
General 20/27
Hardsubx 1/1
Hauppage 3/3
MP4 3/3
NoCC 10/10
Options 77/86
Teletext 20/21
WTV 13/13
XDS 31/34

Your PR breaks these cases:

  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 8e8229b88b...
  • ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2...
  • ccextractor --autoprogram --out=ttxt --latin1 132d7df7e9...
  • ccextractor --autoprogram --out=ttxt --latin1 99e5eaafdc...
  • ccextractor --autoprogram --out=srt --latin1 b22260d065...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 7aad20907e...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65...
  • ccextractor --autoprogram --out=ttxt --latin1 01509e4d27...
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b...
  • ccextractor --out=spupng c83f765c66...
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla 85058ad37e...
  • ccextractor --autoprogram --out=srt --latin1 --ucla b22260d065...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7f41299cc7...

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

@ccextractor-bot
Copy link
Copy Markdown
Collaborator

CCExtractor CI platform finished running the test files on windows. Below is a summary of the test results, when compared to test for commit d56a6be...:
Report Name Tests Passed
Broken 9/13
CEA-708 1/14
DVB 4/7
DVD 3/3
DVR-MS 2/2
General 22/27
Hardsubx 1/1
Hauppage 3/3
MP4 3/3
NoCC 10/10
Options 81/86
Teletext 20/21
WTV 13/13
XDS 31/34

Your PR breaks these cases:

  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 8e8229b88b...
  • ccextractor --autoprogram --out=ttxt --latin1 132d7df7e9...
  • ccextractor --autoprogram --out=ttxt --latin1 99e5eaafdc...
  • ccextractor --autoprogram --out=srt --latin1 b22260d065...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 7aad20907e...
  • ccextractor --autoprogram --out=ttxt --latin1 01509e4d27...
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla 85058ad37e...
  • ccextractor --autoprogram --out=srt --latin1 --ucla b22260d065...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7f41299cc7...

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
  • ccextractor --out=spupng c83f765c66..., Last passed: Never
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants