Request for evaluation scripts used to compute avgRTE, avgRE, and RMSE APE metrics

[ajaystar8]

Hi,

Thank you for the excellent work on cuVSLAM!

I am trying to reproduce the benchmark results reported in Table 2 of the technical report, specifically the avgRTE (%), avgRE (deg), and RMSE APE (m) metrics on the KITTI odometry dataset.

I have been attempting to replicate these metrics using evo's main_rpe and main_ape APIs, but I am having difficulty matching the reported numbers for baseline methods like DPVO for the KITTI dataset. I suspect the discrepancy may be due to differences in RPE delta/unit settings or how the percentage conversion is handled for RTE.

Would it be possible to share the evaluation scripts used to compute these metrics? This would be very helpful for ensuring fair and consistent comparisons with cuVSLAM.

Thank you for your time!

[zwdoescode]

Hi! I got more information from our engineer. Our engineer did that benchmark a year ago and they will check if they can find the script used back then.

Regarding the DPVO, they think the main problem is not in the EVO library (the comparison parameters were common between the methods - the predicted trajectory was simply placed next to the GT in the form of a .txt file), but in the DPVO checkpoint used, since for TUM-RGBD, the DPVO results are very good, as well as angular error for all datasets including KITTI (looks like the model output was correct, but the estimated translation value is rapidly deteriorating)