Skip to content

Conversation

@varshith15
Copy link
Member

@varshith15 varshith15 commented Nov 5, 2025

  • Seperates out raft trt compilation, previously it was part of runtime
  • Will add this raft model compilation to dl_checkpoints in ai-runner
  • fixed the sdxl controlnet compilation bug, was missing a couple of shape fetching methods

Note

Moves RAFT TensorRT compilation to a separate tool, updates TemporalNet TRT to use prebuilt engines and output [prev_input, flow_RGB], and fixes SDXL ControlNet TensorRT dynamic axes/input profiles; also forwards preprocessor params.

  • TemporalNet TensorRT Preprocessor (preprocessing/processors/temporal_net_tensorrt.py)
    • Requires prebuilt RAFT TensorRT engine via engine_path; removes runtime ONNX/TRT build.
    • Adds dynamic buffer allocation and shape reallocation in TensorRTEngine.
    • Changes output to [prev_input_RGB, flow_RGB(prev→curr)] with configurable flow_strength; first frame uses zeros for flow.
    • Tracks prev_input, improves reset/caching, and clarifies error messages.
  • New Tool (tools/compile_raft_tensorrt.py)
    • Adds CLI to export RAFT to ONNX and build a TensorRT engine with dynamic H/W profiles.
  • ControlNet SDXL TensorRT (acceleration/tensorrt/models/controlnet_models.py)
    • Adds SDXL-specific get_dynamic_axes and extends get_input_profile for conditioning_scale, text_embeds, and time_ids.
  • Preprocessor plumbing
    • get_preprocessor now accepts params and forwards to constructors; ControlNetModule.add_controlnet passes preprocessor_params.

Written by Cursor Bugbot for commit fca2666. Configure here.

@varshith15 varshith15 requested a review from victorges November 5, 2025 10:39
@varshith15 varshith15 changed the title fix: temporalnet raft trt script fix: temporalnet preprocessing Nov 7, 2025
@varshith15
Copy link
Member Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Comment @cursor review or bugbot run to trigger another review on this PR

Copy link
Collaborator

@ryanontheinside ryanontheinside left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Is this PR no longer needed? #129

@varshith15
Copy link
Member Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no bugs!


Comment @cursor review or bugbot run to trigger another review on this PR

@varshith15 varshith15 merged commit bd6aa2c into main Nov 13, 2025
1 check passed
@varshith15 varshith15 deleted the vb/temporalnet/raft_trt_script branch November 13, 2025 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants