Bounding box / skeleton lag during camera movement causes severe mouse oscillation

[LostSunset]

Description

When the mouse (or aimbot) moves, the game camera rotates. However, the bounding boxes and skeleton lines drawn on the game overlay use coordinates from the capture time (T0). By the time they are rendered (T2), the camera has already shifted, causing the boxes and skeleton to visibly lag behind the target.

This lag is not just a visual issue — it also affects aiming accuracy. Since the aimbot calculates the target position based on detection coordinates that are already outdated by render time, the aim point oscillates around the actual target rather than converging on it. This is especially noticeable during fast mouse movement or when tracking moving targets.

When stationary, there is no issue because the camera is not moving.

Steps to Reproduce

1. Enable game overlay (bounding boxes + skeleton)
2. In-game, move the mouse rapidly left/right while a target is detected
3. Observe that the bounding box and skeleton visibly trail behind the target
4. Notice the aimbot oscillates / spirals around the target instead of converging smoothly

Expected Behavior

The bounding boxes and skeleton should stay aligned with the target even during camera movement, and the aiming should converge without oscillation.

Possible Solutions?

Are there any recommended approaches to compensate for this capture-to-render delay? Some ideas I've considered:

1. Mouse delta tracking: Track cumulative mouse movement between capture and render, then offset the overlay coordinates accordingly
2. Velocity-based extrapolation: Estimate target movement speed and extrapolate positions forward
3. Reducing pipeline latency: Minimize the time between capture → inference → render

Would appreciate any guidance on the best approach for this project's architecture.

Environment

• Windows 11
• CUDA build with TensorRT
• Game overlay enabled with boxes + skeleton