Skip to content

Commit c657853

Browse files
committed
Add -Zannotate-moves for profiler visibility of move/copy operations
This implements a new unstable compiler flag `-Zannotate-moves` that makes move and copy operations visible in profilers by creating synthetic debug information. This is achieved with zero runtime cost by manipulating debug info scopes to make moves/copies appear as calls to `compiler_move<T, SIZE>` and `compiler_copy<T, SIZE>` marker functions in profiling tools. A new `AnnotateMoves` MIR transform pass runs after MIR optimization and modifies source scopes for statements containing `Operand::Move` and `Operand::Copy` to make them appear as if inlined from profiling marker functions in `core::profiling`. Two marker functions (`compiler_move` and `compiler_copy`) are defined in `library/core/src/profiling.rs`. These are never actually called - they exist solely as debug info anchors. The transform creates synthetic `SourceScopeData` with the `inlined` field set to point to the appropriate profiling marker, leveraging the existing inlining infrastructure. Operations are only annotated if the type: - Meets the size threshold (default: 65 bytes, configurable via `-Zannotate-moves=SIZE`) - Has a non-scalar backend representation (scalars use registers, not memcpy) An early issue was that modifying a statement's SourceInfo to add the `compiler_move` scope affected the entire statement, including function calls when the move was a call argument. This made profilers attribute the whole function call to the move operation, greatly exaggerating its cost. The solution stores argument move/copy SourceInfo separately in `Body::call_arg_move_source_info`. During codegen, this SourceInfo is applied only during argument preparation, then reset to the call site location before emitting the call instruction itself. This ensures profilers see the argument copy attributed to `compiler_move` while the function call retains its proper source attribution.
1 parent 4082d6a commit c657853

40 files changed

+1591
-7
lines changed

compiler/rustc_codegen_ssa/src/mir/block.rs

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1151,6 +1151,16 @@ impl<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> FunctionCx<'a, 'tcx, Bx> {
11511151
_ => {}
11521152
}
11531153

1154+
// Look up stored SourceInfo for this argument if it exists (from annotate_moves pass)
1155+
let bb_info = self
1156+
.mir
1157+
.call_arg_move_source_info
1158+
.iter()
1159+
.find(|&&((block, idx), _)| block == helper.bb && idx == i);
1160+
if let Some((_, arg_source_info)) = bb_info {
1161+
self.set_debug_loc(bx, *arg_source_info);
1162+
}
1163+
11541164
self.codegen_argument(
11551165
bx,
11561166
op,

compiler/rustc_middle/src/mir/mod.rs

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -331,6 +331,13 @@ pub struct Body<'tcx> {
331331
#[type_foldable(identity)]
332332
#[type_visitable(ignore)]
333333
pub function_coverage_info: Option<Box<coverage::FunctionCoverageInfo>>,
334+
335+
/// Debug information for argument moves/copies in Call parameters. Stores pairs of
336+
/// ((BasicBlock, argument_index), SourceInfo) for move/copy operations. Only populated when
337+
/// `-Zannotate-moves` is enabled.
338+
#[type_foldable(identity)]
339+
#[type_visitable(ignore)]
340+
pub call_arg_move_source_info: Vec<((BasicBlock, usize), SourceInfo)>,
334341
}
335342

336343
impl<'tcx> Body<'tcx> {
@@ -374,6 +381,7 @@ impl<'tcx> Body<'tcx> {
374381
tainted_by_errors,
375382
coverage_info_hi: None,
376383
function_coverage_info: None,
384+
call_arg_move_source_info: Vec::new(),
377385
};
378386
body.is_polymorphic = body.has_non_region_param();
379387
body
@@ -405,6 +413,7 @@ impl<'tcx> Body<'tcx> {
405413
tainted_by_errors: None,
406414
coverage_info_hi: None,
407415
function_coverage_info: None,
416+
call_arg_move_source_info: Vec::new(),
408417
};
409418
body.is_polymorphic = body.has_non_region_param();
410419
body

compiler/rustc_mir_build/src/builder/custom/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ pub(super) fn build_custom_mir<'tcx>(
6262
pass_count: 0,
6363
coverage_info_hi: None,
6464
function_coverage_info: None,
65+
call_arg_move_source_info: Vec::new(),
6566
};
6667

6768
body.local_decls.push(LocalDecl::new(return_ty, return_ty_span));

0 commit comments

Comments
 (0)