Drop built-in atomicity support in favour of OverlayFS#701
Conversation
🤖 Augment PR SummarySummary: This PR removes One’s built-in “atomic commit” mechanism for index output generation, shifting responsibility for atomicity to the execution environment (e.g., OverlayFS). Changes:
Technical Notes: After this change, a failed indexing run can leave the output directory partially updated unless an external atomicity layer (like OverlayFS) is used. 🤖 Was this summary useful? React with 👍 or 👎 |
src/index/index.cc
Outdated
| ///////////////////////////////////////////////////////////////////////////// | ||
|
|
||
| sourcemeta::one::Output output{staging_path}; | ||
| sourcemeta::one::Output output{final_output_path}; |
There was a problem hiding this comment.
Now that Output targets final_output_path directly, any failure after this point can leave the real output directory partially updated/created (where previously the atomic swap avoided touching it). If non-OverlayFS runs are still supported, it may be worth documenting/warning about this behavioral change to avoid surprising data loss.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
src/index/index.cc
Outdated
| std::ofstream sentinel_stream(staging_sentinel); | ||
| sentinel_stream.close(); | ||
|
|
||
| PROFILE_END(profiling, "Commit"); |
There was a problem hiding this comment.
PROFILE_END(profiling, "Commit") now appears to measure an empty stage (the commit/swap logic was removed), which can make --time output misleading. Consider removing or renaming this profiling label so timings reflect real work.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
There was a problem hiding this comment.
Benchmark Index (community)
Details
| Benchmark suite | Current: 6144844 | Previous: 2cbd5ce | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
41 ms |
44 ms |
0.93 |
Add one schema (100 existing) |
452 ms |
1000 ms |
0.45 |
Add one schema (1000 existing) |
4566 ms |
9431 ms |
0.48 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Benchmark Index (enterprise)
Details
| Benchmark suite | Current: 6144844 | Previous: 2cbd5ce | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
45 ms |
53 ms |
0.85 |
Add one schema (100 existing) |
484 ms |
1110 ms |
0.44 |
Add one schema (1000 existing) |
4576 ms |
10658 ms |
0.43 |
This comment was automatically generated by workflow using github-action-benchmark.
Signed-off-by: Juan Cruz Viotti jv@jviotti.com