Add accelerator usage field in trace record #6677
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
pditommaso
requested changes
Jan 2, 2026
| } | ||
| // If Fusion is enabled read parse the use of accelerator form .command.log | ||
| if( Global.session && FusionHelper.isFusionEnabled(Global.session as Session) ) | ||
| record.parseFusionAccelerator(task.workDir?.resolve(TaskRun.CMD_LOG)) |
Member
There was a problem hiding this comment.
Doesn't sound great parsing a log file to determine this capability
Contributor
Author
There was a problem hiding this comment.
Agree, but this is how Fusion currently provides the GPU metric. This is why I keep it as draft.
Member
|
@jorgee can you make separate PRs for the accelerator request vs GPU metrics? I mainly wanted to get the accelerator request added because it should be trivial |
Contributor
Author
|
By accelerator request, do you mean the |
Member
|
Yes. The request is separate from the usage |
jorgee
changed the title
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Contributor
Author
|
I have created #6703 to include the accelerator request in the trace record and pass it to Platform. I have also modified this PR to set the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for tracking and reporting the use of hardware accelerators (such as GPUs) in Fusion-enabled Nextflow tasks. The changes add a new transient
acceleratorfield to theTraceRecord, ensure this field is populated by parsing Fusion logs, and propagate this information to Tower reporting. Comprehensive tests are included to validate the new functionality.Fusion accelerator tracking and reporting:
acceleratorfield to theTraceRecordclass, along with getter, setter, and a method (parseFusionAccelerator) to parse the accelerator usage from Fusion log files. The parsing logic checks for theFUSION_GPU_USED=true|falseline in the log. ([[1]](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-9c0cf79e841529211bb8765c37a7c63a11841f9b9294bd0e36982d21b10a9793R125),[[2]](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-9c0cf79e841529211bb8765c37a7c63a11841f9b9294bd0e36982d21b10a9793R631-R674))TaskHandlerto invokeparseFusionAcceleratoron the.command.logfile when Fusion is enabled, ensuring theacceleratorfield is set during task record creation. ([modules/nextflow/src/main/groovy/nextflow/processor/TaskHandler.groovyR256-R258](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-cc608a283b4ad19f97e1926e7b4e60347d59fa544285e1200893893d990e13a3R256-R258))TowerClientto include theacceleratorvalue when creating the task map for reporting to Tower. ([plugins/nf-tower/src/main/io/seqera/tower/plugin/TowerClient.groovyR664](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-4e7968503d8e56d6d5f2eff04e734382b6b79fee49dc22668d9a3803c91113e2R664))Testing and validation:
TraceRecordTestto verify correct management, parsing, and (non-)serialization of theacceleratorfield, including various edge cases for log file content. ([modules/nextflow/src/test/groovy/nextflow/trace/TraceRecordTest.groovyR372-R445](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-a0fd6cbe04bc7f51d64dfd2475161fe33dc5a65b163dcee567e29022ef27e18bR372-R445))TowerClientTestto ensure theacceleratorfield is included in the Tower task map. ([plugins/nf-tower/src/test/io/seqera/tower/plugin/TowerClientTest.groovyR563-R587](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-463f0ed6e146304ec6d2d8252af59dc42ba32ef8851ae68c371708e3823157c3R563-R587))Imports and dependencies:
TaskHandler.groovy. ([modules/nextflow/src/main/groovy/nextflow/processor/TaskHandler.groovyR19-R22](https://github.com/nextflow-io/nextflow/pull/6677/files#diff-cc608a283b4ad19f97e1926e7b4e60347d59fa544285e1200893893d990e13a3R19-R22))