Adding standalone compression workflow #114
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… are not equivalent in object storage!)
Contributor
Author
|
Branch is now ready for review! This is a workflow that we don't plan on supporting indefinitely, but giving it a place to live for future documentation. |
gbeane
reviewed
Nov 10, 2025
| } | ||
| PUT_DATA_TO_CLOUDIAN(sync_names, params.cloudian_prefix, params.cloudian_config) | ||
|
|
||
| } No newline at end of file |
Contributor
There was a problem hiding this comment.
file does not end in a newline
gbeane
reviewed
Nov 10, 2025
|
|
||
| # There are batch files in the folder containing data to process. | ||
| # Batch files are titled batch_#.txt | ||
| BATCH_START=1 |
Contributor
There was a problem hiding this comment.
BATCH_START doesn't appear to be used elsewhere in the file
gbeane
approved these changes
Nov 10, 2025
Contributor
gbeane
left a comment
gbeane
left a comment
There was a problem hiding this comment.
I don't know much about Nextflow but LGTM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the effort related to compressing data from one cloud resource and placing it on another.
Rough workflow follows 3 steps: