Skip to content

Run AMR++ on Workbench#14

Draft
samhornstein wants to merge 6 commits intosamh/amrfrom
samh/amr-dev
Draft

Run AMR++ on Workbench#14
samhornstein wants to merge 6 commits intosamh/amrfrom
samh/amr-dev

Conversation

@samhornstein
Copy link
Contributor

@samhornstein samhornstein commented Jan 9, 2026

Summary

Adapts AMR++ bioinformatics pipeline to run on Verily Workbench with Nextflow orchestration and Google Batch compute.

Key Changes

  • Google Batch resource scaling: Explicit machine type configuration to prevent crashes when processes request more resources than default
  • Workbench integration: GCS storage, Artifact Registry containers, private networking
  • Infrastructure scripts: Setup, data upload, container build, and run automation
  • Dynamic CPU allocation: All processes use ${task.cpus} instead of hardcoded threads
  • Documentation: Setup guides and resource scaling best practices

Environments

  • Workbench (primary), GCP (debugging), Local (testing)

Technical Details

Fixes Google Batch limitation where machine types don't auto-scale based on CPU/memory requests. Config precedence: withName > withLabel > default.

Requires Nextflow v24, Docker, wb CLI, gcloud.

@samhornstein samhornstein requested review from PeterSu92 and removed request for PeterSu92 January 9, 2026 18:15
@samhornstein samhornstein self-assigned this Mar 3, 2026
@samhornstein samhornstein marked this pull request as draft March 3, 2026 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant