Introduction to Pipeline Development with Nextflow

Iowa State University
Genome Informatics Facility, Office of Biotechnology October 21st, 2025 (1pm-5pm)

---

Welcome!

Welcome to the Nextflow workshop! This hands-on tutorial will teach you how to build scalable, reproducible computational pipelines using Nextflow - a powerful workflow management system used across bioinformatics, data science, and computational research.

What You'll Learn

By the end of this workshop, you'll be able to:

• Write Nextflow pipelines from scratch
• Process multiple files in parallel automatically
• Integrate any command-line tool into workflows
• Make pipelines configurable and user-friendly
• Debug and troubleshoot pipeline issues
• Build production-ready analysis workflows

Who This Workshop Is For

This workshop is designed for researchers and students from any computational field:

• Bioinformatics and genomics
• Data science and machine learning
• Image processing and computer vision
• Climate modeling and simulations
• Text analysis and NLP
• Any domain requiring batch data processing

No prior Nextflow experience required! Basic command-line knowledge is helpful.

---

Workshop Structure

The workshop is divided into two main tutorials:

Part 1: Foundational Concepts (hello.md)

Learn Nextflow basics through simple, progressive examples:

• Script 01: Hello World - Your first Nextflow process
• Script 02: Writing to Files - Understanding file outputs and work directory
• Script 03: Publishing Outputs - Making results accessible with publishDir
• Script 04: Process Inputs - Passing data to processes
• Script 05: Parameters - Making pipelines configurable

Prerequisites: Basic command-line knowledge

Part 2: Implementation Tutorial (implementation.md)

Apply your knowledge to build a real bioinformatics pipeline:

• Script 06: FastQC Quality Control - Run FastQC on multiple files in parallel
• Script 07: Paired-End Trimming - Handle paired-end data with Fastp
• Script 08: Parallel Workflows - Run multiple processes simultaneously
• Script 09: Collecting Results - Aggregate files and run custom scripts
• Script 10: Complete Pipeline - Chain processes with channel transformations

Prerequisites: Complete Part 1 (Scripts 01-05)

---

Quick Start

1. Access the HPC Cluster

Use Nova OnDemand to get a compute node:

🔗 Nova OnDemand

Login with your ISU credentials and launch VS Code Server.

2. Clone the Repository

cd /work/short_term/<ISU_NetID>
git clone https://github.com/ISUgenomics/isu-nextflow-workshop.git
cd isu-nextflow-workshop

3. Copy Tutorial Data

cp -a /work/short_term/workshop2_bash/01_data .

4. Load Nextflow

module load nextflow
module list

5. Start the Tutorial

Begin with the foundational concepts:

cat hello.md
# Or open in your editor
code hello.md

---

Workshop Materials

Tutorials

• hello.md - Foundational concepts (Scripts 01-05)
• implementation.md - Real-world implementation (Scripts 06-10)

Pipeline Scripts

All pipeline scripts are in the pipelines/ directory:

pipelines/
├── 01_hello_screen.nf          # Script 01: Hello World
├── 02_hello_redirect.nf         # Script 02: File outputs
├── 03_hello_publishdir.nf       # Script 03: Publishing
├── 04_hello_input.nf            # Script 04: Process inputs
├── 05_hello_default.nf          # Script 05: Parameters
├── 06_implementation_fastqc.nf  # Script 06: FastQC
├── 07_implementation_fastp.nf   # Script 07: Fastp
├── 08_implementation_fastqc_fastp.nf  # Script 08: Parallel
├── 09_implementation_readLenDist.nf   # Script 09: Collecting
└── 10_implementation_complete.nf      # Script 10: Complete pipeline

---

Instructors

Viswanathan Satheesh
Genome Informatics Facility
Office of Biotechnology
Iowa State University

Rick Masonbrink
Genome Informatics Facility
Office of Biotechnology
Iowa State University

Sharu Paul Sharma
Genome Informatics Facility
Office of Biotechnology
Iowa State University

Contact

Email: gifhelp@iastate.edu

For questions about:

• Workshop content and materials
• Nextflow pipelines and troubleshooting
• Follow-up

---

Additional Resources

Official Documentation

• Nextflow Documentation
• Nextflow Training
• Nextflow Patterns
• nf-core Pipelines - Community-curated pipelines

Getting Help

• Nextflow Slack - Active community
• Nextflow GitHub - Issues and discussions
• nf-core Slack - Pipeline-specific help

ISU Resources

• Nova HPC Documentation
• Genome Informatics Facility
• GIF Help

---

Learning Path

graph LR
    A[Start Here] --> B[hello.md<br/>Scripts 01-05<br/>Foundational Concepts]
    B --> C[implementation.md<br/>Scripts 06-10<br/>Real Pipeline]
    C --> D[Build Your Own<br/>Pipeline!]
    
    style A fill:#e1f5ff,stroke:#01579b,stroke-width:2px,color:#000
    style B fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
    style C fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
    style D fill:#f8bbd0,stroke:#c2185b,stroke-width:2px,color:#000

Loading

---