Zenodo DOI:

QuPath Spotiflow extension

This repo adds some support to use Spotiflow within QuPath through a Python virtual environment.

Warning

This extension is developed for QuPath 0.6.x

Citing

Please cite this extension by linking it to this GitHub or to the release you used, and feel free to give us a star ⭐️

As this code is neither novel nor entirely original, there is no linked publication, but you can use the following Zenodo entry:

Code authorship

Author: Rémy Dornier (1)

Contributors: Albert Dominguez Mantes (2), Olivier Burri (1), Nicolas Chiaruttini (1), Romain Guiet (1)

This code heavily mirrors functions and design implemented in QuPath Cellpose.

Affiliations

(1) EPFL BioImaging and Optics Platform (BIOP) (2) EPFL Weigert Lab

If you use this extension, you should cite the following publications

Dominguez Mantes, A., Herrera, A., Khven, I., Schlaeppi, A., Kyriacou, E., Tsissios, G., Skoufa, E., Santangeli, L., Buglakova, E., Berna Durmus, E., Manley, S., Kreshuk, A., Arendt, D., Aztekin, C., Lingner, J., La Manno, G., Weigert, M. Spotiflow: accurate and efficient spot detection for fluorescence microscopy with deep stereographic flow regression. bioRxiv 2024.02.01.578426. https://doi.org/10.1101/2024.02.01.578426

Bankhead, P. et al. QuPath: Open source software for digital pathology image analysis. Scientific Reports (2017). https://doi.org/10.1038/s41598-017-17204-5

Installation

Step 1: Install Spotiflow

Installation from the official Spotiflow documentation

You will need a Python virtual environment with Spotiflow working.
Please follow the instructions to install Spotiflow from the main Spotiflow repository.

Get the path to the Python executable

This extension will then need to know the path (the location in your file system) to (at least) your Spotiflow environment. You will need to enter this information in the QuPath Preferences when configuring the QuPath Spotiflow extension.

Important

Make sure you activate your Spotiflow environment and then check the path!

For example, if you installed spotiflow into a conda environment named spotiflow-biop then you can get the path to the Python executable using the command line or terminal using the following two commands to activate the conda environment and locate the Python binary (the 3rd line is example output).
On Windows:

> mamba activate spotiflow-biop
> where python
F:\conda-envs\spotiflow-biop\python.exe

On macOS or Linux:

> mamba activate spotiflow-biop
> which python
{HOME}/conda/envs/spotiflow-biop/bin/python

where {HOME} will be the location of your home directory, typically /Users/user-name/ on macOS or /home/user-name/ on Linux.

Step 2: Install the QuPath Spotiflow extension

On QuPath 0.6.x

• Install and setup the qupath-biop-catalog first.
• Open the Extension Manager
• Follow steps described in the official documention to browse and install/update the latest version of QuPath Spotiflow extension from the QuPath BIOP catalog.

Note

If your extensions directory is unset, create a new empty folder somewhere on your computer. On QuPath, go under Edit -> Preferences -> Extensions and browse the empty folder you've just created. Extensions will then be added inside this folder.

• You might then need to restart QuPath (but not your computer).

On QuPath 0.5.x

Download the latest qupath-extension-spotiflow-[version].jar file from releases and copy it into your extensions directory.

If your extensions directory is unset, drag & drop qupath-extension-spotiflow-[version].jar onto the main QuPath window. You'll be prompted to select a QuPath user directory. The extension will then be copied to a location inside that directory. You finally need to restart QuPath.

QuPath Extension Spotiflow: First time setup

Go to Edit > Preferences > Spotiflow Complete the fields with the requested information by pasting the path(s) you obtained above. Based on the mamba installation above, this is what it should look like on Windows:

Note

Unless stated otherwise, this extension should work with the latest releases of both Spotiflow. If this is not the case, please open an issue on our GitHub or write a post on the Image.sc forum detailing the problem and the steps necessary to reproduce it.

Running Spotiflow the first time in standalone

Spotiflow needs to download the pretrained models the first time it is run. On some OSes, this does not work from within QuPath due to permission issues.

One trick is to run Spotiflow from the command line once with the model you want to use. The download should work from there, and you can then use it within the QuPath Extension Spotiflow.

spotiflow-predict path/to/your/image

or, to download a specific model, different from the general one

spotiflow-predict path/to/your/image --pretrained-model hybiss

Using the Spotiflow QuPath Extension

Prediction

Running Spotiflow is done via a script and is very similar to the excellent QuPath Cellpose Extension

You can find a template in QuPath in

Extensions > Spotiflow > Spotiflow detection template

Or you can download the Spotiflow_detection_template.groovy script from this repo and open it in the QuPath script editor.

Important

Lines starting with // are commented out and are not used. To enable those parameters, delete the //. To disable a parameter, add // at the start of the line.

Getting all available arguments

All builder options that are implemented are documented in the Javadoc. Examples are provided in the announcement forum post. You can pass additional options to spotiflow by adding .addParameter() before the .build() line.

To get the available additional arguments, call the help from the spotiflow object using spotiflow.helpPredict()

def spotiflow = Spotiflow.builder()...build() 
spotiflow.helpPredict() 

Prediction using custom models

To use a custom model or your own trained model, provide the path to the directory containing the model to the Spotiflow.builder using .setModelDir()

def pathModel = 'C:/spotiflow-custom-models/my-nice-spotiflow-model'
def spotiflow = Spotiflow.builder()
                     .setModelDir(new File(pathModel))  

Prediction using other pre-trained models

To use a pre-trained model, different from the default one (general), provide the name of this model to the Spotiflow.builder using .setPretrainedModelName()

def pretrainedModel = 'hybiss'
def spotiflow = Spotiflow.builder()
                     .setPretrainedModelName(pretrainedModel)  

Training custom models

Requirements: A QuPath project with rectangles of class "Training" and "Validation" inside which the ground truth objects have been painted as annotations with no class.

Important

Training requires dense annotations. This means you cannot just annotate a few objects per Training and Validation rectangle. You must annotate ALL relevant objects within each of those regions!

We typically create a standalone QuPath project for training only. This project will contain the training images along with the ground truth annotations drawn in QuPath. Here are some reasons we do it this way:

1. Separating training and prediction/analysis makes for clean project structures and easier sharing of the different steps of your workflow.
2. In case we need to get more ground truth, we can simply fire up the relevant QuPath project and rerun the training, and then import the newly trained model into any other project that might need it.

Protocol

1. In your QuPath project create at least 2 rectangle annotations.
2. In the Annotations tab, add new classes name "Training" and "Validation" and assign your rectangles to each of them. You do not need an equal number of Training and Validation rectangles.
3. Lock the rectangles (right click > Annotations > Lock).
4. Draw your ground truth annotations within all of the rectangles.
5. Repeat this for as many images/regions of interest as you would like.

Warning

All images with Training or Validation annotations in the project will be used for the training.

Once you have your labeled Training and Validation rectangles with ground truth annotations, make sure you save your project! Then you can run the Spotiflow training template script in Extensions > Spotiflow > Spotiflow training script template

Or you can download Spotiflow_training_template.groovy from this repo and run it from the script editor.

Important

Lines starting with // are commented out and are not used. To enable those parameters, delete the //. To disable a parameter, add // at the start of the line.

The first thing the script will do is create a sub-folder in your project called spotiflow-training containing sub-folders train and val, followed by exporting the image(s) that will be processed by spotiflow. The train folder will contain images of your training rectangles and a csv file including point coordinates. The val folder will contain the Validation data, which is also used by the QC script.

Once the script successfully completes training, you will have a models sub-folder within your Project folder, which will contain your custom model, as well as a QC sub-folder with the output of the QC script.

Once you are happy with your training script, you should save the edited copy to your Project (or another scripts folder) for re-use!

Getting all available training options

All builder options that are implemented are documented in the Javadoc. Examples are provided in the announcement forum post. You can pass additional options to spotiflow by adding .addParameter() before the .build() line.

To get the available additional arguments, call the help from the spotiflow object using spotiflow.helpTrain()

def spotiflow = Spotiflow.builder()...build() 
spotiflow.helpTrain() 

Fine-tuning a pre-trained models

To fine-tune a pre-trained model, provide the name of this model to the Spotiflow.builder using .setModelToFineTune()

def pretrainedModel = 'general'
def spotiflow = Spotiflow.builder()
                     .setModelToFineTune(pretrainedModel)  

Training validation

You can find a run-spotiflow-qc.py python script in the QC folder of this repository.

Basically, when you train using this extension:

1. It will first train your model as expected
2. It will then run your newly trained spotiflow model on your "Validation" images
3. At the end, it will run the run-spotiflow-qc.py python script to output validation metrics.
4. The validation metrics will be saved into a folder called QC in your QuPath Project

Saving training results for publication purposes

In order to be as reproducible and sure of your results as possible, especially when it comes to publishing, these are our current guidelines:

1. Use saveBuilder() which saves a JSON file of your SpotiflowBuilder, which can be reused with SpotiflowBuilder(File builderFile). That way you will not lose the setting your did
2. Save the spotiflow-training and models folders at the end of your training somewhere. This will contain everything that was made during training.
3. Save the training script as well.

Building

You can build the QuPath Spotiflow extension from source with

gradlew clean build

The output will be under build/libs.

• clean removes anything old
• build builds the QuPath extension as a .jar file and adds it to libs