Version 3.6.8

Major:

• Move to Pytorch 2.9.1
• Add TEST.METRICS_IN_CPU option to calculate test metrics in CPU and make it default. Useful for large datasets such as MitoEM

Minor:

• Update notebooks with version 3.6.7
• Update some aux scripts
• Add TEST.METRICS_IN_CPU option to calculate test metrics in CPU and make it default. Useful for large datasets such as MitoEM
• Improve error message when an image couldn't be loaded
• Add TEST.SAVE_MODEL_RAW_OUTPUT option to now save model raw outputs (related to #145)
• Add TEST.POST_PROCESSING.MEASURE_PROPERTIES.EXTRA_PROPS to choose extra properties to be calculated for the predicted instances (related to #144)
• Add SpineDL paper scripts and templates
• Simplify HRNet configuration and delete hrnet2x20
• Add ConvNeXtBlocks to be usable in HRNet
• Add different possible heads to HRNet
• Test checkpoint load process and reduce test14 experiment epochs, as it was taking 1h to complete
• Remove DATA.EXTRACT_RANDOM_PATCH, DATA.REPLICATE, DATA.PROBABILITY_MAP, DATA.W_FOREGROUND, DATA.W_BACKGROUND
• Update model references and improve BMZ documentation created
• Change sigmoid to sofmax when classes are more than 2 for semantic seg
• Add option to some models to add activations layers at the end, so no post-processing can be set during BMZ model creation, leading to more stable and reproducible results
• More robust checkpoint capturation during model checkpoint load

Bugs fixed:

• Change conda package action to be done after PyPI's action is done
• Add minor fix to ensure unique_labels_fast functionality when labels are not integers
• Fix edge case when creating the BMZ cover
• Update BMZ model creation to support models that do not output an image with the same shape as the input as HRNet
• Add all the variables used outside HRNet definition inside so it can be exported properly to BMZ
• Make all the dependencies found by extract_model function be in a consistent order so all the dependencies are correctly found, i.e. aux functions/classes first
• Resolve PATHS.CHECKPOINT_FILE variable bug (related to suggestions to overcome over segmentation)
• Do not force class IoU calculation to be uint16 but uint8 in instance segmentation
• Don't change patch size defined in the model yaml if only the channel axis is different
• Solve critical bug in BMZ models, as they were not changed to .eval() when doing inference
• Make metric calculation in denoising more robust

Full Changelog: v3.6.7...v3.6.8

Version 3.6.7

Minor:

• Update all notebooks to the new version and simplify BMZ cell
• Minor change in fill_holes_in_seg_masks.py
• Speed up the label creation process and display ETA bar when there are more than 200 instances in the image
• Improve error message if something happened extracting the input image shape from model RDF
• Substitute affable-shark with other models as it is currenlty crashing and it is normally the one used to test new things so could not be stable
• Enhance check_masks() error messages
• Clarify when LOSS.CLASS_REBALANCE is used
• Make the YAML loading step more robust
• Add foreground_probs_to_instances.py aux script

Bugs fixed:

• Try to solve conda-forge-bump issue
• Fix halo values for BMZ packages using 3D models
• Improve B and F channels when they are used alone and with weights (We)
• Rename Dc channel type center to centroid as other channels and fix a minor bug setting its default value
• Fix bug with the inference of BMZ models as the activations were not applied if the model were not defining them explicitly
• Fix errors in LOSS.CLASS_WEIGHTS
• Fix bug using hrnet48 and include it in run_checks.py

Full Changelog: v3.6.6...v3.6.7

Version 3.6.6

Major:

• Change the augmentations to work with pure 3D images and completely remove z and c channel merge process (related to #139)
• Update how the instance channels are handled and extend the support to lots of more channels and combinations:
  • Renamed old D channel into Db (distance to boundary)
  • Renamed old B channel into F
  • Add T for the touching areas
  • Add R for the radial distances, i.e. Stardist
  • Add H+V+Z channels for the distance offset, i.e. Cellpose, Hover-Net and Omnipose.
  • Now D channel is Db for the foreground and background
  • Add Dn for the distance to neighbors
  • Add instance refinement post-processing to do filtering and morphological operations before applying any other more complex post-processing (under TEST.POST_PROCESSING.INSTANCE_REFINEMENT)
  • Add embeddings as instance representation option (EmbedSeg, E channel)
  • Add U-Net-like border weighting for the instace segmentation workflow (channel We)
  • Change the names of the instance segmentationn label directory to create. Now the names are long, as they depend on the channels used and in their options. This ensures creating a new folder if any of the channel options were modified as they may lead to different representations
  • Speed up instance representation mask creation with DDP threads already started
• Speed up generator checks and make the gridmask create the grid with the maximum value per channel
• Update how activations are handled. Now there is one per channel to be more clean
• Resize to the original size if the preprocessing was applied in the test images
• Change test generators to now return also Y filename so we can improve the error messages
• Add safetensors as a dependency
• Update BiaPy and BioImage Model Zoo's connection:
  • Improve BMZ documentation creation based on their Model Card Template
  • Ensure an image with some mask information is captures as BMZ test sample
  • Generate suitable cover images based directly on the GT (if provided) and not on the model prediction
  • Now all the functions used in the notebooks are included in the core
  • Simplify some code by merging check_model_restrictions with check_bmz_model_compatibility. Now check_bmz_model_compatibility does everything
  • Add halo to the BMZ model (related to #141)

Minor:

• Update conda package CI
• Update docstrings to build the doc (@iarganda)
• Binarized version of semantic segmentation's prediction now are thresholded using otsu instead, and not at 0.5 as it used to be
• Improve error message when no files are found during instance segmentation target creation data
• Delete AUGMENTOR.BRIGHTNESS_MODE and AUGMENTOR.CONTRAST_MODE
• Delete the old MODEL.LAST_ACTIVATION
• Remove unused DATA.TRAIN.REPLICATE
• Remove TEST.POST_PROCESSING.FILL_HOLES as it now is inside TEST.POST_PROCESSING.INSTANCE_REFINEMENT
• Remove TEST.POST_PROCESSING.CLEAR_BORDER as it now is inside TEST.POST_PROCESSING.INSTANCE_REFINEMENT
• Provide a better message when raw/gt val data does not match
• PROBLEM.INSTANCE_SEG.DATA_REMOVE_SMALL_OBJ_BEFORE moved to PROBLEM.INSTANCE_SEG.WATERSHED.DATA_REMOVE_SMALL_OBJ_BEFORE
• PROBLEM.INSTANCE_SEG.DATA_REMOVE_BEFORE_MW moved to PROBLEM.INSTANCE_SEG.WATERSHED.DATA_REMOVE_BEFORE_MW
• PROBLEM.INSTANCE_SEG.SEED_MORPH_SEQUENCE moved to PROBLEM.INSTANCE_SEG.WATERSHED.SEED_MORPH_SEQUENCE
• PROBLEM.INSTANCE_SEG.SEED_MORPH_RADIUS moved to PROBLEM.INSTANCE_SEG.WATERSHED.SEED_MORPH_RADIUS
• PROBLEM.INSTANCE_SEG.ERODE_AND_DILATE_GROWTH_MASK moved to PROBLEM.INSTANCE_SEG.ERODE_AND_DILATE_GROWTH_MASK
• PROBLEM.INSTANCE_SEG.FORE_EROSION_RADIUS moved to PROBLEM.INSTANCE_SEG.WATERSHED.FORE_EROSION_RADIUS
• PROBLEM.INSTANCE_SEG.FORE_DILATION_RADIUS moved to PROBLEM.INSTANCE_SEG.WATERSHED.FORE_DILATION_RADIUS
• PROBLEM.INSTANCE_SEG.DATA_CHECK_MW moved to PROBLEM.INSTANCE_SEG.WATERSHED.DATA_CHECK_MW
• PROBLEM.INSTANCE_SEG.WATERSHED_BY_2D_SLICES moved to PROBLEM.INSTANCE_SEG.WATERSHED.BY_2D_SLICES
• Remove PROBLEM.INSTANCE_SEG.SYNAPSES.NORMALIZE_DISTANCES and normalize always for now
• Added new auxiliary scripts:
  • biapy/utils/scripts/measure_dataset_size.py
  • biapy/utils/scripts/stardist_wheels.py
  • biapy/utils/scripts/measure_class_weights.py
  • biapy/utils/scripts/PR_plot_instances.py
  • biapy/utils/scripts/agreement_manual_points_vs_dnn_instances.py
  • biapy/utils/scripts/semantic_seg_stats.py
  • biapy/utils/scripts/collect_instance_sizes.py

Bugs fixed:

• Remove heat norm/unnorm steps
• Disable automatic normalization for instance segmentation channels
• Correct bug in zoom data augmentation with heatmaps (related to #139)
• Remove UNETR unused layer (related to #123)
• Repare SE blocks (related to #123)
• Avoid printing early stopping message more than once in DDP setting
• Solve bug with BMZ model exportation when combined with loading a checkpoint as the loaded model's cfg was overriding the current cfg
• Solve bug in HRNet building process in all version but hrnet48
• Fix bug with contrastive learning during TTA postprocessing
• Ensure heatmaps are in their original data range after each transformation
• Fix bug in discarding test samples
• Fix problems related to relative paths during BMZ model exportation (related to #140)
• Correct bug in Denoising workflow for multichannel images
• Improve os_walk_clean function used to sort files/dirs as sometimes the files were not sorted good
• Fix issues in BiaPy's CI due to the changes made to adapt the code to Hypha structure API

Full Changelog: v3.6.5...v3.6.6

Version 3.6.5

Minor:

• Allow usage of safetensors to load model weights. MODEL.OUT_CHECKPOINT_FORMAT is created to decide the output checkpoint format

Full Changelog: v3.6.4...v3.6.5

Version 3.6.4

Major:

• Create conda-forge package of BiaPy. Extra info: meta.yaml + CI to trigger package creation on new release
• BMZ updates:
  • Creates a model.py file for BMZ packge by collecting all functions and imports that are necessary to build the model
  • Change the environment.yaml to only be created when timm is used in the model

Minor:

• Change docstrings so the documentation API is built correctly

Bugs fixed:

• Fix minor error that was forcing some models to return a dictionary due to the contrastive learning code added previously
• Fix a bug in contrastive learning
• Fix Z_DOWN handling for HRNet
• Adapt HRNet to prediction by chunks
• Restrict padding in chunked data generator to avoid edge case errors
• Transpose extra 3D image and mask in generator. Needed for CutMix.
• Change last_block in multiresunet as it was before
• Correct a bug in pair_base_data_generator.py

Full Changelog: v3.6.3...v3.6.4

Version 3.6.3

Major:

• Do not force numpy>2, just relax it with numpy>1.26 so the pyproject.toml can is coherent with the upcoming conda package

Minor:

• Relax test 4 check

Bugs fixed:

• Fix bug in SSL workflow
• Fix bug in Dockerfile due to conda TOS approval that must be done non-interactively

Full Changelog: v3.6.2...v3.6.3

Version 3.6.2

Major:

• Update notebooks to install version 3.6.1
• Add contrastive learning and HRNet
• Update models to return a dict if more than one output are returned
• Go back to the surface_area and sphericity calculation to avoid installing diplib package as it does not support some macOS version plus they have no conda package for it. Mainly done to be able to create a conda package and streamline the installation process. This will help to implement the plugin for Fiji and integrate BiaPy in Galaxy.

Minor:

• Add aux script to remap cityscape dataset label remapping
• Add "superresolution" tag for BMZ models
• Add timm, torchmetrics and pytorch-msssim to pyproject.toml
• MODEL.N_CLASSES renamed into DATA.N_CLASSES
• Add a message when checking the DATA.PATCH_SIZE in the loaded configuration and model, as lower values may raise an error when building/using that model
• Correcting some of the descriptions in some functions to avoid compiling errors in the documentation building process
• Update installation in the code consistency CI

Bugs fixed:

• Fix bug in PatchCoords print function
• Solve bug in fixed mean/std normalization
• Move the translation of old config into the new one for the loaded cfg from the checkpoint
• Fix Dockerfile errors due to the last TOS changes in conda

Full Changelog: v3.6.1...v3.6.2

Version 3.6.1

Minor:

• Disable painting matching instances by default as it takes a lot of time, i.e. TEST.MATCHING_STATS_THS_COLORED_IMG to [] (#135)
• Refine DA notebook
• Update notebooks to install BiaPy v3.6.0

Bugs fixed:

• Fix minor issue when creating the prediction.csv file in classification workflow

Full Changelog: v3.6.0...v3.6.1

Version 3.6.0

Major:

• Transition to numpy 2 (#134)
• Update notebooks by removing the cell that restarts the kernel, as it is no longer necessary after the transition to NumPy 2.
• Changing default normalization to ‘zero_mean_unit_variance’
• imgaug dependency removal by implementing the transformations we were using directly in BiaPy (#134)

Minor:

• Renamed REMOVE_CLOSE_POST_POINTS_RADIUS_BY_MASK to REMOVE_CLOSE_POINTS_RADIUS_BY_MASK so it’s more general and to be used with pre points too
• Add num_workers information for test generator
• Improve classification message when expected classes and number of folder does not match (#124)
• Change all os.walk functions to os_walk_clean so the hidden files/dirs are not listed (#125)
• Add support to n5 files
• Add MODEL.SKIP_UNMATCHED_LAYERS variable to allow finetune models without no SSL workflow pretraining (#132)
• Add check in 3D data cropping to not allow padding greater than half of the patch size
• Try to capture tiff metadata automatically to determine axis order
• Limit zarr to be under 3.0 until it is more developed
• Fix timm version to 1.0.14 as in 1.0.15 there are some errors when calling the ViT forward function. Check that test20 is working whenever we want to update the version
• Add a script to repair the notebooks so they can be visualized in Github

Bugs fixed:

• Avoid errors when no post-points are detected during synapse detection
• Adapt histogram matching to new output from load_data_from_dir method
• Add test data check for classification in check_configuration.py (#124)
• Add the old AUGMENTOR.AFFINE_MODE and DATA.NORMALIZATION.CUSTOM_MODE key conversion to the current config version
• Move model.eval() before BMZ input/output creation in test
• Delete old DATA.NORMALIZATION.APPLICATION_MODE during cfg translation to the current version
• Only write pre/post csv files if at least one point is found
• Fix bug for semantic segmentation with more than 2 classes
• Fix bug when loading prediction files when using MODEL.REUSE_PREDICTIONS, the stored prediction files are always of '.tif' extension (as created by the save_tif method).
• Add div normalization in SR templates as the default normalization was changed it needs to be explicitly specified.
• Add normalization instructions to classification workflow data load

Full Changelog: v3.5.13...v3.6.0

Version 3.5.13

Major:

• Completely restructure the inference by chunks (closing #91):
  • A custom Pytorch dataloader is used to speed up the process taking into account number of workers and multiple GPUs
  • Now this inference always creates a Zarr, even if the inputs are H5 files, thus, TEST.BY_CHUNKS.FORMAT was removed
  • Now to division mask is needed, speeding up the process and saving disk space
• Complete synapse detection pipeline by adding metrics, debug files, etc. Now it should not be experimental anymore
• Add remove close points by distance+mask as post-processing for the synapses (remove_close_points_by_mask function)

Minor:

• Add two optional cells in the inference notebook to transform 3D images/videos into 2D images, and the inverse task after prediction. This allows the users to process 3D data using a 2D model (closing #115)
• Add B channel in synapse segmentation
• Merge train/test/val instance creation into one function
• Add zarr_to_tif.py utility script
• Change SYSTEM.NUM_WORKERS default value to -1 and remove it from all templates (closing #114).
• Make ensure_3d_shape function more robust and write_chunked_data to write in 'ZYXC' axes order always
• Move activations into model_call function, removing redundant code (closing #110)
• Add new losses description in SR, SSL and I2I workflows
• Add option to ignore a value in instance masks (still experimental)
• Add TEST.DET_TH_TYPE option
• Add PROBLEM.INSTANCE_SEG.SYNAPSES.TH_TYPE option
• Add a new feature to export the calculated metrics into a CSV file (closing #116 and #117)
• Change the way saved cfg in checkpoint is merged with the current cfg (now it is more robust as the cfg is always translated into the current version)
• Move to bioimageio.core==0.8.0 version
• Update notebooks:
  • Move installation up to restart notebook after reinstalling numpy
  • Update BMZ cell with latest fields

Bugs fixed:

• Disable norm by image when using Zarr/H5 images
• Fix error when instances were generated using a Zarr/H5, as the axes need to be modified always in the next runs so the user does not need to know how the data was created/axes order
• Solve minor bug in blocks.py that lead to errors while building U-Next models.
• Ensure only the input cfg is copied one time in DDP setting
• Solve problem in during shape mismatch check
• Avoid copying gt_associated_id always, reducing the amount of memory for training
• Make ensemble functions work as before returning a torch.Tensor
• Reduce the code in I2I workflow reusing predict_batches_in_test
• Remove convert_old_model_cfg_to_current_version from misc.py so to use the one from biapy.engine.check_configuration instead
• Add MODEL.BMZ.EXPORT.DATASET_INFO when converting the an old cfg into the current version
• Fix error during old cfg conversion with PATHS.LWR_Y_FILE old key
• Relax test25 constraints and fix Dapi_dataset path
• Correct load_data_from_dir function converting it to the way we load data before by just giving a path (closing #109)
• Solve diff_between_configs function
• Solve edge problem with by_chunks_collate_fn function
• Adapt chunks size of the of the Zarr to be created in by chunks setting by transposing the axes into the Zarr axe order

Full Changelog: v3.5.12...v3.5.13