Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Fractal tasks for object and single-cell registration #99

Merged
merged 46 commits into from
Dec 14, 2023
Merged

Conversation

jluethi
Copy link
Contributor

@jluethi jluethi commented Dec 5, 2023

@nrepina I'm opening the PR so that when we push to the branch, Github CI reruns the automated tests :) Not to be merged yet

What this is adding

  • Label-based registration for organoids (3 tasks)
  • Label-based registration for single cells

Minor changes:

  • Updates scmultiplex measurements task to Fractal-tasks-core 0.14.0a0
  • Updates test data to have new ROI tables with correct metadata
  • Fixes overwrite bug in scmultiplex task
  • Refactor of scmultiplex task to use more library functions, less edge-case handling in the task code

Once fractal-tasks-core 0.14.0 is released:

  • Update the new tasks to 0.14.0
  • Add test data for new tasks

Sorry, something went wrong.

nrepina and others added 30 commits September 28, 2023 16:05
Based on calculate_registration_image_based but uses label images as input and calculate_shifts scmpx function.
Based on calculate_registration_image_based but uses label images as input and calculate_shifts scmpx function.
Correct handing of case where task was already run and shifts already exist in table.
Also correct bug so that correct RX table is loaded (was loading R0).
Calculate linking of objects between pairs of multiplexing rounds and generate linking table.
Calculate consensus across rounds for all rounds, relative to reference round.
Store as table in reference round directory.
… input and arbitrary column names.

Add helper function make_linking_dict to create dictionary from arbitrary pandas df columns.
Relabel label images and tables based on linked consensus IDs.
Merge branch 'nar-fractal' of https://github.com/fmi-basel/gliberal-scMultipleX into nar-fractal
Table name and path to label image was incorrect in zattr. See also open issue #619 in fractal tasks core.
… 3D approach, but covered in 2D tests anyway
… 3D approach, but covered in 2D tests anyway
Note bug fix that loads rx_idlist instead of r0_idlist during load_region for rx.
…ed nuclear sizes.

PlatMatch linking previously relied on estimating ransac error from nuclear size distribution in the organoid.
However, if segmentation results include segmented debris or other outliers, the size distribution can be non-gaussian and highly skewed.
Thus the  method of calculating ransac error using mean/stdev was highly sensitive to these outliers and can even result in negative ransac error, leading to ransac failure.
Instead, now calculate error using a quantile threshold.
Create new function do_ransac_complete without allocation of a process pool.
Kept previous function just renamed to do_ransac_complete_multithread.
Multithreading was causing conflicts with Fractal-based parallelization, can revisit later to improve.
…entation image.

Allows to remove labels ids from numpy array image that were filtered out during size filtering or disconnected component analysis.
Allows affine and free-form deformation registration of segmented nuclei or cells within segmented orbjects (e.g. organoids).
@codecov-commenter
Copy link

codecov-commenter commented Dec 10, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (4b46ca5) 98.91% compared to head (c3dd4af) 98.13%.
Report is 13 commits behind head on main.

Files Patch % Lines
tests/fractal/test_fractal_measurement.py 96.66% 2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #99      +/-   ##
==========================================
- Coverage   98.91%   98.13%   -0.79%     
==========================================
  Files          10       10              
  Lines         461      482      +21     
==========================================
+ Hits          456      473      +17     
- Misses          5        9       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nrepina
Copy link
Collaborator

nrepina commented Dec 12, 2023

Adding 4 new tasks:

  • calculate_object_linking --> organoid linking across rounds with phase cross-correlation of segmentations; outputs linking table
  • calculate_platymatch_registration --> single-cell linking and registration across rounds; outputs linking table and affine or free-form deformation transformation matrix saved to disk
  • calculate_linking_consensus --> calculates master consensus table for organoids or single cells that are successfully linked across rounds, output: master table stored in reference round directory
  • relabel_by_linking_consensus --> relabels segmentation images and ROI tables based on master consensus.

The places where the i/o code could use a refactor is noted with # TODOs. In particular the calculate_platymatch_registration is huge and can benefit from some i/o helper functions.

Sorry, something went wrong.

@nrepina nrepina changed the title [WIP] Add additional Fractal tasks for label-based registration Add Fractal tasks for object and single-cell registration Dec 13, 2023
@nrepina nrepina marked this pull request as ready for review December 14, 2023 13:48
@nrepina nrepina merged commit 281fb78 into main Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants