Transformations Meeting 1

Minutes of meeting 1

Date: 28th April 2020 Membership: M. Jorge Cardoso (Lead), 
Michael Baumgartner, Tom Varsavsky, 
Fernando Perez-Garcia, Richard Shaw, Mark Graham, Mauricio Arteaga, Stephen Aylward, 
Rahul Choudhury, Wenqi Li 

---

Define the scope of the group

Focus on imaging “tensors” as inputs and outputs Input-to-output transformation and data types

• Segmentation: Images/Labels as joint pieces of information
• Classification: Images/classes
• Semantic Regression: Image/image pairs
• Object Detection: Image/Object-descriptor
• Registration: Image/transformation
• AutoML: Requirements regarding differentiability
• Including post-processing?

Cache intermediate representations

• Performance
• Algorithmic purposes

Differentiability

---

Types of Transformations

Voxel/physical-coordinates consideration

• All transformation should have a mm equivalent
• Different orientations
• Different images of the same subject with different orientation
• Should allow the user to bypass all of this

Spatial Transforms

• Rotation,
• Translation,
• Scale
  • Scale factor
  • Target resolution
• Affine,
• Flips,
• Non-linear transformations
  • B-splines
  • Velocity fields
    • Shooting
    • Integration
    • Exponentiation
• Composability of transformations
• Interpolation (linked with data type)
  • Nearest neighbor, linear, b-spline, sinc
• Crop
  • Centre crop
  • Localised Crop
• Padding

Intensity transforms

• Noise
• Whitening (zero=mean, std=1)
• Min-max
• Robust Min-Max
• Outlier removal
• Log
• Equalisation
• Histogram matching
  • Learned?
• Gamma
• Window-Level
• Clipping
• Quantisation
• Filters: CLAHE / AHE / VCD / Curvature / ...
• Blurring
  • Voxel
  • Physical Spaces
• Represent NANs

Acquisition Physics transforms

• MRI
  • Bias-field correction
  • K-space artefacts
  • Motion
• Ultrasound
  • Speckle
  • Occlusions/shadows
  • Poor acoustic coupling
  • Motion
• CT
  • Simulating changes in acquisition (keV)
  • Metal artifacts
  • Motion
• Projections
  • CT -> XRAY
  • CT -> tomosynthesis
• Simulating other modalities
• Simulating image artefacts
• Simulating results from different scanners/manufacturers

Segmentation specific

• Categorical to probabilistic (vice versa)
• Categorical-mapping (forward and backwards)
• Temperature scaling

Patient transforms

• Simulating body habitus variations
• Simulating aging
• Simulation in general and what an API would look like

Sampling strategies

• Sampling 2D images from a 3D volume (maybe part of the “normal” 3d patch pathway)
• Sampling multiple contiguous 2d images (pseudo3d) from 3d volume
• Sampling 2D images with multiple orientations (2.5D)
• Patches from a Volume/Subject from dataset
  • Random
  • Segmentation-aware
  • Masked sampling (only in a subregion)
  • Hard-negative mining
  • Grid sampling
  • Grid-sampling with overlap
  • Aggregation of grid sampling
    • Mapping values
    • Weighted aggregation

---

Interaction with other packages?

• BatchGenerator - numpy/SimpleITK
• TorchIO - Torch/numpy/SimpleITK/nibabel
• Phoenix Rising - torch
• ITK - itk
• Nvidia DALI - 2d, separate ecosystem
• Kornia - torch

.Considerations

• Torch Tensor? => CPU/GPU
• Manage Batches?

---

---

Points of discussion for the next meeting

• Differentiability
• Data type transformation and performance
• Looking into Transformation (code) API
  • 1 slide per method
• Geometric transforms and composability
• Linear vs DAG execution
• Export/import augmentation (random sample) parameters