MONAI_Layer_Design_Discussion

Layers

pytorch defines layers in the following way:

• some layers are defined on a per-dimension basis
  • each layer then expects a tensor in the format 'BFD' where D can be 1 or more dimensions
    • A 1D pytorch layer takes a 3D tensor
    • A 2D pytorch layer takes a 4D tensor
    • A 3D pytorch layer takes a 5D tensor, etc.
• some layers determine their functionality based on the shape of the input tensor
• MONAI will have an expanded set of capabilities focussing in two directions:
  • New layer functionality
  • Extended dimension compatability

Discussion:

• BenM: The design decision that pytorch makes in terms of dimension-specific and dimension-agnostic APIs is confusing:
  • More layers are dimension-agnostic than dimension-specific
  • Dimension-agnostic layers present a problem when expanding the number of dimensions that can be processed
  • Proposals:
    • We provide dimension-agnostic wrappers for all dimension-specific vanilla functions, for optional use
    • We provide dimension-specific extensions for functions that extend dimension-specific vanilla functions
      • We do so in places where pytorch would choose dimension-specific functions
        • TODO: enumerate pytorch API comprehensively
    • We provide dimension-agnostic wrappers for all extended functionality for which we also provide dimension-specific wrappers
    • Layers and layer factories are obviously related:
      • Layer factories might be a good place to present a consistently layer-agnostic view of layer functionality that also ensure that dimension checking occurs at initialisation time rather than run time