LECTURE 6

Lecture 6. Preprocessing, normalisation and fast data loading with MONAI

How a CT or MRI volume becomes a tensor a network can learn from: orientation to RAS, resampling to 1 mm voxels, cropping to the body, the soft-tissue window from −175 to 250 HU, z-scores for MRI, and CacheDataset and PersistentDataset so that the GPU never waits. With diagrams, formulas, interactive examples and 3D animations.

This page is still being reviewed by our scientific, clinical and legal experts. It describes plans, not finished work.

Two CT scans of the same abdomen can differ in everything but the anatomy. One has 0.7 mm pixels and 5 mm slices, stored from the patient’s left; the other has 0.9 mm pixels and 1 mm slices, stored from the right. A network sees only arrays of numbers. Unless every volume is brought to the same orientation, the same voxel size and the same range of values, it will learn the scanners instead of the organs. And if that preparation runs again for every sample in every epoch, the GPU spends most of its time waiting. This lecture covers both halves with MONAI 1.6. It is lecture 6 of the learning programme, after lecture 5 on datasets and splits.

Every image on this page is drawn by code from the phantom of lecture 1. The osv repository does not use MONAI yet; this lecture describes the standard MONAI practice that a 3D pipeline in the project would start from.

The chain

deterministic: computed once and cachedLoadImagedNIfTI or DICOMEnsureChannelFirstdshape C × H × W × DOrientationdaxcodes = "RAS"Spacingd1.0 × 1.0 × 1.0 mmScaleIntensityRanged−175…250 HU → 0…1CropForegrounddthe body onlyrandom: every sample, every epochRandCropByPosNegLabeldpatches around labelsRandFlipd, RandRotate90daugmentationto the modelon the GPU
A typical chain for abdominal CT. The deterministic part gives the same result every time and can be cached; the random part is augmentation and must run anew.

MONAI transforms come in two forms: Spacing works on an array, and Spacingd works on a dictionary such as {"image": ..., "label": ...}, applying the same operation to the keys you name. The dictionary form keeps an image and its label aligned through every step. LoadImaged reads NIfTI or DICOM together with the affine matrix from lecture 1, and EnsureChannelFirstd puts the channel axis first, as PyTorch expects.

Spatial transforms

Orientationd(keys=["image", "label"], axcodes="RAS") reorders and flips the axes so that the first axis points to the patient’s right, the second to the front and the third to the head. DICOM’s LPS and NIfTI’s RAS were covered in lecture 1. Orientation does not interpolate anything; it only reorders the array and updates the affine.

Spacingd(keys=["image", "label"], pixdim=(1.0, 1.0, 1.0), mode=("bilinear", "nearest")) resamples to 1 mm voxels along every axis. The number of voxels along an axis changes in proportion to the spacing:

ni′=round(ni·sisi′) 512·0.781≈400,80·51=400

A CT of 512 × 512 × 80 voxels at 0.78 × 0.78 × 5 mm becomes about 400 × 400 × 400. Isotropic voxels mean that a 3 × 3 × 3 convolution covers the same distance in every direction, and that a model trained on one slice thickness can be applied to another. The price is size: 1 mm is a choice, not a law. MONAI’s BTCV tutorial uses 1.5 × 1.5 × 2.0 mm, which needs about four and a half times fewer voxels.

The interpolation mode matters more than it looks. MONAI’s default is bilinear, which is right for the image and wrong for the label. Between a voxel labelled 0 and one labelled 1, bilinear interpolation produces 0.5, which belongs to no class, and between labels 3 and 5 it produces 4, another organ. Labels are always resampled with "nearest".

The plane runs through the liver from the patient’s right to left and from the feet up, tilted from back to front so that organ edges curve; the label is in orange. With thick slices the stored volume is a staircase. Resampling smooths the image, but cannot bring back detail that the thick slices never recorded. With linear interpolation, the label’s edge fills with fractions.

CropForegroundd(keys=["image", "label"], source_key="image") cuts the volume down to the bounding box of its foreground, by default every voxel greater than zero. Air around the body can be a third of a CT, and cropping it saves memory and time in every later step. Place the crop after the intensity window: once the window has mapped everything below −175 HU to 0, “greater than zero” means the body. On raw Hounsfield units, fat at −100 HU would count as background. Since MONAI 1.5, allow_smaller defaults to False, so a margin never reaches beyond the image’s edge.

Intensity normalisation

CT. Hounsfield units run from −1000 for air to above +1000 for bone, but the organs of the abdomen lie between about −100 and +200. ScaleIntensityRanged(keys=["image"], a_min=-175, a_max=250, b_min=0.0, b_max=1.0, clip=True) maps that soft-tissue window to 0…1; the same window is used in MONAI’s BTCV multi-organ tutorial.

y=clip(x−aminamax−amin·(bmax−bmin)+bmin,bmin,bmax) amin=−175,amax=250:x=40→y=215425≈0.506

clip is off by default. Without it, air becomes −1.94 and cortical bone at 900 HU becomes 2.53, so the network still sees the full range, now merely rescaled. Try it:

With clipping off, values below 0 show in blue and above 1 in orange. The histogram’s height is the square root of the count, so that small groups stay visible next to the large ones; the green band is 0…1.

MRI. MR intensities have no physical unit. The same tissue gives different numbers on different scanners, coils and even sessions, so no fixed window fits. Instead each volume is standardised to zero mean and unit standard deviation with NormalizeIntensityd(keys=["image"], nonzero=True, channel_wise=True):

y=x−μσ,μ,σ:Ω={xj:xj≠0}

nonzero=True computes the mean and standard deviation over the voxels that are not zero, leaving the background out. Otherwise a volume that is mostly empty space would get its statistics from the air. channel_wise=True normalises each sequence (T1, T2, FLAIR) on its own. In the tool, the MRI-like slice is the phantom with made-up intensities. Switch between scanners A and B: the raw numbers change, the normalised image does not.

Fast data loading

Reading a compressed NIfTI file, reorienting, resampling tens of millions of voxels and cropping takes seconds per volume on a CPU. A GPU step on a 96³ patch takes a fraction of that. If every sample goes through the whole chain in every epoch, the GPU waits:

u=min(1,W·tstepB·tsample)

Here u is the share of time the GPU is busy, W the number of loader workers, B the batch size, tsample the time to prepare one sample and tstep the time of one training step. More workers help until the CPU cores run out. The other lever is to stop repeating work whose result never changes.

Datasetfilesdeterministicevery epochrandomevery epochGPUCacheDatasetfilesdeterministiconce → RAMrandomevery epochGPUPersistentDatasetfilesdeterministiconce → local SSDrandomevery epochGPU
All three run the random transforms every time. They differ in where the deterministic part runs and where its result lives.
  • CacheDataset(data, transform, cache_rate=1.0, num_workers=8) runs the chain up to the first random transform once, before the first epoch, and keeps the results in memory. From then on, each sample costs only its random transforms. The first epoch is slow while the cache fills; cache_rate=0.0 skips caching while debugging. cache_rate defaults to 1.0, all of the data.
  • PersistentDataset(data, transform, cache_dir="/local/ssd/cache") does the same but writes each result as a tensor file to cache_dir. It needs no RAM for the cache and survives a restart; reading a tensor from a fast local disk is far cheaper than decoding and resampling the original. The file names include a hash of the transforms, but the documentation advises clearing the directory after changing them.

Both cache only up to the first random transform in a Compose. A random step placed early, a flip before Spacingd for example, stops the caching there and everything after it runs every time. So deterministic transforms go first and random ones last.

How much memory does a full cache need?

M=N·K·∏ini·b 100·2·4003·4≈51GB

N volumes with K arrays each (an image and a label), ni voxels along each axis and b bytes per voxel. A hundred abdominal CTs at 1 mm, about 400³ voxels each, stored as float32 with their labels need about 51 GB. That is why the crop matters, why labels can be stored as uint8, and why PersistentDataset or a cache_rate below 1 exists.

Try the loader below. A plain Dataset with the default times keeps the GPU busy about a third of the time; a cache brings it close to all of it.

The timeline shows 16 GPU steps with a batch of 2. The times are illustrations, not measurements: measure your own with a profiler before tuning. MONAI’s ThreadDataLoader and ToDeviced go one step further for cached data, keeping the random transforms in threads and the tensors on the GPU.

Checklist

  • Use the dictionary transforms on image and label together, with mode=("bilinear", "nearest") in Spacingd.
  • Orient to RAS before resampling, and crop after the intensity window.
  • Set clip=True in ScaleIntensityRanged; use nonzero=True z-scores for MRI.
  • Put every deterministic transform before the first random one.
  • Cache what fits in memory, persist the rest to a local SSD, and clear the cache after changing a transform.
  • Keep the validation chain identical to the training chain, minus the random transforms.

Further reading