LINC Convert — End‑to‑End Demo¶
This notebook is a guided tour of the linc-convert CLI, showing how to:
- Inspect available modalities and pipelines
- Run a conversion (PS‑OCT example)
- Open the resulting OME‑Zarr / NIfTI‑Zarr with different backends
- Compare against the original MATLAB array
- Troubleshoot & tune performance
Note: This demo is built on your local environment. Ensure
linc-convertis installed and on yourPATH.
1) Inspect CLI & Modalities¶
The top-level help lists modalities (data families) and global flags.
In [1]:
Copied!
!linc-convert --help
!linc-convert --help
Usage: linc-convert COMMAND Collection of conversion scripts for LINC datasets ╭─ Commands ───────────────────────────────────────────────────────────────────╮ │ df Converters for Dark Field microscopy │ │ lsm Converters for Light Sheet Microscopy │ │ psoct Converters for PS-OCT .mat files │ │ wk Converters for Webknossos annotation │ │ --help -h Display this message and exit. │ │ --version Display application version. │ ╰──────────────────────────────────────────────────────────────────────────────╯
Each modality contains one or more pipelines (subcommands).
PS‑OCT modality¶
We’ll use PS‑OCT for this demo and preview the available pipelines.
In [2]:
Copied!
!linc-convert psoct --help
!linc-convert psoct --help
Usage: linc-convert psoct COMMAND Converters for PS-OCT .mat files ╭─ Commands ───────────────────────────────────────────────────────────────────╮ │ multi_slice Matlab to OME-Zarr. │ │ single_volume Matlab to OME-Zarr. │ ╰──────────────────────────────────────────────────────────────────────────────╯
Look for pipelines like multi_slice and single_volume.
For this demo, we will run single_volume
Pipeline help: single_volume¶
Skim the options to understand input, output, chunking, sharding and additional flags.
In [3]:
Copied!
!linc-convert psoct single_volume --help
!linc-convert psoct single_volume --help
Usage: linc-convert psoct single_volume [ARGS] [OPTIONS] Matlab to OME-Zarr. Convert OCT volumes in raw matlab files into a pyramidal OME-ZARR (or NIfTI-Zarr) hierarchy. ╭─ Parameters ─────────────────────────────────────────────────────────────────╮ │ * INP --inp Path to the input mat file [required] │ │ --key Key of the array to be extracted, │ │ default to first key found │ │ --meta Path to the metadata file │ │ --orientation Orientation of the volume [default: │ │ RAS] │ │ --center --no-center Set RAS[0, 0, 0] at FOV center │ │ [default: True] │ │ --out -o Output path │ │ --zarr-version Zarr version to use. If shard is used, │ │ 3 is required. [choices: 2, 3] │ │ [default: 3] │ │ --chunk --empty-chunk Output chunk size. Behavior depends on │ │ the number of values provided: │ │ │ │ • one: used for all spatial │ │ dimensions │ │ • three: used for spatial dimensions │ │ ([z, y, x]) │ │ • four+: used for channels and │ │ spatial dimensions ([c, z, y, x]) If │ │ "auto", find chunk size smaller than │ │ 1 MB (TODO: not implemented) │ │ [default: (128,)] │ │ --chunk-channels Put channels in different chunk. If │ │ --no-chunk-channels False, combine all channels in a single │ │ chunk. [default: False] │ │ --chunk-time --no-chunk-time Put timepoints in different chunk. If │ │ False, combine all timepoints in a │ │ single chunk. [default: True] │ │ --shard --empty-shard Output shard size. Behavior same as │ │ chunk. If "auto", find shard size that │ │ ensures files smaller than 2TB, │ │ assuming a compression ratio or 2. │ │ [choices: auto] │ │ --shard-channels Put channels in different shards. If │ │ --no-shard-channels False, combine all channels in a single │ │ shard. [default: False] │ │ --shard-time --no-shard-time Put timepoints in different shards. If │ │ False, combine all timepoints in a │ │ single shard. [default: False] │ │ --dimension-separator The separator placed between the │ │ dimensions of a chunk. [choices: ., /] │ │ [default: /] │ │ --order Memory layout order for the data array. │ │ [choices: C, F] [default: C] │ │ --compressor Compression method [choices: blosc, │ │ zlib, none] [default: blosc] │ │ --compressor-opt Compression options [default: {}] │ │ --no-time --no-no-time If True, indicates that the dataset │ │ does not have a time dimension. In such │ │ cases, any fourth dimension is │ │ interpreted as the channel dimension. │ │ [default: False] │ │ --no-pyramid-axis Spatial axis that should not be │ │ downsampled when generating pyramid │ │ levels. If None, downsampling is │ │ applied across all spatial axes. │ │ [choices: x, y, z] │ │ --levels Number of pyramid levels to generate. │ │ If set to -1, all possible levels are │ │ generated until the smallest level fits │ │ into one chunk. [default: -1] │ │ --ome-version Version of the OME-Zarr specification │ │ to use [choices: 0.4, 0.5] [default: │ │ 0.4] │ │ --nii --no-nii Convert the output to nifti-zarr │ │ format. This is automatically enabled │ │ if the output path ends with │ │ ".nii.zarr". [default: False] │ │ --max-load Maximum amount of data to load into │ │ memory at once during processing. │ │ [default: 512] │ │ --overwrite --no-overwrite when no name is supplied and using │ │ default output name, if overwrite is │ │ set, it won't ask if overwrite │ │ [default: True] │ │ --driver library used for Zarr IO Operation │ │ [choices: zarr-python, tensorstore, │ │ zarrita] [default: zarr-python] │ │ --[KEYWORD] │ ╰──────────────────────────────────────────────────────────────────────────────╯
Key flags (quick reference):
INP/--inp(required): input.matfile--key: MATLAB variable to extract (defaults to the first key)--out/-o: output Zarr store path;.nii.zarrimplies NIfTI‑Zarr layout--zarr-version {2,3}: choose Zarr 2 or 3 (v3 required for sharding)--chunk,--shard: control chunk/shard sizes (1 value → all dims; 3 values →[z y x]; 4+ →[c z y x])--driver: IO backend (zarr-python,tensorstore)
2) Convert a Single Volume (PS‑OCT)¶
Goal: Convert a MATLAB volume to a pyramidal NIfTI‑Zarr (or OME‑Zarr) store.
The cell below runs the conversion. Adjust
INPUT_MAT,MAT_KEY, andOUTPUT_ZARRat the top if needed.
In [4]:
Copied!
!linc-convert psoct single_volume /scratch/sample_input.mat --key Psi_ObsLSQ --out /scratch/output.nii.zarr
!linc-convert psoct single_volume /scratch/sample_input.mat --key Psi_ObsLSQ --out /scratch/output.nii.zarr
2025-09-24 14:55:22,379 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 0, 0) of 2025-09-24 14:55:25,268 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 1, 0) of 2025-09-24 14:55:25,543 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 2, 0) of 2025-09-24 14:55:26,680 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 0, 0) of 2025-09-24 14:55:26,951 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 1, 0) of 2025-09-24 14:55:27,229 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 2, 0) of 2025-09-24 14:55:27,439 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 0, 0) of 2025-09-24 14:55:28,669 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 1, 0) of 2025-09-24 14:55:28,939 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 2, 0) of 2025-09-24 14:55:29,134 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 0, 0) of 2025-09-24 14:55:29,268 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 1, 0) of 2025-09-24 14:55:29,401 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 2, 0) of 0%| | 0/4 [00:00<?, ?it/s]2025-09-24 14:55:29,510 - linc_convert.utils.io.zarr.abc - INFO - Compute level 1 with shape [855, 660, 1] [########################################] | 100% Completed | 1.50 sms 25%|███████████▎ | 1/4 [00:01<00:05, 1.74s/it]2025-09-24 14:55:31,245 - linc_convert.utils.io.zarr.abc - INFO - Compute level 2 with shape [427, 330, 1] [########################################] | 100% Completed | 401.06 ms 50%|██████████████████████▌ | 2/4 [00:02<00:01, 1.01it/s]2025-09-24 14:55:31,709 - linc_convert.utils.io.zarr.abc - INFO - Compute level 3 with shape [213, 165, 1] [########################################] | 100% Completed | 200.56 ms 75%|█████████████████████████████████▊ | 3/4 [00:02<00:00, 1.56it/s]2025-09-24 14:55:31,935 - linc_convert.utils.io.zarr.abc - INFO - Compute level 4 with shape [106, 82, 1] [########################################] | 100% Completed | 100.36 ms 100%|█████████████████████████████████████████████| 4/4 [00:02<00:00, 1.57it/s] 2025-09-24 14:55:32,051 - linc_convert.modalities.psoct.single_volume - INFO - Write OME-Zarr multiscale metadata 2025-09-24 14:55:32,070 - py.warnings - WARNING - /autofs/space/megaera_001/users/kchai/linc-convert/linc_convert/utils/io/zarr/drivers/zarr_python.py:194: FutureWarning: Pass shape=[348], dtype=u1 as keyword args. From version 3.1.0 passing these as positional arguments will result in an error arr = self._zgroup.create_array(name, shape, dtype, **kwargs)
By default, zarr files are handled with zarr-python, we can try using tensorstore as well, which currently has much better writing performance with sharding.
In [5]:
Copied!
!linc-convert psoct single_volume /scratch/sample_input.mat --key Psi_ObsLSQ --out /scratch/output-ts.nii.zarr --zarr-version 3 --shard 1024 --driver tensorstore
!linc-convert psoct single_volume /scratch/sample_input.mat --key Psi_ObsLSQ --out /scratch/output-ts.nii.zarr --zarr-version 3 --shard 1024 --driver tensorstore
2025-09-24 14:55:34,234 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 0, 0) of 2025-09-24 14:55:34,321 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 1, 0) of 2025-09-24 14:55:34,423 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (0, 2, 0) of 2025-09-24 14:55:34,498 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 0, 0) of 2025-09-24 14:55:34,608 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 1, 0) of 2025-09-24 14:55:34,686 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (1, 2, 0) of 2025-09-24 14:55:34,735 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 0, 0) of 2025-09-24 14:55:34,824 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 1, 0) of 2025-09-24 14:55:34,915 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (2, 2, 0) of 2025-09-24 14:55:34,987 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 0, 0) of 2025-09-24 14:55:35,031 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 1, 0) of 2025-09-24 14:55:35,095 - linc_convert.modalities.psoct.single_volume - INFO - Processing chunk (3, 2, 0) of 0%| | 0/4 [00:00<?, ?it/s]2025-09-24 14:55:35,147 - linc_convert.utils.io.zarr.abc - INFO - Compute level 1 with shape [855, 660, 1] [########################################] | 100% Completed | 300.80 ms 25%|███████████▎ | 1/4 [00:00<00:01, 2.48it/s]2025-09-24 14:55:35,551 - linc_convert.utils.io.zarr.abc - INFO - Compute level 2 with shape [427, 330, 1] [########################################] | 100% Completed | 100.70 ms 50%|██████████████████████▌ | 2/4 [00:00<00:00, 4.08it/s]2025-09-24 14:55:35,686 - linc_convert.utils.io.zarr.abc - INFO - Compute level 3 with shape [213, 165, 1] [########################################] | 100% Completed | 100.52 ms 75%|█████████████████████████████████▊ | 3/4 [00:00<00:00, 5.35it/s]2025-09-24 14:55:35,803 - linc_convert.utils.io.zarr.abc - INFO - Compute level 4 with shape [106, 82, 1] [########################################] | 100% Completed | 100.65 ms 100%|█████████████████████████████████████████████| 4/4 [00:00<00:00, 5.20it/s] 2025-09-24 14:55:35,917 - linc_convert.modalities.psoct.single_volume - INFO - Write OME-Zarr multiscale metadata
3) Compare with the original MATLAB array¶
In [6]:
Copied!
import numpy as np
import zarr
import scipy.io as sio
# Paths & keys
INPUT_MAT = "/scratch/sample_input.mat"
MAT_KEY = "Psi_ObsLSQ"
OUT_ZARR_ZP = "/scratch/output.nii.zarr"
OUT_ZARR_TS = "/scratch/output-ts.nii.zarr"
# Load MATLAB source array
mat = sio.loadmat(INPUT_MAT)
a_mat = np.asarray(mat[MAT_KEY])
# Load Zarr arrays (zarr-python), dataset "0"
a_zp = np.asarray(zarr.open(OUT_ZARR_ZP, mode="r")["0"])
a_ts = np.asarray(zarr.open(OUT_ZARR_TS, mode="r")["0"])
# Optional: cast outputs to MATLAB dtype for strict comparison
if a_zp.dtype != a_mat.dtype:
a_zp = a_zp.astype(a_mat.dtype, copy=False)
if a_ts.dtype != a_mat.dtype:
a_ts = a_ts.astype(a_mat.dtype, copy=False)
# Report shapes/dtypes
print("[info] MAT :", a_mat.shape, a_mat.dtype)
print("[info] Zarr :", a_zp.shape, a_zp.dtype)
print("[info] TStore:", a_ts.shape, a_ts.dtype)
# Compare with NumPy testing (adjust tolerances if needed)
rtol, atol = 1e-5, 1e-8
np.testing.assert_allclose(a_mat, a_zp, rtol=rtol, atol=atol, err_msg="MAT vs output.nii.zarr differ")
print("[ok] MAT vs output.nii.zarr match within tolerances.")
np.testing.assert_allclose(a_mat, a_ts, rtol=rtol, atol=atol, err_msg="MAT vs output-ts.nii.zarr differ")
print("[ok] MAT vs output-ts.nii.zarr match within tolerances.")
import numpy as np
import zarr
import scipy.io as sio
# Paths & keys
INPUT_MAT = "/scratch/sample_input.mat"
MAT_KEY = "Psi_ObsLSQ"
OUT_ZARR_ZP = "/scratch/output.nii.zarr"
OUT_ZARR_TS = "/scratch/output-ts.nii.zarr"
# Load MATLAB source array
mat = sio.loadmat(INPUT_MAT)
a_mat = np.asarray(mat[MAT_KEY])
# Load Zarr arrays (zarr-python), dataset "0"
a_zp = np.asarray(zarr.open(OUT_ZARR_ZP, mode="r")["0"])
a_ts = np.asarray(zarr.open(OUT_ZARR_TS, mode="r")["0"])
# Optional: cast outputs to MATLAB dtype for strict comparison
if a_zp.dtype != a_mat.dtype:
a_zp = a_zp.astype(a_mat.dtype, copy=False)
if a_ts.dtype != a_mat.dtype:
a_ts = a_ts.astype(a_mat.dtype, copy=False)
# Report shapes/dtypes
print("[info] MAT :", a_mat.shape, a_mat.dtype)
print("[info] Zarr :", a_zp.shape, a_zp.dtype)
print("[info] TStore:", a_ts.shape, a_ts.dtype)
# Compare with NumPy testing (adjust tolerances if needed)
rtol, atol = 1e-5, 1e-8
np.testing.assert_allclose(a_mat, a_zp, rtol=rtol, atol=atol, err_msg="MAT vs output.nii.zarr differ")
print("[ok] MAT vs output.nii.zarr match within tolerances.")
np.testing.assert_allclose(a_mat, a_ts, rtol=rtol, atol=atol, err_msg="MAT vs output-ts.nii.zarr differ")
print("[ok] MAT vs output-ts.nii.zarr match within tolerances.")
[info] MAT : (1711, 1320, 2) float64 [info] Zarr : (1711, 1320, 2) float64 [info] TStore: (1711, 1320, 2) float64 [ok] MAT vs output.nii.zarr match within tolerances. [ok] MAT vs output-ts.nii.zarr match within tolerances.
Appendix — CLI Cheatsheet¶
# List all modalities
linc-convert --help
# List pipelines for a modality
linc-convert psoct --help
# Show detailed help for a pipeline
linc-convert psoct single_volume --help
# Minimal conversion
linc-convert psoct single_volume IN.mat -o OUT.zarr
# NIfTI-Zarr + custom key
linc-convert psoct single_volume IN.mat --key Psi_ObsLSQ -o OUT.nii.zarr
# Zarr v3 with sharding
linc-convert psoct single_volume IN.mat -o OUT.zarr --zarr-version 3 --shard 1024