Open source · microscopy file formats
Judging Olympus by the cover
Olympus has a good reputation for hardware and a poor one for data formats. Anyone who has tried to put their files into an automated pipeline knows the pain. So when spinning-disk data landed on my desk and I heard it was Olympus, my enthusiasm collapsed. It turned out I had judged the book by its cover.
Underneath, a sensible format
There is no public specification: the vendor has one, but doesn’t distribute it. The structure, though, is sound.
- The
.vsifile is a small metadata tree. The pixels sit next to it in.etsfiles, tiled and indexed, so a single plane can be read without loading the whole stack. - The order of time, z and channel differs between files from the same microscope (a reminder that it’s still Olympus), but each file declares its own order.
- Large scans carry a resolution pyramid, so a viewer can fetch the level of detail it needs without touching full resolution.
One trap: the .vsi is also a valid TIFF that holds only a thumbnail. A generic TIFF reader will cheerfully “open” it and hand you 512 pixels.
Why not Bio-Formats?
In Python, bioio could reach VSI files only through Bio-Formats, which means Java. I have spent enough time with javabridge not to touch that with a three-metre pole. So I wrote bioio-vsi: a pure-Python bioio plugin, no JVM.
Reverse-engineered, carefully
The reader is built from sample files. The tree layout follows slideio’s BSD-licensed VSI driver, checked against those files, and Bio-Formats and slideio serve as test oracles: the same file must give the same pixels. A separate document records what is known about the format and how each fact was established, so the next person doesn’t have to rediscover it.
It reads what it has seen and refuses the rest. Several images in one file, an unknown compression, an incomplete tile grid, truncated metadata: each produces a named error instead of a plausible-looking wrong image. In a pipeline that runs unattended, a loud failure is worth more than a quiet guess.
Reporting a file without sharing it
Microscopy data is often unpublished or confidential, which makes bug reports hard. bioio-vsi can describe a file with paths and names redacted, or produce a structure fixture: headers, tile table and metadata layout, with pixels, text and dates removed and measured values replaced by synthetic ones. That is usually enough to reproduce a failure.
pip install bioio-vsi Sample files are welcome, especially from older cellSens versions, slide scanners and long time series.