Python Imgstore

Written on Tuesday May 30, 2017

Imgstore is a library that we built for efficient reading and writing of large video recordings. By splitting long recordings into chunks and implementing efficient indexing alongside, it allows frame accurate seeking and random frame access.

Imgstore supports videos recorded with loopbio’s Motif recording system.

If you need to store hundreds, to thousands to millions of frames, and metadata with each frame (including but not limited to framenumber and timestamp), then check out the project page.

Sample frame from a 7-day, 120Gb, 9-million frame recording (more information)

ImgStore supports two different backends; DirectoryImgStore and VideoImgStore which are designed for lossless and lossy recordings respectively. DirectoryImgStore stores each frame as a single compressed file supporting raw numpy npy arrays, png images, compressed blosc arrays, raw pgm images, and many more. Files are grouped into directories ‘chunks’ to retain good filesystem performance.

In VideoImgStore, each ‘chunk’ is an individual movie file with a fixed number of frames. This chunking strategy limits the size of each file to make copying and archival easier (respecting FAT32 file size limits for USB sticks for example).

In both cases the Imgstore API allows continuous playback through all frames as though it were a single file on disk, and allows seeking to any frame within.

Reading files is easy:

from imgstore import new_for_filename

store = new_for_filename('mystore/metadata.yaml')

print('%d frames in store' % store.frame_count)

# read the best frame from the video
img, (frame_number, frame_timestamp) = store.get_image(42)
print('framenumber:', frame_number, 'timestamp:', frame_timestamp)

For more information check out the project page.