Swissknife tool for ONI files

The ONI file format is the mechanism provided by OpenNI for supporting the recording of videos from RGB-D cameras. This format is generally designed to support multiple compressed streams among which color and depth are the most important. Inside ROS the rosbag fulfils most the data recording and playback needs, but outside ROS the OpenNI is a valuable tools for dealing with RGB-D videos.

Starting from a simple tool for fixing truncated ONI files, I have prepared a more general Python-based tool for manipulating ONI files, motivated by the needs of our research activity and limitations in the C++ API. Some of the features are in common with video editing while others are more specific to the 3D nature of the stream:

• Filtering: removal of the depth or color channel
• Cutting the file in a given range
• Time Manipulation: rescaling of the time, insertion of duplicate frames or dropping frames
• Extraction of the Color channel as MJPEG video or the depth as PNG images (16-bit depth)
• Registration of Color with Depth streams

Registration

The registration feature is used to fix the ONI files that have not been recorded with the registration flag active. This means that the color and depth streams are not spatially aligned. The operation relies on the presence of intrinsics and extrinsic parameters of the color and depth cameras obtained from a calibration procedure. The tool supports the registration from color to depth or viceversa. In this way one of the two streams is rewritten by the tool and the registrationType flag is marked in the file.

At the moment the code embeds only the Asus xtion camera parameters and an option for specifying external camera parameters (via yaml) needs to be added. In addition there is no rectification adjustment.

Implementation

The tool has a general oni module that exposes the basic processing of the ONI file (see appendix for details on the ONI format) over which the several manipulation operations have been implemented. Two elements are written in C++: the first is the code for dealing with the special depth compression algorithm used by ONI files that is directly taken from the OpenNI implementation and exposed via ctypes, the second is the registration code that is written in C++ via Eigen for performance reason. Image compression is managed with Pillow except for the support of 16-bit PNG that requires the png package.

Source Code

Download the source from github.

For the moment the C++ part has to be compiled manually using CMake but is quite trivial provided the presence of Eigen3. 

Appendix - ONI Format

The ONI file format is a block based structure in which each block has a type identifier, a size and the identifier of the associated sensor node. After the initial phase in which descriptive blocks are present, starts the stream of data block coming from the differente nodes. The file is closed by one table for every node describing the mapping from time to the file offset. The block with the contents are compressed with a codec that is specified in the metadata blocks, that is jpeg for the color part, or custom xndec for the depth part.