Bandwidth usage for USB cameras: ZED tests under Linux

Robotics and Computer Vision applications are taking advance of a range USB3 cameras that are characterised by large resolution, high-frame rate and low-latency. Examples of such cameras we worked with are the StereoLabs ZED that provides a side-by-side low-latency stereo image, or monocular PointGrey cameras.

These cameras belong to the USB Video Class that has, since long time, reduced the need of specific drivers. A typical challenge in the use of this camera is the management of USB bandwidth in particular when multiple cameras have to be integrated over a same computer.

The theoretical bandwidth of USB3 is 5 Gbps but then it is known that the effective bulk transfer bandwidth is 440MB/s. In addition to this there are issues of protocol overhead that affect the use of a given resolution and rate selection.

When using multiple cameras there are also limitations in the chipset used in the motherboard. Some chipset have one single USB3 bus shared among multiple ports, instead, if an addon PCI card is used, there could be issues of PCI transfer rate.

This post aims at providing a tool for Linux and example usage with ZED for the analysis of the bandwidth used by a camera and understanding what happens when two cameras are connected over the same bus.




The debugfs (kernel doc) of Linux allows for detailed analysis of the kernel and it can be extended by means of modules. One of such modules is usbmon that allows to intercept the USB packets for each bus.

Assuming debugfs is already present, the module can be loaded with:

modprobe usbmon

Now it is possible to ready data from one path that publishes the USB packets (e.g. for the Bus 03):

cat /sys/kernel/debug/usb/usbmon/3u

The format of the resulting output is documented here and with general introduction here.

The bus of the camera can be identified using lsusb, or better lsusb -t. This is an extract from an Ubuntu 14.04 with two ZED connected:

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 14, If 0, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 1: Dev 14, If 1, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 17, If 0, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 2: Dev 17, If 1, Class=Video, Driver=uvcvideo, 5000M

The tool discussed here and available on github parses live the stream from the above usbmon entrypoint and prints for every device on the bus (14 and 17 in the listing above) the bytes and packets produced every second.

The tool accepts the bus number, a filename, or "-" for getting data from standard input. The following two are equivalent:

cat /sys/kernel/debug/usb/usbmon/3u | python bandwidth.py -
python bandwidth.py 3

An example output for 2560x720@30Hz is the following:

device time(s) MB packets
17 3012 95.911936 2927
17 3013 148.471808 4531
17 3014 148.43904 4530
17 3015 148.504576 4532
17 3016 148.43904 4530
17 3017 148.43904 4530
17 3018 148.504576 4532
17 3019 148.43904 4530
17 3020 148.43904 4530

For what concerns the ZED here are some examples of bandwidth and packets for different configurations. The listing is: resolution@rate, bandwidth MB/s, packets.

• 4416x1242@15 230.4 7032
• 3840x1080@30 332.2 10140
• 3840x1080@60 166 5070
• 2560x720@60 296.8 8288
• 2560x720@30 148.4 4530
• 2560x720@15 74 2265
• 1280x480@30 50.0 1530
• 1344x376@100 133 4064
• 1344x376@60 80.6 2452
• 1344x376@30 40.3 1254
• 1344x376@30 21 632

In practice data is transferred as YUYV 16bit with effective data rate that depends on the number of packets (see wireshark source and specification) resulting in about 2.6 bytes per pixel in average.

In a robot computer with a single USB3 bus (Intel QM77 mobile chipset) the use of two ZED with resolution 2560x720 at 30Hz is relatively stable with little oscillations (taking about 300MB/s for 9000 packets) while higher configurations saturate at about 400 with oscillating behavior.

Grab the script on github.