The essentials:
Why is there an issue: The Realsense camera uses the YUYV422 pixel format for it’s streams. Unfortunately, zoom does not currently support streams of this format. This is why you see a black square in zoom.
What is the solution: Take the Realsense video stream and re-stream it in YUV420P which zoom does accept.
What configuration is this tested for: Intel Relasense D435, Ubuntu 20.04
The process:
This assumes that you already have the Intel Realsense camera drivers installed and working. You will also need ffmpeg which we will use for re-coding the video stream, and v4l2loopback which we will use to create the virtual video stream. Once you have the required packages, you can begin by creating a dumy video, and then listing all the video streams to identify it’s name (for me this is /dev/video8). You can then use ffplay to find the intel realsense’s rgb stream (for me /dev/video6) and send it to your dummy video. At this point you should be good to go! In practice I use the bash function (copied into .bashrc).
The commands are below, and if you run into any issues feel free to contact me through the “about” page above.
Installation:
sudo apt install ffmpeg
sudo apt install v4l2loopback-dkms
Helpful Commands:
To list video devices:
v4l2-ctl --list-devices
To view a particular video:
ffplay -f v4l2 /dev/[video name]
ffplay -f v4l2 /dev/video1
To create a dummy video:
sudo modprobe v4l2loopback
To clone your real sense onto it:
ffmpeg -hide_banner -f v4l2 -i /dev/[realsense rgb stream] -vf format=yuv420p -f v4l2 /dev/[dummy video stream]
ffmpeg -hide_banner -f v4l2 -i /dev/video6 -vf format=yuv420p -f v4l2 /dev/video8
To play your desktop over a video stream:
ffmpeg -f x11grab -framerate 25 -video_size 1920x1200 -i :1 -f v4l2 /dev/[dummy video stream]
ffmpeg -f x11grab -framerate 25 -video_size 1920x1200 -i :1 -f v4l2 /dev/video8
In practice I use the following commands:
sudo modprobe v4l2loopback
ffmpeg -hide_banner -f v4l2 -i /dev/video6 -vf format=yuv420p -f v4l2 /dev/video8
Bash Function:
function realsense_zoom () {
local target=${1:-'video16'}
local source=${1:-'video6'}
sudo modprobe v4l2loopback video_nr=16
ffmpeg -hide_banner -f v4l2 -i /dev/${source} -vf format=yuv420p -f v4l2 /dev/${target}
}
Source:
This page is essentially all based on a delightful blog post by Roman Soldatow which is linked below. He also has several other very useful commands I don’t use and thus didn’t copy over, so you should definitely check it out.
https://rmsol.de/2020/04/25/v4l2/
Last Updated: 10/30/2021 (improved bash function)