Hardware Accelerated Point Cloud Streaming

See also follow-up and extension with infrared textured point cloud streaming:
https://www.youtube.com/watch?v=zVIuvWMz5mU

The technology builds up on open source and open standard for interfacing hardware encoders.

The source code:

https://github.com/bmegli/realsense-network-hardware-video-encoder
https://github.com/bmegli/unity-network-hardware-video-decoder

The idea behind depth encoding:
https://github.com/bmegli/realsense-depth-to-vaapi-hevc10/wiki/How-it-works

The technical side of decoding:
https://github.com/bmegli/unity-network-hardware-video-decoder/wiki/How-it-works

The latency measurement detailed description:
https://github.com/bmegli/unity-network-hardware-video-decoder/issues/9

The above final solution builds up on my other libraries (as git submodules) grouped here:
https://github.com/bmegli/hardware-video-streaming

A word of caution:
The encoder uses lossy video codec for depth map encoding. It will not work perfectly. See this issue for the details:
https://github.com/bmegli/unity-network-hardware-video-decoder/issues/13

Note for users:
The code currently works on Unix-like operating systems only (e.g. - Linux) and with Intel on encoding side (HEVC Main10 encoding through VAAPI).
See also:
https://github.com/bmegli/unity-network-hardware-video-decoder/wiki/Hardware

Errata:
In bandwidth scene it is 1 Mb/2 Mb/4 Mb/8 Mb (not 8 MB)

Meta information (on making the video)

Public Domain music used (thank you!):

After the rain by Rexlambo https://soundcloud.com/rexlambo

Creative Commons — Attribution 3.0 Unported — CC BY 3.0
Free Download / Stream: https://bit.ly/_after-the-rain
Music promoted by Audio Library https://youtu.be/IcH1Qc7NkBo

Polaroid by extenz https://soundcloud.com/extenz
Creative Commons — Attribution 3.0 Unported — CC BY 3.0
Free Download / Stream: http://bit.ly/2RjYE6U
Music promoted by Audio Library https://youtu.be/YnipBtKvnAo

Level Plane - Riot
Creative Commons — Attribution 4.0 International — CC BY 4.0

Birds by Scandinavianz | https://soundcloud.com/scandinavianz
Music promoted by https://www.free-stock-music.com
Creative Commons Attribution 3.0 Unported License
https://creativecommons.org/licenses/by/3.0/deed.en_US

Takayama by Niwel https://soundcloud.com/niwel-516897768
Creative Commons — Attribution 3.0 Unported — CC BY 3.0
Free Download / Stream: https://bit.ly/_takayama
Music promoted by Audio Library https://youtu.be/fBI4-eTBMqo

Public Domain sound effects used (thank you!):

The Slow Mo Guys slow motion sound effect https://www.youtube.com/watch?v=a4Vu5latvTI

Hardware:

Video recording/photographs: Nikon D5300 (DSLR)
Samsung S9+ (slow motion in "The Measurement" scene)

Voice recording ("The Overview" scene) Behringer C2 condenser microphones and PreSonus AudioBox iTwo hardware interface

Software:

FFMpeg with OpenGL flipping disabled in X Server Settings.

ffmpeg -video_size 1920x1080 -framerate 30 -f x11grab -i :1 -draw_mouse 0 -c:v libx264 -crf 0 -preset ultrafast ~/Videos/$1

Voice recording: PreSonus Studio One Artist DAW
Video editing: VSDC Free Video Editor

Meta meta information: (on making the materials for the video)

All depth maps were grabbed with Realsense D435 at 848x480@30 FPS without any post processing.
Some were streamed remotely between:
- LattePanda Alpha
- Laptop
Some were streamed locally (encoded and decoded on Laptop).

Various Depth Units for Realsense were set (including those only available from 5.12.1.0 development firmware).
Various bitrates for depth encoding were used, mainly 2Mbit, 4Mbit, 8Mbit.
B frames were always set to 0 (for low latency, if you prefer quality or smaller size over latency use positive value for B frames).

The voice quality in DSLR recorded videos was obtained by first recording with DSLR to get good voice timings then over-dubbing the voice with condenser microphones . Finally original layer of sound was deleted, leaving only the clapping sound ("The Overview" scene)