Support for MTAP16 and FU-B NALUs in H264 depacketizer for OBS support

[biglittlebigben]

Summary

The LiveKit team has received reports of some LiveKit Ingress sessions coming from OBS failing mid stream. Logs show the following messages out of the Pion depacketizer:

NALU Type is unhandled: 26
NALU Type is unhandled: 29

This seems to indicate the OBS is generating MTAP16 and FU-B NALUs, that the Pion H264 depacketizer doesn't support.

Motivation

This seems required to support media coming from OBS 30.1.2, which seems like a fairly common use case for Pion.

[Sean-Der]

libdatachannel's RTP packetization code is here

I don't believe it has support for MTAP16/FU-B (not familiar with either though)

Is this reproducible? Does it consistently happen for a certain customer?

My guess is the H264 bitstream is unexpected, and is below the fragment size. Falling into this case here maybe?

[biglittlebigben]

Indeed, it looks like libdatachannel cannot generate such NALUs. It's possible we're dealing with corrupted data. We'll dig deeper on our side. Feel free to close (happy to close myself too if you rather).

[Sean-Der]

I would leave it open! I am not sure what the problem is yet, I don't have any good ideas on what to debug next :(

If you are able to get more details from that customer would love to dig in.

[biglittlebigben]

My current theory is that the corruption happens somewhere on the server logic above Pion, but indeed, I'm not sure yet.

[biglittlebigben]

Latest we have on this is this is that we seem to receive sporadically a H264 packet with a valid RTP header, but an invalid payload (no valid NALU header) out of WebRTC.TrackRemote.Read. The source is OBS 30.1.2. I could repro on OBS 30.1.0, but not on OBS 30.0.x. I could also not reproduce over localhost, only when OBS connects to a remote server...

To me this points to either an issue in pion/webrtc, OBS, or our integration (in an odd way since we seem to get corrupted data directly out TrackRemote.Read). Either way, this doesn't look like an issue in pion/rtp.

Being able to dump payload out of OBS before SRTP level encryption, or any other method to look at the (unencrypted) raw data coming over the wire would help narrowing this down.

Attaching a raw dump of the buffer returned by TrackRemote.Read

rtp.zip