Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bandwidth Estimation (BWE) for subscribers (WIP) #3278

Draft
wants to merge 26 commits into
base: master
Choose a base branch
from
Draft

Conversation

lminiero
Copy link
Member

@lminiero lminiero commented Oct 13, 2023

This is a first experimental draft of my BWE efforts on Janus: although it already "works", it's not in a finished state, since it needs a lot of tweaking and additions. I'm sharing this as a PR mostly to give a pointer to those who've seen (or will see, as I'm writing it hasn't happened yet!) my RTC.ON presentation on the subject.

Slides: https://www.slideshare.net/LorenzoMiniero/bwe-in-janus
Video: https://www.youtube.com/live/n6igQ2J3s80?si=mpvXUcCY0FeCjkp1&t=9804

Rather than implementing any of the existing algorithms (e.g., GCC), I followed the same approach @murillo128 followed for Medooze, meaning I decided to come up with a "homemade" approach myself starting from the fundamental concepts needed for the purpose. I actually need to thank Sergio a lot for that, not only because his effort convinced me it would be a good idea (when you know how things wotk, you can tweak more easily, which is definitely useful!), but also because navigating his code helped me a lot not getting lost in some of those BWE seas. There's other people I need to thank too (especially at the beginning, where I exchanged ideas about how BWE works at its core), but they know who they are!

In a nutshell, this PR creates a new concent, a BWE context that can be added to a PeerConnection in Janus. When created, Janus will try and estimate the available outgoing bandwidth for that handle using TWCC, monitoring the acked rate, losses, delays, determining probing, etc. When an estimate is available, the related plugin will be notified about it, so that it can decide what to do (e.g., switch simulcast layer for that subscriber). Using the Admin API, stats on the BWE context can be saved to a CSV file, and/or sent in real-time via UDP to an external application, where it can be graphed (I created a separate tool for that, that I'll probably share later on for those interested in using it for testing). If you want to learn more, you can check my presentation: I'll probably also write a blog post on all that in a few weeks.

As anticipated, this is not 100% done and is still quite experimental, but all the basics are all there: at the moment, I've partially integrated this in the VideoRoom, where you can test this working with a simulcast subscriber (as long as there's a single video in the PeerConnection). Next steps will be fine tuning this scenario as much as possible, until we're happy with it, which basically means testing it as much as possible in different conditions. Once that's done, it will be a matter of making the plugin integration "smarter" too, e.g., extending the VideoRoom support to multiple subscriptions in the same PeerConnection, SVC besides simulcast, etc.

If this is something that interests you (and it should!), please do take the time to test this! But again, please remember this is experimental: I'm not interested in generic feedback that just says "it works", "it doesn't work", etc. I need feedback I can actually use to improve things, which means providing more info, so please only contribute something if you're serious about testing this new stuff properly.

lminiero and others added 21 commits July 6, 2023 10:57
- parse recv delta as signed int from feedbacks
- detect first recv delta in a chunk
- fix cleanup of pkt queues
- evaluate average delay in a time window of 1 second
@lminiero lminiero added the multistream Related to Janus 1.x label Oct 13, 2023
@Odinvt
Copy link

Odinvt commented Feb 23, 2024

@lminiero Extremely useful. Thanks ! Reading through the blog post and the code, i have a couple of questions if you don't mind :

  • Is this usable in the Streaming Plugin watch handle or not yet ? We're running rooms at scale and the VideoRoom Plugin doesn't allow for that, so the bulk of our subscribers are actually on the Streaming Plugin.
  • Does one have a way to enable/disable this per handle ? I'm asking to avoid using it for subs to pubs not providing Simulcast.
  • Does one have an API to set a target bitrate in order to trigger probing ?
  • Are these events published as handle events like "slowlink" and does the admin handle_info API provide BWE stats ? We have an ELK stack heavily poking this info so that's the only way i might be able to help provide feedback with real users.

Thanks for your time. I apologize if these questions have obvious answers in the code. I tried reading the code but my C knowledge is very limited 🙏.

@lminiero
Copy link
Member Author

  • Is this usable in the Streaming Plugin watch handle or not yet ? We're running rooms at scale and the VideoRoom Plugin doesn't allow for that, so the bulk of our subscribers are actually on the Streaming Plugin.

No, not yet, and I won't integrate it in other plugins until I get enough feedback to consider the work close to done. If you're interested in getting this added to the Streaming plugin, please test the existing integration and provide feedback we can use.

  • Does one have a way to enable/disable this per handle ? I'm asking to avoid using it for subs to pubs not providing Simulcast.

Plugins will have a way to enable/disable this dynamically. In the current version of the code, for testing purposes the VideoRoom plugin always enables it for subscriptions. Again, once we're close to being done, I'll add APIs or configuration tweaks to make it configurable, but that's not what I'm interested in right now. I need to know if and how it works when enabled.

  • Does one have an API to set a target bitrate in order to trigger probing ?

Plugins will have a way to set probing targets but there's no API for that yet, and I'm not sure there will be. At the moment the VideoRoom plugin automatically sets a target for the next simulcast layer you're interested in.

  • Are these events published as handle events like "slowlink" and does the admin handle_info API provide BWE stats ? We have an ELK stack heavily poking this info so that's the only way i might be able to help provide feedback with real users.

The Admin API has the current estimate, while we don't push this event to participants or event handlers yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
multistream Related to Janus 1.x
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants