Write a SocialTV app @ OpenSIPS 2021

1. Having fun with Janus and WebRTC! Lorenzo Miniero @elminiero OpenSIPS Summit Distributed September 13th 2021,

2. Who am I? Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero • https://soundcloud.com/lminiero

3. Just a few words on Meetecho • Co-founded in 2009 as an academic spin-off • University research efforts brought to the market • Completely independent from the University • Focus on real-time multimedia applications • Strong perspective on standardization and open source • Several activities • Consulting services • Commercial support and Janus licenses • Streaming of live events (IETF, ACM, etc.) • Proudly brewed in sunny Napoli(*), Italy

4. ((*) I’m legally obliged to show this)

5. A quick reminder on what Janus is! Janus General purpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://groups.google.com/forum/#!forum/meetecho-janus

6. Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.

10. A ton of scenarios done today with Janus! • SIP and RTSP gatewaying • WebRTC-based call/contact centers • Conferencing & collaboration • E-learning & webinars • Cloud platforms • Media production • Broadcasting & Gaming • Identity verification • Internet of Things • Augmented/Virtual Reality • ...and more!

11. How do you talk to Janus? https://janus.conf.meetecho.com/docs/rest

12. How do you talk to Janus? https://janus.conf.meetecho.com/docs/rest

13. Session, Handles and PeerConnections https://janus.conf.meetecho.com/docs/rest

16. Our practical use case for today: SocialTV!

17. A few requirements for a SocialTV demo • An interesting scenario • A few friends connect to the same web page to watch TV • They can talk to each other while watching • Any of them can change channel at any time • Effective example of combining different plugins • We’ll need something for the TV broadcasting... • ... something to let them mingle ... • ... and something for the interactive features (e.g., channel surfing) • This workshop will guide you through the process • Choosing plugins, combining handles, etc.

20. First obvious choice: VideoRoom plugin! https://janus.conf.meetecho.com/docs/videoroom

21. Chatting with friends using the VideoRoom • The VideoRoom plugin is an SFU (Selective Forwarding Unit) • Publish/Subscribe approach • Participants can send their media and subscribe to others • Very widely used in conferencing scenarios • Obvious choice for the friends mingling feature • They’re basically in a “conference” with each other • We may want to constrain it a bit compared to a traditional conference, though • Maybe a limited number of participants? (e.g., 4) • Capping the bandwidth is a good idea too (they’ll be thumbnails)

24. Sample configuration for the room room-1000: { description = "Social TV Room" secret = "verysecret" pin = "zapp" require_pvtid = true publishers = 4 bitrate = 200000 fir_freq = 10 audiocodec = "opus" videocodec = "vp8" transport_wide_cc_ext = true }

25. Broadcasting with the Streaming plugin https://janus.conf.meetecho.com/docs/streaming

26. Using the Streaming plugin for TV channels • The Streaming plugin is an effective RTP-to-WebRTC broadcaster • It works by creating so called “mountpoints” to identify a stream • A “mountpoint” can receive RTP from any source (FFmpeg/GStreamer/VLC/others) • Multiple participants can subscribe to receive the same stream via WebRTC • Easy to generate WebRTC streams using non-WebRTC tools • Tools only need to understand RTP and support the right codecs • Janus “wraps” RTP in WebRTC coat, but won’t transcode • Browsers need to be able to decode the media • Only need a single copy of the stream • The Streaming plugin duplicates it to interested subscribers

30. Different ways of implementing TV channels • One mountpoint per room? • RTP source of the mountpoint must change when surfing TV • Requires server side management of the media • No need for channel synchronization among participants, but... • ... can’t re-use mountpoints/channels for different rooms • One mountpoint per channel? • RTP source of the mountpoint never changes • Participants change mountpoint they receive when surfing TV • Does require channel synchronization among participants, but... • ... allows channels re-use for different rooms

38. Mapping TV channels to mountpoints • Each channel can be mapped to a separate mountpoint • Independent channels and separate RTP media sources • A cool feature called mountpoint “switching” • If you’re watching mountpoint #1, you can “switch” to #2 • No need to create a new PeerConnection for the purpose, it’s the source that changes • Only works if the two mountpoints were configured with the same codecs, of course • For our SocialTV, different mountpoints can be different TV channels • Participants start with a channel and can change dynamically • Logic used to change channels can be delegated to a different plugin

41. Sample configuration for a mountpoint rtp-sample-12: { type = "rtp" id = 12 description = "Sports" audio = true video = true audioport = 6002 audiopt = 111 audiortpmap = "opus/48000/2" videoport = 6004 videortcpport = 6005 videopt = 100 videortpmap = "VP8/90000" secret = "verysecret" }

42. Sample configuration for another mountpoint rtp-sample-13: { type = "rtp" id = 13 description = "Comedy" audio = true video = true audioport = 7002 audiopt = 111 audiortpmap = "opus/48000/2" videoport = 7004 videortcpport = 7005 videopt = 100 videortpmap = "VP8/90000" secret = "verysecret" }

43. Implementing a shared TV remote • The Streaming plugin can switch from channel to channel dynamically • ... but this only applies to an individual participant • How should we synchronize this among all of them? • A ton of different ways to implement this • Maybe an HTTP/WS based web application (e.g., via node.js) • Could leverage an existing framework (e.g., Firebase) • Why not an instant messaging protocol? (e.g., XMPP) • WebRTC datachannels are an option too, of course!

44. Implementing a shared TV remote • The Streaming plugin can switch from channel to channel dynamically • ... but this only applies to an individual participant • How should we synchronize this among all of them? • A ton of different ways to implement this • Maybe an HTTP/WS based web application (e.g., via node.js) • Could leverage an existing framework (e.g., Firebase) • Why not an instant messaging protocol? (e.g., XMPP) • WebRTC datachannels are an option too, of course!

45. Using datachannels for our TV remote • When looking at datachannels, many options available • Peer-to-peer connections, without going through Janus? • Piggybacking existing VideoRoom streams (audio/video AND data) • Using the TextRoom instant messaging functionality • Writing a new ad-hoc plugin (in C, Lua or JavaScript) For the sake of this workshop, let’s write a new plugin! • Writing one in JavaScript (Duktape plugin) is trivial and informative • Exchanging datachannel messages is a simple logic to implement • We can keep some server-side state too (list of channels, current channels, etc.)

46. Using datachannels for our TV remote • When looking at datachannels, many options available • Peer-to-peer connections, without going through Janus? • Piggybacking existing VideoRoom streams (audio/video AND data) • Using the TextRoom instant messaging functionality • Writing a new ad-hoc plugin (in C, Lua or JavaScript) For the sake of this workshop, let’s write a new plugin! • Writing one in JavaScript (Duktape plugin) is trivial and informative • Exchanging datachannel messages is a simple logic to implement • We can keep some server-side state too (list of channels, current channels, etc.)

47. Writing a Janus plugin in JavaScript https://janus.conf.meetecho.com/docs/duktape

48. Sequence diagram for the interactions

49. Event on existing channels and active one { "channels": { "1": { "name": "Tech", "title": "Meetecho Spot" }, "12": { "name": "Sports", "title": "Napoli Milan 4-2" }, "13": { "name": "Comedy", "title": "SNL - Cold Commercial" }, "14": { "name": "Music", "title": "Iron Maiden - Be quick or be dead" } }, "currentChannel": 1 }

50. Changing TV channels Request (sent by Bob): { "request": "change", "channel": 12 } Response: { "changed": { "name": "Bob", "channel": 12 } }

51. Putting it all together in a web application • Three different Janus plugins used at the same time • Custom Duktape plugin (one handle, datachannel PeerConnection) • Streaming plugin (one handle, recvonly audio/video PeerConnection) • VideoRoom plugin (multiple handles, sendonly publisher + N recvonly subscribers) • Potential workflow for the UI 1 Prompt participant for a display name 2 Use the display name to join as a VideoRoom publisher 3 Create a datachannel with the custom Duktape plugin 4 Create subscribers for other VideoRoom participants in the same room 5 Update UI with list of channels received via datachannels 6 Subscribe to the mountpoint associated with the current channel 7 In case of channel changes, switch the mountpoint source

59. Handles/PeerConnections per participant

60. Time to see the demo running!

61. Thanks! Questions? Comments? Get in touch! • https://twitter.com/elminiero • https://twitter.com/meetecho • http://www.meetecho.com

Write a SocialTV app @ OpenSIPS 2021

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Write a SocialTV app @ OpenSIPS 2021

Semelhante a Write a SocialTV app @ OpenSIPS 2021 (20)

Mais de Lorenzo Miniero

Mais de Lorenzo Miniero (9)

Último

Último (20)

Write a SocialTV app @ OpenSIPS 2021