Modern Web Weekly #15

Staying up to date with the modern web

Screen sharing like a pro: remote control the captured side with Capture Handle

If you have ever done a screen share and recording of a presentation you know how annoying it can be when you have to switch back and forth between both sides.

Imagine you're in a video conferencing app like Teams and you share your screen that shows a presentation. You want to stay inside Teams to interact with the others in the call but when you need to move the presentation to the next slide, you need to switch to that presentation, change the slide, and then switch back to Teams. You have to do this each time you need to change the slide, which is clearly not very user-friendly.

With Capture Handle, you can now control that presentation from the capturing side so you can stay there, no more switching back and forth!

How Capture Handle works

Each web app that wants to use Capture Handle needs to opt-in, which means that it allows any web app that captures it to communicate with it.

A web app can opt-in through a call to navigator.mediaDevices.setCaptureHandleConfig with a configuration object:

// captured side
const config = {
  handle: crypto.randomUUID(),
  exposeOrigin: true,
  permittedOrigins: ['*'],
};

navigator.mediaDevices.setCaptureHandleConfig(config);

The config object has three members:

• handle: a string of up to 1024 characters that is used to identify the captured web app
• exposeOrigin: a boolean that defines if the origin of the captured web app is available to the capturing app
• permittedOrigins: an array of origins of web apps that are allowed to observe the captured web app. The value '*' indicates any web app is allowed

In the example above, handle is set to a random string. This member is the most important one as handle will be used by the capturing app to identify the captured app.

The capturing side will first get a MediaStream that represents the captured web app and then call getCaptureHandle on the VideoTrack of the stream:

// capturing side
const stream = await navigator.mediaDevices.getDisplayMedia();

const [videoTrack] = stream.getVideoTracks();
const captureHandle = videoTrack.getCaptureHandle();

if(captureHandle) {
  console.log('capturing web app with handle:', captureHandle.handle);
}  

The capturing web app can now initiate communication with the captured web app, for example through BroadcastChannel but you can also use other methods that provide cross-window communication. It can then send messages to the capturing web app that contain the handle of that app and a command. In the following example, the capturing web app opens a communication channel with BroadcastChannel. The captured web app shows a presentation and the capturing web app has two buttons. When the buttons are clicked, a message is sent to the captured web app to move to the previous or next slide of that presentation:

const broadcastChannel = new BroadcastChannel("capture-handle");

previousButton.addEventListener('click', () => {
  broadcastChannel.postMessage({
    handle: captureHandle.handle,
    command: 'previous',
  });
});

nextButton.addEventListener('click', () => {
  broadcastChannel.postMessage({
    handle: captureHandle.handle,
    command: 'next',
  });
});

The captured web app now connects to the same BroadcastChannel and adds a message event handler to listen for the commands from the capturing side:

const broadcastChannel = new BroadcastChannel("capture-handle");

broadcastChannel.addEventListener('message', ({data}) => {
  const {handle, command} = data;

  if(handle === config.handle) {
    switch(command) {
      case 'previous':
        gallery.previous();
        break;
        
      case 'next':
        gallery.next();
        break;
    }
  }
});

This way, the user can remote control the captured web app and doesn't have to switch back and forth anymore.

The VideoTrack also emits a capturehandlechange event whenever the captured web app calls setCaptureHandleConfig or navigates to another URL:

videoTrack.addEventListener('capturehandlechange', e => {
  captureHandle = e.target.getCaptureHandle();
  // do something with the new capture handle
  
});

Capture Handle is supported in Chrome and Edge 102+.

Demo

I added a demo to What PWA Can Do Today and below you can watch a screen recording of it:

Wut? That's it???

Unfortunately, yes...

I've been very short on time this week so this edition ends here. Stay tuned for a full edition next week!