Comment

WebRTC in Safari 11 and iOS 11

webRTC finally found its way in the safari mac and iOS port of webkit. While it had been in the GTK port for quite some time, based on openWebRTC, the Safari port reused all the bindings and most of the webcore work done by the webrtc-in-webkit project, but used the library from webrtc.org (a.k.a. libwebrtc) for the lower layers. It is great news for the webrtc ecosystem, as now most of the major browsers support WebRTC. That was the number one concern according to several surveys.

Some additional work was needed to incorporate libwebrtc in webkit. Those interested in a review of all the commits one by one, and to track the port of the commits upstream to webrtc.org can look at this document maintained by Dr Alex has he was, and still is helping the project go through. 

First, libwebrtc uses its own configuration and compilation tools (depot_tools, GN, Ninja) which are not only different from those used by the linux port of webkit (cmake) and these used by the safari port (Xcode). The apple engineers first exported the configuration file from GN to a JSON format (using a GN option) and translated that into a CMake files to generate the Xcode project file to reach very quickly a compilable version. Then, the work to start compiling webrtc within the webkit system could start, which required changing a lot of compilation flags, and adding within libwebrtc the same kind of flag to differentiate when libwebrtc is compiled as a standalone library, within chrome and within FireFox. Now a "webkit" flag has been introduced. That took most of January 2017.

Most of February was spent stabilising libwebrtc builds, adding iOS builds, and adding builds options not supported in webrtc.org like support for dynamic libraries and dynamic loading. By the end of February 2017, most of the build were stable, and the C++ webcore classes could start being connected to the libwebrtc C++ classes. The JS APIs and the C++ to JS bindings where already existing and reused from the webrtc-in-webkit project. Both capturers and renderers for MacOS and iOS needed to be created from scratch. By the end of february, the first production builds were tested.

The first half of march was spend adding as many tests as possible, including importing the W3C tests, and enabling the network portion (STUN, TURN, Sockets, ..). By mid-march 2017, fully functional alpha stage was reached, and the ironing of the remaining bugs started. Decision was made at that time to move to a track only design, closer to current stage of the specifications. From there on the focus was on polishing the implementation, and eventually, when everything reached beta quality toward the end of may, add the stats for each objects (yeah ... you want to keep the stats for last .....).

Today webrtc is out there for everybody to use. It is not a full implementation (do not ask about VP8), but it's already very close. While there will be no major release for a year, the safari tech previews go out almost every 2 weeks, and of course, webkit nightly can also be used for the most adventurous.

Youenn Fablet, the main engineer behind the libwertc-in-webkit implementation spoke publicly about their design, and called for feedback during the SF webrtc meet up at Mozilla HQ on June 27th, 2017. The video is accessible at this link. We thanks him again for the recognition of the hard work that was done within this project, mainly by Adam B. from Ericsson Research and Igalia in terms of coding. 

The project is still active, and there should be more news next week, in August 2017, and then in October 2017 as the webrtc webkit developers from all ports (Linux, Safari, Samsung, ...) will get together at the (on-invitation only) web engine Hackfest.

Have fun with webrtc!
 

 

Comment

1 Comment

OpenWebRTC in WebKit Upstream Complete

It's been a long time since the last blogpost, but that doesn't mean that there haven't been any progress. On the contrary, a bunch of new features from WebRTC 1.0 have been implemented and the upstreaming to the official WebKit repository is complete. Let's start by looking at a new feature.

One major update is the introduction of the RTCRtpTranscevier object. It ties together an RTCRtpSender and an RTCRtpReceiver and represents a media description, also known as a "m= line", in SDP. When you do addTrack() and get an RTCRtpSender, an RTCRtpTransceiver is also created automatically. With the transceiver, you also get an RTCRtpReceiver with a track, representing incoming media, that you can start rendering right away, but it will remain muted until you have negotiated media with the remote peer. There's also a dedicated API, addTransceiver(), to work directly with transceivers. This API lets you do more advanced stuff like offering to receive five video tracks without sending any video yourself.

As described in previous blogposts, we have kept our experimental code in a fork of WebKit, but the intention has always been to upstream and work in the main WebKit repository. We are happy to say that we have reached a point where we can retire our fork and continue the development upstream. This means that you will get all our code when you do a regular checkout, or clone, of WebKit. We will keep the fork around for a while since it still contains some experimental code and hacks that we want to keep easily accessible.

Finally, some other great news. Philippe and Alex (Igalia) decided make the WebKit GTK+ MiniBrowser work in the wild with some real WebRTC-based services. As of this time, they have AppRTC and appear.in working.

1 Comment

2 Comments

WebKitGTK+ and Chrome WebRTC Interop

The process to upstream an RTCPeerConnection implementation, based on OpenWebRTC, to WebKit is under way. We are starting off with the platform generic additions to WebCore (see previous blog post). Before we can move on to the actual OpenWebRTC WebRTC backend we need to bump WebKitGTK+'s GStreamer version to 1.6. That work is ongoing and you can read about it here.

But if you would like to test it before it lands in upstream WebKit you can. The WebKit fork, where we keep our work-in-progress code, is publicly available on github. We also provide some simple test applications and I will show you some videos of two of them in action below. If you would like to run them yourself, go to our test repository on github and follow the instruction on how to run the "Peer-to-peer Example". Those steps also contain build instructions, so when completed, you can also run the other test applications.

The Peer-to-peer example is a simple video chat application that connects two browsers. The example code comes with a server that acts both as a web server, to serve the application itself, as well as a signaling server, to exchange messages such as SDPs and ICE candidates between the clients. Setting up a video call between the WebKitGTK+ MiniBrowser and Chrome looks something like this:

If you do not have two webcams, you can try the "One tab p2p" example instead. It uses different setup methods to make calls internally in one browser tab between two RTCPeerConnection objects. Take a look at the source code and compare the simple promise chain and the more complicated setup with the callback based API. The video below shows how media can be set up in one direction at a time compared to everything being done in a single offer/answer dialog as in the previous example:


We will provide more information about the upstream as we make progress. The next step is the infrastructure to run and test createOffer().

2 Comments

Comment

Upstream of RTCPeerConnection started

RTCPeerConnection is the main object in WebRTC for sending media and data peer-to-peer. The API is constantly evolving and a recent trend has been to add accessors to more "low-level" information, such as ICE and DTLS transport information. A major update is the RTCRtpSender/Receiver objects, that let the script have more control over how a MediaStreamTrack is sent, and even replace the track with an other - without negotiation.

Another big API update is the move to a promise based API for the asynchronous function calls. Don't worry, the callback-based API is still supported, but it is considered legacy, and you should really use the promise-based API.

The RTCPeerConnection implementation in WebKit was much in need of an update. That is why we are happy to announce that we have upstreamed a polished RTCPeerConnection API to the WebKit repository. It does not have all of the latest features, but it is in good shape with some of the new additions/changes listed below:

  • Overloaded calls that support both the new promise API as well as the old callback-based one
  • The new promise-based operations queue
  • RTCPeerConnection.addTrack() that replaces addStream()
  • RTCRtpSender/Receiver objects
  • The new "current" and "pending" local- and remoteDescription attributes

In addition to the updated RTCPeerConnection API, we have also upstreamed the PeerConnectionBackend interface. It is a WebCore (which in turn is the library handling layout and rendering in WebKit) interface that lets WebKit ports use different WebRTC backends more easily. So OpenWebRTC could run alongside Google's webrtc.org backend for example.

This is the first step in the process to upstream a runnable WebRTC solution to the WebKitGTK+ port powered by OpenWebRTC. Other ports will also be able to reuse a lot of functionality if they wish.

More information will be posted here as we progress.

Comment

Comment

Updated MediaStream API Upstreamed

The MediaStream API, with its primary interfaces MediaStream and MediaStreamTrack, is the glue of the WebRTC APIs. You use it to direct real-time media from your camera and/or microphone to a media element for rendering or an RTCPeerConnection to be sent over the network. Several other APIs in the Web Platform use the MediaStream API to control real-time media. You can capture video and still images with MediaStream Recording and MediaStream Image Capture and the WebAudio APIs lets you process real-time audio. The MediaStream API is also a control surface where you can temporarily mute audio or make a video track render blackness.

Quite a lot has happened since the MediaStream API was first implemented in WebKit back in 2013. Some things have been removed, for example the specific types for audio and video tracks, and other things have been added and refined.

We recently took a big step forwards in the WebRTCinWebKit project when we upstreamed our updated MediaStream API to WebKit. Some things still need more work, like the Constraints mechanism and the functionality to enumerate local devices, but the foundation is now up to date and testable. 

We will keep you updated on our future progress.

Comment

Comment

Mid-April Updates!

This blog post is long overdue. As WebRTC Expo is about to begin in Miami, I thought it would be a good time to show the progress done until April 15th. There will be a follow up post soon for the progress made from mid april to mid may.

Adam B. (Ericsson) is hard at work on the underlying PeerConnection Implementation and the new promise-based JS API, while Philippe N. (Igalia) is taking care of the media engine, and Lewis L. and Adam T. (Temasys) are handling the DataChannel API and implementation as of now.

The development has recently been opened and moved to github (https://github.com/WebRTCinWebKit) to let more people contribute back through pull request. There is always some delay between the availability of a feature in the latest code in github, and the code that get pushed in webkit because of the additional work to complete the review process. Usually the review process is also slowed down by the lack of available reviewer, but in our case, reviewers from Igalia and Apple have made this their priority and are helping bringing the patches upstream quickly. Right now, the GStreamer library used by webrtcinwebkit and openWebRTC is quite recent, and updating it in upstream webkit has a big impact. That is the main patch under review.

For everybody interesting in the detail and the code, here are the corresponding bugs.

Comment

2 Comments

Bringing WebRTC to WebKit

For years now, the WebRTC developers and users community has been asking for support for WebRTC on more platforms and Operating Systems.  While native app developers can always use an existing stack, namely webrtc.org or OpenWebRTC, web app developers have to either limit their service offering to certain platforms or use browser plugins.           

One important browser engine that does currently not support WebRTC is WebKit. WebKit is the engine behind a number of web browsers, notably Safari and Web (Epiphany), and also powers the iOS WebView as well as numerous other applications.

The goal of this site is to inform the WebRTC community of an ongoing project to bring WebRTC to WebKit, gather feedback, and provide a forum for discussions and contributions.

Let's dig a little bit deeper on the technical side of things. Most browsers have a similar architecture, with an application level (the browser itself) on top of a reusable engine to handle web pages (DOM / CSS / JS). In our case, this reusable engine is WebKit.

There are quite a few things to do in different underlying parts of WebKit to support WebRTC:

- the HTML <video> element (HTMLMediaElement) needs to be extended to support MediaStream as a possible source.

- the WebRTC APIs (as defined by the W3C specifications) must be implemented in WebCore. This is a key part of this project, as it is directly reusable by other WebKit ports without modification to add support for WebRTC.

- the bindings part needs to be able to translate those interfaces in JS for the browser to use. Not for the faint of heart when you want to support both callback-based and promise-based versions of the APIs.

Some work needs to be done at the browser level as well:

- the browser UI part needs to implement all the security prompts and other pop ups we have learned to love and cherish when trying to share one's camera, microphone, or screen.

- Hardware access needs to be done at browser level to be able to share access to e.g. the camera to several tabs at the same time, or to achieve browser-wide echo cancellation.

Some of the above work had been done in the past by several contributors including Google, Nokia, Apple, Ericsson, Igalia, Samsung, at different points in time. However, with the corresponding W3C and IETF specifications still being in flux, the work needs to be kept synchronized. 

Once all this is done, you have JavaScript APIs available from your browser that does ... absolutely nothing. You still need to have a WebRTC stack, implementing all the necessary IETF specifications hooked up to the platform-independent API implementation for anything to work. In WebCore, this is called the platform layer or Back-End. This is where OpenWebRTC (first) and webrtc.org (maybe later) come in.

The focus of this effort for the time being is to:

1. Bring WebCore's implementation of WebRTC APIs (and corresponding UI elements) up to date with the specifications and keep it there.

2. Implement the glue layer needed to use OpenWebRTC as the core of the corresponding platform implementation

3. Implement the browser level changes in the WebKitGTK+ port, on Linux.

And we’ve already gotten started – here is a list of recent WebKit bugs and patches:

Eventually, more ports (browsers) and more WebRTC stacks should be supported as the groundwork will be done and included in WebKit source tree. 

It will take a lot of effort to test, polish and bring the whole project to completion, and of course, to convince your favorite browser vendor to adopt this WebRTC-enabled version of WebKit once it's done, so please contact us if you want to contribute in any way.

 

 

2 Comments