Diving into WebKit

First of all, I want to thank Haiku, Inc. for giving me the opportunity to concentrate fully for a while on the WebKit port and browser! This is an awesome chance that I intend to make full use of.

At the moment, I have mixed feelings. Not about writing blogs. Not about working on WebKit. But about using the new WebKit browser to write the blog entry, haha! I've seen it crash, although in the last days, it has become pretty stable. After we upgraded to a newer WebKit version as the basis for the port, the frequent random crashes have almost disappeared and I saw only one crash in three days. Compared to one every few minutes before.

But still, I am of course pushing it a bit by eating my own dog food so early on. Lucky for me, I have integration with the system clipboard already working, so I can at least copy & paste my writing into Pe from time to time.

I'll try to give an overview of how I came to work on the WebKit port and what I've done so far. Basically, I woke up one day and thought Haiku really needs a modern, efficient and well integrated browser - now. In the beginning it wasn't clear to me yet, what the best approach for this would be. I started out with NetSurf and looked into that port. Francois has done some solid work there, but the flickering annoyed me and I've looked a bit into how this could be fixed. At the same time, I tried a few websites and it just wasn't convincing to me that NetSurf would be a full featured browser soon. It's certainly an awesome project, but it looked to me like it's not in the same advanced state as other engines. So I've started to consider Firefox and WebKit. For Firefox, I've heard that the problem is the Cairo port for Haiku. So I've downloaded the Cairo source tree and pretty much rewrote the BeOS port under the presumption that the existing Cairo software rendering backend could be used to draw into BBitmaps to get an instantly fully compliant implementation. Then however I failed to get the Makefiles set up correctly for building the testing framework. At the same time, it became clear through talks with Fredrik Holmqvist, that porting a current Firefox would be much more work than just porting Cairo to Haiku.

So WebKit began looking more and more attractive. I've quickly learned that Maxime Simone was keeping the WebKit port updated since his work on it during the Google Summer of Code 2009. I checked out the repo and began working on getting it to compile. Maxime was stuck at some linking problems, but after a while I figured out what those were and then Maxime helped me get the rest of the port into compilable state again. After I finally looked at the HaikuLauncher, which is the testing application to embed a WebKit view into an application, render a website for the first time, I've been working on the port and the browser ever since.

I think the infrastructure for ports in the WebKit source tree has improved a lot since Ryan Leavengood did the initial port in 2007. One of the last things that Maxime did was enabling the pthread threading backend. When I ran the HaikuLauncher for the first few times, I ran into all kinds of asserts, because WebCore code was executed on the wrong thread. My first work was therefor to design a proper threading concept for the WebKit code. Maybe it's a good time to mention that WebCore and JavaScriptCore are the core frameworks, the actual web engine. WebKit is a thin layer on top of WebCore that provides an API for embedding and controlling WebCore on each respective platform. So WebKit is where our BWebView and BWebPage will live, while WebCore is the actual backend. The Haiku port is thus separated into these parts. Three parts actually: There are all the files in WebCore which provide platform native implementations for various aspects of WebCore.There is for example GraphicsContextHaiku, which implements the WebCore rendering backend API via BView drawing functions. In WebKit, there is two sets of files. One part is the actual WebKit public API that the Haiku port provides, the other part are the so called WebCore support classes which bridge between WebCore and WebKit.

Work had and has to be done in all three parts. Some stuff was already finished from when Ryan worked on the port, other things were finished by Maxime, but a lot of the code was still stubbed out. Lucky for me, the port also consist of it's own jam based build system, which I presume Ryan wrote. It's nice not having to deal with that as well. After I fixed and implemented a great deal more code, HaikuLauncher became somewhat usable for surfing, and more people began to be excited about the WebKit browser. Michael Lotz has been crucial in tracking down some hard to fix memory corruption bugs and in extending the CURL network layer backend in WebCore to support htdigest authentication. Rene Gollent has implemented FileChoser support and tabbed browsing in HaikuLauncher. It was of course very encouraging to get support from so many people. Each day, the browser became more usable. And sometimes fixing one little thing, even if it was very hard to track down, can suddenly enable a whole bunch of websites.

So far I think my choice to invest into WebKit was a good one. It's possible to make a very native feeling browser, although some compromises have to be made. While WebCore is essentially single threaded, at least the native drawing happens in one app_server thread per page. And this will of course improve over time, as WebCore itself will make more use of threading. In terms of feeling native, I believe HaikuLauncher has left the Firefox port far behind. It even renders faster in many situations, sometimes drastically faster.

Today, I've finally found the problem with frames and iframes not being loaded and you can thank Ingo Weinhold and praise his source level debugger for that. This too made a great deal of sites work suddenly, like maps.google.com, the full version of mail.google.com, www.gmx.net finally works... and I've found the cause of some annoying clipping problems. If you want to give the HaikuLauncher a spin, you need to run a GCC4 or hybrid Haiku system. And it needs to be pretty recent, since we've also fixed bugs and added features in Haiku to support stuff for WebKit and HaikuLauncher. Have fun! But keep in mind it can still crash. Just because I managed to finish this blog doesn't mean you should trust it, yet. :-D

Download WebPositive (SVN revision: 444)

P.S. I've stolen the awesome icon from Meanwhile's great BB icons package.