WebKit Port Complications

Blog post by leavengood on Mon, 2007-08-20 23:31

I have been documenting my progress on porting the WebKit project to Haiku on the Haikuware site, but decided to also post information here. You may want to read my previous blog article about this port and also the information at the WebKit bounty on Haikuware. Please consider donating to a bounty.

Before I started work on this port I asked in the #webkit IRC channel what the mimimum version of GCC that was required to compile WebKit. The general consensus was 4.0. So what does that mean?

Well let's talk a little bit about BeOS compilers. I won't go into too much history because I don't know it all. What is relevant to the current discussion is the newest GCC which can be used to compile BeOS-compatible programs is GCC 2.95.3 . This version was released on March 16, 2001. Yes it is that old, older even than the Haiku project. The original GCC 2.95 was released on July 31, 1999. Of course this ancient GCC has been nicely maintained and updated as much as possible by Oliver Tappe and others.

The problem with using newer GCCs for BeOS programs is the GCC project changed various important aspects of the compiler which affects how C++ is compiled after 2.95.3. These changes were to fix bugs and design flaws, so they were necessary, but the end result is older C++ programs and libraries will not work with newer compiled code. This isn't a big deal on fully open source systems with well-maintained software. Projects like FreeBSD and Debian Linux went through the "growing pains" of changing to newer GCCs years ago. They had to make sure everything was recompiled and packages all updated, so it wasn't trivial, even for them.

Haiku, on the other hand, has a bigger problem: it needs to support very old, unsupported "abandonware" like Gobe Productive. Though we can dream, it is doubtful the Gobe Productive code will ever be released, or even that someone who has the code will recompile it for Haiku. There may be several other such applications that at least a few people find useful and would like to use on Haiku. So for Haiku to truly be the successor to BeOS which can support these programs, it must use the older GCC 2.95.3.

So how does all this relate to my WebKit port project? Well like I said above WebKit is a modern code-base and it requires a modern compiler, probably at least GCC 4.0 (released April, 20 2005, not exactly brand new.) In fact this weekend I attempted to compile JavaScriptCore with GCC 2.95.3 and it failed almost immediately. The preprocessor couldn't handle even a fairly simple macro in the Platform.h header file, where various parameters for the WebKit-supported platforms are kept. Now I know this problem can be worked around. I'm sure the many other errors I would encounter could be hacked around too. So why don't I just do this? Several reasons:

  • It will be tedious, frustrating and could take a very long time.
  • The resulting mess would not ever be accepted by the meticulous developers on the WebKit project. This would turn my port into a fork, meaning I would have to manually integrate any changes from the "real" Webkit or maintain loads of patches. I do not want to do this. A clean port will most likely make it into the WebKit SVN repository.
  • It is unnecessary because Haiku can already be built with GCC 4.1.2!

That's right, the Haiku code-base was "ported" to compile on GCC 4.1.2 quite some time ago. Using GCC 4.1.2 requires Linux though, and in fact I had never bothered to use this GCC until I got my new Linux computer which became my default Haiku development machine. This is also why I decided to port WebKit to Haiku on Linux, not BeOS.

So that solves this WebKit problem then, right? Well no, because as I said above a GCC 4.1.2 compiled Haiku would not be able to load older BeOS applications and the whole "backwards compatible dream" would be lost.

But wait, I forsee another solution, of course (hey I'm a problem solver.) I believe it is possible to create a "legacy" sub-system in Haiku to run older BeOS applications. All the core libraries that programs need when running, like libroot.so, libbe.so, etc. would be compiled with both GCC 2.95.3 and GCC 4.1.2. Newer applications would have a special code embedded in their ELF header to indicate they are newer. The runtime_loader, which is responsible for loading applications, would detect this code and link the application with the newer GCC 4.1.2 libraries. If the application does not have this code, the loader assumes it is a legacy application, and it is linked with the older GCC 2.95.3 libraries. Now there may be a flaw in this whole idea, but I believe it is worth trying.

But for the moment this issue can wait, because my priority right now is to get WebKit ported. I can continue on this work by compiling my own Haiku with GCC 4.1.2 to test the port, which is exactly what I've been doing.

My current issue is trying to figure out how to compile external libraries with the Haiku GCC 4.1.2 compilers. The Haiku build system is pretty well designed, but is specialized to do what it does: compile Haiku using the Jam build tool. I integrated WebKit into this system by writing Jamfiles for it, but that really isn't an option for all the various libraries WebKit depends on. So I need to find a way to either use the current compilers or compile my own for this specific purpose. Dealing with compilers isn't trivial so this may take me a while.

But I will report that I got JavaScriptCore to compile successfully last week. But I need to test it and that is what requires I compile external libraries (specifically I'm trying to compile ICU, the International Components for Unicode library.) If anyone has any tips or tricks, please let me know! :)

Comments

Re: WebKit Port Complications

Personally speaking, I think you're going in the right direction. The binary compatibility, while a worthy goal back in 2001, is no longer a good goal if all you're trying to get is Productive to work on Haiku. It's been a good goal for setting a standard for the code to adhere to. The rest of the apps that are commercial are either abandonware or actively developed (e.g. WonderBrush). I really doubt that it's going to be an issue when we're ready for R1. By then, the dust and cobwebs will just be thicker. Coming up with Jamfiles for the dependencies might be a worthwhile, if not tedious, effort -- it would make integration into the tree *much* easier. WebKit integrated with Haiku? Ouch. That sounds almost so nice that it hurts. :) I wish you the best of luck.

--Jon

Re: WebKit Port Complications

darkwyrm wrote:

Coming up with Jamfiles for the dependencies might be a worthwhile, if not tedious, effort -- it would make integration into the tree *much* easier. WebKit integrated with Haiku? Ouch. That sounds almost so nice that it hurts. :)

Well start hurting, because I already wrote working Jamfiles for JavaScriptCore. I plan to do the same for WebCore, though they will certainly be more complicated. But still not that bad.

My idea is for someone to be able to download the WebKit SVN into the 3rdparty directory, add a UserBuildConfig I'll create and then "jam webkit". Easy peasy.

My browser shell will probably end up in the Haiku SVN, and until then it will be as easy to use as WebKit. Just plop it in 3rdparty.

I plan to make the other dependencies (which do not change nearly as much as WebKit) into binary downloads. Things like ICU, libxml2, curl, etc.

Re: WebKit Port Complications

I'd personally rather the deps were in the tree also...

If anything - because the underlying OS apis may change in the future - not to mention, you never know when someone will be porting to PPC, X86-64, etc...

Build systems

If external dependencies require its own build system anyway, why bother with Jam for WebKit? As you wrote in your previous blog post, every port currently has its own build system. IMHO that’s a bit messy. How about working with the KDE guys and use CMake as WebKit’s build system on Haiku?

Re: WebKit Port Complications

Firefox is the same, we are the only ones using an older compiler. So far it's been quite easy to fix these issues. It appears though as our compiler is more strict than the newer ones, so most errors is just small corrections.

Here is a typical one:
https://bugzilla.mozilla.org/show_bug.cgi?id=365118

Re: WebKit Port Complications

I agree with DW that binary compatibility sounded like a good idea when the project started but is of less interest now. In some ways I think it would actually be better to release R1 with GCC 4 - otherwise it adds to the impression that Haiku is out-of-date before it's even released (supported by the "why clone a 7 year old OS?" crowd).

Another issue is that I believe mozilla are dropping support for GCC 2.95 (along with getting rid of the platform-specific widget code and adding lots of extra dependencies) which will make getting later versions of Firefox running on Haiku non-trivial also. I think this is a debate for the mailing list though.

On the dependency issue - everything else is in the tree as it ensures a stable base common to everyone. Things like AGG, Freetype, Mesa, glib, gdb, etc, etc all have a snapshot of their code in the tree. I don't really see why WebKit should be any different.

Re: WebKit Port Complications

Well we can see about truly integrating WebKit into Haiku when the port is further along. But WebKit itself is still a huge moving target, with 10-20 commits a day, more even than Haiku. When it stabilizes a bit more I would be more willing to integrate it. Still though I question the logic of integrating every interesting open source technology into our tree, when they have their own perfectly good trees. I guess it depends on how "core" the technology is. If I or someone else creates a BHTMLView that uses WebKit I suppose we won't have much choice, especially if Mail or a future help application make use of it. I guess I can't complain if my "pet project" were to become integrated like that.

As per build tools, at first I was going to try to use one of the existing systems in WebKit. But since I wanted to use the Haiku cross compilers I wasn't quite sure how to use those with other build systems. Plus for future integration Jamfiles are better. Of course since I need to solve the first issue for ICU and other libs, I suppose that is no longer a valid reason. Still the Jamfiles (for JSC at least) aren't even that complicated, so I don't think it is a big deal.

Re: WebKit Port Complications

If core Haiku doesn't eventually come with a web browser of some sort, it won't be taken seriously.

If WebKit is too big, I would recommend putting something like FlyingTroll into the tree and adding network support to it since it already does basic HTML rendering (good enough for documentation at least).

It seems like once WebKit is integrated into the Haiku build system, all you should have to do is update the relevant code files from the official WebKit repo...

For example, how Haiku currently updates the coreutils, or Mesa, etc.

Anyhow, those are obviously issues that don't need to be dealt with until the port actually works ;)

Re: WebKit Port Complications

Hmm, for sure it's a shame that Gobe Productive is abandonware but I'd rather have GP then any web stuff because for the small things I need an office suite for it does it's jobs excellent.

However it means we won't be able to use Web kit on R5 I hope I assume correctly and for the time being I use R5 as a (more or less) stable base with Haiku things in it.

The legacy and newer compiled stuff on one OS sounds a lot like how MicroSoft does it in their Windows hope it won't be as shit as their crap.

Nice article, made me a bit more knowledgeable which I appriciate (as well as all Haiku and OSS related efforts).

Best of luck!

Re: WebKit Port Complications

"hope it won't be as shit as their crap."

wow there's some words of encouragement :P ... seriously though, could the 2.95 GCC stuff not run off some subsystem, kinda like how FreeBSD runs Linux apps via LBC ?

If it could that would mean a way forward with legacy support. It's almost impossible to