Blogs

Hello there,

Well, somehow quiet and regular activty this week. Not too much exciting things, but progress is being made.

I updated WebKit to early december version. This is not the latest one, but the guys at WebKit started using even more C++11 as Visual Studio on Windows finally gets more support for it. So, enter std::chrono and some std::thread stuff. Unfortunately, our version of gcc4 seems to be missing some of these. I'm now closely watching the work of korli and mt to bring us gcc4.8.

I made some long overdue cleanup to the build system, removing some manual (and undocumented) steps and fixing some bugs in the process. The about box will now report the proper WebKit version, once again using the standard CMake scripts to get it, instead of a custom shell script. Moreover, this seems to have fixed Google detecting us as a mobile browser. Some other websites still show that problem, however.

The work on completing the testsuite expectations continues. The progress is slow, as reviewing each of the tests, and looking for possible solutions before marking them all as failing is a very time consuming process. It is also a bit boring and unrewarding, so I'm also spending some time looking at some other things.

I attemted to fix the lockup issue. If you use Web+ you probably have noticed that it oftens becomes unresponsive and seems frozen for a minute or so. I now know what's happening: this triggers when you try to navigate away from a page thet's still loading, or performing an XmlHttpRequest or some other kind of network activity. If the socket is blocked on a connect() call, we have no way to unlock it from there. What WebKit does is it tries to close the connection, and the way it's implemented in our HTTP backend, we have to wait for the network activity to stop, before we can free the connection object. If we free the object without waiting for the thread to terminate first, the thread will crash when the connect call finally times out.

There is a well-known trick to avoid this issue, making the connect non-blocking, and then using select or poll to wait on it. Unfortunately, the Haiku network stack has some bugs that prevent this from working. I'm not yet up to speed on that area of the Haiku code yet, but I'll try to improve the situation.

On the rendering side, I'm also hitting various limitations of the BView API. First of all, the latest build I uploaded last week introduced two new rendering glitches, but fortunately I could get them fixed without reintroducing the box-shadow bug htat led to black rectangles drawing all over the place.

The missing bits in BView currently are support for transform matrices, which are used a lot for drawing SVG and maybe canvas elements ; and support for arbitrary clipping. I had a try at implementing the latter using ClipToPicture, but this lacks antialiasing and is implemented in a suboptimal way, leading to laggy scrolling. It also has problems with the way scrolling is implemented in WebKit, because the clipping is itself clipped (doh!) to the visible part of the view. So, when you try scrolling, it isn't updated and prevent things from drawing if they weren't already visible before scrolling.

I discussed these BView issues with Stippi and there is clearly no way to solve these issues without improving the BView API to add support for arbitrary transformations and clipping shapes. While the initial implementation of this shouldn't be too hard, there may be a few difficulties, for example with the way view coordinates are converted to window or screen ones. It may be a bit tricky to get this all working well.

There are other issues that can be solved only on the WebKit side, for example the history navigation problems. I'm not sure why this is happening, but sometimes, clicking on a link doesn't add it to the navigation history, making the next/previous buttons a bit unreliable to use.

Finally, there are a few known crashes that I'm still tracking. One of them happens when decoding big images or animatd gifs, another one when trying to attach a file to a mail on gmail, and one happens when trying to stop an https connection, and tends to be triggered easily when navigating gmail or github. I'll try to get these crashing issues fixed before I start playing with new features, so we can at least have a stable version of WebKit to do some web browsing, even if some features are still missing.

It's been quite a long time since my last report so I think it is a good time to describe what I have been doing in the last two months. The main scheduler logic has been completed and now I am concentrating mainly on bug fixes, adjusting tunables and some minor improvements. I also removed gSchedulerLock, a spinlock I mentioned in my last post, and replaced it with more fine grained locking. An new interfaces for cpufreq and cpuidle modules has been created together with a cpufreq module for Intel Sandy Bridge or newer cores and cpuidle module for all processors that support C-states and invariant TSC. Furthermore, IRQs (including MSI) can be now directed to an arbitrary logical processor. Implementation of inter-processor interrupts has been improved so that it avoids acquiring any lock if it is not necessary and supports multicast interrupts. And, last but not least, 8 processor limit has been removed.

With the advent of package management and hrev46391, it has become possible to prevent a package from being extracted at boot time.From a suggestion of Matt, and with the contribution of Luroh (thanks!), i would like to explain you how to blacklist a package file in Haiku.

In Haiku's boot menu , there is a 'Blacklist entries' option available. This method will only let you disable system packages, and only until the next time you reboot.

You may know, since the package manager has been added to Haiku, some directories are read only, so unlike the past, isn't possible to just delete the driver. The Blacklist functionality addresses the issue that it may be necessary to remove a problematic file such as a library or a bugged driver, which would otherwise require editing the containing package file.

So let's go to the very simple steps to do that :

1. Figure out which file in which package you want to blacklist, especially keep in mind if the file is contained in a system package or in a user one.
2. The second step is to create a text file named 'packages' in /boot/system/settings or in /boot/home/config/settings/global, respectively the first directory is used to blacklist system packages the second is used for user packages.

The final step is to fill the packages file with something like that :

Package 'packagename' {
	EntryBlacklist {
		'entrypath'
		...
	}
}

'packagename' is the name of the package without version, for example 'haiku'.

'entrypath' is an installation location relative path, e.g. "add-ons/Translators/FooTranslator".

This way, blacklisted entries will be ignored by the package_fs. So that they won't appear in the file system.

Let me give a pratical example.

In my case i had the broadcom570x driver to blacklist, so i created the packages file under /boot/system/settings with this content :

Package haiku {
	EntryBlacklist {
		add-ons/kernel/drivers/bin/broadcom570x
	}
}

Then i saved it and rebooted. Once Haiku start the package is re-mounted and the file is ignored by the packagefs.

Hope it was interesting!

Hello world!
You may have heard I'm in for another month!

So, this week I made some more progress with the testsuite system. The major improvements are finished support for the image comparison system I was talking about in last week report. The DumpRenderTree tool now dumps a PNG image of the webpage (this was quite easy to do thanks to the offscreen BView to BBitmap rendering and the translation kit) when requested (not all tests use the feature). Then, another tool called ImageDiff reads two images, and computes the difference. If the difference is big enough, it generates a new image highlighting areas that don't match.

I had to re-add the x86 translators to the x86_gcc2hybrid build so I could make use of them in the gcc4-only WebKit. Fortunately this wasn't too hard either, with just some jamfile tweaks required to make it work again.

The other thing I worked on on DRT side is support for saving crash reports. The problem is, several tests are crashing the DumpRenderTree tool. Without further handling, this results on the testsuite waiting on the debug_server alert asking wether to save a debug report, kill the team, or open Debugger. It is quite annoying to click "kill" an hundred of times as the testsuite runs. So, I'm now catching the signals (SIGSEGV and SIGABRT) from DRT, and from the signal handler I ask Debugger to save a crash report for the current thread. I also modified the python side of the testsuite so it reads reads these reports and move them to the right place. They are now stored with the test results, renamed to the name of the test that made things crash. This is quite useful to see what happened, and execute the crashing tests manually. There's still one problem: Debugger will currently not save any stacktrace, because it does that only for stopped/crashed threads. And since I'm catching the signals, the threads aren't in stopped state anymore when using this trick.

With these fixes out of the way, we're now passing over 52% of the tests. This may look quite bad at first but there are several factors to this, other than failing tests. Most of these come from features that we don't implement yet, such as WebGL, video and audio support, accessibility, battery status (yes, there is a javascript API allowing webpages to query that), web notifications, and so on. There are also a few minor problems like off-by-one size errors that make a lot of the CSS tests fail. And there are strage things like some colors not matching what was requested (a 0,255,0 green gets drawn as 0,254,0 for some reason, making most of the canvas test fail).

The next part of the process is reviewing these results and completing the TestExpectation file. This involves saying things like 'yes, we expect "battery status" tests to fail'. And there are some more fine-grained indications, either "we don't plan to implement it" or "here's the bug in our bugtracker". The testsuite tool can then notify people that some tests passed when they shouldn't have (meaning a bug got fixed somewhere), or on the reverse, some test doesn't pass anymore (so something was broken by your changes). People can update the TestExpectations in the first case, or fix the problems in the second one. There is also a way to tell that we expect some tests to crash the test engine, or to skip them altogether (I use this to skip the one test that crashes app_server).

While reviewing the test results, I also catched and fixed one bug in our WebKit drawing code. This was a corner case of canvas drawing functions, probably not very important, but it would have not been easy to catch without the testsuite. This set of simple pages focusing on a single feature at a time are great to hint you as to where to look for problems.

So, on to analysing the results and maybe fixing some more issues.

On a more interesting note, I finished the support for data: URIs, and added cookie storage for file:// ones. I will provide updated haikuwebkit packages next week including these changes and fixing the easy-to-trigger crashes in the current Web+.