WebKit weekly report #21

Blog post by PulkoMandy on Fri, 2014-02-28 07:48

Hello there.

Well, the good news first: for some time I had a bug with GMail, where the top part of the interface (with the search bar, trash button, identity and GMail logo would disappear after the page loaded. This is why I didn't do any release in a while. Well, this bug is now mostly fixed. There is some flickering of the same area, but at least it doesn't completely disappear. I'll be researching the flickering, however it isn't an usability problem anymore, so I can package a release with all the improvements done over the last weeks.

WebKit weekly report #20

Blog post by PulkoMandy on Fri, 2014-02-21 08:10

Hello everyone!

So, as advertised last week, I spent some time running the testsuite again. And as usual, it helped spot and even fix a few bugs.

WebKit weekly report #19

Blog post by PulkoMandy on Fri, 2014-02-14 07:37

Hello everyone!

This week I worked on stabilization and small improvements of WebKit. There are a few new features, as well.

The crash with cursors I mentionned last week is fixed. I had forgotten to copy an object in the copy constructor, leading to a double delete. I continued working on the clipping code, and fixed the issues with www.haiku-os.org and a few other websites. But, I can't get it to work with haikuports, Trac, and now gmail is also broken. I don't want to do a release until we have a fix for that.

I also did some more work on supporting css shadows. With the updated clipping code, the ugly black box that was sometimes visible is now gone (it's clipped out). However, we also must draw the shadow itself. I did most of the implementation on WebKit side, but it needs support for a drawing mode (SourceIn) that app_server doesn't handle yet. So, I left that disabled for now, until we can get it to render the expected way.

An easy fix was adding the support for HTML5 "file" API. I just had to turn a compile flag on, and this gives us 10 more points on html5test.com. It also gets imgur.com image upload working.

I merged some changes from WebKit, without much problems. This brings the usual small fixes and cleanup, without too much code breakage this time. There are a lot of code cleanups going on at WebKit, making the codebase simpler and also faster.

I also did some work on the Network Kit HTTP backend. We now support gzip/deflate compression of HTTP data. This make some web pages load much faster, and also fixes issues with some websites serving compressed data even though the browser doesn't advertise support for it. While working on this, another problem in the HTTP code was discovered, there was a possible stack overflow because we were using a gcc extension to C++. This is now fixed, hopefully improving stability of the web browser.

With all those small issues fixed, it's time for another testsuite run. I hope I can find some test that fail because of our remaining clipping problems, as this would help me identify the issues more easily than with a complex web page such as gmail. I'd really like to squash at least the new issues introduced by this clipping change, so we can have a release that I'd qualify as stable. I'm trying to not start work on too much other features until we get this sorted out. Once that release is done, we can resume way on features that need more changes.

Next up on the TODO list: support for affine transformations (stippi added this to the app_server), which will improve SVG rendering a lot and possibly fix some other issues. Support for shadows, and the missing SourceIn composite drawing mode. When we get these two out, we should have much better rendering. There are probably other missing features, but they are yet to be identified.

See you next week!

WebKit weekly report #18

Blog post by PulkoMandy on Fri, 2014-02-07 10:17

Hi there!
As you can read on the frontpage, I'll continue working for Haiku in february.This will be the 5th month of this contract. Thanks to everyone who donated to Haiku, Inc for making this possible!

So, I've sorted out my filesystem issues over the week-end (no important data was lost), and I'm back to full-speed work. As I was saying last week, we had a problem with gcc4.7 not compiling the most recent WebKit code. I expected an update to gcc4.8 to solve this, but it didn't. What was needed is an extra configure option to enable C++11 threads support, as WebKit started using that and gcc doesn't autodetect the required OS support.

So, I rebuilt gcc with the needed option, and could get WebKit updated again, merging the work done at WebKit in december and january. As usual, not much crazy new features, but a lot of refactoring and cleanup. The API to manage the mouse cursor was changed to a simpler one that wastes less time and memory allocating stuff, some compile-time options were removed as all ports used the same value, and some custom classes were replaced with C++11 standard equivalents. This is made possible because of the release of a new version of Visual Studio, which still lacked some of these features. Of interest to us is the use of C++11 override. This comes from Java and allows to tell the compiler that a given method in a class should replace one from a base class. If it doesn't, you get a compile error. This is very useful in WebKit, as it allows detecting when the base class API changed (method removed or renamed, parameters added or removed, types changed, ...). I started adding the "override" keyword to some of the Haiku specific classes, and could remove a dozen of useless methods. This is one little change that will make further upgrades much easier. Another change is the deprecation of the history API we were using. This was the occasion to clean up our old code for this and get the back/forward buttons to work more reliably.

So, I started testing the new WebKit and noticed it was very crashy, with testsuite results as low as 4000 passing tests out of 32000 (ouch!). At least part of this was found to be caused by stricter stack alignment requirements on gcc side. gcc4.8 started using more x86 instructions that need 16-byte stack alignment. Before this only happened in some well-defined parts of the code, and I could fix this on a function-by-function basis. Now, all the code using floating point numbers is potentially affected. I can work around this by compiling all of WebKit with the -mstackrealign option, however this is something that should be fixed on Haiku side. Fortunately, the fix shouldn't be too complicated, it's just a change of the alignment constraint we have to do when creatign a new thread. If the initial alignment is correct, gcc generates code that always preserves it, unless you have some non-gcc compiled code in your function call stack (hand-written assembly code is one possible case of this).

There is apparently another crash related to the new mouse cursor code, which I haven't investigated yet. With these two out of the way, I'll have to run the testsuite again and see if there are other problems. With so few passing tests, the result html page (which list everything that failed) is too slow to browse and barely useable.

On to the new features now: it was more than time we fix the drawing glitches known as "border bleeding". You probably have noticed this problem on the side menu of this very website. There are some other places affected by this. Anyway, stippi did an amazing job of implementing ClipToPicture the right way. We now have a very fast implementation that also supports antialiasing. Bridging the gap between old and new apps, this improves the situation for both Gobe Productive (one of the few apps to use this API in BeOS days) and WebKit.

I had the code using ClipToPicture mostly ready in WebKit, waiting for the working implementation in Haiku. I could finally test it this morning and... well it doesn't work perfectly, yet. While it fixes the border bleeding, and we get our gradients where they should be again, when scrolling the page too fast (with pageup/pagedown buttons for example), the text above the gradients isn't drawn at all. Other websites also get new drawing problems in similar situations. I'm not sure what happens yet.

While stippi was working in app_server internals again, he also started implementing arbitrary view transforms. We had most of the API ready, with the BAffineTransform class available but only used to transform BPolygons. You can now set a transformation on a view and arbitrary rotate, scale, translate and otherwise distort all the drawing. This is all new and not yet completely tested (and in fact, there are some known bugs). But, it will allow a huge improvement of WebKit SVG rendering once it gets plugged in WebKit's GraphicsContext class.

I'm also trying to get some other devs into WebKit development, as working alone isn't fun. I've opened one "easy" issue on our bugtracker. It's about implementing Web Sockets support. There are some other things I would like to see done by others, for example support for web notifications using the BNotification API. Wouldn't it be nice to have a pop-up showing messages from gmail web page show next to the deskbar? Send in your patches!

WebKit weekly report #17

Blog post by PulkoMandy on Fri, 2014-01-31 08:50

Hello everyone!

The work started last week on ClipToPicture made some progress this week. We discussed this further with Stippi and now have a solution that doesn't involve rewriting half of app_server code, and is also a bit simpler and faster than what I tried to do first. I wrote a test application and some boilerplate code, then Stippi jumped in and implemented the missing bits. There are still some missing features like the ability to stack multiple clippings using PushState/PopState, and some problems when scaling and translating the view, as expected. We also met a drawing glitch when moving or resizing the window, however, we're not sure what's happening yet.

With Haiku switched to gcc4.8, I tried updating our WebKit to a newer version again. But, this doesn't work yet, and it seems the problem is a missing option in our gcc configure script invocation. I wanted to rebuilt gcc with the proper options, but I hit some filesystem corruption on my data partition. I'm now trying to backup everything, but a bug in Haiku makes this incredibly slow. Of course, I paused my contract since wednesday, and until I can get this issue sorted out and resume working. No data was lost, but touching some files on that partition triggers a KDL. So, it's time I reformat it and put the data back on it.

As a result of these FS problems, I haven't got much work done this week. So, this report is short.

WebKit weekly report #16

Blog post by PulkoMandy on Wed, 2014-01-22 11:33

Hello world!

As I said last week, the remaining drawing glitches are because of BView limitations. Well, it's time to solve those as well!

I'll start with what is now known as the "border bleeding" bug. You have encountered it if you tried opening the Haiku website, or the bugtracker, in Web+. You will easily notice that some items are completely filled with the border color, instead of the expected background one. To understand what's going on, let's have a look at the way WebKit draws things.

WebKit weekly report #14

Blog post by PulkoMandy on Fri, 2014-01-10 08:34

Hello there,

Well, somehow quiet and regular activty this week. Not too much exciting things, but progress is being made.

I updated WebKit to early december version. This is not the latest one, but the guys at WebKit started using even more C++11 as Visual Studio on Windows finally gets more support for it. So, enter std::chrono and some std::thread stuff. Unfortunately, our version of gcc4 seems to be missing some of these. I'm now closely watching the work of korli and mt to bring us gcc4.8.

I made some long overdue cleanup to the build system, removing some manual (and undocumented) steps and fixing some bugs in the process. The about box will now report the proper WebKit version, once again using the standard CMake scripts to get it, instead of a custom shell script. Moreover, this seems to have fixed Google detecting us as a mobile browser. Some other websites still show that problem, however.

The work on completing the testsuite expectations continues. The progress is slow, as reviewing each of the tests, and looking for possible solutions before marking them all as failing is a very time consuming process. It is also a bit boring and unrewarding, so I'm also spending some time looking at some other things.

I attemted to fix the lockup issue. If you use Web+ you probably have noticed that it oftens becomes unresponsive and seems frozen for a minute or so. I now know what's happening: this triggers when you try to navigate away from a page thet's still loading, or performing an XmlHttpRequest or some other kind of network activity. If the socket is blocked on a connect() call, we have no way to unlock it from there. What WebKit does is it tries to close the connection, and the way it's implemented in our HTTP backend, we have to wait for the network activity to stop, before we can free the connection object. If we free the object without waiting for the thread to terminate first, the thread will crash when the connect call finally times out.

There is a well-known trick to avoid this issue, making the connect non-blocking, and then using select or poll to wait on it. Unfortunately, the Haiku network stack has some bugs that prevent this from working. I'm not yet up to speed on that area of the Haiku code yet, but I'll try to improve the situation.

On the rendering side, I'm also hitting various limitations of the BView API. First of all, the latest build I uploaded last week introduced two new rendering glitches, but fortunately I could get them fixed without reintroducing the box-shadow bug htat led to black rectangles drawing all over the place.

The missing bits in BView currently are support for transform matrices, which are used a lot for drawing SVG and maybe canvas elements ; and support for arbitrary clipping. I had a try at implementing the latter using ClipToPicture, but this lacks antialiasing and is implemented in a suboptimal way, leading to laggy scrolling. It also has problems with the way scrolling is implemented in WebKit, because the clipping is itself clipped (doh!) to the visible part of the view. So, when you try scrolling, it isn't updated and prevent things from drawing if they weren't already visible before scrolling.

I discussed these BView issues with Stippi and there is clearly no way to solve these issues without improving the BView API to add support for arbitrary transformations and clipping shapes. While the initial implementation of this shouldn't be too hard, there may be a few difficulties, for example with the way view coordinates are converted to window or screen ones. It may be a bit tricky to get this all working well.

There are other issues that can be solved only on the WebKit side, for example the history navigation problems. I'm not sure why this is happening, but sometimes, clicking on a link doesn't add it to the navigation history, making the next/previous buttons a bit unreliable to use.

Finally, there are a few known crashes that I'm still tracking. One of them happens when decoding big images or animatd gifs, another one when trying to attach a file to a mail on gmail, and one happens when trying to stop an https connection, and tends to be triggered easily when navigating gmail or github. I'll try to get these crashing issues fixed before I start playing with new features, so we can at least have a stable version of WebKit to do some web browsing, even if some features are still missing.