Haiku is a new open-source operating system that specifically targets personal computing. Inspired by the BeOS, Haiku is fast, simple to use, easy to learn and yet very powerful.

Fundraising 2014

Goal: $35,000
  $23,862

WHAT'S NEW IN HAIKU DEVELOPMENT

The Haiku source is continually built and released for testing purposes nearly every day. You can download and install these latest snapshots to check out the latest features and bug-fixes.

Be aware that nightly images may be unstable. Additionally, some packages included with official releases need to be installed separately.

If you're OK with this, you can find further instructions at our Nightly image page.

WebKit weekly report #18

Blog post by PulkoMandy on Fri, 2014-02-07 10:17

Hi there!
As you can read on the frontpage, I'll continue working for Haiku in february.This will be the 5th month of this contract. Thanks to everyone who donated to Haiku, Inc for making this possible!

So, I've sorted out my filesystem issues over the week-end (no important data was lost), and I'm back to full-speed work. As I was saying last week, we had a problem with gcc4.7 not compiling the most recent WebKit code. I expected an update to gcc4.8 to solve this, but it didn't. What was needed is an extra configure option to enable C++11 threads support, as WebKit started using that and gcc doesn't autodetect the required OS support.

So, I rebuilt gcc with the needed option, and could get WebKit updated again, merging the work done at WebKit in december and january. As usual, not much crazy new features, but a lot of refactoring and cleanup. The API to manage the mouse cursor was changed to a simpler one that wastes less time and memory allocating stuff, some compile-time options were removed as all ports used the same value, and some custom classes were replaced with C++11 standard equivalents. This is made possible because of the release of a new version of Visual Studio, which still lacked some of these features. Of interest to us is the use of C++11 override. This comes from Java and allows to tell the compiler that a given method in a class should replace one from a base class. If it doesn't, you get a compile error. This is very useful in WebKit, as it allows detecting when the base class API changed (method removed or renamed, parameters added or removed, types changed, ...). I started adding the "override" keyword to some of the Haiku specific classes, and could remove a dozen of useless methods. This is one little change that will make further upgrades much easier. Another change is the deprecation of the history API we were using. This was the occasion to clean up our old code for this and get the back/forward buttons to work more reliably.

So, I started testing the new WebKit and noticed it was very crashy, with testsuite results as low as 4000 passing tests out of 32000 (ouch!). At least part of this was found to be caused by stricter stack alignment requirements on gcc side. gcc4.8 started using more x86 instructions that need 16-byte stack alignment. Before this only happened in some well-defined parts of the code, and I could fix this on a function-by-function basis. Now, all the code using floating point numbers is potentially affected. I can work around this by compiling all of WebKit with the -mstackrealign option, however this is something that should be fixed on Haiku side. Fortunately, the fix shouldn't be too complicated, it's just a change of the alignment constraint we have to do when creatign a new thread. If the initial alignment is correct, gcc generates code that always preserves it, unless you have some non-gcc compiled code in your function call stack (hand-written assembly code is one possible case of this).

There is apparently another crash related to the new mouse cursor code, which I haven't investigated yet. With these two out of the way, I'll have to run the testsuite again and see if there are other problems. With so few passing tests, the result html page (which list everything that failed) is too slow to browse and barely useable.

On to the new features now: it was more than time we fix the drawing glitches known as "border bleeding". You probably have noticed this problem on the side menu of this very website. There are some other places affected by this. Anyway, stippi did an amazing job of implementing ClipToPicture the right way. We now have a very fast implementation that also supports antialiasing. Bridging the gap between old and new apps, this improves the situation for both Gobe Productive (one of the few apps to use this API in BeOS days) and WebKit.

I had the code using ClipToPicture mostly ready in WebKit, waiting for the working implementation in Haiku. I could finally test it this morning and... well it doesn't work perfectly, yet. While it fixes the border bleeding, and we get our gradients where they should be again, when scrolling the page too fast (with pageup/pagedown buttons for example), the text above the gradients isn't drawn at all. Other websites also get new drawing problems in similar situations. I'm not sure what happens yet.

While stippi was working in app_server internals again, he also started implementing arbitrary view transforms. We had most of the API ready, with the BAffineTransform class available but only used to transform BPolygons. You can now set a transformation on a view and arbitrary rotate, scale, translate and otherwise distort all the drawing. This is all new and not yet completely tested (and in fact, there are some known bugs). But, it will allow a huge improvement of WebKit SVG rendering once it gets plugged in WebKit's GraphicsContext class.

I'm also trying to get some other devs into WebKit development, as working alone isn't fun. I've opened one "easy" issue on our bugtracker. It's about implementing Web Sockets support. There are some other things I would like to see done by others, for example support for web notifications using the BNotification API. Wouldn't it be nice to have a pop-up showing messages from gmail web page show next to the deskbar? Send in your patches!

Contract extension and 2013 financial review.

News posted on Fri, 2014-02-07 03:01

Pawel's contract has concluded with his work being merged into Haiku's master repository. Hopefully within the next few days, he will be able to post another blog post to summarize the improvements.

Now the even better news -- the recent donations from everyone has made it possible for Adrien to continue for another month! If another $1200 USD is raised by the end of this month, then there will definitely be enough funds to keep him coding through March (At the moment, our reserves would drop to below $800 USD, which is something we try to avoid.)

WebKit weekly report #17

Blog post by PulkoMandy on Fri, 2014-01-31 08:50

Hello everyone!

The work started last week on ClipToPicture made some progress this week. We discussed this further with Stippi and now have a solution that doesn't involve rewriting half of app_server code, and is also a bit simpler and faster than what I tried to do first. I wrote a test application and some boilerplate code, then Stippi jumped in and implemented the missing bits. There are still some missing features like the ability to stack multiple clippings using PushState/PopState, and some problems when scaling and translating the view, as expected. We also met a drawing glitch when moving or resizing the window, however, we're not sure what's happening yet.

With Haiku switched to gcc4.8, I tried updating our WebKit to a newer version again. But, this doesn't work yet, and it seems the problem is a missing option in our gcc configure script invocation. I wanted to rebuilt gcc with the proper options, but I hit some filesystem corruption on my data partition. I'm now trying to backup everything, but a bug in Haiku makes this incredibly slow. Of course, I paused my contract since wednesday, and until I can get this issue sorted out and resume working. No data was lost, but touching some files on that partition triggers a KDL. So, it's time I reformat it and put the data back on it.

As a result of these FS problems, I haven't got much work done this week. So, this report is short.

WebKit weekly report #16

Blog post by PulkoMandy on Wed, 2014-01-22 11:33

Hello world!

As I said last week, the remaining drawing glitches are because of BView limitations. Well, it's time to solve those as well!

I'll start with what is now known as the "border bleeding" bug. You have encountered it if you tried opening the Haiku website, or the bugtracker, in Web+. You will easily notice that some items are completely filled with the border color, instead of the expected background one. To understand what's going on, let's have a look at the way WebKit draws things.

Google Code-In 2013 wrap up report

Blog post by scottmc on Tue, 2014-01-21 05:48

Google has now announced the 20 winners for Google Code-In 2013, with Freeman Lou and Puck Meerburg being the two winners from Haiku.

This was the fourth year of Google's Code-In, and the fourth for Haiku. This contest came at a good point this year for Haiku as the package management merge happened just a few weeks prior to the start of the contest and thus gave us plenty of ideas for tasks. Nearly half of our tasks were somehow related to writing recipes for packages to be built into .hpkg files. We also opened up the coverity scan results for students to try their hand at fixing some of those issues for the first time. Along with these tasks there were several others which ranged from fixing specific bugs from Haiku's trac tickets, to writing new programs such as a blogging program and a spider solitaire game, and even a few for the artistic type students who created a new flyer and some new icons.

This year we had 5 students who completed 20 or more tasks, this is more than any of our students completed during GCI 2012. We had 42 students who completed a total of 245 tasks for Haiku which is more than have been completed in any previous year for Haiku, so it was a very good year for us. Of the 42 students, 19 of them completed 3 or more tasks which qualified them to receive a Google Code-In 2013 t-shirt, this was also the most students we've have that completed 3 or more tasks.

I'd like to thank the 19 Haiku mentors, which included 3 former Google Code-In students, and all 42 students who completed at least one task for Haiku this year. Also a special thanks to those who were on irc to help handle the flood of students during the contest, for their patience in handling of all the questions and such that the students were asking. And lastly, a special thanks to Stephanie Taylor and the rest of the team at Google for having Google Code-In, and for picking Haiku to participate again. It was another very productive and fun Code-In.

WebKit weekly report #15

Blog post by PulkoMandy on Fri, 2014-01-17 08:46

Hello again!

No big new changes this week, but a lot of small fixes and improvements.

I reviewed the growing issues list for Web+ on the bug tracker, and fixed several of them. Most of these were small and rather easy to fix bugs (I kept all the harder ones for later). Here is a list with comments, not that the issues were hard to track, but this is also a way to learn a bit more about the WebKit codebase.

Web+ crashed when trying to upload a file to GMail. This was a bug in our BFormDataIO code we use for serializing the form data into the HTTP stream. It missed the case where the first element in the form was a file, and tried to read from it without initializing it first. The FormDataIO class is used so we don't have to put the whole form data in memory in order to send it. It handles each form element one at a time, with special case for files, which are streamed from disk in small chunks, rather than loaded into memory.

Web+ also crashed when trying to decode a huge image. The test case for this is a 93MB JPEG file that expands to 700+MB of pixel data. ShowImage manages to display that, however Web+ tries to do incremental decoding, showing an incomplete view of the image as it is loaded. Our implementation of this is not optimal, as there are at least two copies of the data, one in a BBitmap and one as a raw byte array. For now I fixed the crash, but we abort the decoding and just show a blank page instead. We may want to review the image code to lower the memory use.

I implemented listing local directories in the Services Kit. WebKit has support for rendering directories as part of their FTP handling. Returning files list in one of the formats FTP listings use (there is no standard for this, but a few common formats in use) makes WebKit parse it and generate an HTML page for listing. There are still some problems with encoding (WebKit doesn't seem to expect UTF-8 filenames in those listings), but things should be working now. I also fixed some problems with symlinks in the file:// protocol handler.

Some drawing glitches were fixed (again). We're now in a state where all improvements will require adding support to BView.

There was also a problem with opening links in a new tab from inside a frame. I also added shift+middle click as a way to open a link in a new tab and immediately switch to it (middle click alone opens the tab in the background).

I did several fixes to Cookie management. The most important one is there was a bug in the code for getting cookies for a specific website. A misuse of our StringHash class (this is a simple class that allows using a string as a hash for a hash map) led to memory corruption. We were trying to set the key for the map to a substring of the previous key to implement domain exploration (so a site at www.example.com can access cookies set at example.com - but not for just 'com'). Basically, the HashString freed the old key, then tried to copy characters from it to the new key, using memcpy. This is a classic use-after-free problem, that didn't always create problems in normal run, but was very obvious when running the browser with libroot_debug. Another fix was the proper implementation of CookiesForDOM. This is one of the two methods for accessing cookies. We used the same code as for the access from Javascript, but CookiesForDOM must also include "HttpOnly" cookies. Finally, a third bug was wrongly parsing the expiration date for cookies using the local time zone, whereas they need GMT dates. Depending on your timezone, this lead to cookies expiring too late (you probably didn't notice) or too early, sometimes right in the past. For example, some banking website use short-lived cookies (1 hour or less) as a timeout system. In my GMT+1 timezone, the cookie was expired immediately and I couldn't even access the login screen.

Some fixes went into the SSL support. One case of crashing was fixed, we were deleting the OpenSSL connection context before the network thread had a chance to exit, leading to a crash when leaving an https page before it finished loading. I also started work towards proper support for certificate checking. SSL connections didn't do any checking for certificates, and actually didn't even load the certificate store, making the SSL host authentication useless (you still get the encryption, but you can't make sure you're sending things to the right server). I implemented the Network Kit side of things, but now I must get this exposed in the Services Kit, then in WebKit, and finally add a nice dialog in Web+ asking what to do. Then, I must get the answer back to the network kit and continue or stop the connection with the unsafe host.

On WebKit side, I did a lot of small - but useful! - usability enhancements.

We got the error reporting for non-http connections working again. When trying to open a non-existing file:// URL, you now get a "file not found" message instead of a blank page.

The URL bar now always has an icon (the default is a little globe), to avoid the URL jumping to the right when the favicon gets shown. I also fixed some glitch pixels below the text in that bar, when using small font sizes.

I reworked some of the bookmark loading code. Now, bookmarks load in the current window, instead of the first window they can find in the workspace. If you open several of them at the same time, it works as expected. There was a race condition leading Web+ to try opening several bookmarks in the same tab, with of course only the last one showing up. Another problem was it was not possible to use symlinks in the bookmarks folder, as the BNavMenu we use for bookmarks wasn't traversing them. This now works as expected.

The search page in Web+ is now configurable. This means you can switch to goodsearch.com and help raise some money for Haiku while searching! Or, you can use the local version of Google or whatever search engine you prefer. The bug that made us unable to search for UTF-8 strings was also fixed, so you don't have to search in english anymore. And, there was also some progress with IDN domains, but the complete fix for this will have to wait for the next update to the WebKit package.

So, what's next? I will continue working on better SSL support, as this is currently set as an alpha blocker. I also plan to have a look at doing a bookmark bar. I tried doing this as a BMenuBar + BNavMenu, but these classes aren't designed for multiple inheritance, so I have some refactoring to do there. Or maybe I should go with another approach.

The "network lock-up" bug and missing BView features are also still fairly high on the TODO list, but these will need more time as I'm not as familiar with the code in these areas.

I didn't do this for some time, but let's also talk about non-working-hours time I also spent on Haiku. I did some Haikuporter recipes for XRick, OpenTTD, and a few other games. I also finally made a recipe for libusb, and others have used this to compile libftdi and avrdude. This isn't quite working yet, but I hope someone gets it going so I can finally do some hardware hacking on Haiku (did I hear blinkenlights?).

See you next week!

WebKit weekly report #14

Blog post by PulkoMandy on Fri, 2014-01-10 08:34

Hello there,

Well, somehow quiet and regular activty this week. Not too much exciting things, but progress is being made.

I updated WebKit to early december version. This is not the latest one, but the guys at WebKit started using even more C++11 as Visual Studio on Windows finally gets more support for it. So, enter std::chrono and some std::thread stuff. Unfortunately, our version of gcc4 seems to be missing some of these. I'm now closely watching the work of korli and mt to bring us gcc4.8.

I made some long overdue cleanup to the build system, removing some manual (and undocumented) steps and fixing some bugs in the process. The about box will now report the proper WebKit version, once again using the standard CMake scripts to get it, instead of a custom shell script. Moreover, this seems to have fixed Google detecting us as a mobile browser. Some other websites still show that problem, however.

The work on completing the testsuite expectations continues. The progress is slow, as reviewing each of the tests, and looking for possible solutions before marking them all as failing is a very time consuming process. It is also a bit boring and unrewarding, so I'm also spending some time looking at some other things.

I attemted to fix the lockup issue. If you use Web+ you probably have noticed that it oftens becomes unresponsive and seems frozen for a minute or so. I now know what's happening: this triggers when you try to navigate away from a page thet's still loading, or performing an XmlHttpRequest or some other kind of network activity. If the socket is blocked on a connect() call, we have no way to unlock it from there. What WebKit does is it tries to close the connection, and the way it's implemented in our HTTP backend, we have to wait for the network activity to stop, before we can free the connection object. If we free the object without waiting for the thread to terminate first, the thread will crash when the connect call finally times out.

There is a well-known trick to avoid this issue, making the connect non-blocking, and then using select or poll to wait on it. Unfortunately, the Haiku network stack has some bugs that prevent this from working. I'm not yet up to speed on that area of the Haiku code yet, but I'll try to improve the situation.

On the rendering side, I'm also hitting various limitations of the BView API. First of all, the latest build I uploaded last week introduced two new rendering glitches, but fortunately I could get them fixed without reintroducing the box-shadow bug htat led to black rectangles drawing all over the place.

The missing bits in BView currently are support for transform matrices, which are used a lot for drawing SVG and maybe canvas elements ; and support for arbitrary clipping. I had a try at implementing the latter using ClipToPicture, but this lacks antialiasing and is implemented in a suboptimal way, leading to laggy scrolling. It also has problems with the way scrolling is implemented in WebKit, because the clipping is itself clipped (doh!) to the visible part of the view. So, when you try scrolling, it isn't updated and prevent things from drawing if they weren't already visible before scrolling.

I discussed these BView issues with Stippi and there is clearly no way to solve these issues without improving the BView API to add support for arbitrary transformations and clipping shapes. While the initial implementation of this shouldn't be too hard, there may be a few difficulties, for example with the way view coordinates are converted to window or screen ones. It may be a bit tricky to get this all working well.

There are other issues that can be solved only on the WebKit side, for example the history navigation problems. I'm not sure why this is happening, but sometimes, clicking on a link doesn't add it to the navigation history, making the next/previous buttons a bit unreliable to use.

Finally, there are a few known crashes that I'm still tracking. One of them happens when decoding big images or animatd gifs, another one when trying to attach a file to a mail on gmail, and one happens when trying to stop an https connection, and tends to be triggered easily when navigating gmail or github. I'll try to get these crashing issues fixed before I start playing with new features, so we can at least have a stable version of WebKit to do some web browsing, even if some features are still missing.

Syndicate content