Haiku is an open-source operating system that specifically targets personal computing. Inspired by the BeOS, Haiku is fast, simple to use, easy to learn and yet very powerful.

Fundraising 2016

Goal: $35,000


The Haiku source is continually built and released for testing purposes nearly every day. You can download and install these latest snapshots to check out the latest features and bug-fixes.

Be aware that nightly images may be unstable. Additionally, some packages included with official releases need to be installed separately.

If you're OK with this, you can find further instructions at our Nightly image page.

WebKit weekly report #17

Blog post by PulkoMandy on Fri, 2014-01-31 08:50

Hello everyone!

The work started last week on ClipToPicture made some progress this week. We discussed this further with Stippi and now have a solution that doesn't involve rewriting half of app_server code, and is also a bit simpler and faster than what I tried to do first. I wrote a test application and some boilerplate code, then Stippi jumped in and implemented the missing bits. There are still some missing features like the ability to stack multiple clippings using PushState/PopState, and some problems when scaling and translating the view, as expected. We also met a drawing glitch when moving or resizing the window, however, we're not sure what's happening yet.

With Haiku switched to gcc4.8, I tried updating our WebKit to a newer version again. But, this doesn't work yet, and it seems the problem is a missing option in our gcc configure script invocation. I wanted to rebuilt gcc with the proper options, but I hit some filesystem corruption on my data partition. I'm now trying to backup everything, but a bug in Haiku makes this incredibly slow. Of course, I paused my contract since wednesday, and until I can get this issue sorted out and resume working. No data was lost, but touching some files on that partition triggers a KDL. So, it's time I reformat it and put the data back on it.

As a result of these FS problems, I haven't got much work done this week. So, this report is short.

WebKit weekly report #16

Blog post by PulkoMandy on Wed, 2014-01-22 11:33

Hello world!

As I said last week, the remaining drawing glitches are because of BView limitations. Well, it's time to solve those as well!

I'll start with what is now known as the "border bleeding" bug. You have encountered it if you tried opening the Haiku website, or the bugtracker, in Web+. You will easily notice that some items are completely filled with the border color, instead of the expected background one. To understand what's going on, let's have a look at the way WebKit draws things.

Google Code-In 2013 wrap up report

Blog post by scottmc on Tue, 2014-01-21 05:48

Google has now announced the 20 winners for Google Code-In 2013, with Freeman Lou and Puck Meerburg being the two winners from Haiku.

This was the fourth year of Google's Code-In, and the fourth for Haiku. This contest came at a good point this year for Haiku as the package management merge happened just a few weeks prior to the start of the contest and thus gave us plenty of ideas for tasks. Nearly half of our tasks were somehow related to writing recipes for packages to be built into .hpkg files. We also opened up the coverity scan results for students to try their hand at fixing some of those issues for the first time. Along with these tasks there were several others which ranged from fixing specific bugs from Haiku's trac tickets, to writing new programs such as a blogging program and a spider solitaire game, and even a few for the artistic type students who created a new flyer and some new icons.

This year we had 5 students who completed 20 or more tasks, this is more than any of our students completed during GCI 2012. We had 42 students who completed a total of 245 tasks for Haiku which is more than have been completed in any previous year for Haiku, so it was a very good year for us. Of the 42 students, 19 of them completed 3 or more tasks which qualified them to receive a Google Code-In 2013 t-shirt, this was also the most students we've have that completed 3 or more tasks.

I'd like to thank the 19 Haiku mentors, which included 3 former Google Code-In students, and all 42 students who completed at least one task for Haiku this year. Also a special thanks to those who were on irc to help handle the flood of students during the contest, for their patience in handling of all the questions and such that the students were asking. And lastly, a special thanks to Stephanie Taylor and the rest of the team at Google for having Google Code-In, and for picking Haiku to participate again. It was another very productive and fun Code-In.

WebKit weekly report #15

Blog post by PulkoMandy on Fri, 2014-01-17 08:46

Hello again!

No big new changes this week, but a lot of small fixes and improvements.

I reviewed the growing issues list for Web+ on the bug tracker, and fixed several of them. Most of these were small and rather easy to fix bugs (I kept all the harder ones for later). Here is a list with comments, not that the issues were hard to track, but this is also a way to learn a bit more about the WebKit codebase.

Web+ crashed when trying to upload a file to GMail. This was a bug in our BFormDataIO code we use for serializing the form data into the HTTP stream. It missed the case where the first element in the form was a file, and tried to read from it without initializing it first. The FormDataIO class is used so we don't have to put the whole form data in memory in order to send it. It handles each form element one at a time, with special case for files, which are streamed from disk in small chunks, rather than loaded into memory.

Web+ also crashed when trying to decode a huge image. The test case for this is a 93MB JPEG file that expands to 700+MB of pixel data. ShowImage manages to display that, however Web+ tries to do incremental decoding, showing an incomplete view of the image as it is loaded. Our implementation of this is not optimal, as there are at least two copies of the data, one in a BBitmap and one as a raw byte array. For now I fixed the crash, but we abort the decoding and just show a blank page instead. We may want to review the image code to lower the memory use.

I implemented listing local directories in the Services Kit. WebKit has support for rendering directories as part of their FTP handling. Returning files list in one of the formats FTP listings use (there is no standard for this, but a few common formats in use) makes WebKit parse it and generate an HTML page for listing. There are still some problems with encoding (WebKit doesn't seem to expect UTF-8 filenames in those listings), but things should be working now. I also fixed some problems with symlinks in the file:// protocol handler.

Some drawing glitches were fixed (again). We're now in a state where all improvements will require adding support to BView.

There was also a problem with opening links in a new tab from inside a frame. I also added shift+middle click as a way to open a link in a new tab and immediately switch to it (middle click alone opens the tab in the background).

I did several fixes to Cookie management. The most important one is there was a bug in the code for getting cookies for a specific website. A misuse of our StringHash class (this is a simple class that allows using a string as a hash for a hash map) led to memory corruption. We were trying to set the key for the map to a substring of the previous key to implement domain exploration (so a site at www.example.com can access cookies set at example.com - but not for just 'com'). Basically, the HashString freed the old key, then tried to copy characters from it to the new key, using memcpy. This is a classic use-after-free problem, that didn't always create problems in normal run, but was very obvious when running the browser with libroot_debug. Another fix was the proper implementation of CookiesForDOM. This is one of the two methods for accessing cookies. We used the same code as for the access from Javascript, but CookiesForDOM must also include "HttpOnly" cookies. Finally, a third bug was wrongly parsing the expiration date for cookies using the local time zone, whereas they need GMT dates. Depending on your timezone, this lead to cookies expiring too late (you probably didn't notice) or too early, sometimes right in the past. For example, some banking website use short-lived cookies (1 hour or less) as a timeout system. In my GMT+1 timezone, the cookie was expired immediately and I couldn't even access the login screen.

Some fixes went into the SSL support. One case of crashing was fixed, we were deleting the OpenSSL connection context before the network thread had a chance to exit, leading to a crash when leaving an https page before it finished loading. I also started work towards proper support for certificate checking. SSL connections didn't do any checking for certificates, and actually didn't even load the certificate store, making the SSL host authentication useless (you still get the encryption, but you can't make sure you're sending things to the right server). I implemented the Network Kit side of things, but now I must get this exposed in the Services Kit, then in WebKit, and finally add a nice dialog in Web+ asking what to do. Then, I must get the answer back to the network kit and continue or stop the connection with the unsafe host.

On WebKit side, I did a lot of small - but useful! - usability enhancements.

We got the error reporting for non-http connections working again. When trying to open a non-existing file:// URL, you now get a "file not found" message instead of a blank page.

The URL bar now always has an icon (the default is a little globe), to avoid the URL jumping to the right when the favicon gets shown. I also fixed some glitch pixels below the text in that bar, when using small font sizes.

I reworked some of the bookmark loading code. Now, bookmarks load in the current window, instead of the first window they can find in the workspace. If you open several of them at the same time, it works as expected. There was a race condition leading Web+ to try opening several bookmarks in the same tab, with of course only the last one showing up. Another problem was it was not possible to use symlinks in the bookmarks folder, as the BNavMenu we use for bookmarks wasn't traversing them. This now works as expected.

The search page in Web+ is now configurable. This means you can switch to goodsearch.com and help raise some money for Haiku while searching! Or, you can use the local version of Google or whatever search engine you prefer. The bug that made us unable to search for UTF-8 strings was also fixed, so you don't have to search in english anymore. And, there was also some progress with IDN domains, but the complete fix for this will have to wait for the next update to the WebKit package.

So, what's next? I will continue working on better SSL support, as this is currently set as an alpha blocker. I also plan to have a look at doing a bookmark bar. I tried doing this as a BMenuBar + BNavMenu, but these classes aren't designed for multiple inheritance, so I have some refactoring to do there. Or maybe I should go with another approach.

The "network lock-up" bug and missing BView features are also still fairly high on the TODO list, but these will need more time as I'm not as familiar with the code in these areas.

I didn't do this for some time, but let's also talk about non-working-hours time I also spent on Haiku. I did some Haikuporter recipes for XRick, OpenTTD, and a few other games. I also finally made a recipe for libusb, and others have used this to compile libftdi and avrdude. This isn't quite working yet, but I hope someone gets it going so I can finally do some hardware hacking on Haiku (did I hear blinkenlights?).

See you next week!

WebKit weekly report #14

Blog post by PulkoMandy on Fri, 2014-01-10 08:34

Hello there,

Well, somehow quiet and regular activty this week. Not too much exciting things, but progress is being made.

I updated WebKit to early december version. This is not the latest one, but the guys at WebKit started using even more C++11 as Visual Studio on Windows finally gets more support for it. So, enter std::chrono and some std::thread stuff. Unfortunately, our version of gcc4 seems to be missing some of these. I'm now closely watching the work of korli and mt to bring us gcc4.8.

I made some long overdue cleanup to the build system, removing some manual (and undocumented) steps and fixing some bugs in the process. The about box will now report the proper WebKit version, once again using the standard CMake scripts to get it, instead of a custom shell script. Moreover, this seems to have fixed Google detecting us as a mobile browser. Some other websites still show that problem, however.

The work on completing the testsuite expectations continues. The progress is slow, as reviewing each of the tests, and looking for possible solutions before marking them all as failing is a very time consuming process. It is also a bit boring and unrewarding, so I'm also spending some time looking at some other things.

I attemted to fix the lockup issue. If you use Web+ you probably have noticed that it oftens becomes unresponsive and seems frozen for a minute or so. I now know what's happening: this triggers when you try to navigate away from a page thet's still loading, or performing an XmlHttpRequest or some other kind of network activity. If the socket is blocked on a connect() call, we have no way to unlock it from there. What WebKit does is it tries to close the connection, and the way it's implemented in our HTTP backend, we have to wait for the network activity to stop, before we can free the connection object. If we free the object without waiting for the thread to terminate first, the thread will crash when the connect call finally times out.

There is a well-known trick to avoid this issue, making the connect non-blocking, and then using select or poll to wait on it. Unfortunately, the Haiku network stack has some bugs that prevent this from working. I'm not yet up to speed on that area of the Haiku code yet, but I'll try to improve the situation.

On the rendering side, I'm also hitting various limitations of the BView API. First of all, the latest build I uploaded last week introduced two new rendering glitches, but fortunately I could get them fixed without reintroducing the box-shadow bug htat led to black rectangles drawing all over the place.

The missing bits in BView currently are support for transform matrices, which are used a lot for drawing SVG and maybe canvas elements ; and support for arbitrary clipping. I had a try at implementing the latter using ClipToPicture, but this lacks antialiasing and is implemented in a suboptimal way, leading to laggy scrolling. It also has problems with the way scrolling is implemented in WebKit, because the clipping is itself clipped (doh!) to the visible part of the view. So, when you try scrolling, it isn't updated and prevent things from drawing if they weren't already visible before scrolling.

I discussed these BView issues with Stippi and there is clearly no way to solve these issues without improving the BView API to add support for arbitrary transformations and clipping shapes. While the initial implementation of this shouldn't be too hard, there may be a few difficulties, for example with the way view coordinates are converted to window or screen ones. It may be a bit tricky to get this all working well.

There are other issues that can be solved only on the WebKit side, for example the history navigation problems. I'm not sure why this is happening, but sometimes, clicking on a link doesn't add it to the navigation history, making the next/previous buttons a bit unreliable to use.

Finally, there are a few known crashes that I'm still tracking. One of them happens when decoding big images or animatd gifs, another one when trying to attach a file to a mail on gmail, and one happens when trying to stop an https connection, and tends to be triggered easily when navigating gmail or github. I'll try to get these crashing issues fixed before I start playing with new features, so we can at least have a stable version of WebKit to do some web browsing, even if some features are still missing.

WebKit weekly report #13

Blog post by PulkoMandy on Sat, 2014-01-04 12:16

Hello everyone!

You probably already read the news on the homepage: I'm continuing to work on WebKit for January.
Maybe you noticed there was no report last week, as I was visiting family and didn't get much work done. I'm not counting that week as paid work for Haiku.

Most of the work I did during the last two week revolves around the testsuite stuff. The testsuite engine got support for tests that need some time before the reports are parsed. We used to dump the page right after loading, but in some cases the test wants us to wait a bit and notify us when it's done. The test uses Javascript to notify the testsuite system of this. We now get better (and more reliable) results for many tests that run some javascript code before giving their results. On the downside, the testsuite now needs more time to run, about 3 hours on my computer (using 2 cores).

I also added some support for dumping pages with frames. Only the top level would be dumped before, and of course that didn't quite match what the references for the tests expected.

I'm now starting to get a list of mostly actual test failures. A lot of them are because our font metrics aren't exactly the same as on other platforms. This leads to slightly different size for some text runs, and ultimately a different dump for the page. Some of the tests use the Ahem font to get predictable results, but not all of them. The ones that don't may need platform-specific test results to avoid the problem.

We are also failing a lot of the tests that use SVG. While the fixed gradient support improves things, it's not quite enough. The SVG drawing in WebKit is one of the places where transform matrices are used to scale, pan and rotate objects when drawing them to the view. This is another thing our BView can't do yet. I started looking at what's needed for this, and briefly discussed with stippi, our app_server expert, over IRC. However, I think it's better to get the testsuite ready before I do the change. This will allow seeing how much tests we can get to pass, and if it breaks other things (giving me an hint on what to work on next). The testsuite will also be useful when merging changes from WebKit again. We're now some month behind, but our current builds are fairly stable, and I don't want to break everything by merging the latest changes without having a way to track the regressions (and possibly the bugfixes).

I'll be uploading a new webkit build next week. This will include the gradient and box-shadow fixes, as well as the web worker crash fix, and some other things I did since last time. This should be a "reasonably good" version with enough stability to last for some time. I'll try to fix the remaining issues with the border color bleeding on the background, and the broken back/forward management, as these are the most visible and annoying of the remaining problems. I'll have to learn about the back/forward code, which, as you expect coming from WebKit, is more complicated than you'd think, for performance reasons. There is an in-memory cache for recently viewed pages, that avoids reloading everything when you press the "back" button. This includes the page itself, but also the position of the scrollbars and a few other things. While this seems very natural when using the browser, it's actually quite tricky to implement. The page can't be easily snapshotted because of JavaScript and other things that may be running, and the anchor used for restoring the scrolling may itself point to javacript-generated (or otherwise dynamically added) content.

Well, quite a lot of work to do, still.

Happy Contract Extensions!

News posted on Wed, 2014-01-01 19:17

This may be the final chapter of contract extensions, as the available funds will be plummeting to less than $1500 USD.

Syndicate content