serviceskit

Hello everyone!

This week I worked on stabilization and small improvements of WebKit. There are a few new features, as well.

The crash with cursors I mentionned last week is fixed. I had forgotten to copy an object in the copy constructor, leading to a double delete. I continued working on the clipping code, and fixed the issues with www.haiku-os.org and a few other websites. But, I can't get it to work with haikuports, Trac, and now gmail is also broken. I don't want to do a release until we have a fix for that.

I also did some more work on supporting css shadows. With the updated clipping code, the ugly black box that was sometimes visible is now gone (it's clipped out). However, we also must draw the shadow itself. I did most of the implementation on WebKit side, but it needs support for a drawing mode (SourceIn) that app_server doesn't handle yet. So, I left that disabled for now, until we can get it to render the expected way.

An easy fix was adding the support for HTML5 "file" API. I just had to turn a compile flag on, and this gives us 10 more points on html5test.com. It also gets imgur.com image upload working.

I merged some changes from WebKit, without much problems. This brings the usual small fixes and cleanup, without too much code breakage this time. There are a lot of code cleanups going on at WebKit, making the codebase simpler and also faster.

I also did some work on the Network Kit HTTP backend. We now support gzip/deflate compression of HTTP data. This make some web pages load much faster, and also fixes issues with some websites serving compressed data even though the browser doesn't advertise support for it. While working on this, another problem in the HTTP code was discovered, there was a possible stack overflow because we were using a gcc extension to C++. This is now fixed, hopefully improving stability of the web browser.

With all those small issues fixed, it's time for another testsuite run. I hope I can find some test that fail because of our remaining clipping problems, as this would help me identify the issues more easily than with a complex web page such as gmail. I'd really like to squash at least the new issues introduced by this clipping change, so we can have a release that I'd qualify as stable. I'm trying to not start work on too much other features until we get this sorted out. Once that release is done, we can resume way on features that need more changes.

Next up on the TODO list: support for affine transformations (stippi added this to the app_server), which will improve SVG rendering a lot and possibly fix some other issues. Support for shadows, and the missing SourceIn composite drawing mode. When we get these two out, we should have much better rendering. There are probably other missing features, but they are yet to be identified.

See you next week!

Time for a report again !

So, over last week-end and monday morning I finally got Ninja working. I already said some words about it, Ninja is meant to replace Make for building projects. It has less features, because it is designed to run files that are generated, rather than hand-written. In my case, CMake generate the Ninja failes. I had problems building Ninja that turned out to be abug in our Python port. I didn't fix it, but I found a way to work around it.

The use for all this buildsystem work? Well, we're now using one of WebKit standard build systems (CMake), and using it in a way that's fast! CMake can also generate makefiles, but running these would take about a minute just to figure out which files to recompile. With Ninja, this is done in a matter of seconds, because the simpler rules are faster to parse. Ninja was actually written just for that purpose. One minute may not seem much when the full build of WebKit takes about an hour, but when I'm working on just one or two files to track down a bug, the edit/compile/test cycle is now much faster.

So, with this out of the way I was able to more easily try out things and fix the issues as I found them. I mentionned the Opera cookie testsuite in last week report : most of the tests are now passing, and we even get better results than the old Curl backend in some places. The remaining issues are rather minor ones, but I can fix them if you can find a website where they actually matter.

In one of the blog post comments someone mentioned that Windows Live Mail (or Outlook, as it's now called) didn't allow to display messages. I tried to do that with HaikuLauncher and the new code and noticed I couldn't even login to the website. Getting this to work was a bit tricky, leading to fixes in 3 different places: settings cookies from Javascript code, proper handling of URL redirections, and telling WebKit that "aspx" files served without a mimetype are web pages.

I already told you about the cookies, so here are some more details about the two other issues: the URL redirection problem happened when an HTTP server gave us a 302 answer (that is, a redirection) with a relative URL. The HTTP specification says an URL should be used, most likely starting with "http://". But the URL specification was later replaced with the URI one, which allows URIs without that part, for example "/path/to/page.aspx?param=foo". Our BUrl class wasn't able to understand such things and was desperately looking for the "//" part, leading to an endless loop. I replaced the homegrown parser with a regular expressionbased one, using the regex that is kindly provided inthe RFC for URIs. I had to fix our RegExp class in the shared kit (this is a collection of internal classes that are not, yet, part of the official API). So, I had to fix the RegExp class as it didn't handle optional capture groups in the regular expression properly. This includes things like: the URI may, or may not start with "scheme:", then, maybe there will be an authority part, of the form "//authority", then a path, which is anything following the authority and up to th first ? or # or the end of the URI, and then there is the query and fragment, and they are also optional. RegExp allow to write this in a more compact and computer understandable format and get the parts extracted to separate strings.

As for the "aspx" file handling, the HTTP protocol says that the server should give a "content-type" header with the MIME type of the file it's going to send. This fits well with the use of MIME types in Haiku and would be very nice. However, Microsoft servers for login.live.com sometimes don't include the header. Web+ used to assume the file was to be displayed on screen in that case. You may already have seen it trying to render a video file as plain text, as a huge page full of strange characters. Well, that issue was fixed, and if in doubt, it will now download the file to disk instead. A list of well-known file extensions is used to attempt a guess when the "content-type" is missing. I added aspx to that list and mapped it to text/html, so, the URLs ending in aspx will now be shown on screen.

I had a run of testing after all these fixes and things are getting much better. It helped fix some more issues in github and gist, and I could also browse some documents in Google Drive. I think we still have some problems with file upload, however, I'll have to look into that. Uploading video to youtube, as someone pointed out in the forums, also fails.

I caught a crash when opening a new window in HaikuLauncher, but I'm not sure if that would be a problem for Web+. Also, I wanted to work on HTTP authentication (the login/password prompt) and this is not available in HaikuLauncher, so a working Web+ will be needed. To get there I wanted to build a package for WebKit. To do this, I first tried writing an haikuporter recipe. I didn't get very good results with this. First,it means working separately from my development git checkout, so it takes twice the sapce on disk (and we're talking multiple gigabytes) and it takes forever to checkout. Second, there's no easy way to use distcc with haikuporter, so things are keeping my CPU very busy for a long time. And since this isn't my development tree, there are a lot of things to rebuild.

So, I decided to try a different solution: using CMake built-in support for generating packages. I tried that yesterday and found out that it wasn't working. I tried on a very small test project, and with help from the CMake IRC channel I found the issue: our CMake port is lacking some features that are available in the Linux version, and not needed on Windows. This confuses the Ninja generator which isn't ready for that case. This is a rather technical issue related to the way executables and libraries are linked together. This is different in the tests (the lib and executable sit together in a directory) and installed version (they are installed somewhere on the filesystem and the OS, more precisely the runtime_loader) must bind them together. On Linux, CMake makes up for that by rewriting part of the executables when installing or packaging them. This isn't enabled in the Haiku version, so it will instead try to link executables again when installing them. However, the Ninja generator is skipping the link step (it was never tested in that case) and tries to install or package non-existing files. This shouldn't be too hard to get working just like on Linux, more news next week !

There is another missing step to make this complete: CPack, the CMake component that generates packages, knows how to make tar archives (with various compression schemes), zip archives, and deb and rpm packages. This is not very useful for us, so I started adding support for hpkg files there. This will make it super-easy to get a package built from any project using cmake: "make package", or "ninja package", depending on the generator you decided to use.

Anyway, I've uploaded an unsupported build in a manually crafted zip file so you can try things for yourself: http://pulkomandy.tk/drop/HaikuWebKit_20131018.zip
I'm waiting for your reports on how well it's working. Note you need a fairly recent build of Haiku (a nightly from today or yesterday should do) as I made some changes to the BHttpRequest class again, to fix some API design issues that led to memory leaks. That class was also added to the Haiku Book.

That's it for this week!

In these weeks i have improved the contacts kit core in order to have enough support for the formats supported. The vcard and people translators can now translate and exchange many types of field, though photo and groups aren't yet supported.

Main functionalities of the classes :
BRawContact
Their functionality is to deal with the BTranslatorRoster and keep track of basic informations, like the final format. The final destination is represented as a BPositionIO object. Basically the class find a suitable translator when initialized and provide two methods : Commit() and Read(). The first has as argument a BMessage that contain Flattened BContactFields, the second translate the source file (that can be a B_PEOPLE_FORMAT, B_VCARD_FORMAT, B_CONTACT_FORMAT) into a BMessage and return it.

BContact
BContact is the high level class representing a contact, it has the job to store informations about the raw contact, provide methods to add/remove/replace/compare the BContactFields. When initialized it use the BRawContact::Read() function to read fields from a location, and use BRawContact::Commit() to provide BContact::Commit().

BContactField and childs

BContactField is designed to support as well both People and VCard formats. The class is not intended to be used directly, but a common interface for the different types of fields that are supported. The usage of a field is defined by two enums at ContactDefs.h, it is a code that allow to specify additional informations for the data, for example if you want to add a phone number that is a fax you'll use this code :

...
BContactField* field = new BStringContactField(B_CONTACT_PHONE, value);
field->SetUsage(CONTACT_PHONE_FAX_WORK);
contact.AddField(field);
...

The class also accept any number of string parameters, the BFlattenable interface, and the BContactFieldVisitor interface.

BStringContactField
This is used for the major part of the fields, the value is represented as a BString object it is enough for fields like B_CONTACT_NAME, B_CONTACT_ORG...
BAddressContactField
It represent an address, allow to initialize a well-formed address via the constructor or alternatively the programmer can use the provided methods to set all the informations.

There are also other types of fields in the plans, one of these is the BCustomContactField.

Translators
As said while not supporting all fields, they can translate and exchange fields with a common format. There are some bugs, but the whole process is working as well and i consider this part of the project complete.

Services Kit
Unfortunately this side is not close to be completed, i have a draft and i'm working to get it enough complete.

At the moment the main classes are :
BServicesRoster
A simple class that allow to manage the addons in a more low-level manner...something similar to BTranslatorRoster giving access to the add-ons so nothing specific to the contacts support but a base for many uses (including contacts addons).

BServicesAddon
This will be the class specifying the API used by the addons, using a generic set of methods...version, name, friendlyname, services_addon_type, init and so more. Providing a Process() method used to receive data as BMessages from the server. In the same manner, will be instantiated with a messenger object to talk with the server.

BContactRoster,
basically store the BContacts provided in the address book. The user will instantiate an own object reading only the contacts they want. In future would be nice to see classes like BContactQuery...at the moment they will specify simple access to the fields stored in the address book. Internally will use an instance of the BServicesRoster talking with contacts addons.

I have created a git branch of the Haiku's tree at the url :

https://github.com/Barrett17/Haiku-services-branch

At the moment it is a plain copy of the tree, in the next day's i'll update the code since using the osdrawer repo was limitative.