GSOC 2019 Final Report
Hey there beautiful person reading this post. We are in the endgame now (Ha get it avengers reference!). Yes, Google summer of code 2019 is coming to an end. Phew couldn’t say how 3 months passed by, but this is one of my most memorable experiences I will never forget. So let me wrap GSOC with this final report. Buckle up tight it’s going to be a long post…
I am trying to port webkit2 to Haiku.
Why webkit2 when we already have awesome webkit?
Duh, 2 is always better than 1 isn’t it. Now jokes aside, WebKit2 is now webkit and webkit1 is called webkitlegacy. Webkit2 features a multi-process model so that means jobs are split across different processes. This improves high reliability, less crashes, super fast performance. So obviously webkit2 is better than webkit1.
Will it be available for my favourite browser WebPositive?
Lol, definitely thats the whole point of this project.
Did we really ask you these questions?
No, you but I believe it wakes up the inner nerd inside you to ask more questions.
Link to the source code
I’ll go through what has been done till now on webkit2. I know I know that you guys would have read the same stuffs in my previous posts, please don’t get bored, I promise it’s so interesting this time. I have added stats for nerds cough, cough (that is link to commits that implements a specific feature)
By this time, I have talked with lot of haiku dev’s mostly chitchat. I have never felt so welcomed anywhere. They accepted me even if I asked any stupid question.
Sorry don’t know any github links related to this.
Enable build fix
Ah, yes I still remember that nostalgic moment, my stepping stone into webkit2 - was as simple as enabling cmake switch. picture me giving nostalgic reactions for more dramatic effect
Same old drama, had to get webkit to build on haiku before anything could be even done. Initially to get it to compile I had to just define all the platform dependent functions which were just declared and of course lot of time staring at screen to see if there is any error.
Few files were required to be added as a part of forwarding headers. I was confused with local and system includes and then came to know about dilemma between “Rect.h”(Webkit2) and <Rect.h>(BRect).
At this time we had stumbled a problem with deciding type of ProcessID. Our pid_t (long) and other platforms pid_t(int32_t) didn’t go well I dont remember the problem exactly, But then we came up with a fix that went well with the webkit2 model too.
So our code was like around 5 years old before I started doing it. After dusting I mean updating our fork with latest upstream code. Before doing any commits I had to have the latest code in my fork. At that point of time all I knew was push,pull,commit and soon realized these are not enough. Took a week to train myself with advanced git commands like rebase,cherry-pick and then tried on the big picture.
To be honest I messed up the first time, Pulkomandy cleaned up my mess as usual and the saying Second times the charm is actually true.
A Minibrowser to satisfy our needs
The compilation is done what now? A browser to test our stuff (even if we have nothing). I stole the design from HaikuLauncher( a testing browser that uses webkitlegacy ). Was just a basic browser with messed up schema but felt so happy that I have accomplished something.
This is the part where it loads “blank url”.
Birth of WebView
This the API that actually lets us connect our browser with webkit. It had simple functionality to load a html string as we haven’t implemented network stuff yet.
It hit my brain at a later point of time that this is a multiprocess engine. So where are they?
So now the processes are up and running each process should have their main Runloop up and running. They were attached to Applications looper. At this point All BApplication, BLooper, BHandler started making sense to me!
IPC (Inter process Communication)
We wish processes had mouths to speak with other processes instead all they had was BMessage. Other platforms used sockets for the same.
Initially we hit some kind of IPC deadlock as we tried to use mainloop as proxy. So we came to know that if we have BMessenger object then we can send message directly. In this way mainloop had to interfere only once for handshaking. psst It would be nice if we could directly send to any looper in a different process without much work.
Few fun facts related to IPC:
The initialization is done by first exchanging 'inil'-> sent to application looper (details of particular BLooper where messages from other process is expected) 'inig'->sent to other process( BMessenger object of this looper). When the other process recieves the bmessenger it forwards to correct looper where message should be recieved. Then once connection is successful each process starts sending messages. Each IPC message is tagged 'ipcm' with Data Attached to them.
And yes I started to give meaningful commit messages. Plus here is one of a kind limited edition doodle by me.
Sharing is caring
Shared memory is pretty much used everywhere in webkit2 starting from sharing history to bitmap across different processes. I learnt about areas in Haiku. It’s pretty cool.
Redesign of Minibrowser
I realized halfway through that the design of Minibrowser is weird so I added widgets like url bar, buttons etc. All the widgets were even added to respective callbacks i.e pressing back button goes back in history. Score!!
Now that our browser is ready to post Load a url request to webkit we need to get network process up and running. This process will actually communicate with the INTERNET(WOAH!!) - makes request using
BUrlRequest . Html and other resources is received and added to a shared buffer and sent to WebProcess. This has to be called on the runloop( main thread) so we used an explicit call to
callOnMainThread provided by WebTemplate Framework. But Pulkomandy suggested the use of SynchronousListener so which makes the BMessage end up on main thread but in this case it doesnt work consistently(Maybe this is a bug where some work is required!).
Have you guys ever tried BeDC its a cool logging tool in haiku (Give it a try!). We couldn’t perform debug build because it always run of memory at that point of time. Maybe it is fixed with the advent of rpmalloc. But we used BeDC to log and to find which part requires tinkering. It is really colorful too.
And can anyone point me to cool terminal designs? maybe like awesome color combination?
There comes our final part of browser engine - “Rendering”. A backingstore holds ownership of a BBitmap that was painted and shared from WebProcess. Only rendering is done till now, resizing is slow because everytime bits have to imported into bitmap on the UI Process side. It would be nice to have Bitmap access from one process to another through app_server maybe that would be faster.
Check this out!
So my goal was to get webkit2 rendering and I could proudly say I achieved the base work required. Which we could build on top off it…
Hence by the end of GSOC we wanted to have clean code with all debugging statements removed,Indentations were fixed (4 spaces for webkit and tab for haiku), Fixed lot of randomness with the code as well as the MiniBrowser.
It was really a nice summer for me. I learnt a lot throughout especially how to use communication platforms efficiently, Work on a complex project, How to not loose my hopes anytime (seriously I doubted myself many times before taking up this project that its going to be impossible). But when we got that page rendered beautifully no words to speak…
It is a very pleasant experience for me in Haiku and that was made possible again by sweet developers of haiku for making feel welcomed and helped us when things were seeming to be bad.
A special thanks to my mentor Pulkomandy for patiently listening to me, teaching me even smallest of things and also having some chit chat with me despite his busy schedule. Learnt a lot not only with code but related to life too… Readers may think I’m being overly dramatic but I really mean it. Im just so lucky to have got Pulkomandy as my mentor😀.Lots of love from me!! Especially when we are breaking our heads when browser initialization took about 15 seconds, and it was all because of me where I added sleep(15) and that was the reason for such a mess. Not to mention we were working with this 15 seconds lag until the end. When I said this to Pulkomandy, I expected him to be mad but he handled it so cool embracing my mistake(despite the fact that he told me many times not to mess up platform generic code). I would really like to thank Pulkomandy, well thanks can never be enough.I really owe you.
I would also like to thank a developer from Qtwebkit and his name is annulen(Konstantin Tokarev ) for helping me initially to get through webkit code, patiently telling me what each stuff did. Im really really grateful for this. Thank you so much!!
I would also like to thank people from webkit for giving us solutions when we were stuck.
And finally, I promise you guys I’m not going to leave Haiku after GSOC. I really want to grow along with haiku sharing all its successes and failures.
English is my pretty weak spot. I’m sorry if there is any spelling mistakes or grammatical errors or any understanding problems(Hey this is just my 5th post so I’m improving). Also I’m sorry if it was too technical and too long…
I had lot to talk in my mind and I didnt make any plans on what to talk just went with the flow so sorry if I missed something important(Please let me know in the comments)
I have only setup a base for us to work upon there is a lot of pending things that needs to be done before it can be put into use.
- Network process crashes when we visit sites like youtube.com
- Few sites only show white page that needs attention(probably solved by painting the next layer).
- Certificate should be handled properly
- Only single layer is getting painted so compositing is yet to be done.
- Mouse and keyboard events has to be added.
- Webprocess and Networkprocess should be made as a background process
- Resizing and memory consuming is slow an alternate mechanism for sharing bitmaps should be figured out.
- API refactoring - should make API’s easier for developers to include in their apps.
- Get rid of WKRetainPtr - it is soon going to be deprecated and will become difficult to maintain in long run so it should be replaced with RefPtr’s.
- Features like zoom in, zoom out needs to be added.
- Cleanup forwarding headers
- Enable tests for webkit2
- Once all is done(hopefully) update WebPositive
This post is a gist of everything I did in this summer. Steps of work I followed goes like this.
- Enable build fix
- Get code to compile
- Created Minibrowser to test webkit2 API’s
- Got webkit to paint webpage!!
And other emotional,technical stuff. Check out successfully merged PR’s
https://github.com/haiku/webkit/pull/8 https://github.com/haiku/webkit/pull/13 https://github.com/haiku/webkit/pull/14 https://github.com/haiku/webkit/pull/15 https://github.com/haiku/webkit/pull/16 https://github.com/haiku/webkit/pull/23 https://github.com/haiku/webkit/pull/25
List of sites that currently work:
https://www.google.com http://www.gmail.com http://bing.com http://example.org https://text.npr.org/
List of sites that doesn’t crash but shows white page
List of sites that crash
http://github.com http://firefox.com http://youtube.com
THANK YOU FOR SCROLLING BY!!
- GSoC 2020 Project: Improving and Extending Services Kit
- GSoC 2020 Project: Adding XFS file system in Haiku
- GSoC 2020 Project: Input Preferences
- GSoC 2020 Project: Adding UFS2 file system in Haiku
- Haiku activity report - April 2020
- Haiku activity report - February and March 2020
- Haiku field trip report - January 2020
- Google Code-in 2019 finished
- No, I'm not Haiku's lead developer
- Haiku monthly activity report - 12/2019