Most of us are used to how BeOS handles multiple workspaces: each workspace represents a configuration for the screen, including resolution, colours, and desktop background. The visual appearance, that is UI colours, scroll bar design, etc. is maintained per desktop, though - all workspaces share the same properties in this regard.
It's maybe not so obvious that every workspace also knows which windows are open on it, and in what order they are. When you switch between workspaces the position and order of the windows is usually preserved. There are exceptions when it comes to windows that are visible on multiple workspaces, but I never quite understood why this is and how it is working. Adi Oanca has implemented a similar behaviour for the Haiku app_server, and I hope that he figured out how it worked or even found a better way how these windows are handled.
Since it would probably be boring if I just wrote which bugs I fixed in the app_server today, or which function I renamed to make it easier on the eyes, I think I can better use this forum to introduce some app_server concepts, in varying detail, and in small steps.
The first concept I will introduce you to is the Desktop. Not the Desktop you see as part of your daily (?) computer experience, but the Desktop as the app_server manages it. Every user logged into your system will get a different Desktop object. Every user? Well, for now, this is just you - but the app_server can manage as many desktops as needed. If you notice this for R1 remains yet to be seen, but the chances for this are not too bad.
As many of you know, we are using FreeType as our font renderer. It is already able to produce very nice font renderings, but the quality of the rendering still seem to improve with every further version.
Until today, we were using FreeType in a wrong way, though. That didn’t affect the rendering quality, but let the app_server crash pretty regularly (note, it still does that, but for different reasons and even a tiny bit less often 8-)). Somehow we assumed that FreeType was safe to be used in a multi-threaded environment. It even kind of is, but not in the way we used it.
FreeType provides an FT_Face object for every font face you are using - these objects are completely separated and can be used from different threads without worries. You just can’t share one of these objects between several threads, and that’s exactly what we were doing. Of course, that was a more or less stupid oversight on our behalf. But it’s also a major pain that FreeType simply doesn’t allow you to access a font face in a shared manner - it gives you a very easy to use API, but also a very limited one.
We just started discussing this topic in the app_server mailing list, so I don’t know yet to what conclusion we’ll come. We will probably need to introduce another abstraction layer between FreeType and our font sub-system, so that we can efficiently use FreeType without being locked in its single threaded design. Right now, I’ve introduced a per font face lock that might not even cover all uses of the object, but greatly lifts the situation.
If you know FreeType well, and feel that I missed something looking at the API it provides, feel free to tell me :-)
As I advertised yesterday, I had a short look at hyper threading today. Well, that short look was actually a bit longer than what I hoped for - and it even still doesn’t work.
While starting up the second logical processor is working almost fine - for some reason it needs some longer timeouts between the initialization commands - I get no interrupts anymore, when both processors are running. Have I said none? Not true, as the inter-CPU messages (which are interrupt based) are still working. But even the APIC timers don’t trigger an interrupt anymore (the timer interrupt is what makes the CPU switch to another thread from time to time).
So both processors are running the thread they were given at some point. Sometimes they went through until the disk manager is trying to read something from a disk (as you need interrupts for that), sometimes they hang before - probably depending on which CPU gets which thread. The page daemon for example, isn’t likely to get us that far :-)
Anyway, the APICs are correctly initialized and enabled - the only thing that I still don’t touch is the I/O APIC which is responsible for dispatching interrupts to the local CPU APICs. But since it’s working fine in a uni-processor environment, it should also just work with both CPUs running.
So even though Hyper Threading seems to be popular, I am postponing it a bit - it would definitely take some more time to get right. I’ll probably tackle it again when the system doesn’t use the virtual wire mode anymore for SMP (in a few weeks, I’d guess). Also, to really configure the system correctly, we should look at ACPI which isn’t available in the boot loader right now.
If you want to give it a try yourself, Hyper Threading is currently disabled: to enable it, you’d have to uncomment lines 228 to 231 in src/system/boot/platform/bios_ia32/smp.cpp. But don’t get your hopes up too much :-)
Now, it’s app_server time.
Tomorrow, I’ll have a short look at implementing Hyper Threading support - not the full monty, but it would still be nice to have it start using the two logical processors in my system; after all, I bought this system with enabling Hyper Threading in Haiku in mind.
After that excursion, I will start looking at the app_server again. I had started to refactor the code some weeks ago, but got interrupted, and didn’t find the time to continue this effort. I hope to make the app_server stable in the next few weeks - it’s one of the biggest show stoppers for Haiku right now: the kernel should decide when the time for a reboot has come (read KDL), not some bloody userland application :-)
Anyway, the app_server is the single most important application running under Haiku, and it’s in many regards as critical as the kernel. When the Application Kit/Interface Kit/app_server triumvirate works as expected, we should be able to run almost every R5 or Haiku application under Haiku. And that should enable us sooner or later to start distributing official alpha releases - not that we’ll be able to work with these in a productive manner, but it’ll be a major step forward.
I just booted into Haiku working on an SMP machine. Unfortunately, I am not really sure what change exactly triggered this - I’ve tried so much and all of a sudden it started to work, after I disabled setting up the APIC (the advanced programmable interrupt controller) to use ExtINT delivery mode - that shouldn’t tell you anything, I know, but it’s still remarkably that this code was originally disabled as well.
It took me quite a number of hours to get it working, so it’s a bit frustrating not to know what was actually responsible for the hickup, but it still didn’t make me that curious to start an investigation on this topic for now…
Anyway, our SMP configuration is pretty weak right now - it only supports virtual-wire mode which is only one of two possible modes every compatible IA-32 MP system should support. We don’t yet support to run the system in so called symmetrical MP mode - that would require us to do some more APIC programming for interrupt redirection, which I obviously didn’t need to do to get my machine up and running. Bad for Haiku, but good for me :-)
Next on the list are some more SMP related changes, as some things like call_all_cpus() are not yet working. I expect to finish the remaining SMP work tomorrow, and that’s when all the testing can begin on your end. A serial debugging cable (with a second machine) would be very helpful, though, in order to get me useful information about what went wrong. Your effort is surely appreciated!
Older SMP machines could or even should work now, but I would be surprised if the same could be said for current SMP machines - but as I don’t have access to such a machine, it’s not on me to find out about that now.
Even though I usually don’t work at the weekend, I had to, since I didn’t manage to work 8 hours on friday.
Unfortunately, I still haven’t got SMP to work yet. I’ve investigated the issue, and came to the simple conclusion that the APIC interrupts doesn’t reach their goal (it just took me some time to get there, and exlude all other possible faults). I can trigger such an interrupt manually, so the second CPU is setup correctly, but its APIC doesn’t seem to be. You don’t understand a word of what I just said? Well, let’s just say one CPU doesn’t manage to talk to the other CPU (through the APIC, the “advanced programmable interrupt controller”).
Basically, the boot process is halted as soon as the first CPU tries to tell the second CPU what to do, and then waits for an answer - until you stop it.
I haven’t been able to find a bug in the initialization code yet, and I haven’t even started looking at the I/O APIC, but I still hope I can figure out what’s going wrong on monday.
It took a bit longer to get the dual machine up and running again - it has two 500 MHz PIIIs and the hard drive is a bit older as well, so it took about two hours to update the source repository and get it compiled.
While waiting for the machine to complete its task, I had the time to look into some other known issues of our code, and clean the signaling code a bit. We are now able to handle signals with interrupts turned on, some minor bugs went away, and there is now support for sigsuspend() and sigpending() - nothing earth shaking, but definitely a step into the right direction.
There were some other distractions, so I played around with SMP only shortly - I am just sure now that it still doesn’t work :-)
Shortly after the second CPU steps in, both CPUs seem to come to a halt. I don’t know yet what’s causing this, but it seems to be a single basic problem - let’s just hope I don’t waste too much time searching for it.
I’m done implementing sub transactions for now - I haven’t yet tested detaching sub transactions, but everything seems to work fine. Time will tell :-)
A complete Tracker build now dropped from 13.5 minutes to 5.4 minutes - that’s great, but BeOS R5 does the same job on this machine in around 2.5 minutes, so even while this is an improvement, we still have a long road ahead of us. I can only guess where we lose those 3 minutes for now, but I am sure we’ll find out well before R1. One of the responsible components should be the caching system, as it still only looks up single blocks/pages, instead of doing some bigger reads and read-ahead.
Anyway, since Adi is still working on the app_server, my next assignment is getting Haiku to work again on SMP machines. While it may seem like luxury right now, having an SMP machine to test a multi-threaded system on is almost mandatory. Let’s see how many related bugs have sneaked into the system - I only know about one particular piece of code that won’t work well on those machines (and I am to blame for that one, out of pure laziness).
The machine I am testing on is a dual PIII (with Intel BX chipset) that was generously donated (or lent :-)) by Ingo, one of the best developers we have on the team.
A small update to the BFS incompatibility: I’ve now ported the original logging structure to the R5 version of BFS as well, so that the tools like bfs_shell can now successfully mount “dirty” volumes, too. I also found another bug in Be’s implementation, and needed to cut down the log entry array by one to make it work with larger transactions.
Now I am working on implementing sub transactions. If you have tried out Haiku and compiled some stuff or just redirected some shell output to a file, you undoubtedly are aware that this takes ages on the current system.
The reason for this is that BFS starts a new transaction for every write to a file that enlarges its file size - and that’s indeed a very common case. Since writing back a transaction also includes flushing the drive caches, this isn’t a very cheap operation - it slows down BFS a lot.
The original approach taken by Be Inc. was to combine several smaller transactions to a bigger transaction - problem solved. The downside to this approach is that you lose the ability to undo a transaction. If you need to undo some actions, you have to manually undo the changes in the transaction that would have belonged to the small transaction.
That works but also complicates the code a lot, and is a welcome for any kind of bugs (and that’s one more reason why file systems take ages to become mature).
In Haiku, we introduce the concept of a sub transaction: you can start a transaction in the context of the current transaction, and then abort only the sub transaction instead of the whole thing. As soon as the sub transaction is acknowledged, its changes are merged with the parent transaction - at that point, you cannot revert its changes anymore, you can only still revert the whole transaction.
The only downside of this approach is that it uses more memory, as it has to store the changes of the sub transaction alongside those of the parent. The largest transaction that is possible with a standard BFS volume currently consists of 4096 blocks - so even the worst case should be acceptable.
If a sub transaction grows too much, it can be detached from its parent - since the parent transaction itself is done already, it can safely be written back to disk.
I hope to finish implementing sub transactions and use them in BFS until some time tomorrow. Depending on the number of bugs I add to the code, it might also go faster, though :-)
