Blogs

VBox guest additions: 3/4 term report

Blog post by scgtrp on Thu, 2011-08-04 04:12

New status report:
major feature dropped; bugs fixed;
did some screen research.

At the start of the third term, it was pointed out to me that Haiku does not actually support hardware 3D acceleration, and to add it would be a larger project than I have the time (or knowledge) for. Therefore, I've had to drop host-accelerated OpenGL from the planned features. I'm somewhat annoyed by this, but looking back it was probably a bit too ambitious anyway, and I'm not convinced I could have finished it in time.

As a result of this quite large decrease in scope, I decided to mentally revise the timeline and spend most of my coding time in the third quarter fixing some of the annoying bugs that had been following me around for a while:

  • Icons can now be dragged around without having to disable MPI first.
  • VBoxService no longer implodes on exit.
  • vboxsf no longer refuses to load on hosts that are running VBox 3 (but symlinks will not work properly as VBox 3 didn't support them).

I've also added guest control support for executing programs - see $(VBoxManage help guestcontrol) for usage instructions. Note that you'll have to set a password on your Haiku user account to use this if you haven't already, as VBox's guest control requires authentication.

This leaves screen resizing, which will be my primary focus in these last few weeks, followed by a bit of cleanup and finally sending a patch off to the VBox people.

UVC Driver -- GSoC Three-quarter-term Report

Blog post by gabriel.hartmann on Mon, 2011-08-01 20:25

Not so long ago, at the half way mark of the GSoc, I was optimistic that I was near to actually interpretting data from the camera in such a way as to produce images on screen. I was successfully grabbing payload data from the camera, the camera's in-use light was on, things were looking good. Since that point, progress has been repeatedly stalled by strange and difficult to debug behaviour.

The first problem which I encountered was that the vast majority of the data received from the camera consisted only of headers with very little actual image data ever attached. In a few seconds of contact with the camera as an example, only 6 out of 7714 packets (0.000778%) contained content. That was worrying. Furthermore the bit indicating an error was often set in the headers, although not perpetually. Investigation of that problem has revealed one of two possible scenarios.

1) Protected content -- This situation occurs if the data source device detects that the video or still-image data is protected and cannot be transmitted. In this case, empty packets containing only headers will be sent for the duration of the protected content.

2) Input buffer underrun -- If the data source device is not able to supply data at the requested rate, it will transmit empty packets containing only headers for the duration of the buffer underrun.

There are other possible error modes, but these are the only two described in the specification which indicate the empty packets with only headers behaviour. Both I and my mentors believe that the second scenario is most likely. However we do not know for certain, which leads to another problem.

Requesting information about what error mode exactly is being encountered of the camera, should be very nearly in exactly the same format as the probe / commit routine which directly precedes the initiation of data transfer. However, whenever the camera is asked for further information about the error mode it is indicating, both the media_addon_server and the camera lock up. Now, it is far from abnormal for the media_addon_server to lock. This has happened countless times during development. But the camera has never locked up befor. Even after a restart, the camera remains locked. A full power cycle is required to bring things back to relative normal. So that's one major intractable issue at the moment, but it is not the only one.

Another major issue was an essentially infinite loop in the VideoProducer's Datapump thread. A division by zero was causing a very large negative number to be included in a sum. The result of this sum was supposed to be a time in the future for which a semaphore grabbing loop should wait. However since the time was hugely negative, at no point could a constantly positive and climbing system time reach this "future" date. Investigations into this issue were going on at the same time as I was investigating the empty packets error. Because my camera was locking up, I had been unplugging and replugging my camera repeatedly. In order to make this easier I had started using the usb plug on the front of my machine. This movement has, I believe, solved the division by zero problem. I haven't done rigorous testing yet, but it was a consistent problem before, and now that I've switched usb plugs, it has gone away. Do you see what I mean about strange and difficult to debug behaviour?

Finally, in an effort to see if the empty packets problem was confined to the beginning of data transfer, and perhaps worked itself out later, I began watching printouts of actually full packets being received. After about 20 seconds of this, Haiku reliably crashes into Kernel Debug Land (KDL). I'm now looking into kernel debugging advice my mentors have given me. Just judging by the kernel debug messages, it looks like an inappropriate NULL pointer problem in the ehci isochronous transfer code.

So, while it would be nice to be writing a deframer or some code for negotiating resolution between the user applications and the driver, with a view towards getting images on the screen at user requested qualities, I think it's probably more important to get the back end working properly. There are only about 3 weeks until the firm pencils down date and I haven't entirely given up hope of one day seeing images, but at the very least I'll hope to have made thorough informative investigations into a series of problematic behaviours that would certainly not be acceptable for end-users to encounter.

On spatial mode and the document-centered interface

Blog post by PulkoMandy on Wed, 2011-07-27 19:37

Once again, the idea that tracker should use single-window mode was raised as a trac ticket. This discussion was made multiple times on the mailing list, and each time the answer from the developper was no. However, users still seem to prefer the single window mode, and other OS are switching to it. Maybe we just need to explain how to efficiently use this mode, and why we think it's better. I'll try to do that in this blogpost, with my own point of view on it.

A bit of history

The spatial metaphors was used quite early in graphical user interfaces. You can already see it in the first version of Mac OS. It was adopted in windows 95, made optional and non-default on windows 98.

The Mac OS version of spatial navigation was quite well done. The desktop is at the root of everything and shows a list of volumes (or disks). In each volume, you can find folders and files. When a folder is already open, it will be grayed out in the parent so you can see that's the case. Double clicking on a grayed icon will raise the already open window.

From the very start, there were other ways of working. While Windows 3.1 program manager was sort of spatial, it didn't match the filesystem hierarchy at all and offered only one level of windows. The file manager was working with a tree view, a concept still visible in later versions of Windows in "explore" mode, and also part of OS/2 way of working. This kind of browser is now known as "single window" or "navigational". Typical features include not opening a window for each folder, allowing a folder to be open multiple times, and a web-browser like history allowing to go back in time.

BeOS did only allow spatial mode. When OpenTracker got released, one of the first changes done by the community was adding a single-window navigational mode to it. This shows how even BeOS users wanted this feature.

Still, 10 years later, we're still using spatial mode in Haiku as the default one.

Spatial navigation

As I said, in spatial mode, a window is binded to a single folder, and each folder can only be shown in a single window. When opening another folder, another window is opened. Each window stores its position and size, as well as the content (icons) positions inside itself. This makes navigating the filesystem a tangible operation. Beginner users quickly get what "opening" a folder means, and how the hierarchy of files work.

There are some quirks to make it more convenient. On Haiku, these include allowing different workspaces to have the same folder open, possibly at different positions ; and symlinks, which are shortcuts in the directory tree to get somewhere quickly. These are hinted by a slight underlining.

The main complaint we hear about spatial mode is that one always end up with a lot of windows opened. This is a learning problem. When programming in C++, each object you create with new must be released by delete (sorry for the technical comparison). Likewise, in a spatial mode file browser, every folder-window you open by clicking must be closed by clicking - keyboard shortcuts also work. It's just done this way. If you end up with too many windows (in C++ this would be called a memory leak), either you open too much, or you don't close enough of them.

In Haiku, we have a nice feature called drill-down menus. When you right click any folder icon, you can see what's inside and navigate through popup menus to your target. The target is then opened in a window, while the path isn't. Learning how to use that allows you to quickly et anywhere without opening more windows than you need. This really seems to be the key, along with getting used to close windows when you don't need them anymore.

On the other hand, there is a lot of material on the internet showing the drawbacks of navigational mode. I just want to point out I'm so used to spatial mode that I now find it annoying to have to explicitly open a new window when navigating to another directory if I want to keep the one I'm in at the same time. Some file browsers (I use pcmanfm on linux) even added tabs in the windows, which make things sometimes even more confusing, as you may have something open in a background tab which itself is somewhere in an hidden window, and thus won't show up in the taskbar. In Haiku I get a list of all the open folders at a glance. I don't see people really using the navigation support besides the "parent folder" button, or even trying to use the "back" button when they actually mean "parent". Going to the parent folder in spatial mode is as easy as typing alt+up, we could have a toolbar button for it if people really want that.

Document-centered

Another feature of Haiku is that it is document-centered. This goes outside of Trancker and file navigation and defines how applications work and look like. The idea is that a window in Haiku maps to a document on the filesystem. You see how this matches spatial mode navigation.

With the document-centered system, you get more windows on screen than when using other systems like MDI (Multiple Document Interface). But you get to manage all these window the same way, and you can mix them together as you need. It doesn't make snese to group your documents in applications, by filetypes, however, it does make sense to put together a sourcecode editor and a debugger, or a word processing application and a drawing application used to insert diagrams in the document.

The problem is a lot of GUI elements are duplicated : toolbars and menus are shown for each document. Mac OS tried to solve that by moving the menu outside of windows, at the top of the screen. This way, it is shown only once for all documents in an application. However, activating a window suddenly shows the associated menu, which is sometimes a bit confusing, and you can't use toolbars.

A later alternative to MDI and SDI appeared later, it's called TDI for Tabbed Document Interface. The idea is to have tabs for each document in an application. You can see that in any modern web browser, or in Haiku's terminal. TDI avoids nested windows like MDI does it, and makes it a bit easier to manage the documents. However, it still doesn't allow to mix documents from different apps in the same area. Haiku has a solution for that, it's called Stack&Tile. Stack&Tile allows to use tabbing accross applications. This makes it a lot more powerful and allows to group windows by their actual use, not by the document type/associated application. Stacked windows suddenly make it a lot easier to manage all these windows. And tracker in spatial mode just fits this perfectly. Tracker is then the application for opening folders, and it works just like any other application.

Note this is very different from the way other OS work. They tend to have one single application take the whole screen space, while we have multiple ones sharing the space, with ability to operate them together (drag and drop, copy and paste, and other ways of sharing data). This is why tracker windows zoom button will make them shink to fit the contents, instead of growing to hide the whole screen. Any application zooming to the whole screen can get very annoying, because that's of no use, unless you're trying to view a really, really big document. With modern high-resolution screens, it's time to share the space ! StyledEdit should enlarge the window to ensure there are no forced line breaks; showimage will fit the current image, some apps are even not scalable at all since the contents always has the same size. Making the zoom button work that way helps getting things done much more efficiently.

Making it more visible

Working the document-centered way takes some learning, but it's reallyworth it. Having tracker fitting in with spatial mode may be of some help, but it won't do everything. Here are some more things we could do (or try to do) :

  • Icon showing open/closed state : this is something that was available in Mac OS since the very beginning. An open folder is grayed out, or shown with an "open" icon. So you know it's already somewhere. The same applies for other documents.
  • Document-centered deskbar : currently the deskbar groups things together by application. This makes it hard to see the document nature of things. Grouping the documents could be done by projects, or S&T groups instead. This would help making the grouping more visible.
  • S&T groups persistence : I already said tracker windows keep their position and size. All windows should also keep their S&T grouping state accross reboots. Even better, this state could be stored as a "project" and reopened later, as needed, restoring the windows as they were. Projects could be mapped to folders, and show all the documents inside the folder. Or they could be an independant stuff on another layer. Projects would be the grouping criterion in Deskbar, so the deskbar menu for a project would allow to save it, besides closing and hiding it.
  • Single window mode has some points

    There are still things that work really well in single window mode and can't be done in spatial. They relate to the ability to decouple windows from folders. So you can have multiple windows showing the same folder. While harder to understand for beginners, this allows for quite powerful use, as each window can then offer a different "view" on the same folder : one may be sorted by filename, and another by date, so you can quickly grab what you're after. The only way to do that in spatial mode is to cheat using workspaces. A partial answer to this is queries : they are window with a different point of view on the whole filesystem. An application called Trax allows you to do queries with a constraint on the filepath.

Bits and Pieces: Notifications and Menu Builders

Blog post by Sil2100 on Wed, 2011-07-27 17:59

It's been a while since I last wrote something here on Blog-O-Sphere. Probably most of you don't remember me anymore - but I'm still around, still experimenting with things Haiku in my free time.

During the weekends, I'm working on enhancing a very old BeOS application long lost in time. While browsing the Haiku kit and application source tree, sometimes I stumble upon some new (at least for me) but also interesting small elements that Haiku added to the Haiku API during its development. I like to try these elements out. Most of these API additions might change or even disappear in the nearest future, since I understand their development process is not yet finished, but they're interesting to know nevertheless.

I know some of these additions might be obvious to those up-to-date with the Haiku source code. But maybe some readers will find this at least a bit informative.

Contacts Kit, Mid-term

Blog post by Barrett on Tue, 2011-07-19 17:16

From my latest post, i had to do more work on the base classes, i realized that my implementation of BContactField was too inflexible for the use so my progresses were not fast as i hoped.
The main problem was to provide something that can fit the simplicity of the Person format (people files) and the complexity of VCard. The result was BContactField. Starting from a number of fields (defined in ContactDefs.h) the class provide some methods to access the field-type of a particular fields and give an interface able to manage properties and parameters as well. At the bottom there are some classes that implements BContactField, they will be used directly by the user, let me give some examples :

BStringContactField
This merge every fields that does not have any other object or special functionalities like B_CONTACT_NAME or B_CONTACT_EMAIL.

BUrlContactField
This is the first example of a field that will incorporate an object from the Haiku's API, in this case it's simple but will be helpful when the BAddressContactField will be added.

How the API help to distinguish between the type of the fields?

You can use the method BContactField::FieldType() but there's a more simple solution, using the visitor pattern i have created a BContactFieldVisitor class, it is nothing more than an interface that specify one Visit() method for every BContactField child class. Implementing a your own class, when you receive a BContactField you can pass your visitor into the BContactField::Accept() method without having any troubles converting the base class to the derived class. One pratical use is for example the EqualityVisitor that allow to implement the method BContactField::IsEqual() in a nice manner. Also the VCardTranslator use a private visitor as a class that convert every field into a VCard string.

While i should commit some changes for BContactField, BContact and BRawContact are basically complete and usable and seems working as well.

Translators

The state of translators is summarily good, at the moment VCard can read and write some basic properties, People is in full development state. The main problem with it was a intrinsic limitation of the translation kit, basically you have to pass a BPositionIO (BFile, BMemoryIO..) to a translator but people files are recorded as attributes. You cannot edit an attribute using a BPositionIO since they are data written outside the file, so at the moment the translator will check if the input/output (as the case) is a BFile and if not it will fail.

Future
At this point i will not make forecasts, i'm working on a plan for the services kit and the addon API in order to begin discuss it with my mentor(s), the first part of the project seems close to be complete so i'll update you with a blog post in the next days.

ZFS Port: Midterm Report

Blog post by GeneralMaximus on Tue, 2011-07-19 09:44

My midterm goal was porting libzpool -- which contains most of the ZFS code -- to Haiku. Another midterm goal was to get ztest -- the ZFS testing tool --- to run on Haiku. Being able to run ztest in a loop for an entire day means that about 80% of the ported code is working fine (though the remaining 20% is the most difficult part of the entire porting process). ztest is a userland test, so actual file system modules or disks are not involved in the testing procedure -- ztest creates block files in a temporary directory and treats them as disks.

It took me days of fighting with the linker and the compiler, but I'm happy to report that both ztest and libzpool build on Haiku without errors! Does that mean I can run ztest for a day without problems? Sadly, that is not the case. ztest is unable to create ZFS storage pools and fails within a second of starting up. I am currently trying to investigate and fix this crash. Fixing this one crash will reveal more crashes, and running ztest with several threads will reveal subtle threading issues. This means I have my work cut out for me ;)

Meanwhile, I have also started porting libzfs, which is the library used by the zfs and zpool administration tools to communicate with ZFS code in the kernel. This communication occurs as ioctl() calls on /dev/zfs. My goals for the quarter term are getting libzfs, along with zpool and zfs, to build on Haiku. Of course, an additional goal is to get ztest to run without crashing.

You can follow the project at http://github.com/GeneralMaximus/zfs-haiku. Building ztest is as easy as cloning the repository, changing into the zfs-haiku directory, and typing "jam ztest". The ztest executable is generated in the debug.X86 directory.

Batisseur Midterm: Gravatars, packaging sugar, and achievements

Blog post by jrabbit on Mon, 2011-07-18 20:03

I’m in the middle of my planned vacation in New Mexico. I recently wrote a new feature for haikuporter emulating the style of homebrew’s create command. (It makes as much of the bep file for you as can be automated.) I’ve written a haikuporter (or later pkgman) wrapper to handle the achievements. It works like git-achievements you alias over the command and check the switches/commands. I’ve moved over some code to redis, and have gotten their development branch to build on alpha3 (Stable used to work back a few revs.). If you have ideas for achievements please file them in the issues tracker

I’ve implemented Gravatar support in the points board web app, it needs to be fleshed out more from the html side, if anyone has design questions or suggestions on how to implement this HaikuPoints system with regard to existing Haiku website features let me know.

The build drone “queen” controller I’ve been thinking about and will use a unique identifier for build jobs to allow cross-referencing with a list of available builds. (i.e. delegating responsibility of picking a useful build up to the client.) The unique ID would be based on GCC, Haiku version (release or nightly), and git-sha or a GUID. This will also allow for the build names to reference a camlistore sha1-blob with build information and binaries. I will be investigating if Jenkins is a good fit for receiving this data or if I will write a viewer for packagers to get build logs.

Syndicate content