Haiku QA team: looking for volunteers!

I don't have much time or any QA experience but I'm happy to help as much as I can. Perhaps a good start would be to simply create a Haiku "bug squad" to go through old bugs and see if they still exist.

I think your list sounds great (though ambitious) and I would be happy to act as a Haiku developer liaison with the QA team, if such a thing is necessary. While I should probably spend most of my time developing, I do have a great ability to find bugs, so I could also play some QA role.

One thing which I may work on soon is an automated test tool, with the first initial goal of having an automatic test suite of the Haiku menu system. Right now our "menu tracking" code leaves a lot to be desired, and it needs to be replaced, but it has years of subtle bugs fixes in it. Plus menus are a pain to test manually, and our current code doesn't even fully emulate the BeOS menu behavior, so having a full menu test suite would greatly increase the speed at which a replacement could be made, and also ensure it works how we like (emulating BeOS or not.) I know I sure have more confidence in refactoring my day job Ruby code when there are good RSpecs behind it.

Given that there is now renewed discussion of the alpha4 release, and I may be the release coordinator, I may not have time for the above soon, but it is definitely on my list.

Regards,
Ryan Leavengood

Working off of #1 from DCATT's list, the next step is to focus on what specifically needs a regression test. Ryan, what would you recommend as high value targets for testing? I would suggest that we make a manual first pass. This is gets us acquainted with the tasks at hand, and also gets us started a bit quicker, while we think about automation as a phase 2.

Hi,

I am eager to join any testing effort for Haiku. Please see my bio for details. I would like to help out with automation or exploratory testing, although doing both always seem to produce the best results.

Wouldn't it be great if Haiku was so good it became a standard for some apps? I'd like to see it be a (the?) standard for cross-platform development work with a very wide selection of languages and APIs all working as seemlessly as possible. I like to work in many different languages (perl, python, groovy, etc.)

The future may be running multiple OSes, each on its own virtual machine. We will be going to 8 and 16 processors on the desktop, it would be nice to be able to exploit all this power using VMs. And it would be really great to see Haiku on a VM. Even better would be if Haiku was the VM, but who knows.

James Goss
Pasadena, CA

Well, I guess "overqualified" is better than nothing ... a bottle in front of me is better than a frontal lobotomy, Tom Waits might still say.

It is, after all, an operating system and there is lots that can go wrong - showstoppers, non-reproducibles, performance issues, usability, stability, boot times, APIs and compatibility, drivers, and so on. Some things with operating systems you don't really test, you just verify that after the changes are made, stuff still works about the same.

Anyway, I'd love to help out and it would give me a chance (excuse?) to really get into Haiku. I used to lead projects (as technical lead at Sun Microsystems, and as project lead sometimes, too) like this, so I feel right at home.

When can we start? :-)

James

In case everyone has not subscribed to the QA list yet (I just did myself), I figured I'd post a last message here, but from now on I'd suggest we discuss things on the mailing list.

Also someone should feel free to create a #haiku-qa IRC channel on irc.freenode.net.

Anyhow, we are starting the plans for an Alpha 4 release, which is currently scheduled for September 1, but that may be overly optimistic and certainly is not set in stone. Obviously I seriously doubt we can get a full QA process going for the Alpha 4 release, but if those of you who want to participate in the QA team could play a role in testing the release and see how we do that, I think it will be a great experience for all of us.

I am the release coordinator for Alpha 4, which mainly means being the developer who manages the release branch, with merging and cherry-picking relevant bug fixes from the master branch into the release branch. We have not yet done a release while using Git, but I'm reasonably experienced with Git and I expect the branch management to be much, much, much more pleasant with Git than it was with Subversion.

Right now fortunately Haiku is in pretty decent shape, and while plenty of bugs remain, overall things run pretty well. WebPositive was recently upgraded to a newer WebKit though, and there may be various regressions there, beyond all the bugs and lacking features it had (or didn't have) before ;)

But there is no way all of those can be fixed in the next few weeks, so we will just triage and do the best we can, and I'll just have to put some work into WebPositive and WebKit after the release.

Also one major feature which will be released as part of Alpha 4 is better wireless support, in particular support for WPA security, which was a pretty major lack in Alpha 3. Unfortunately the GUI side of wireless management is just not there at all really, and again I don't think we can pull that off for the release. But frankly we have gone too long without a release so it really just needs to get done.

While there definitely are quite a few changes from Alpha 3, I don't know if the release notes will be quite as impressive as they have been in the past. In a sense I consider this a maintenance release, if such a thing makes sense for something called Alpha (but as has been said our alphas are like some people's released product.)

Anyhow, I'm probably rambling a bit. Back on point: let's start with a basic manual testing process, where the various QA members can divide and conquer testing Haiku itself and the various included applications. As a quick example, we need to make sure ShowImage can:

1. Open images when they are double-clicked in Tracker.
2. Open images from the menu.
3. Open all images formats supported by the Translation Kit.
4. All menu items work as expected.
5. Drag and drop works.
6. Selection mode works.
7. And much more...

If the tests are all documented they can later serve as a template for automated or semi-automated tests. The Haiku User Guide would be a great place to start to better understand some of the major features of the included Haiku applications (even I learn stuff when reading it.) Everything in the User Guide should work as expected (unless the User Guide is out of date, which is something else which should be found and fixed for the release.)

OK, I think that is enough for now, let's try to talk more on the mailing list.

Regards,
Ryan Leavengood

I would be happy to join a QA team as I have been on many in the past (I also love breaking things and trying to fix them) I have subscribed to the mailing list so I would be happy to start straight away.

-Matt

I would be happy to assist. I actually work in the field of consumer software products and support about 10 myself as well as write up documentation. I subscribed to the list so let me know.

-Chad