Displaying Newsletter
Issue 11, 09 Feb 2002


  In This Issue:
 
Binary Compatibility in 3 Easy Steps! by Erik Jakowatz 
In the early days of the OpenBeOS project, a debate raged concerning one of the projects primary goals: maintaining binary compatibility with BeOS R5. The idea was that the only way an effort to rewrite BeOS would be successful was if folks could continue running the apps they already had.

Certainly, a lot of software available for BeOS is open source or actively maintained -- these apps could just be recompiled if necessary. Others -- PostMaster, Gobe's Productive suite and a few other crucial apps -- weren't likely to get rebuilt, either because the original author had stopped maintenance without being kind enough to release the source, or because it just wouldn't be commercially feasible.

Some said that we were crazy; that it couldn't be done. Thankfully, cooler heads prevailed and we're well on our way to a binary compatible clone of R5.

"But wait!" you cry. "How did the cooler heads which prevailed know that the holy grail of binary compatibility was achievable?" I'm so glad you asked! Keeping reading and be enlightened, Grasshopper.

There are three basic issues that have to be addressed to ensure binary compatibility:

  • Names must be identical
    This includes class and structure names as well as public, protected and global function and variable names.
  • Object sizes must be identical
    Classes must contain the same number of bytes of data; global variables must be the same size. Maybe BGlobalVar should've been an int32 instead of an int16, but we're stuck with it now.
  • Virtual function table layout must be identical
    The most cryptic and confusing aspect of maintaining binary compatibility. The issue essentially boils down to this: for any given class, there must be the same number of virtual functions, declared in the same order as the original.

The Nitty Gritty

"Good grief!" you say. "How on earth do I keep all this stuff straight?" Ah, Grasshopper, it is easier that you might imagine. Just follow these steps, and you should be binary compatible in no time!

  1. Make a copy of the appropriate Be header file
    This is now your header file. You may need to change a thing or two, but what you can (or will need to) change is quite limited, and discussed below.
  2. Implement public, protected and virtual functions
    In the course of doing this, you may discover that there are some private non-virtual function declarations that you just don't use. Feel free to axe them! Since they're private, nobody using the class will miss them, and because they're not virtual, they don't effect the vtable layout. Conversely, if you find a need to add some private, non-virtual functions, go right ahead (for the very same reasons).
  3. Make sure you don't change the number of bytes used for data
    There are two situations that can make this seem difficult. First, there may be data members that you don't use in your reimplementation. You can just leave them (safe, but a little messy) or you can add the extra members' bytes to the class's "unused" data array. An example will make this clear.
    Let's say we have a class BFoo:
        class BFoo {
            public:
                BFoo();
                void SomeFunc();
            private:
                int32 fBar;
                int32 fQux;
                char  fZig;
                int32 fUnused[2];
        };
            
    The Be engineers that originally wrote this BFoo purposely added some data padding in the form of an array of 2 int32s (they did this with most classes in the API). Now let's suppose in your implementation, you really didn't need fQux. You can add fQux's bytes into fUnused:
        class BFoo
        {
            ...
            private:
                int32 fBar;
                char  fZig;
                int32 fUnused[3];
        };
            
    Nothing could be easier!

    "But what if I don't need fZig, either?" you wonder. "It's only one byte, not four like an int32!" Have no fear! Just rename it "fUnusedChar" and be done with it.

    The second situation that can make preserving object size tricky is if there aren't enough bytes of data available. Building on our cheesy BFoo example, let's suppose that rather than getting rid of fQux and fZig, you actually needed to add another 4 int32s worth of data: fNewData1 through fNewData4. The original implementation of BFoo has two extra int32s which we can use, but that leaves us two int32s short. What to do? The easiest thing to do is create a data structure to hold your new data and convert one of the fUnused items into a pointer to that structure:

        // Foo.h
        struct _BFooData_;
        class BFoo
        {
            public:
                BFoo();
                ~BFoo();
                void SomeFunc();
            private:
               int32 fBar;
               char  fZig;
               _BFooData_* fNewData;
               int32 fUnused[1];
        };
        // Foo.cpp
        struct _BFooData_
        {
            int32 fNewData1;
            int32 fNewData2;
            int32 fNewData3;
            int32 fNewData4;
        };
        BFoo::BFoo()
        {
            fNewData = new _BFooData_;
        }
        BFoo::~BFoo()
        {
            delete fNewData;
        }
            
    Voila! More data without making the class bigger. Notice the added destructor; make sure you're cleaning up your new (dynamically allocated) data.
And there you have it: Binary Compatibility in 3 Easy Steps!

Questions, comments, or corrections? Please let me know!

 
Generic Programming: Encouraging Code Re-Use by Matthew Hounsel 
Generic Programming is a simple ideal. Generic Programming is the art of taking a set of common code and generalising an abstraction that performs the general case and which can be easily specialised to handle unusual cases.

The best examples of Generic Programming are Design Patterns. Design Patterns are a general solution to a set of problems commonly encountered in computer programs. The definitive text is "Design Patterns: Elements of Reusable Object-Oriented Software." This text defines the solution to many architecture problems, such as The Singleton - A class with only one instance e.g. BApplication, or Iterators - Sequential access to a collection e.g. BEntryList::GetNextEntry()

Design patterns are a solution; there is no code for you to use.

While reading many journal articles on the STL and generic programming, I realised I didn't understand any of it. When I read the explanation of Patterns and Iterators in particular Generic Programming began to make sense.

Generic Programming is about not writing code. We hate rewriting code; it is time consuming and prevents us from doing the work we want. Generic programming is simply abstracting the common general solution from a set of them. To use generic code you only need to specialise obscure details. "Modern C++ Design: Generic Programming and Design Patterns Applied" shows how this can be done, and I hope to introduce these ideas to you. FYI: This book discusses how to use the generic capabilities of C++ to implement your design. The author provides an implementation of The Singleton, Functors and other patterns.

I recommend you read both these books, although you might want to brush up on basic templates before reading "Modern C++ Design" and this article.


Patterns of BeOS

Beginning an application for BeOS is not difficult. The applications mostly follow a single pattern.

Define the type of the main window, this will inherit BWindow.
Define the type of the application, this will inherit BApplication.

Define the applications constructor to create a main window.
Define the applications run function to show the created window.

Define the main routine to create an instance of the application
      then run that application.

The general case varies only on the type of main window to create and the application signature. To get a clearer idea lets look at the code for a sample application.

class MBApplication : public BApplication {
    typedef BApplication super;
public:
    MBApplication( void )
        : super( SIGNATURE_VALUE )
    {
        the_win = new WINDOW_TYPE();
    }
    // No destructor as it crashes app.
    virtual void ReadyToRun(void) {
        the_win->Show();        
    }
protected:
    WINDOW_TYPE * the_win;
};
int main( void ) {
    be_app = NULL;
        
    int errorlevel = 0;
    try {
        // Automatically assigned to be_app
        new MBApplication( );
        
        if( B_OK != be_app->InitCheck() ) {
            errorlevel = APP_INIT_FAIL_VALUE;
        } else {
            be_app->Run();
            errorlevel = 0;
        }
        
    } catch( ... ) {
        // Ensure children clean up.
        delete be_app;
        throw;
    }
    delete be_app;
        
    return errorlevel;
}

Many applications only differ in the values of SIGNATURE_VALUE, INIT_FAIL_VALUE and the type WINDOW_TYPE. C programmers might define them as pre-processor macros, that's generally not a good solution. As C++ programmers, we have templates. A template is essentially a compiler variable. We can define these three values as template parameters of the class and function, T for template type and TV for template value.

template< class T_Window, char const * TV_SIGNATURE >
class MBApplication : public BApplication {
    typedef BApplication super;
public:
    MBApplication( void )
        : super( TV_SIGNATURE )
    {
        the_win = new T_Window();
    }
    // No destructor as it crashes app.
    virtual void ReadyToRun(void) {
        the_win->Show();        
    }
private:
    T_Window * the_win;
};
template< class T_Application, int TV_INIT_FAIL >
int MBMain( void ) {
    be_app = NULL;
        
    int errorlevel = 0;
    try {
        // Automatically assigned to be_app
        new T_Application();
        
        if( B_OK != be_app->InitCheck() ) {
            errorlevel = TV_INIT_FAIL;
        } else {
            be_app->Run();
            errorlevel = 0;
        }
        
    } catch( ... ) {
        // Ensure children clean up.
        delete be_app;
        throw;
    }
    delete be_app;
        
    return errorlevel;
}

Notice how little code changes to make it generic. There is still the issue of how to use this generic code. You will still need to have your Application Window and your main function. Except your main function now looks like this one.

int main( void ) {
    return MBMain< MBApplication< AWindow, APP_SIG >, OOPS >();
}

This is syntax is called template instantiation. You are telling the compiler, you want this template entity created with these values. Above, you are creating a MBApplication class that uses an AWindow and has APP_SIG as its signature. You then tell the compiler to create a MBMain function that uses your application and returns OOPS if it can't start. Generic code is shown here by the fact you can use the MBMain function it with any application type. Even an application not derived from MBAppplication. We will try to keep our function and class separate.


There are always problems

There is a problem with the above code: it restricts the signature and initialisation failure value to compile time constants. Also GCC on Be doesn't like that, any attempts to get them to play nicely result in internal compiler errors.

The solution to these problems are to consider that the signature of an application and its initialisation failure as characteristic traits of the class. Generic algorithms have to deal with widely varying types and usage patterns, often needing to vary themselves depending upon a class's traits. If your application had to run a special loading function before it could start you might want to disable the default constructor. As such the construction would be a trait of the class, it would be implemented as a call to new for most classes and specialised for different classes.

Since the implementation of template classes can vary dependant upon their arguments, C++ allows you to specialise templates. That is, you can provide a special implementation for a certain type. The standard library has a specialised version for vectors of boolean values. This allows the vector to save space and compact a thirty two element vector from one hundred and twenty eight bytes down to four bytes, a 32:1 saving.

We declare a trait class as an external type that takes the type it describes the character for, as a template parameter. We then specialise it for our application. To specialise you copy the existing code you supply the parameter for the template. Please Note: that >> will be interpreted as shift by the compiler so remember to leave a space.

template< class T_Application >
class MBApplicationTraits {
public:
    static inline char const * Signature( void ) {
        return "application/x-vnd.unknown-mbapp.unknown";
    }
    static inline int InitFailureValue( void ) {
        return 1;
    }
};
template< >
class MBApplicationTraits< MBApplication< AWindow > > {
public:
    static inline char const * Signature( void ) {
        return "application/x-vnd.unknown-mbapp.unknown";
    }
    static inline int InitFailureValue( void ) {
        return 1;
    }
};

We remove the fixed parameters from the generic application and main routine above, replace the parameters with calls to the trait class.

template< class T_Window >
class MBApplication : public BApplication {
    MBApplication( void )
        : super( MBApplicationTraits < MBApplication < T_Window > >
        		::Signature() )
    // ...
}
// And
template< class T_Application >
int MBMain( void ) {
    // ...
        if( B_OK != be_app->InitCheck() ) {
            errorlevel = MBApplicationTraits< T_Application >
            			::InitFailureValue();
    // ...
}

The above code works perfectly out of the box. A macro, MB_APPLICATION_TRAITS_M, is provided to make it easier to use. Now you only need the sample code, a window class and a reworked version of the following code to create a simple application for BeOS. You can sub-class the application class, with a fixed or template parameter, if you wish to augment it.

MB_APPLICATION_TRAITS_M(
	MBApplication< AWindow >,
	"application/x-vnd.mfh-mbapp.sample",
	1 )
int main( void ) {
    return MBMain< MBApplication< AWindow > >();
}

Generic Programming is about not writing code. Once you have written the code several times you can step back and see its general form. You can use templates to provide "out of the box" code to use. You can use the powerful idiom of traits to extract the type specific and design specific code and generalise your solution further still.


Wait there's more ...

The current trait class is limited by being unable to access any runtime data. The InitFailValue function for example can not retrieve a value from the application class. It can use be_app, but using global variables often indicates poor design and in this case, there is a better one.

If we pass a pointer to the application to both trait functions, the application can be accessed as a normal class. Remember the trait class is not a friend by default. If we pass each function a pointer and it is the pointers type that is the template parameter then we no longer need to have a class template. This is because we can make the functions' templates.

Not only does this give us cleaner access to class data but it also cleans our code up substantially. We no longer need to explicitly specify the trait class for the signature or the return value. Nor do we have to specialise the entire class. We can just specialise the trait functions we need. It is not much of a saving, but it is a good example of template functions. There is one caveat, however, we can not just pass be_app to InitFailValue we must cast it to the correct type.

class MBApplicationTraits {
public:
    template< class T_Application >
    static inline char const * Signature( T_Application * the_app ) {
        return "application/x-vnd.unknown-mbapp.unknown";
    }
    template< class T_Application >
    static inline int InitFailureValue( T_Application * the_app ) {
        return 1;
    }
};
template< class T_Window >
class MBApplication : public BApplication {
    MBApplication( void )
        : super( MBApplicationTraits::Signature( this ) )
    // ...
};
// And
template< class T_Application >
int MBMain( void ) {
    // ...
        if( B_OK != be_app->InitCheck() ) {
            // Cast or it will use the BApplication version
            errorlevel = MBApplicationTraits::InitFailureValue(
                static_cast< T_Application* >( be_app ) );
    // ...
}
// Specialize traits for MBApplication< AWindow >
template< >
    char const * MBApplicationTraits::
    Signature< MBApplication< AWindow > >
    ( MBApplication< AWindow > * the_app ) \
        { return "application/x-vnd.mfh-mbapp.sample"; }
// The main stays the same.
int main( void ) {
    return MBMain< MBApplication< AWindow > >();
}

Generalising the creation of the window and application will be dealt with in another article. There is a new concept called policies that perfectly capture that problem.


Source Code:
MBApplication.zip


Bibliography

Andrei Alexandrescu (1995)
"Modern C++ Design: Generic Programming and Design Patterns Applied."
Boston: Addison Wesley. ISBN: 0-201-70431-5.
WWW:
http://www.awl.com/cseng/titles/0-201-70431-5


Gamma, Erich, Richard Helm, Ralph Johnson, John Vlissides (1995)
"Design Patterns: Elements of Reusable Object-Oriented Software."
Reading, MA: Addison Wesley.

 
State of the Union by Michael Phipps 
OpenBeOS was proposed as an idea on August 18, 2001. We are quickly approaching our 6 month anniversary. I thought that this would be an appropriate time to discuss where we are, where we are going and how to get there.


So. Where are we?

We are far more organized. Basically, we have built an organization from the ground up. An evolving organization that is neither too rigid nor too large. We have a great team. Aaron, our recruiter gets volunteers placed in teams and gets their info to Daniel and I for the web site and SourceForge access. Daniel designed and built our web site from the ground up. Rich is our new project manager; he will be keeping track of team statuses and helping to put a more organized overall project plan together. Deej is our newsletter editor, possibly one of our most well received ideas - the newsletter, that is.    ;)    We have a dozen or so team leaders, each responsible for a portion of the OBOS. They communicate with their team memebers, with each other and with Rich and I. We will be updating the web site more as we get into the groove with statuses.

We have a well defined plan. We are recreating R5, to begin with. We are using a home made network stack and application server. We are basing our kernel on NewOS. We are not building browsers, Windows emulators, games, graphics packages, movie makers or anything else. Not that any of those are bad, but they aren't what we are doing today. We are focused on the OS, and we will stay focused on the OS. We will be binary compatible, barring any huge, impossible-to-overcome, provable issues. We will (obviously) be source code compatible. We promise that we will work on x86 machines. Not every one, but as many as possible. PPC support would be nice, if enough people join who are interested in doing so.

We have code! By the time you read this, there should be a new build system in source control. The old system was a blind stab in the dark. The newer system was designed and built to fix the badness in the old system. There are several kits that have made massive progress, others that are making progress and some that need more help and time. Some of the kits are large and difficult, while others are more easy to implement. Some pieces, as well, are "freebies". Mail kit, courtesy of Dr Zoidberg, and OpenTracker, courtesy of Be and Axel are good examples.

We have interest! I can't even begin to tell you about all of the interesting stuff going on. Some of it is confidential. Companies that want to use OBOS, but can't disclose that to the general public, yet. I get about half a dozen emails a week about this sort of thing. We have a couple of distributors who are interested in building OBOS distros, when we have everything in place. We have a mailing list that is often exceptionally busy. We have tons of hits on the web site.


All good. Where do we go from here?

The time is coming (very soon) for a first, partial release. Some of the easier kits and some of the freebies will be released shortly. THIS IS NOT A FULL RELEASE! There is no kernel. No app_kit. No networking. These will be kit by kit replacements for R5. This is a chance to get our work out to the beta testers, to draw a line in the sand and say that some things are done, and to transition people to their next part of the project. Look for this soon!

The other, bigger kits are making a lot of progress. I am sure that you have seen some of these things on the web site:

  • Media Kit plays sounds
  • App Server prototypes
  • Kernel reading a Be created driver

While these things do not a release make, they are milestones that let you know that progress is being made. And that is a good thing. While we don't want to release any code before its time, we certainly also realize how important it is to have regular updates.


How do we get there?

The way I see it, we are in the middle of the project. The initial excitement is over. Most of the people who are going to hear of OBOS have. Most of the people who are going to join have done so. This is the part of the race that separates the dedicated from the band wagon members. At the beginning, there is a lot of excitement about what can be done. At the end, there is a lot of excitement about what has been done. Results are visible and fixing bugs can be fun at that point. The middle is the part of the race where people realize that running is hard work. Harder than they expected. That they won't have as much time or be able to code as fast as they thought. It is easy to become discouraged during these times. This is the danger in the middle of the project, and the reason that this interim release is so important.

If you are a developer, I say GO GO GO! For years, we watched Be and advised them. This is our chance to *BE* them. Pardon the pun. If you are a tester, do your backups and get ready. If you are an interested observer, help out! There are still a lot of things that need to be done. Can't code? Test. Can't test? Buy apps. Can't buy apps? Volunteer with BeUnited. They are a volunteer driven organization who wants to make an OBOS distro. They need a lot of ***DEDICATED*** help.

This project has been a lot of work and a lot of fun, so far. I have learned more than I ever thought possible. And I learn more every day. I hope that it has been fun and educational for everyone else. Keep your chins up and heads held high. We *WILL* get through R1. We *CAN* do this. And when we are finished, we will have earned a place in the industry that few can claim. After all. How many of your friends can say that they wrote their own OS?