Issue 4-51, December 15, 1999

Be Engineering Insights: Using C++ in the Kernel

By Jeff Bush

A while back, Ficus wrote a touching newsletter article about a kernel engineer venturing out into the big world of user space. As he didn't have time to finish his latest article before leaving for Christmas, he asked me to fill in for him, so I thought I would tell the mirror story.

I am but a lowly apps engineer, comfortable in my world of object oriented code and fancy tools, and the kernel is a big, strange place. Perhaps the biggest oddity I find with the kernel is the lack of C++. No multiple virtually inherited templatized functors, nor polymorphic container iterators. Most kernel people I talk to shudder and turn white when I mention using C double-plus in the kernel. Undeterred, I'm going to write about issues that bear on doing just that.

Be does not currently support C++ in the kernel, nor has it officially made any plans ever to do so. What I've been told (in a rather serious, fatherly tone) is that you should use C for drivers. So, with that disclaimer out of the way, and I hope without having brutally offended Cyril or Ficus (or any other kernel engineers; they're a very sensitive group), let's examine this more closely.

A driver is nothing more than a shared library that's loaded by the kernel as an add-on. There are three requirements for a kernel add-on:

  1. The binary is of the ELF or PEF format (depending on your platform).

  2. Calls to kernel functions, or from the kernel to driver functions use the standard C style invocation; arguments are pushed onto the stack right to left and the caller cleans up the stack.

  3. The binary has all the runtime support it needs compiled in.

That's it. You may notice that this is a pretty broad definition. Language features such as name mangling, virtual method dispatch, and templates are not taken into consideration. That's because these things are totally internal to the compiler. The (gcc) compiler simply converts a text file containing a high-level language into a text file containing assembly language. The assembler knows nothing about C++ or its many colorful features. You can look at the assembly output of the compiler by using the -S flag for GCC. If you're unsure about the implementation of some feature, don't guess. Use -S and pick through the assembly. It can be very educational.

For starters, I'll point out that using high-level libraries like STL is out of the question. Remember that memory used by the kernel is wired in place and can't be paged out like application memory can. This reduces the amount of physical memory available to the rest of the system. It can't be emphasized enough how very important it is to be frugal with kernel memory.

Although, as mentioned earlier, the assembler (and consequently the binary) is language agnostic, it's important to consider the third point mentioned above. C++ is a little more runtime-library intensive than C. Certain low-level language primitives are partially implemented in a runtime library that is statically linked automatically with user space apps. The assembly output will have calls to these functions embedded when needed. For example:

throw 2;

produces the following code on Intel (I've condensed it a bit):

pushl   %ebx
pushl   $4
call    __eh_alloc
addl    $4,%esp
movl    %eax,%ebx
movl    $2,(%ebx)
pushl   $0
call    __tfi
pushl   %eax
pushl   %ebx
call    __cp_push_exception
addl    $12,%esp
call    __throw
movl    -4(%ebp),%ebx

The functions __eh_alloc, __tfi, __cp_push_exception, and __throw are implemented in a library. On Intel, this library is called libgcc.a. Also, certain C++ specific initializations are done when an image is loaded. The initialization code is implemented in crt0.o. These modules are linked into user space apps automatically. However, neither of these libraries gets linked into a driver. They can't, because they make assumptions about being in user space. Some important C++ features that are dependent on this runtime library support are:

  1. Exceptions

  2. Run time type information (RTTI); this includes dynamic_cast and typeinfo.

  3. new and delete

  4. The handler for pure_virtual()

  5. Static object instantiation

There are two ways to work around the absence of these functions. One could avoid using the feature, or reimplement it in the driver in a kernel-friendly way. The latter can be more work, so consider carefully before embarking on this path. The runtime library is shipped with the compiler and statically linked into executables. Thus, if your driver runs on both platforms, you'll potentially have to implement the same feature for both Metrowerks and GCC, which usually differ in implementation. Worse, if some compiler implementation changed in some subtle way, in order to compile the driver with the new compiler, implementation of these runtime functions would have to be updated.

It's probably best to avoid exceptions, for example. The current exception implementation on GCC generates a lot of large static tables when exceptions are enabled, and can increase the binary size significantly. I've seen increases of around 25% with exceptions enabled, and that's even if you never use them. As mentioned earlier, kernel memory is a precious resource, not to be taken for granted. Also, the effects of uncaught exceptions propagating out of your code and into the kernel proper are fatal. Besides, implementing the stack unwinding code in your driver would take a fair amount of work and be tedious to debug.

new and delete are arguably important C++ features, and you'll probably want to implement them. Luckily, the compiler gives you an easy (and relatively portable) way to do this, by treating them as global operators. For example:

void* operator new(size_t size, const nothrow_t&)
     throw() { return malloc(size); }

void* operator new[](size_t size, const nothrow_t&)
     throw() { return malloc(size); }

void operator delete(void *ptr) { free(ptr); }

void operator delete[](void *ptr) { free(ptr); }

Note the use of nothrow. This is defined in the new header. This is important for handling out of memory conditions. You'll need to call new like so:

SomeClass *obj = new (nothrow) SomeClass;
if (obj == 0)
    // handle this politely

This version of new will generate code to check that the returned pointer is not NULL before calling your constructor or setting vtable pointers (which occurs before your constructor is invoked). You'll have to check to see if the result is NULL anywhere that you call new and handle it properly.

Note also that if you have instance variables that are objects, you must be very careful to check and make sure they initialize properly. This is very subtle, but very important.

The handler for pure_virtual is just a C function. It can be implemented simply as

extern "C" void pure_virtual() { panic("pure virtual
function call"); }

This generally only happens when something is really hosed anyway, say, if you've trashed memory. But you'll get a linker error if you don't include it.

Finally, if you have declare global instances in your driver, their constructors will *not* be called (ever). It's probably not a good idea to do this anyway, as initialization order in a driver is generally important, and it is fragile to depend on the compiler to initialize things in a predefined order. It's cleanest and most prudent to explicitly instantiate everything.

It's important to mention that we often use a two- component driver model in BeOS, where a user level add-on lives in a server, with a smaller portion in the kernel. It's generally better to put all your C++ in the user level add-on and write a thin driver in C to bang registers and handle interrupts. This model can be faster (as you can perform certain operations on the driver without having to enter the kernel), more memory friendly (because the code in user space is swappable), and more stable, because bugs in the user level add-on are potentially less fatal.

As you can see, writing a driver in C++ is more complex than writing one in C. The official Be-sanctioned practice is to write drivers in C, keeping them small and simple. However, it's important to understand the issues involved.

Source Code: <ftp://ftp.be.com/pub/samples/drivers/alphabet.zip


Developers' Workshop: BeOS Driver FAQs (Part 1)

By Todd Thomas

Over the years, Be tech writers and engineers have produced a substantial amount of prose and sample code on the subject of drivers for the BeOS. Really. The problem is that it's not so easy to find it all.

Enter the BeOS Driver FAQs, which will give you a crib sheet that should provide concise answers to your basic questions and serve as a launching pad for your deeper explorations of the BeOS driver universe. This document is making its initial appearance here in the Newsletter, but will soon go to live in the Be Book's "Drivers" section DeviceDrivers_Introduction.html. Without further ado, I give you the BeOS Driver FAQs (Part 1). Look for Part 2 in the next Newsletter.

1. Drivers

Q:

What is a driver?

A:

In general, a driver is software that directly controls a hardware device, and may also provide an interface to the device for higher level software.

In BeOS parlance, a driver is one of three types of kernel add-ons. The other two are modules and file systems. As add-ons, drivers, modules, and file systems can be loaded and unloaded by the kernel as needed at runtime.

For more information on drivers, see "Device Drivers" DeviceDrivers_Introduction.html and "Writing Drivers" DeviceDrivers_WritingDrivers.html in the Be Book. Also, you must read Jon Watte's article "Be Engineering Insights: An Introduction to Input Method Aware Views" in the Be Newsletter (a newer version is available at <http://www.b500.com/bepage/driver.html>. It has very useful discussions of topics not strictly related to the interface in drivers/Drivers.h but essential to the task of writing a driver, such as how to write your driver's interrupt handler.

Q:

Where do BeOS drivers live?

A:

The driver binaries that ship with BeOS live in /boot/beos/system/add-ons/kernel/drivers/bin. Driver binaries provided by third parties should be placed in /boot/home/config/add-ons/kernel/drivers/bin/ because the /boot/beos/system/ hierarchy should not be modified.

For access from user space, the driver is published in the appropriate locations in the /dev hierarchy. /dev is managed by the devfs file system. For more information on devfs and how drivers are published in the /dev hierarchy, see the section "devfs" DeviceDrivers_Introduction.html#DeviceDrivers_Introduction_devfs in "Device Drivers" in the Be Book.

Q:

What is the driver API?

A:

Drivers must implement the API declared in drivers/Drivers.h, which has two parts. One part is used by devfs DeviceDrivers_Introduction.html#DeviceDrivers_Introduction_devfs to manage the driver and publish it in the /dev hierarchy. This part consists of the following exported symbols:

init_hardware()

Called when the system is booted, to let the driver detect and reset the hardware.

init_driver()

Called when the driver is loaded, so it can allocate needed system resources.

uninit_driver()

Called just before the driver is unloaded, so it can free allocated resources.

publish_devices()

Called to obtain a list of device names supported by the driver.

find_device()

Called to obtain a list of pointers to the hook functions for a specified device.

api_version

This exported value tells the kernel what version of the driver API it was written to, and should always be set to B_CUR_DRIVER_API_VERSION in your source code.

Only devfs should call the above functions. So how does other code in kernel space or user space manipulate the driver? Via the second part of the driver API, which is a set of hook functions (the set returned from find_devices()) that maps directly to the familiar posix file-handling functions open(), close(), read(), write(), ioctl(), etc. The full set of hooks can be seen in the device_hooks structure defined in drivers/Drivers.h:

typedef struct {
    device_open_hook open;
    device_close_hook close;
    device_free_hook free;
    device_control_hook control;
    device_read_hook read;
    device_write_hook write;
    device_select_hook select;
    device_deselect_hook deselect;
    device_readv_hook readv;
    device_writev_hook writev;
} device_hooks;

Thus you can manipulate a driver from kernel or user space by using the API defined in unistd.h and fcntl.h on its entry in the /dev hierarchy.

Some drivers may also implement a third kind of API: standard opcodes for the control hook function (which maps to ioctl()). drivers/Drivers.h defines some fairly generic opcodes your device can support if it makes sense, such as B_GET_MEDIA_STATUS and B_EJECT_DEVICE. Arve describes these opcodes as of R4.5 in his article, "Be Engineering Insights: Common ioctls and Error codes for Drivers.

There are also suites of opcodes that must be supported by devices wishing to conform to specific Be protocols. For example, a driver that wants to be compatible with the multichannel audio media node discussed in Jon Watte's article "Be Engineering Insights: Do You Have 24 Ears?" needs to support the opcodes defined in multi_audio.h. This header file has not yet been finalized, but if you would like to work with a preliminary version of it, send a note to trinity@be.com.

For complete documentation of the driver API, see "Writing Drivers" in the Be Book.

Q:

Where can I find driver sample code?

A:

In /boot/optional/sample-code/drivers/ if you installed the optional sample code from the BeOS CD, or at <ftp://ftp.be.com/pub/samples/drivers/>. The Digit driver is a good starting place; the Sonic Vibes driver is meatier but demonstrates how to work with a real device.

2. Modules

Q:

What is a module?

A:

A module is a kernel add-on that exports an API for use by drivers or other modules. This API cannot be accessed from user space. A module is useful for providing services to a class of similar devices so that each device's driver does not have to implement those services independently. Modules can also provide services to other modules.

For more information about modules see "Writing Modules" and "Using Modules" in the Be Book.

Q:

What is an example of a module?

A:

Every binary in the /boot/beos/system/add-ons/kernel/ hierarchy except the binaries in drivers/ and file_systems/ is a module.

For example, in /boot/beos/system/add-ons/kernel/bus_managers/ you'll find usb. This is the USB manager module. It exports an API declared in drivers/USB.h which implements useful functions just about every USB device driver will need, such as install_notify() and get_device_descriptor().

On its backend, the USB manager interfaces with another example of a module—a bus module—which knows the implementation details of a particular USB host controller. For example, /boot/beos/system/add-ons/kernel/busses/usb/uhci is the bus module that knows how to work with UHCI-compliant USB host controllers. Thus individual USB device drivers can be written to one API and yet work with potentially many different USB host controllers. For more information on bus managers, see Arve's article, "Be Engineering Insights: Splitting Device Drivers and Bus Managers" and Brian Swetland's article, "Be Engineering Insights: BeOS Kernel Programming Part IV: Bus Managers."

Another example of a module is the Atomizer, which implements a virtual device that returns a unique token for a null- terminated UTF8 string. In this case, a module is being used not to support hardware devices, but to extend the logical feature set of the kernel. For more information on the Atomizer module, see Trey Boudreau's article, "Be Engineering Insights: Creating Your Own System Services—the Modular Way"

Q:

Where do modules live?

A:

Two BeOS modules—the PCI and ISA bus managers—are built into the kernel.

All other BeOS modules live in the directories found in /boot/beos/system/add-ons/kernel/ (excepting the drivers/ and file_systems/ directories, whose residents are of course drivers and filesystems), and in the analogous directories at /boot/home/config/add-ons/kernel/.

Q:

How do I use a module?

A:

Call get_module() to get a structure that has the module's info and pointers to its hook functions. Call put_module() when you're done. Modules are ref-counted, so if you're the last remaining client, the module will be unloaded from memory when you call put_module().

Here's a quick chunk of sample code that demonstrates how to use the PCI bus module to see if your device is on the bus:

#include <drivers/PCI.h>

char pci_name[] = B_PCI_MODULE_NAME;
pci_module_info *pci;
pci_info info;

int ix = 0;

if (get_module(pci_name, (module_info **)&pci) != B_OK) {
    // handle error
}

while ((*pci->get_nth_pci_info)(ix, &info) == B_OK) {
    if (info.vendor_id == YOUR_VENDOR_ID &&
        info.device_id == YOUR_DEVICE_ID) {
        // device is on the bus
    }
    ix++;
}

put_module(pci_name);
Q:

Where are public module API header files located?

A:

All public module API headers are found in /boot/develop/headers/be/drivers/. Here are some examples:

Bus Managers
PCI: PCI.h
ISA: ISA.h
SCSI (via Common Access Method): CAM.h
USB: USB.h

Miscellaneous Atomizer: atomizer.h area_malloc: area_malloc.h

Q:

Where can I find module sample code?

A:

Module sample code is mixed in with the driver sample code at <ftp://ftp.be.com/pub/samples/drivers>, although it should get its own directory soon. atomizer.zip and xyz5038.zip are good starting points, while buslogic.zip and symbios.zip contain working bus modules for actual SCSI controllers.

Look for BeOS Driver FAQs (Part 2) in the next Newsletter.

Creative Commons License
Legal Notice
This work is licensed under a Creative Commons Attribution-Non commercial-No Derivative Works 3.0 License.