Language Bindings for the C++ API: Mid-term Report and Third Quarter Goals

Blog post by jalopeura on Sat, 2011-07-09 23:45

The following objects have at least partial bindings:

From the Application Kit: From the Interface Kit: From the Support Kit:
Application
Clipboard
Cursor
Handler
Invoker
Looper
Message
Alert
Box
Button
Control
Font
Menu
MenuBar
MenuField
MenuItem
Point
PopUpMenu
Rect
Screen
StringView
TextControl
TextView
View
Window
Archivable

At the moment, some of the classes are not very usable; they're necessary because other objects inherit from them. They will later be expanded to allow creation of custom objects based on them, so users could subclass a Looper or a View the same way they can now subclass Application and Window.

Python has a minimal test program that uses Application, Window, and Button, but the other objects have not been tested. It would be helpful if interested Python programmers could start coding (and thereby discovering bugs). In addition, I get a large number of warnings about multi-character constants when compiling. Despite following the instructions in the documentation, I cannot get the installer to pass options to the compiler to turn these off. There are also a few other warnings I'm working on eliminating, but the extensions successfully compile and the test program works.

For the time being, the Python objects are all named Haiku.Object instead of Haiku.Kit.Object. This allows me to split up the kits into different extensions; since an extension does not know what kit a foreign object is defined in, the kit name cannot be part of the object name. Since this naming scheme is apparently not good Python practice, I am still looking into alternatives; currently I am considering either placing everything into a single extension or adding some data to my definition files that lets an extension know what kit/extension a foreign object belongs to.

Perl has the minimal test program and it also has a slightly more complex test program (a small Person viewer app). There are also a few compiler warnings I'm still working on eliminating.

The bindings can be found at http://dev.osdrawer.net/projects/perl-haiku-kits. The downloadable files are under "Files"; there are also Forums and a Wiki, and (under "Issues") a bug tracker. Anyone can download the files and look at the content, but if you want to report bugs, post to the forums, or edit the wiki, you need to be added to the project. Interested users with an OsDrawer.net account can email me or post a comment here, and I will add you as a member of the project.

There may be some trial and error involved in adding members; OsDrawer.net has defined some roles for members of a project, but I can't find any documentation on what permissions each role has. There is a role called "Reporter" that I assume lets the user post new bugs, and there's one called "Wiki editor" that's self-explanatory. But there's no role for Forum poster, so I assume one or more of the other roles has that permission included. But I don't know which one(s).

Okay, let's take a look at my second-quarter goals:

- Bring the Python bindings to minimal functionality (Done)
- Write a minimal Python test program (Done)
- Continue to test threading issues (See below)
- Expand preliminary bindings and add new bindings (Done)
- Write test programs for the bindings (Partially done)
- Write documentation for the bindings (See below)
- If there is sufficient time, select a third target language (Insufficient time)

Threading: I have found several issues I thought might be threading issues, but upon closer examination, they were not. I did find one genuine threading issue - but it was because I forgot to lock a window before updating data.

Documentation: I have written some documentation and I am working on programmatically adding it to the bindings. Python has fields in the underlying C++ structures to add documentation, and Perl allows documentation to be mixed in with the code.

Now for my upcoming goals. In general, during the next quarter I will continue to keep an eye out for threading issues, add new bindings, and fix reported bugs. There are also a few more specific things I want to work on:

There are a number of methods that haven't been implemented yet because they have structs as input or output, and the bindings do not handle structs yet. It should not be too difficult to map these to appropriate data types in the target language (Perl hash, Python dictionary).

I'm not sure what to do about globals like be_app and be_clipboard; on the one hand, they could be treated like constants and restricted to a particular namespace, in order to not pollute the global namespace. This is the way I'm leaning right now. On the other hand, there are relatively few of them, and so it would probably not result in a great deal of pollution if I were to put them in the global namespace.

Several of the C++ objects have overloaded operators. I would like to expand the bindings to support these overloaded operators. I'm not sure how much time I want to spend on this issue right now, though. It depends on how much users want this feature.

- Continue to test threading issues
- Expand existing bindings and add new bindings
- Fix bugs reported by users
- Enable structs
- Expose globals
- Add documentation
- Write additional and more complex test programs
- If there is sufficient time, select a third target language
- If there is sufficient time and user interest, work on overloaded operators

Comments

Re: Language Bindings for the C++ API: Mid-term Report and ...

Glad to see that you're getting ahead. But if I may comment once more over the namespace issue? I promise to be brief.

The way it is now, you define the Python classes as part of the modules, but with names outside the namespace of the modules. In Python, this is not just bad practice, it's actually a violation of the language definition. It is impossible to do in pure Python, and can only be done in extensions due to relaxed checks.

What I'm getting to is this: You're not just risking to confuse programmers, but to confuse the interpreter, too. This may lead to all sorts of nasty side effects like the weird "name not defined" errors you experienced earlier on. It may even stop working entirely with future Python releases.

Re: Language Bindings for the C++ API: Mid-term Report and ...

__import__('__builtin__').kittens = 42 ;-)
(Not that I'm disagreeing that it's bad practice.)

Re: Language Bindings for the C++ API: Mid-term Report and ...

P2501 wrote:

Glad to see that you're getting ahead. But if I may comment once more over the namespace issue? I promise to be brief.

Please, comment as much as you want. I learned just enough Python to write the little test program; I learned the C++ API just so I could write this extension. So comments from people who are part of the target community and aware of its conventions are helpful to me; after all, that's who I'm writing this for.

Quote:

The way it is now, you define the Python classes as part of the modules, but with names outside the namespace of the modules. In Python, this is not just bad practice, it's actually a violation of the language definition. It is impossible to do in pure Python, and can only be done in extensions due to relaxed checks.

Well, on the C++ side, I'm not actually defining them as part of the loaded module; I'm getting a reference to the already-existing Haiku module and adding them to that, with an internal name that reflects that namespace hierarchy. But from a Python perspective, I agree, in the sense that the user loads Package.Module and gets Package.Class instead of Package.Module.Class.

Is it really impossible to do in pure Python? I can access a module via its fully-qualified namespace, and once I have access to it, can't I add classes directly as an attribute of the module? (As I mentioned, I don't know Python itself very well, so I could be wrong about this.)

Anyway, it looks like I'm going to have to go with a single global extension after all. It violates the principle of modularity, but it will be a lot less complex and remove the need for "cheating" via eval'd Python code. And it will not be any less modular than the OS itself, since Haiku compiles the C++ objects in question into a single file anyway (libbe.so).

So how do you feel about having a single module extension Haiku, with objects named Haiku.Object. Is this acceptable, or would you still rather have the objects split into kit submodules (Haiku.Kit.Object)? (I say "you" because you're the one who really seams to care; I don't mean to exclude other interested Python users.)

Splitting into kits would be more complex for me to code. It would require some of what jrabbit called "voodoo" in another comment, to add submodules to the single loaded module. It would also require some code changes in the generator to allow keeping track of what kit an object was defined in. It shouldn't be too difficult, just more time-consuming than the changes required to make a single extension where now there are several.

Re: Language Bindings for the C++ API: Mid-term Report and ...

jalopeura wrote:

Is it really impossible to do in pure Python? I can access a module via its fully-qualified namespace, and once I have access to it, can't I add classes directly as an attribute of the module? (As I mentioned, I don't know Python itself very well, so I could be wrong about this.)

Well, there are tricks around it, as scgtrp mentioned above. But basically it's like this: Python code may modify the contents of it's own namespace as well as child namespaces, but not of parent namespaces. You can import names, but you can't export them.

The import statement works quite differently if it refers to a Python module instead of a C/C++ extension. import Haiku.ApplicationKit for instance would normally go check in the library path for a subdirectory named "Haiku" containing a file named "ApplicationKit.py". That file would then be run in the "Haiku.ApplicationKit" namespace, so it can't access the "Haiku" namespace. Now, it is possible with code hooks to partially rewrite the import rules, and the "Haiku" package may do just that. But everyone I know who did this got into major compatibility problems with different Python interpreters.

Quote:

So how do you feel about having a single module extension Haiku, with objects named Haiku.Object. Is this acceptable, or would you still rather have the objects split into kit submodules (Haiku.Kit.Object)? (I say "you" because you're the one who really seams to care; I don't mean to exclude other interested Python users.)

No problem. After all, the Haiku API is designed to live in one single namespace. My concerns were about technical problems and unexpected behaviour, not aesthetics. I don't care too much about proper style unless I suspect problems. For instance, I probably will do this in my test programs:

from Haiku import *

That's enough to make some of my fellow programmers shudder. *g*

Re: Language Bindings for the C++ API: Mid-term Report and ...

setup.py build seems to fail on gcc4 https://gist.github.com/1076454

You should bail out in setup.py if setgcc doesn't return gcc 2. (Or get gcc4 to work)

It looks like you're doing voodoo to make Haiku.* work. Just import the modules in Haiku/__init__.py [with a relative import statement] If you were on github I'd fork and do a pull request with the simple fix.