Language Bindings for the C++ API: First Quarter Report and Second Quarter Goals

Blog post by jalopeura on Mon, 2011-06-13 16:46
, , ,

During the first quarter, I defined an interface language to use for creating the bindings. I had to create my own for several reasons. Probably the biggest factor was the need to know whether the target language has the right to destroy an object. Most of the target languages has some kind of automatic garbage collection; the programmer never needs to worry about whether to delete an object to free up memory. I didn't want to force programmers to worry about it when they don't normally need to. Therefore, I had to be able to mark whether an object was delete-able, so the generated bindings could delete it automatically if necessary.

I also defined preliminary bindings for a minimal test program, wrote a preliminary generator to create the bindings, and wrote the program itself (a simple application with a window which has a single button). The preliminary generator and the test program targeted Perl.

I also tried changing some data from the test program (with the interpreter running in the main thread) while in the MessageReceived method for the window (which should theoretically lock the window's thread). I was unable to make the program block, so I will need to continue testing for thread issues as I go along. Suggestions are welcome.

I in an earlier post that I would make a decision on target languages based on two criteria, popularity with the Haiku user base and ease of writing extensions. The popularity, based on a poll on the Haiku home page, is as follows:

42 51% Python
14 17% Ruby
12 14% Lua
06 07% Haskell
04 05% Perl
03 04% Scheme
02 02% Squirrel

As you can see, Python won in the popularity category, with over half the total votes and three times as many votes as the next highest language. Extension writing looks like it will be relatively easy as well. With good scores in both categories, Python will be one of the target languages. (In fact, I've already started work on them.)

I already know how to write extensions for Perl, which means it wins in the ease category. Since I did my initial tests with Perl, it will continue to be one of the target languages as I expand the number of bindings.

However, Perl scored quite low in popularity, so I would like to add another of the more popular languages as well. Ruby and Lua both scored more than 10%. If I am able to get the Python bindings working in time, I will research extensions for those two languages. Their scores were close enough that ease of writing extensions will be the deciding factor here.

Report on first-quarter goals:

- Define an interface definition language (Done)
- Define preliminary bindings for a minimal test program (Done)
- Write a preliminary generator to create the bindings (Done)
- Write the minimal test program (Done)
- Test threading issues (Will continue into the second quarter)
- Make a final choice on target languages (Done)
- Expand preliminary bindings and add new bindings (Not done)
- Write test programs for the bindings (Not done)
- Write documentation for the bindings (Not done)

My goals for the second quarter (including the ones brought forward from last quarter) are:

- Bring the Python bindings to minimal functionality
- Write a minimal Python test program
- Continue to test threading issues
- Expand preliminary bindings and add new bindings
- Write test programs for the bindings
- Write documentation for the bindings
- If there is sufficient time, select a third target language

Comments

Re: Language Bindings for the C++ API: First Quarter Report ...

by fengshaun - 2011-06-13 17:34

Wow! Awesome news! Can't wait for the Python bindings. I would say making both Python and Ruby or Lua bindings would be a rather non-optimal use of time, since at least Python and Ruby are quite similar in many aspects. For a change, maybe Haskell or Scheme would be better choices because of their distinction from other languages listed.

Thank you very much! Keep up the great work!

Re: Language Bindings for the C++ API: First Quarter Report ...

by nielx - 2011-06-14 01:49

I also tried changing some data from the test program (with the interpreter running in the main thread) while in the MessageReceived method for the window (which should theoretically lock the window's thread). I was unable to make the program block, so I will need to continue testing for thread issues as I go along. Suggestions are welcome.

Depends on what you really want to do. The Python interpreter does run in multiple threads, but not simultaneously. The interpreter has a Global Interpreter Lock (GIL) that prevents it from running in multiple threads simultaneously. This means that if from both the Application and the window threads the interpreter is called (for example in a callback), one of them will be waiting to acquire the GIL and thus there is not any additional need to use additional locking. Note that this does not release you from locking running loopers when you manipulate their associated handlers, like Windows and Views. Basically you are using the interpreter's lock as a global application lock.

The good thing about languages with a GIL is that you don't really have to worry about much there. The bad thing is that there will be a lot of what I would call context switching when it comes to callbacks like the MessageReceived or Draw() hooks, if that method were to call back to Python when it is invoked. This might lead to some blocking if the python interpreter is doing some heavy stuff in another thread. The obvious solution is to have callback hooks be actively set up so that they are only called if explicitly set. Thus callbacks (like MessageReceived and Draw) do not cause any blocking unnecessarily.

This is of course valid for Python and any other language that employs a global interpreter lock, most notably Ruby. I am curious how Perl handles this problem. Do they provide their own automated locking? Does perl require the user to do the accounting and the locking when running multithreaded? Do they spawn a new (unrelated) interpreter for each thread?

Anyway, to summarize, have a good look at the Python docs on locking. Curious to hear your thoughts on it.