jack2

Jack2 : A Personal Analysis (Part #2)

Blog post by Barrett on Mon, 2013-09-30 16:28

Hello, it has been passed some time from the Part 1 of this article, I've continued my investigations as well, and I have finally some more clear plans for such a hypotetic Jack2 port. Unfortunately i had not enough time to research a bit more in the latency differences between the media_kit and jack, sorry, this should be post-poned until i have precise emphyrical measuring methods.

To better understand this article i suggest you to read the first part.

The Wrapper

So, after lots of temptations, a night i decided to take the jack API headers and begin to stubb a jack wrapper.

This gave me the possiblity to explore in a better way the internals of Jack2.

The result of this work-night, is the jack_simple_client seamless working. Don't misunderstand me, the code is just a proof-of-concept, actually the backend media_node support only producers hooks, and have some problems such as the sound hear a bit streched. Other than being funny, this project has permitted me to understand better how the BMediaNode class work. I would like to synthetically make a resume of the work,

it consists of three layers :

C-API bindings to c++ internal classes
Jack2 emulation which is composed of two classes at the moment, JackPort and JackClient.
JackNode, a native media_kit node controlled by the emulation layer, which serves as the gate between the Jack emulation and the media_kit.

This is a map of the functions on which i worked and their status :

E = empty
P = partially implemented
OK = fully implemented / sufficient implementation ATM.

jack_activate OK
jack_client_close OK
jack_client_name_size OK
jack_client_open OK
jack_connect P
jack_deactivate P
 
jack_get_buffer_size OK
jack_get_client_name OK
jack_get_ports P
jack_get_sample_rate OK
 
jack_midi_clear_buffer E
jack_midi_event_get E
jack_midi_event_write E
jack_midi_get_event_count E
 
jack_port_get_buffer P
jack_port_name OK
jack_port_register P
jack_port_type_get_buffer_size OK
 
(the original implementation is mantained)
jack_ringbuffer_create OK
jack_ringbuffer_free OK 
jack_ringbuffer_mlock OK 
jack_ringbuffer_read OK
jack_ringbuffer_read_space OK
jack_ringbuffer_write OK
jack_ringbuffer_write_space OK
 
jack_session_event_free E
jack_session_reply E
 
jack_set_buffer_size_callback OK
jack_set_process_callback OK
jack_set_session_callback E
jack_shutdown E
 
jack_transport_query E

You can see the code here.

Intrinsic Limitations

There are some intrinsic limitations due to different designs, which might be a bit complicated to deceive.

Jack2 support multiple connections to one port.
There's not a lot to do about it, the only solution without explicit support from the media_kit is to map one jack port into multiple input (or outputs). For example, in Haiku a stereo output is a unique port, you can't connect two mono channels to a stereo one without using a mixer node, in Jack2 instead stereo channels are separated by default.

Anyway i do think a feature to separate stereo channels should be taken into consideration for a future media_kit version.
**EDIT** To be more clear all implementations of JACK permit connections between ports with any valence, they can be 1:1, N:1 or 1:N.
Jack2 support glitch-free ports reconnection.
Well, as far as i know the media_kit just is unaware of this concept.

**EDIT** In those hours, emerged that it relates only a jack2 implementation detail of
the API, so it will be a limitation for a native port but not for libjackcompat. That's a good news indeed.
Jack2 features a session API which is simply missing in Haiku.
This is something already discussed in the mailing list, basically other than being a low-priority part of the API, it could be just emulated until Haiku features a session management system, which will probably come out after R1.

The Jack port and the wrapper

I discovered that there is not need of an external app routing audio between jack2 and the media_server.

With the premise that the incompatibilities in libjackcompat and the issues i explained in the last article are still valid, since the jack_ports management is done by the driver i think an Haiku driver could work that way :

When a jack client is registered a media_node is created, for every port there will be a input/output in the node.
When a media_node is created, it will be mapped into a jack_client, and every i/o mapped onto a jack_port.
The phantom clients will be internally managed to work as audio streams mirrors.
By default there will be a media_node wrapping the jack system_mixer into the Haiku's one.

But i would like to show you an use case where it may be complicated to create an efficient and transparent strategy :

Imagine the jack_client1:port1 is connected to jack_client2:port2. jack_client2 is a media_node in the back-end, so it will wrap the port2 into a native input.
So, now imagine a new jack client, called foo_client, what happens if it try to connect to jack_client2:port2? In jack it's completely legal, but in the media_kit there will be only a input, and no way to make a double connection.

As you may understand, this is a headache. To my mind comes only two different solutions :

Make the behaviour somewhat configurable in settings and in the media_node settings panel. So that you can for example map two jack ports into one media_kit stereo connection.
Force users to use appropriate mixer nodes between connections.

So, What's The Need for a wrapper?

Except the fact that with some degree various simple apps should work out of the box, i think a wrapper is still important for a lot of reasons, if it will work for basic functionalities, it could be used by third-party developers as a initial layer to base their native Haiku port.

For example, as start you can compile your jack2 program, then you include the wrapper into your source tree and begin to modify it to fit your needs.

The last but not less important thing is that the libjackcompat backend will provide most of the functionality needed to create the backend-nodes, so it may be possible that a media_kit jack2 driver will be based itself on this library.

As before, i just tried to explain my vision, hope it was interesting...as ever
any opinion, question and correction is appreciated!

Jack2 : A Personal Analysis (Part #1)

Blog post by Barrett on Tue, 2013-07-02 12:40

jack2, media_kit

Intro

In the last year i managed to play a bit more with the Haiku media_kit.
It was already discussed in various places if jack2 should be adopted, ported, or someway integrated into Haiku. There are various opinions out there, and more or less going down into the topic i want to show you what i think about.

Jack2 is a real-time audio server for UNIX systems, to be more specifical a smp focused reimplementation of Jack (which is single-threaded), it provides a protocol used by audio apps for inter-communication. It is gaining a very good number of apps supporting it and it's becoming interesting also for professional audio.

Besides GNU/Linux there are ports available for various operating systems, such as FreeBSD, Windows, Solaris and Mac OS X. The interesting thing about the OS X port, is the availability of a JackRouter which allow to route audio buffers between jack apps and core audio apps.

The idea is a lot similar to what the Haiku media_kit provide.
There are various differences, which includes jack lacking of video support and multiple formats.

Jack force every app to use 32 bit floating point values for their buffers, and the motivation of that is probably to prevent any latency due to resampling algorithms. This is very different from what the media_kit do, since in Haiku a mechanism is provided to support format negotiation and painless audio resampling.

The following text is a resume of some articles i wrote in my blog.

The Narrow Way

So, a day i decided to checkout the source and i started some exploration, to see how it's feasible a port of jack2, the resulting work is located at my github page and the haiku specific code is located in a subdirectory as other operating systems. Unexpectly it worked, trying to connect jack_rec and jack_simple_client produce the expected sinusoidal sound in the file.

Unfortunately excluding for now the noticeable lack of any media_kit backend, there are some other problems to be resolved before :

The jack shm implementation is not working as should (probably due to http://dev.haiku-os.org/ticket/2657)
Haiku is not supporting posix real time thread priorities (http://dev.haiku-os.org/ticket/8600)
I had to comment some memory locking related code, due to the lack of mlock/mlockall support.
Jack warn about lots of graph reorders.
jack_test is not passing some tests.

In the meantime, i've also done some brainstorm for possible solutions (in the same order) :

The best solution is probably to just fix the bug.
The idea is to replace the actual HaikuThread class (substantially a temporarily hack) and set
it to inherit directly from JackThread instead of JackPosixThread, this would allow to use internally
the Haiku implementation of threads, just emulating what jack is expecting.
Don't know if Haiku lack something preventing to have the system support them. AFAIK Haiku areas are obviously supporting this functionality, so it might be possible to just insert some ifdefs in the Jack implementation, or an Haiku implementation alongside with the POSIX and System V ones.
The latests two are probably related to the others.

So without an app to route sounds between jack and the media_server, we will still have
two separate worlds, since i see very difficulty to make jack2 and the media_server friends (since both are asocial animals).

I don't think this is a very good scenario to have and i don't see it haiku-stylish. But at the same time, i would love to see a hybrid jack2/media_server client, publishing a tunnel allowing to pass audio between the two servers.

Said that, my personal opinion is that reaching a stable and performant Jack2 could potentially require a lot of work for poor results, the amount of work might easily vary depending on the problems encountered in the steps. This was my feeling when i tried to create a simple jack capture node playing buffers using BSoundPlayer in the backend.

Rise And Shine

We are not lost, there is already another way, which *may* be better. The idea is as simple as difficult : force Jack2 applications to use the media_server by wrapping the jack API.

This consist of creating an ad hoc a set of code emulating the jack API but doing things using the media_server in the back-end. The idea is to have an application publishing BMediaNodes, those nodes will just wrap the functions needed by a client to process audio buffers.

Let me show some pratical examples :

jack_get_buffer_size
jack_get_sample_rate

Those could be trivially replaced using the info we have from the format negotiation done by the background node.

jack_midi_clear_buffer
jack_midi_event_get
jack_midi_event_write
jack_midi_get_event_count

I have never used the midi2_kit neither the midi_kit, but i bet the kits provides the needed functionality to emulate it. This is room for the next article (sorry!).

jack_port_get_buffer
jack_port_register
jack_port_type_get_buffer_size

Those functions are used by jack clients to get data out/in. When a BufferReceived() call is received by the BBufferConsumer the wrapper could theoretically call the process() callback the client has set using jack_set_process_callback function and at the same time provide the data in a stack which will be accessed by the client using the jack_port_* functions. Ideally a jack_port registration will result in a input / output for the media_node.

Those examples are taken from my personal research on jalv, and actually there are something like 20/30 functions in total to be wrapped for a theoretical layer allowing jalv to run as a media node. This is not a lot compared to big beasts such as Ardour, but enough to demonstrate that it's feasible.

My major concerns are about the API contained in the control.h header, and some other “minor” functionalities such as graph reorder callbacks, i’m not sure about how to emulate them. For the control API probably the best way is to do some conversion in background to make the BMediaRoster controllable by the jack control API. However, i need more research to evaluate how it’s feasible.

Additionally, taking this approach there would be other advantages, for example since the jack headers are LGPL, a layer of such type could be integrated into Haiku, more like what we already do with the VST media_addon or the freebsd compatibility layer for network cards.

A latency testing

In the end i just want to show you a little latency comparation i've done :

Buffer length	Haiku (media_server)	Linux (Jack2)
1024	13.7 ms	42.7 ms
256	7.75 ms	10.7 ms

The results are encouraging, but i've not a realtime kernel in my debian linux, i'm pretty sure that with a realtime kernel jack2 and Haiku will be more close. I don't think jack and the media_kit can do a lot better, due to the hw limits of my poor integrated Intel HDA card. The values are catched from the system mixer in Haiku (using Cortex). For linux they are found in the QJackctl config window.

Hope it was interesting, i just wanted to say what i think/discovered in the hope that
it could be helpful to realize what's the state of things, and maybe make the way more clear for a developer which want to do this. Additionally, it may be that there's some error in my article since i'm before everyone in learning phase, so any correction/suggestion/opinion is appreciated!