Barrett's blog

How To Permanently Blacklist a Package File

Blog post by Barrett on Sun, 2013-12-15 19:17
, , , ,

With the advent of package management and hrev46391, it has become possible to prevent a package from being extracted at boot time.From a suggestion of Matt, and with the contribution of Luroh (thanks!), i would like to explain you how to blacklist a package file in Haiku.

In Haiku's boot menu , there is a 'Blacklist entries' option available. This method will only let you disable system packages, and only until the next time you reboot.

You may know, since the package manager has been added to Haiku, some directories are read only, so unlike the past, isn't possible to just delete the driver. The Blacklist functionality addresses the issue that it may be necessary to remove a problematic file such as a library or a bugged driver, which would otherwise require editing the containing package file.

So let's go to the very simple steps to do that :

  1. Figure out which file in which package you want to blacklist, especially keep in mind if the file is contained in a system package or in a user one.
  2. The second step is to create a text file named 'packages' in /boot/system/settings or in /boot/home/config/settings/global, respectively the first directory is used to blacklist system packages the second is used for user packages.

The final step is to fill the packages file with something like that :

Package 'packagename' {
	EntryBlacklist {
		'entrypath'
		...
	}
}

'packagename' is the name of the package without version, for example 'haiku'.

'entrypath' is an installation location relative path, e.g. "add-ons/Translators/FooTranslator".

This way, blacklisted entries will be ignored by the package_fs. So that they won't appear in the file system.

Let me give a pratical example.

In my case i had the broadcom570x driver to blacklist, so i created the packages file under /boot/system/settings with this content :

Package haiku {
	EntryBlacklist {
		add-ons/kernel/drivers/bin/broadcom570x
	}
}

Then i saved it and rebooted. Once Haiku start the package is re-mounted and the file is ignored by the packagefs.

Hope it was interesting!

Jack2 : A Personal Analysis (Part #2)

Blog post by Barrett on Mon, 2013-09-30 16:28
, ,

Hello, it has been passed some time from the Part 1 of this article, I've continued my investigations as well, and I have finally some more clear plans for such a hypotetic Jack2 port. Unfortunately i had not enough time to research a bit more in the latency differences between the media_kit and jack, sorry, this should be post-poned until i have precise emphyrical measuring methods.

To better understand this article i suggest you to read the first part.

The Wrapper

So, after lots of temptations, a night i decided to take the jack API headers and begin to stubb a jack wrapper.

This gave me the possiblity to explore in a better way the internals of Jack2.

The result of this work-night, is the jack_simple_client seamless working. Don't misunderstand me, the code is just a proof-of-concept, actually the backend media_node support only producers hooks, and have some problems such as the sound hear a bit streched. Other than being funny, this project has permitted me to understand better how the BMediaNode class work. I would like to synthetically make a resume of the work,

it consists of three layers :

  1. C-API bindings to c++ internal classes
  2. Jack2 emulation which is composed of two classes at the moment, JackPort and JackClient.
  3. JackNode, a native media_kit node controlled by the emulation layer, which serves as the gate between the Jack emulation and the media_kit.

This is a map of the functions on which i worked and their status :

E = empty
P = partially implemented
OK = fully implemented / sufficient implementation ATM.

jack_activate OK
jack_client_close OK
jack_client_name_size OK
jack_client_open OK
jack_connect P
jack_deactivate P
 
jack_get_buffer_size OK
jack_get_client_name OK
jack_get_ports P
jack_get_sample_rate OK
 
jack_midi_clear_buffer E
jack_midi_event_get E
jack_midi_event_write E
jack_midi_get_event_count E
 
jack_port_get_buffer P
jack_port_name OK
jack_port_register P
jack_port_type_get_buffer_size OK
 
(the original implementation is mantained)
jack_ringbuffer_create OK
jack_ringbuffer_free OK 
jack_ringbuffer_mlock OK 
jack_ringbuffer_read OK
jack_ringbuffer_read_space OK
jack_ringbuffer_write OK
jack_ringbuffer_write_space OK
 
jack_session_event_free E
jack_session_reply E
 
jack_set_buffer_size_callback OK
jack_set_process_callback OK
jack_set_session_callback E
jack_shutdown E
 
jack_transport_query E

You can see the code here.

Intrinsic Limitations

There are some intrinsic limitations due to different designs, which might be a bit complicated to deceive.

  • Jack2 support multiple connections to one port.

    There's not a lot to do about it, the only solution without explicit support from the media_kit is to map one jack port into multiple input (or outputs). For example, in Haiku a stereo output is a unique port, you can't connect two mono channels to a stereo one without using a mixer node, in Jack2 instead stereo channels are separated by default.

    Anyway i do think a feature to separate stereo channels should be taken into consideration for a future media_kit version.
    **EDIT** To be more clear all implementations of JACK permit connections between ports with any valence, they can be 1:1, N:1 or 1:N.

  • Jack2 support glitch-free ports reconnection.

    Well, as far as i know the media_kit just is unaware of this concept.

    **EDIT** In those hours, emerged that it relates only a jack2 implementation detail of
    the API, so it will be a limitation for a native port but not for libjackcompat. That's a good news indeed.

  • Jack2 features a session API which is simply missing in Haiku.

    This is something already discussed in the mailing list, basically other than being a low-priority part of the API, it could be just emulated until Haiku features a session management system, which will probably come out after R1.

The Jack port and the wrapper

I discovered that there is not need of an external app routing audio between jack2 and the media_server.

With the premise that the incompatibilities in libjackcompat and the issues i explained in the last article are still valid, since the jack_ports management is done by the driver i think an Haiku driver could work that way :

  1. When a jack client is registered a media_node is created, for every port there will be a input/output in the node.
  2. When a media_node is created, it will be mapped into a jack_client, and every i/o mapped onto a jack_port.
  3. The phantom clients will be internally managed to work as audio streams mirrors.
  4. By default there will be a media_node wrapping the jack system_mixer into the Haiku's one.

But i would like to show you an use case where it may be complicated to create an efficient and transparent strategy :

Imagine the jack_client1:port1 is connected to jack_client2:port2. jack_client2 is a media_node in the back-end, so it will wrap the port2 into a native input.
So, now imagine a new jack client, called foo_client, what happens if it try to connect to jack_client2:port2? In jack it's completely legal, but in the media_kit there will be only a input, and no way to make a double connection.

As you may understand, this is a headache. To my mind comes only two different solutions :

  1. Make the behaviour somewhat configurable in settings and in the media_node settings panel. So that you can for example map two jack ports into one media_kit stereo connection.
  2. Force users to use appropriate mixer nodes between connections.

So, What's The Need for a wrapper?

Except the fact that with some degree various simple apps should work out of the box, i think a wrapper is still important for a lot of reasons, if it will work for basic functionalities, it could be used by third-party developers as a initial layer to base their native Haiku port.

For example, as start you can compile your jack2 program, then you include the wrapper into your source tree and begin to modify it to fit your needs.

The last but not less important thing is that the libjackcompat backend will provide most of the functionality needed to create the backend-nodes, so it may be possible that a media_kit jack2 driver will be based itself on this library.

As before, i just tried to explain my vision, hope it was interesting...as ever
any opinion, question and correction is appreciated!

Jack2 : A Personal Analysis (Part #1)

Blog post by Barrett on Tue, 2013-07-02 12:40
,

Intro

In the last year i managed to play a bit more with the Haiku media_kit.
It was already discussed in various places if jack2 should be adopted, ported, or someway integrated into Haiku. There are various opinions out there, and more or less going down into the topic i want to show you what i think about.

Jack2 is a real-time audio server for UNIX systems, to be more specifical a smp focused reimplementation of Jack (which is single-threaded), it provides a protocol used by audio apps for inter-communication. It is gaining a very good number of apps supporting it and it's becoming interesting also for professional audio.

Besides GNU/Linux there are ports available for various operating systems, such as FreeBSD, Windows, Solaris and Mac OS X. The interesting thing about the OS X port, is the availability of a JackRouter which allow to route audio buffers between jack apps and core audio apps.

The idea is a lot similar to what the Haiku media_kit provide.
There are various differences, which includes jack lacking of video support and multiple formats.

Jack force every app to use 32 bit floating point values for their buffers, and the motivation of that is probably to prevent any latency due to resampling algorithms. This is very different from what the media_kit do, since in Haiku a mechanism is provided to support format negotiation and painless audio resampling.

The following text is a resume of some articles i wrote in my blog.

The Narrow Way

So, a day i decided to checkout the source and i started some exploration, to see how it's feasible a port of jack2, the resulting work is located at my github page and the haiku specific code is located in a subdirectory as other operating systems. Unexpectly it worked, trying to connect jack_rec and jack_simple_client produce the expected sinusoidal sound in the file.

Unfortunately excluding for now the noticeable lack of any media_kit backend, there are some other problems to be resolved before :

  • The jack shm implementation is not working as should (probably due to http://dev.haiku-os.org/ticket/2657)
  • Haiku is not supporting posix real time thread priorities (http://dev.haiku-os.org/ticket/8600)
  • I had to comment some memory locking related code, due to the lack of mlock/mlockall support.
  • Jack warn about lots of graph reorders.
  • jack_test is not passing some tests.

In the meantime, i've also done some brainstorm for possible solutions (in the same order) :

  • The best solution is probably to just fix the bug.
  • The idea is to replace the actual HaikuThread class (substantially a temporarily hack) and set
    it to inherit directly from JackThread instead of JackPosixThread, this would allow to use internally
    the Haiku implementation of threads, just emulating what jack is expecting.
  • Don't know if Haiku lack something preventing to have the system support them. AFAIK Haiku areas are obviously supporting this functionality, so it might be possible to just insert some ifdefs in the Jack implementation, or an Haiku implementation alongside with the POSIX and System V ones.
  • The latests two are probably related to the others.

So without an app to route sounds between jack and the media_server, we will still have
two separate worlds, since i see very difficulty to make jack2 and the media_server friends (since both are asocial animals).

I don't think this is a very good scenario to have and i don't see it haiku-stylish. But at the same time, i would love to see a hybrid jack2/media_server client, publishing a tunnel allowing to pass audio between the two servers.

Said that, my personal opinion is that reaching a stable and performant Jack2 could potentially require a lot of work for poor results, the amount of work might easily vary depending on the problems encountered in the steps. This was my feeling when i tried to create a simple jack capture node playing buffers using BSoundPlayer in the backend.

Rise And Shine

We are not lost, there is already another way, which *may* be better. The idea is as simple as difficult : force Jack2 applications to use the media_server by wrapping the jack API.

This consist of creating an ad hoc a set of code emulating the jack API but doing things using the media_server in the back-end. The idea is to have an application publishing BMediaNodes, those nodes will just wrap the functions needed by a client to process audio buffers.

Let me show some pratical examples :

jack_get_buffer_size
jack_get_sample_rate

Those could be trivially replaced using the info we have from the format negotiation done by the background node.

jack_midi_clear_buffer
jack_midi_event_get
jack_midi_event_write
jack_midi_get_event_count

I have never used the midi2_kit neither the midi_kit, but i bet the kits provides the needed functionality to emulate it. This is room for the next article (sorry!).

jack_port_get_buffer
jack_port_register
jack_port_type_get_buffer_size

Those functions are used by jack clients to get data out/in. When a BufferReceived() call is received by the BBufferConsumer the wrapper could theoretically call the process() callback the client has set using jack_set_process_callback function and at the same time provide the data in a stack which will be accessed by the client using the jack_port_* functions. Ideally a jack_port registration will result in a input / output for the media_node.

Those examples are taken from my personal research on jalv, and actually there are something like 20/30 functions in total to be wrapped for a theoretical layer allowing jalv to run as a media node. This is not a lot compared to big beasts such as Ardour, but enough to demonstrate that it's feasible.

My major concerns are about the API contained in the control.h header, and some other “minor” functionalities such as graph reorder callbacks, i’m not sure about how to emulate them. For the control API probably the best way is to do some conversion in background to make the BMediaRoster controllable by the jack control API. However, i need more research to evaluate how it’s feasible.

Additionally, taking this approach there would be other advantages, for example since the jack headers are LGPL, a layer of such type could be integrated into Haiku, more like what we already do with the VST media_addon or the freebsd compatibility layer for network cards.

A latency testing

In the end i just want to show you a little latency comparation i've done :

Buffer length Haiku (media_server)  Linux (Jack2)
1024 13.7 ms 42.7 ms
256 7.75 ms 10.7 ms

The results are encouraging, but i've not a realtime kernel in my debian linux, i'm pretty sure that with a realtime kernel jack2 and Haiku will be more close. I don't think jack and the media_kit can do a lot better, due to the hw limits of my poor integrated Intel HDA card. The values are catched from the system mixer in Haiku (using Cortex). For linux they are found in the QJackctl config window.

Hope it was interesting, i just wanted to say what i think/discovered in the hope that
it could be helpful to realize what's the state of things, and maybe make the way more clear for a developer which want to do this. Additionally, it may be that there's some error in my article since i'm before everyone in learning phase, so any correction/suggestion/opinion is appreciated!

Virtualize a Physical Haiku Partition With Virtualbox

Blog post by Barrett on Sun, 2011-11-06 17:54
,

In some situations, for example when we are using linux, can be extremely annoying to reboot into Haiku every time we need something (for example when we have a ppp connection).

I've written this article and i decided to post it here, in the hope that will help users and developers to have the life a bit simple.

There's a fast method to boot a physical Haiku partition using VirtualBox, it require only a few commands.
My commands refers to linux, anyway the operation is possibile under Windows (and presumably all supported platforms), changing the disk path. Remember also that you don't have to set permissions under Windows.

  1. Open Terminal and log in as root (or use sudo before every command) using "su"
  2. Type fdisk -l and choose the target device (for example /dev/sda)

Add to your own user the needed permissions :

usermod -a -G disk,vboxusers username

Then create the vdmk file using this command :

VBoxManage internalcommands createrawvmdk -filename file.vdmk -rawdisk /dev/sda

Where file.vdmk is the file that will be used to set up the virtual machine and /dev/sda your device, it can be an hard disk, usb drive and any other mass storage device.

If our command worked as well, we'll have a new file for example "file.vdmk".

Now you should set the file permissions, type into terminal :

chmod 777 file.vdmk

At the end open VirtualBox, create a new virtual machine and set the file as master storage.

If you have a bootloader, you will choose the operating system as when you boot the device, in the case you need a bootloader you can use an Haiku's livecd adding it in VirtualBox and pressing shift or space to boot from another partition before the bootscreen.

This method is also useful when you have a BFS disk with a lot of files and you want to transfer it to your physical partition.

Note : the created file can be used as well with VMWare.

It's available an italian version of the tutorial at this link.
Enjoy!

What Will Happen....

Blog post by Barrett on Sun, 2011-09-04 02:23
, ,

You probably know that i have not passed the GSoC final evaluation. Although i am a bit discouraged (it's natural i think), as said from the begin, it's not my intention to abandon my project. Money wasn't my first motivation to work, and it will not be in any case.

It's just a short post to tell you what is the state of my code, and about which i'm working on.

At the end of gsoc, contacts kit's base classes are working fine, People were refactored to use my contact API, so i'm writing here my goals for the next period (deadline is presumably before january) :

* Move attribute indexing from People to the PeopleTranslator.
* Adjust file changes monitoring in People.
* Add a PeopleFilePanel class to export contacts into formats different than Person format.
* Fix all TODO and my late mentor's suggestions

* Create BContactList and BContactFieldList classes to make basic operations on collections of objects...like merging of contacts.
* Finalize BContactGroup by using the mentioned classes.
* Create the BContactRoster class, and make working the BAddressBook class (that will wrap /boot/home/people).
* Integrate Mail with BAddressBook.

Sorry for few details, my first need was to notice you that i'm already working on the project.

Below some screenshots about my latest gsoc work :

http://imageshack.us/f/856/screenshot2kz.png/
http://imageshack.us/photo/my-images/853/screenshot2km.png/

Contacts Kit, quarter-term report

Blog post by Barrett on Mon, 2011-08-08 23:44
, ,

In these weeks i have improved the contacts kit core in order to have enough support for the formats supported. The vcard and people translators can now translate and exchange many types of field, though photo and groups aren't yet supported.

Main functionalities of the classes :
BRawContact
Their functionality is to deal with the BTranslatorRoster and keep track of basic informations, like the final format. The final destination is represented as a BPositionIO object. Basically the class find a suitable translator when initialized and provide two methods : Commit() and Read(). The first has as argument a BMessage that contain Flattened BContactFields, the second translate the source file (that can be a B_PEOPLE_FORMAT, B_VCARD_FORMAT, B_CONTACT_FORMAT) into a BMessage and return it.

BContact
BContact is the high level class representing a contact, it has the job to store informations about the raw contact, provide methods to add/remove/replace/compare the BContactFields. When initialized it use the BRawContact::Read() function to read fields from a location, and use BRawContact::Commit() to provide BContact::Commit().

BContactField and childs

BContactField is designed to support as well both People and VCard formats. The class is not intended to be used directly, but a common interface for the different types of fields that are supported. The usage of a field is defined by two enums at ContactDefs.h, it is a code that allow to specify additional informations for the data, for example if you want to add a phone number that is a fax you'll use this code :

...
BContactField* field = new BStringContactField(B_CONTACT_PHONE, value);
field->SetUsage(CONTACT_PHONE_FAX_WORK);
contact.AddField(field);
...

Another interesting function of BContactField is the Label() method, that return a friendly description of the field allowing to create easily apps like People without having care to translate a label for any field.

The class also accept any number of string parameters, the BFlattenable interface, and the BContactFieldVisitor interface.

BStringContactField
This is used for the major part of the fields, the value is represented as a BString object it is enough for fields like B_CONTACT_NAME, B_CONTACT_ORG...
BAddressContactField
It represent an address, allow to initialize a well-formed address via the constructor or alternatively the programmer can use the provided methods to set all the informations.

There are also other types of fields in the plans, one of these is the BCustomContactField.

Translators
As said while not supporting all fields, they can translate and exchange fields with a common format. There are some bugs, but the whole process is working as well and i consider this part of the project complete.

Services Kit
Unfortunately this side is not close to be completed, i have a draft and i'm working to get it enough complete.

At the moment the main classes are :
BServicesRoster
A simple class that allow to manage the addons in a more low-level manner...something similar to BTranslatorRoster giving access to the add-ons so nothing specific to the contacts support but a base for many uses (including contacts addons).

BServicesAddon
This will be the class specifying the API used by the addons, using a generic set of methods...version, name, friendlyname, services_addon_type, init and so more. Providing a Process() method used to receive data as BMessages from the server. In the same manner, will be instantiated with a messenger object to talk with the server.

BContactRoster,
basically store the BContacts provided in the address book. The user will instantiate an own object reading only the contacts they want. In future would be nice to see classes like BContactQuery...at the moment they will specify simple access to the fields stored in the address book. Internally will use an instance of the BServicesRoster talking with contacts addons.

I have created a git branch of the Haiku's tree at the url :

https://github.com/Barrett17/Haiku-services-branch

At the moment it is a plain copy of the tree, in the next day's i'll update the code since using the osdrawer repo was limitative.

Contacts Kit, Mid-term

Blog post by Barrett on Tue, 2011-07-19 17:16
, , ,

From my latest post, i had to do more work on the base classes, i realized that my implementation of BContactField was too inflexible for the use so my progresses were not fast as i hoped.
The main problem was to provide something that can fit the simplicity of the Person format (people files) and the complexity of VCard. The result was BContactField. Starting from a number of fields (defined in ContactDefs.h) the class provide some methods to access the field-type of a particular fields and give an interface able to manage properties and parameters as well. At the bottom there are some classes that implements BContactField, they will be used directly by the user, let me give some examples :

BStringContactField
This merge every fields that does not have any other object or special functionalities like B_CONTACT_NAME or B_CONTACT_EMAIL.

BUrlContactField
This is the first example of a field that will incorporate an object from the Haiku's API, in this case it's simple but will be helpful when the BAddressContactField will be added.

How the API help to distinguish between the type of the fields?

You can use the method BContactField::FieldType() but there's a more simple solution, using the visitor pattern i have created a BContactFieldVisitor class, it is nothing more than an interface that specify one Visit() method for every BContactField child class. Implementing a your own class, when you receive a BContactField you can pass your visitor into the BContactField::Accept() method without having any troubles converting the base class to the derived class. One pratical use is for example the EqualityVisitor that allow to implement the method BContactField::IsEqual() in a nice manner. Also the VCardTranslator use a private visitor as a class that convert every field into a VCard string.

While i should commit some changes for BContactField, BContact and BRawContact are basically complete and usable and seems working as well.

Translators

The state of translators is summarily good, at the moment VCard can read and write some basic properties, People is in full development state. The main problem with it was a intrinsic limitation of the translation kit, basically you have to pass a BPositionIO (BFile, BMemoryIO..) to a translator but people files are recorded as attributes. You cannot edit an attribute using a BPositionIO since they are data written outside the file, so at the moment the translator will check if the input/output (as the case) is a BFile and if not it will fail.

Future
At this point i will not make forecasts, i'm working on a plan for the services kit and the addon API in order to begin discuss it with my mentor(s), the first part of the project seems close to be complete so i'll update you with a blog post in the next days.

Syndicate content