HTTP Network Services Preview in R1 Beta 4
The newest beta of Haiku includes a preview of a redesigned, modern HTTP library as the initial part of an renewed Network Services Kit. The primary goal of including the library is to get developers to experiment with it and give feedback on how it works within their software. The secondary goal is to gather some feedback on the use of modern C++ and some additional experimental features. This article provides a background to the new kit, some pointers on how to get started, and some notes on experimental designs that utilize modern C++ features.
What’s Included in the Preview
The preview that is part of Haiku R1 Beta 4 includes a basic HTTP 1.1 library. It supports the following:
- HTTP 1.1 Requests with:
- Standard and custom methods (GET, POST, etc.)
- Optional request body with a known size and content type
- Custom HTTP Header Fields
- HTTP Basic Authentication
- HTTP 1.1 responses with:
- Request headers
- Response bodies as a string, or written to specific files
- Configurable automatic handling of HTTP redirects
- Configuration of the number of concurrent requests per host and in total
Protocols other than HTTP 1.1, or additional functionality like proxies, cookie handling and other authentication types are not supported yet.
Getting Started with the Preview
The new HTTP library was first merged in
hrev56572, and is part of the Haiku R1 Beta 4 development package.
There is a specific section in the API documentation. The headers can be found in
/boot/system/develop/headers/private/netservices2/ and in order to use the API, you should link against
If you use the Makefile engine to build your application, then you should tweak the
LIBS to add the
libnetservices2.a library, as well as the applicable C++ runtime libraries (
$(STDCPPLIBS)) and the network libraries (
libbnetapi.so). You also need to add the headers to the
SYSTEM_INCLUDE_PATHS. For example:
LIBS = be netservices2 $(STDCPPLIBS) network bnetapi SYSTEM_INCLUDE_PATHS = /boot/system/develop/headers/be \ /boot/system/develop/headers/cpp \ /boot/system/develop/headers/private/netservices2
If you are building your applications on a 32 bit Haiku image, you will have to make sure you use the modern compiler kit. If you are getting unexpected/odd build errors, double check you have run
In the nightly images and the master branch, development continues, and therefore there may be incompatible changes to the kit, so be sure to test your software whenever you build it against a newer (unstable) version of the kit. The advice is that if you want to release software based on the preview, then make sure to build it against a known Haiku image. Because the preview functionality is statically linked in your application, it is likely any binaries will continue to work on more modern Haiku builds.
Experimental Design Features
The Preview of the HTTP Network Services Kit uses some of the C++ language and library features that are currently (mostly) unused in the rest of the project. Most of Haiku follows the design standards originating from Be, which means that most of the API and the code is based on the (pre-) C++98 version of the language. Furthermore, because of Haiku’s promise of binary compatiblity, most of the code will still have to build on GCC 2.95.3, which was released on 16 March 2001. This compatibility guarantee is not there for any new APIs and kits. While the new Network Services Kit is in development, it will include some experiments in its design and use of the language.
This section highlights three of those changes: the use of modern C++ language and library features, the introduction of exceptions for error handling, and an experiment in data access and control mechanisms through exclusive borrows.
Modern C++ (C++17)
||Arguably move semantics is the largest new feature in the language in C++11. The libnetservices2 kit utilizes move semantics to the fullest, meaning in the implementation all classes abide by the rule of six, which makes it possible to pass objects by value in most cases. The example of the
||The API uses
||C++ lambda’s allows the definition of anonymous functions. In the example, this is used by a the
In addition to these features, the implementation of the kit also uses some other modern language features internally, like initializer lists, the
auto keyword for variable declaration,
constexpr, and string literals.
Exceptions for Error Handling
The Preview of the HTTP Network Services Kit uses exceptions for error handling, as an alternative to the traditional way of error handling in the APIs inherited from BeOS. In the current public API, error codes are the way to identify and handle errors: functions and methods that may fail have a
status_t return type, as well as the use of negative values in
off_t return values to signal errors. For many objects, there is an
InitCheck() method that should be called in order to check if the object is valid and working correctly. For consistency reasons, all new public APIs and most other code do not use exceptions. There is some precedent for experimentation with exceptions, for example in the private parts of the package kit.
The preview of the HTTP Network Kit requires the user of the API to do all error handling through exceptions. It has the following characteristics:
- All error handling is done through exceptions. This means that no method will return
status_tor overload any other return value to signal an error has occurred. It will no longer be necessary to call
InitCheck()after creating a new object, any issues in the constructor will raise an exception.
- The API consistently shows which methods and functions can throw exceptions, by the consistent use of the
noexceptkeyword. This is also part of the documentation.
- The API documentation explicitly states which types of exception can be thrown, as to allow the user of the API to determine which exceptions to handle.
- The API introduces the
BPrivate::Network::BErrorabstract base class, which defines a standard API for all derived exceptions. Part of this base class is the
originproperty, which holds the location of where the exception occurred in order to aid debugging.
- Next to the exceptions derived from
BError, users of the API will also find exceptions of the
std::bad_alloctype for when allocations fail due to the fact that the system is low on memory.
In the internal implementation of the kit, exceptions are also used to handle error conditions and other exceptional states. Inside the code that runs in the threads that execute HTTP requests, the
BNetworkRequestError is thrown when there are network or protocol errors. For example, when there is a parsing error of an incoming response, the exception is raised and caught, and then saved as part of the
HttpResult object which is then re-thrown whenever the user tries to access data on that object.
The discussion on whether or not exceptions should be used are mostly ideological and fundamental in nature. This preview allows developers to get some hands-on first hand experience of using it in the context of Haiku development, in order to better form their arguments on whether or not the future of the Haiku API is in way of exceptions, or through alternative to
status_t return values like the
std::expected type, which is going to be part of C++23. Or a mixture; there might be good arguments to apply both strategies.
Thread-safe Data Access with Exclusive Borrows
One of the biggest challenges in software development is about writing code that handles reading and writing data safely over multiple threads. The main rule is that objects should be owned, read and written by one single thread. As soon as data is shared with more than one thread, there should be mechanisms to synchronize read and write access to the data. The most common methods of synchronization are atomic data structures and semaphores/mutexes. You need to employ these synchronization mechanisms, otherwise you will introduce nasty bugs in your code.
The previous iterations of the network services kit has experimented with several ways of delivering the response body of a request. Initially, there were callback functions that allowed a consumer to directly copy the contents of the response in the context of the network request’s thread. This was efficient, but it was up to the user to determine and implement a safety strategy for multithreading. The second option was to deliver the incoming data using BMessages. This was extremely safe, but it was also very inefficient. The current implementation allows the user to provide a
BDataIO object to write the data to, which is efficient. However, the
BDataIO interface itself does not presume or implement any locking mechanism, so that means that it is still up to the user of the API to iplement safety, which is not easy.
In the case of the Preview of the HTTP Network Services kit, the response body can either be stored as the
BHttpBody::text string member, or it can be saved to an object that supports the
BDataIO interface. When the latter option is used, the API forces the user to do this in a thread-safe manner, by implementing a synchronization mechanism with the following properties:
- The owner should be able to create an object, and manipulate while they have that in their control.
- The borrower can then exclusively borrow their object from the owner, so that they then can manipulate the object. The borrow is exclusive, meaning that the owner can no longer access or write to the object, until the borrower releases it.
- The owner can decide that it is no longer interested in the object. However, if the object is still borrowed, the memory cannot be freed until the borrower is done with it.
This model is implemented in the
BExclusiveBorrow smart pointer which provides this functionality. It is used when scheduling HTTP requests for execution, and requesting the response body to be written to a custom target, in
Note that the problem of streaming data needs to be solved differently, as the mechanisms of
BExclusiveBorrow do not allow for writing in one thread and reading in the other. This will be a future improvement of the kit.
Legacy Network Services Kit
The legacy network services kit is still bundled with Haiku, though with Haiku R1 beta 4 it is no longer part of
libbnetapi.so. The headers can still be found in
/boot/system/develop/headers/private/netservices and the library to link to is
libnetservices.a. Note that in a lot of cases, applications will also have to link against
libshared.a, due to a separate change where symbols of static libraries are hidden in the operating-system supplied shared libraries.
Reporting Bugs & Other Feedback
In order report bugs, please use the Haiku issue tracker and report it as a Network Kit bug. Make sure to note that the issue was found in Haiku R1 Beta 4.
Other feedback can be posted on the Haiku 3rd Party mailing list.
Note that the forums are not regularly monitored by the developers of this kit.
- HTTP Network Services Preview in R1 Beta 4
- Rust on Haiku: the Case of the Disappearing Deceased Threads
- The State of Rust on Haiku - July 2018
- Haiku has No Future
- Alpha 1: A Week Later
- A Jocund Eulogy
- Git for Haiku (#1)
- Haiku Alpha 1 Status Update (#2)
- Haiku Alpha 1 Status Update (#1)
- Mindmap of the discussion on alpha 1