diff --git a/doc/release_notes-13-11.txt b/doc/release_notes-13-11.txt new file mode 100644 index 000000000..2468fbf8f --- /dev/null +++ b/doc/release_notes-13-11.txt @@ -0,0 +1,908 @@ + + + =============================================== + Release notes for the Genode OS Framework 13.11 + =============================================== + + Genode Labs + + + +Unlike most Genode releases, which address two or three major topics, version +13.11 brings plentiful new experimental features and improvements of details. +The experimental ground covered by the new release reaches from the use of the +Linux TCP/IP stack as user-level library, over the native use of Qt5 QML on +Genode, new FUSE-based file systems, to the exploration of C++11 for the +framework. In line with the previous releases, the new version extends the +coverage of device drivers, particularly for Exynos-5 SoCs and the Raspberry +Pi. + +Genode keeps steadily growing. With the code base approaching the number of +150 components and as many test cases, we are concerned to keep the code +orthogonal and void of bugs. Of course, automated testing helps a lot. But +even more importantly, we are constantly exploring possible ways to refine the +Genode API such that users of the framework are saved from writing +boiler-plate code, falling into C++ pitfalls, or solving complicated problems +again and again. One of those problems is lock-based synchronization, which +seems to be omni-present in low-level systems code, particularly on classical +L4-based systems. However, taking our existing components as use cases, we +came to the realization that for most of our system services, in particular +device drivers, multi-threading is more of a burden than a benefit. The +current framework API, however, imposes the need to use multiple threads on +many components that do not benefit from parallelism. This raises the question +of how we can evolve the API to become a better fit for typical use cases and +relieve the component developer from dealing with complicated synchronization +problems. We seek the answer to such questions through experimentation. The +new server API described in Section [New server API] and the use of this new +facility by components like the nitpicker GUI server is such an experiment. + +For our mission to use Genode as a general-purpose OS, robust protocol stacks +such as file systems or networking stacks are fundamental. So we are exploring +different ways to bring existing implementations to Genode. For this release, +we took a closer look at FUSE as a provider for file systems. Section +[File systems based on FUSE] gives a brief overview of the results. For +networking, we undertook the challenge of turning the Linux TCP/IP stack into +a user-level library that can be used by Genode programs. Section +[Gigabit networking using the Linux TCP/IP stack] explains how this TCP/IP +stack can be used as an alternative to lwIP to improve gigabit-networking +performance. + +The recently added support for Qt5-based applications raised repeated +questions about using QML, which is the most distinctive feature of Qt5 +compared to Qt4. We are happy to report to have a positive answer to this +question with the inclusion of initial QML support. Even though the support +for QML is not mature yet, it is already possible to run the usual QML +examples on Genode. Section [Qt5 with support for OpenGL and QML] outlines the +steps for running simple QML applications. + +With regard to the support for hardware platforms, version 13.11 comes with +extended driver support for Exynos-5 covering HDMI and USB 3.0 mass storage, +as well as all drivers needed to execute interactive Genode scenarios on the +Raspberry Pi. Furthermore, we enabled the use of ARM TrustZone on i.MX53 via +our custom base-hw kernel platform. + + +Base framework +############## + +Dynamic resource balancing +========================== + +We have addressed the general problem of reassigning memory resources between +subsystems in a dynamic fashion. The Genode architecture devises the explicit +assignment of resources by a parent process to its children. Once assigned, +the amount of memory used to remain at the disposal of the respective child +subsystem until the subsystem gets destroyed by the parent. + +Because Genode passes resources among processes along the branches of the +process tree, the parent interface is the natural place for providing a +protocol for dynamically upgrading and withdrawing resources between a parent +process and its children. The protocol consists of the following functions +that were added to the parent interface. + +! void resource_avail_sigh(Signal_context_capability) + +By calling this function, the child can install a custom signal handler that +gets notified on the arrival of additional resources. Such a signal gets +fired by the parent after upgrading the resource quota of the child. + +! void resource_request(Resource_args const &) + +Using this function, the child is able to apply for additional resources +at its parent. The request takes the amount of desired resources as argument. +A child would invoke this function if it detects scarceness of resources. +If a custom signal handler is installed via 'resource_avail_sigh', the +function returns immediately and the child must wait for the reception +of the corresponding signal. If no signal handler was explicitly installed, +the 'Genode::env()' handles the request via a built-in signal handler that +blocks until the parent satisfies the request. + +! void yield_sigh(Signal_context_capability) + +By calling this function, a child installs a signal handler that gets +notified about so-called yield requests from the parent. A parent issues +a yield request to a subsystem to express its wish for regaining resources. +It is up to the child to comply with a yield request or not. Some kinds +of subsystems have meaningful ways to handle yield requests. For example, +an in-memory block cache could write back the cached information and release +the RAM consumed by the cache. + +! Resource_args yield_request() + +The 'yield_request' function returns the amount of resources desired by the +parent. This information can be used by the child to tailor the resource +deallocation strategy according to the needs of the parent. + +! void yield_response() + +Once the child has taken steps to yield resources, it calls the +'yield_response' function to inform the parent about the availability of +released resources. The parent may respond to this function by withdrawing +those resources from the child's RAM quota. + +The protocol described above has been implemented into the base system +(i.e., 'Genode::Parent', 'Genode::Child', 'Genode::env()') as well as +the interactive console called _cli_monitor_. For the latter, we introduced +new commands for dynamically balancing RAM between subsystems. The 'status' +command prints a table with the RAM status of each subsystem. The 'ram' +command changes the quota or a quota limit of a given subsystem. The quota +limit can be defined to allow the on-demand expansion of the quota. Finally, +the 'yield' command can be used to instruct a subsystem to yield a specified +amount of resources. + + +C++11 enabled by default +======================== + +As more and more individual Genode components started using C++11 features, +it is time to enable the most recent C++ standard by default. This step +clears the way for using C++11 in the base API of the framework. There are +many ways we may leverage the new language features to improve our code, +for example variadic templates may help us to overcome current restrictions in +the RPC framework regarding the number of supported RPC function arguments. + +The C++11 standard is enabled by default for software built via the Genode +build system. If you hit incompatibilities with existing software that +relies on a prior version of the C++ standard - as is the case for some +parts of Qt4 - you can override the default by manually defining +'CC_CXX_OPT_STD' in the build-description file of the offending target. + + +Improved event tracing +====================== + +In the previous release, we introduced Genode's event tracing facility that +enables us to capture inter-process communication and component behavior +with negligible overhead. To make the mechanism usable in practice, an +easy-to-use front end is desired. With the current release, we have taken +the first steps in this direction. The front-end API is located at +_base/include/base/trace/buffer.h_. It hides the technical details of +how a trace buffer is realized behind a convenient interface for +iterating over events. Furthermore, we have added a mechanism for detecting +buffer wraps and expose this information through the API. + +Thanks to this convenience API, the creation of custom trace monitors becomes +much more straight-forward. A simple trace-monitor implementation using the +new API can be found at _os/src/test/trace/_. + + +New support for bit sets in MMIO API +==================================== + +Genode's API for accessing MMIO registers helps a lot to simplify the +access to individual bits or consecutive bit fields of hardware registers. +However, we still stumbled over use cases that the current API does not +cover, particularly the access to bit fields that are not consecutive or even +distributed over multiple registers. For example, values of the HDMI +configuration of the Exynos-5 SoC are scattered over multiple hardware +registers. Hence, we extended the MMIO API to hide those peculiarities behind +an easy-to-use interface. + +The idea is best illustrated by the following example of a hypothetical +timer device. The bits of the clock count value are scattered across +two hardware registers, the lower 6 bits of +the 16-bit-wide register 0x2, and two portions of the 32-bit-wide register +0x4. A declaration of those registers would look like this: + +! struct Clock_2 : Register<0x2, 16> +! { +! struct Value : Bitfield<0, 6> { }; +! }; +! +! struct Clock_1 : Register<0x4, 32> +! { +! struct Value_2 : Bitfield<2, 13> { }; +! struct Value_1 : Bitfield<18, 7> { }; +! }; + +Writing a clock value needs consecutive write accesses to both registers +with bit shift operations applied to the value: + +! write(clk); +! write(clk >> 7); +! write(clk >> 20); + +The new 'Bitset_2' and 'Bitset_3' class templates contained in +_util/register.h_ allow the user to compose a logical bit field from 2 or 3 +physical bit fields. The order of the template arguments expresses the order of +physical bits in the logical bit set. Each argument can be a register, a +bit field, or another bit set. The declaration of such a composed bit set for +the example above looks as follows: + +! struct Clock : Bitset_3 { }; + +With this declaration in place, the driver code becomes as simple as: + +! write(clk); + +Under the hood, the MMIO framework performs all needed consecutive write +operations on the registers 0x2 and 0x4. + + +New utility for handling character buffers +========================================== + +Throughout Genode, we find manually instantiated char arrays and checks for +null-termination scattered throughout the code base, mostly in connection with +RPC functions. To avoid repetitions of such C-fashioned and bug-prone code, we +added a new 'String' template class to _util/string.h_. It plainly holds a +null-terminated string to be stored as a member variable (e.g., a session +label) or passed as RPC argument. The string class is not intended to become +a full-fledged string-handling API though. + + +Low-level OS infrastructure +########################### + +Gigabit networking using the Linux TCP/IP stack +=============================================== + +On Genode, we used to rely on lwIP as TCP/IP stack. Network-using applications +would simply link against the lwIP stack and our libc_lwip_dhcp plugin to +access a NIC session, as provided by a NIC driver. For using multiple +networking application on the same machine, there is a NIC bridge component, +which multiplexes one physical NIC to multiple virtual NICs. + +While experimenting with gigabit networking, we discovered that lwIP lags far +behind the performance of other modern TCP/IP stacks such as the one found in +the Linux kernel. This observation prompted us to create an alternative to +lwIP, which uses the TCP/IP stack of the Linux kernel. The result is called +LXIP and can be found in the _dde_linux_ repository. LXIP consists of two +parts, a port of the Linux TCP/IP stack to Genode and glue code to Genode's +C runtime. It comes in the form of a shared library named _lxip.lib.so_. The +IP stack can be interfaced using Genode's version of libc by linking your +application to the _lxip_libc_ plugin in your 'target.mk' file. + +From the application developer's point of view, the use of LXIP instead of +LwIP is as simple as replacing the 'lwip_libc_dhcp' library with the +'lxip_libc' library. Note however, that LXIP has larger memory demands +compared to lwIP. So quota adjustments may be needed. For a quick test drive +of LXIP, we have prepared a set of run scripts at _libports/run/_ that execute +netperf using LXIP in different settings. The _netperf_lxip.run_ script +connects netperf directly with a NIC driver, the _netperf_lxip_bridge.run_ +script adds a NIC bridge as an indirection between the netperf application and +the driver, and the _netperf_lxip_usb30.run_ script targets a Gigabit +network-over-USB3 network adapter. + + +Configuration and session-label utilities +========================================= + +The often-used convenience utility for accessing a process configuration used +to come in the form of the header file _os/config.h_. But this causes aliasing +problems if multiple compilation units access the config while the +configuration gets dynamically updated. Moving the implementation of the +accessor to the singleton object into a library solves those problems. +Because of this change, all programs that rely on the 'Genode::config()' +utility need to have the 'config' library specified in the 'LIBS' declaration +of their _target.mk_ file. + +Closely related to the process configuration, the utilities for handling +session labels have slightly changed. By splitting the former 'Session_policy' +into two classes, we make it more flexible. Originally, the constructor +solely accepted an args string, which made it unusable for situations where we +already had extracted the session label (e.g., stored in the session meta +data of a server). Now, the extraction of the label from the args string is +performed by the new 'Session_label' class instead, which, in turn, can be +passed to the constructor of 'Session_policy'. This change causes a minor API +change. The following code +! Session_policy policy(session_args); +must be turned into +! Session_label label(session_args); +! Session_policy policy(label); + + +Serialization of signal and RPC dispatching +=========================================== + +In Genode, inter-process communication can be implemented synchronously +via RPC calls or asynchronously via signals. Most services use a combination +of both mechanisms. For example, a block driver provides an RPC interface +for querying the size of the block device. However, the actual transactions +are communicated via shared memory and asynchronous notifications to maximize +throughput. For this reason, most servers need to handle both, incoming RPCs +and signals. + +RPCs are dispatched by special threads called RPC entrypoints whereas signals +can be waited-for by any thread, most commonly the main thread. Consequently, +RPC functions and signal dispatchers are executed in the contexts of different +threads. So access to the state shared by both signal handlers and RPC functions +must be synchronized. This synchronization can be performed by using locks. +However, lock-based synchronization is complicated and hard to test. For this +reason, we introduce an alternative mechanism to serialize signal handlers +with RPC functions by the means of the so-called 'Signal_rpc_dispatcher' +utility in _os/signal_rpc_dispatcher.h_. A signal RPC dispatcher acts as a +proxy, which delegates the handling of signals to the context of an RPC +entrypoint. This way, signals received by the main thread result in a +process-local RPC call to an RPC entrypoint, which is naturally serialized +with ordinary RPC calls. This way, both signals and RPCs are effectively +handled by the same thread, which alleviates the need for synchronization. + + +New server API +============== + +The new signal RPC dispatcher described above offers a glimpse into the +direction we want to see the Genode API evolve. Instead of handling +synchronous and asynchronous communication via different mechanisms, we +would like to streamline both into one solution. Right now, we have RPC +entrypoints for handling RPC requests and signal receivers for handling +signals. In the future, we would like to have just entrypoints, which +can be associated with both RPC objects and signal dispatchers. + +Because we cannot change the Genode API over night, we take gradual +steps. One step is the new server library with the interface located +at 'os/server.h'. This library contains a skeleton for a common +single-threaded server that wants to respond to both incoming RPC +requests as well as signals. The interface provides a single 'Entrypoint'. +In contrast to an 'Rpc_entrypoint', a 'Server::Entrypoint' can be +associated to signal dispatchers. + +Over the course of the next year, we will evaluate this idea by migrating +existing servers to the new API and refining the server library as needed. +If the new approach stands the test of time, we will possibly replace +parts of the existing Genode API with the much simpler 'Server::Entrypoint' +interface. + + +Nitpicker GUI server +==================== + +The nitpicker GUI server is the first server adapted to the new server +API described above. Besides this change under the hood, nitpicker gained +the following functional improvements: + +First and foremost, the use of the server API cleared the way to let +nitpicker respond to signals, particularly configuration changes at +runtime. The new version is able to immediately respond to such changes +including the definition of global keys, session colors, and the +background color. + +Global keys for the X-ray and kill modes are no longer hard-coded. +The keys for toggling those functions can be defined in the nitpicker +configuration as follows: + +! +! +! +! +! +! + +The '' node contains the policy for handling global keys. Each +'' sub node expresses a rule for a named key. The 'operation' attribute +refers to nitpicker's built-in operations. In the example above, the X-ray +mode can be activated via the scroll-lock key and the kill mode can be +activated via the print key. + +Alternatively to specifying an 'operation' attribute, a key node can contain +a 'label' attribute. If specified, all events regarding the key will be +reported to the client with the specified label. This enables clients to +handle global shortcuts. The client with the matching label will receive +all events until the number of concurrently pressed keys reaches zero. +This way, it is possible to handle chords of multiple keys starting with +the key specified in the '' node. For the routing of global keys to +clients, the order of '' nodes is important. If multiple nodes exists for +different labels, the first match will take effect. For example: + +! +! +! +! + +The "launchpad" client will receive all key sequences starting with F11 unless +the "launchpad -> testnit" program is running. As soon as testnit gets started +by launchpad, testnit will receive the events. If the order was reversed, +launchpad would always receive the events. + +Furthermore, the new version allows clients to dynamically allocate virtual +framebuffers during the lifetime of the session. This solves two problems: + +First, it enables a client to create a connection to nitpicker without +donating much session quota in advance. The old interface required each +screen-size-dependent client to donate as much memory as needed to +allocate a screen-sized virtual framebuffer. For clients that are +interested in the screen size but cover just a small portion of the +screen (e.g., a banner, a menu, or an applet that sits in the screen +corner), this over-provisioning is painful. The new interface allows such +clients to upgrade the session quota for an existing session as needed. + +Second, because each nitpicker session used to have a virtual frame +buffer with a fixed size over the lifetime of the session, a client that +wanted to implement a variable-sized window had to either vastly +over-provide resources (by opening a session as large as the screen just +in order to be prepared for the worst case of a maximized window), or it +had to replace the session by a new one (thereby discarding the stacking +order of the old views) each time the window changes its dimensions. The +new interface accommodates such clients much better. + + +New terminal services +===================== + +The new file terminal located at _gems/src/server/file_terminal/_ is a service +that provides Genode's 'Terminal_session' interface for a given file via a +file-system session. +! +! +! h +! +To keep things simple, a client can open only one file at the moment. + +The new log terminal located at _os/src/server/log_terminal/_ forwards +terminal output to a LOG session. + + +New C-runtime plugin for accessing block devices +================================================ + +Certain third-party code, which mostly originates from POSIX-like systems, +uses normal file operations to access a block device from userspace , e.g. +fsck(8) or mkfs(8). For this reason, access to a block device by using +Genode's block-session interface through the normal C-runtime file operations +is needed. The solution comes in the form of the new _libc_block_ plugin. A +program linked against this plugin obtains access to a block-session by +opening the special device file "/dev/blkdev". Reading and writing from and to +the block session - even on offsets that are not aligned to the block boundary +- is handled by this plugin. For the actual program, the use of the plugin is +completely transparent. + + +New file-system server for hybrid Genode/Linux systems +====================================================== + +So far, scenarios depending on file accesses via the file-system session +could not be run on Genode/Linux. But with the growing complexity of +software that is ported to the framework, the demand for a full-fledged +rapid-prototyping environment on Linux grows too. For example, our +former approach to load required files as ROM modules stretches its +limits with file sizes of several gigabytes. In the current release, +we added a file-system server that provides transparent access to +Linux files for Genode applications. As with other file-system +servers, 'lx_fs' can be configured to provide access to a specific +subdirectory tree of the current working directory via a 'root' policy +entry. + +! +! +! +! + +The provided service is comparable to "chrooting" a client. +Accesses outside of the tree are denied by the server. Like the common +practice with chroot, the client directory trees can be assembled by +linking files into the visible directory tree. As we do not depend on +hard links, the client tree may include links to entire sub-trees of +the host file system via symbolic links to directories. Note that the +current implementation does not support to create symlinks via the +file-system interface. + + +Block-session extended by sync call +=================================== + +We extended the block-session interface with the ability to indicate that +synchronization with the device is necessary. By now, all block device drivers +in Genode work fully synchronous. Hence, there was no need for an explicit +sync request by the client. When reworking block-device drivers to work +asynchronously to potentially increase the performance or when adding a block +cache component, the need for a synchronization call came up. Clients, like +for instance a file system, have to ensure that their data is written back to +the device at certain points of execution. The new 'sync()' call enables +block-server developers to implement their components asynchronously, while at +the same time account for the integrity needs of block-session clients. + + +Configurable NIC buffer sizes for lwIP-based applications +========================================================= + +Whenever a network client opens a NIC-session, it has to provide buffers for +the receive/transmit (RX/TX) packet queues. By now, we've used relatively low +default values for dimensioning those buffers. +However, we identified that the demands by different applications vary a lot. +Applications that require high throughput need large buffers whereas +applications that issue sporadic network traffic can live with small buffers. +Therefore, we extended the lwIP library's initialization with the ability to +manually define the buffer sizes. Moreover, as lwIP is mostly used as network +backend for Genode's libc, we extended the libc lwip plugin with a proper +configuration option. To increase the default buffer sizes for a network +application, the following snippet can be added to its configuration section + +! + + +Support for recursive mutexes within the pthread library +======================================================== + +As argued by one of the authors of the POSIX thread API, +[http://www.zaval.org/resources/library/butenhof1.html - recursive mutexes are bad]. +Until now, we happily got away with not supporting them in Genode's pthread +library. However, during our work on enabling QML on Genode, we hit the problem +of ported 3rd-party code relying on recursive-mutex acquisition, which +prompted us to finally implement the semantics as specified in the standard. + + +Device drivers +############## + +Raspberry Pi +============ + +We improved the platform support for the Raspberry Pi to a level where +Genode's graphical demo scenario works well on the Pi. This required +us to supplement device drivers for USB (for USB HID), Videocore mboxes +(power management and HDMI), and for the framebuffer. + +The USB host-controller driver is ported from the official fork of the Linux +kernel for the Raspberry Pi. Unfortunately, it is not part of the normal Linux +kernel. For this reason, we need to download it separately. There exists a +'prepare_rpi' rule in the 'dde_linux/Makefile' to automate this process. + +The Raspberry-Pi-specific platform driver is used to access the features +provided by the Videocore mboxes, i.e., power configuration and framebuffer +setup. The framebuffer driver uses the platform interface to setup a screen +mode of 1024x768. + +The demo scenario at _os/run/demo.run_ can be executed on the Raspberry +Pi using the following steps: + +* Download the USB driver + ! make -C dde_linux prepare_rpi +* Download the C library (needed for setjmp/longjmp as used by the USB + driver) + ! make -C libports prepare PKG=libc +* Create a build directory + ! ./tool/create_builddir hw_rpi BUILD_DIR=build.rpi +* Change to the new build directory + ! cd build.rpi +* Enable the _dde_linux_ and _libports_ repositories by uncommenting the + corresponding lines in your build directory's _etc/build.conf_ file. + Optionally, you may also enable parallelized building by adding the line + ! MAKE += -j4 +* Build the demo scenario + ! make run/demo +* When the build process is finished, you can find the resulting ELF + image at _var/run/demo/image.elf_. There are multiple options to + boot the image on the Raspberry Pi, for example by using the u-boot + boot loader or using JTAG. The simplest way, however, is to put an image + directly on an SD-card. For doing so, prepare an SD-card with the + regular firmware for the Raspberry Pi as found in the official + Git repository at [https://github.com/raspberrypi/firmware]. + + The first boot stage is executed by the so-called Videocore GPU, which + fetches the boot code (called _bootcode.bin_) from the SD-card. At the + second stage, the boot code loads the actual OS kernel from the SD-card + (called _kernel.img_). Normally, this is the Linux kernel. To boot Genode + instead of Linux, we have to persuade the boot code to load our image + instead of the regular '_kernel.img_' file. Because the Raspberry Pi boot + code´ is not able to load ELF files, we first need to convert the Genode ELF + image to a raw binary using the _objcopy_ tool: + ! /usr/local/genode-gcc/bin/genode-arm-objcopy -Obinary \ + ! var/run/demo/image.elf \ + ! genode.img + Finally, we have to tell the boot code to load our image instead of the + Linux kernel. This can be done by putting a simple _config.txt_ file + with the following content onto the SD-card: + ! kernel=genode.img + ! kernel_address=0x00800000 + +[image rpi_demo] + Live from this year's Genode Hack'n'Hike: the graphical Genode demo running + on the Raspberry Pi + + +Samsung Exynos 5 +================ + +We extended the support for the Exynos-5 SoC with a new HDMI driver located at +_os/src/drivers/framebuffer/exynos5_ as well as support for USB 3.0 mass +storage. Thereby Genode reaches a pretty thorough driver coverage for the +Exynos-5 platform including SATA 3.0 and 2.0, USB 3.0, DVFS, eMMC, +networking, and HDMI. + +To exercise those features, we have assembled an integrated system scenario +for the Exynos-5-based Arndale board. The run script is located at +_ports-foc/run/l4linux_dynamic.run_. It comes in two stages. At the first +stage, the user can interact with the system over UART using the terminal_mux +terminal multiplexer and a command-line interface. It is possible to start +multiple instances of L4Linux and assign different block devices to the +instances. It is also possible to run VIM using the Noux runtime environment +to edit files on a VFAT-formatted SATA disk. The second stage can be started +via the 'start graphical_demo'. It presents another instance of the +command-line interface. This instance allows the start of the GNU debugger or +the scout demo program. + + +Libraries and applications +########################## + +Qt5 with support for OpenGL and QML +=================================== + +The inclusion of Qt5 in Genode 13.08 apparently spawned interest in using QML +as QML is widely regarded as the most important improvement of Qt5 compared +with Qt4. With the current release, we are happy to announce the principal +support for QML on Genode. The porting process was more elaborate than we +had hoped for. First, the V8 Javascript engine as used by QML comes with its +own platform abstraction (rather than building on top of Qt), which we had +to implement for Genode. While doing so, we needed to complement our POSIX +thread library to support recursive mutexes. Because QML relies on OpenGL, we +had to integrate Qt with Genode's port of Mesa, which implied work on our EGL +driver. + +[image qt5_qml_samegame] + The QML based Qt5 example game called "samegame" running on Genode. + +The QML support is still in an experimental state. It has been primarily +developed and tested on Genode/Linux on x86_64. We plan to enable QML for +the other Genode platforms and optimize performance during the upcoming +release cycle. + +If you want to start experimenting with QML on Genode right away, here are +the basic steps for realizing a QML application: + +# The easiest way to get started is using a copy of the + _libports/src/app/qt5/qt_quicktest_ as a template. + +# _main.cpp_ creates a 'QQuickView' object, which loads the main QML file and + starts to kick off the interpreter. Here, you may customize the + name of the main QML file according to your needs. + +# The Qt resource file _qt_quicktest.qrc_ contains QML files as well as + data files as referenced by the QML code (such as images or Javascript + files). All files required by your QML project must be listed here. + +# A copy of the QMake project file _qt_quicktest.pro_ should be named after + your project and customized according to your changes of the qrc file. + Add further C++ sources and headers as needed. + +# The Genode build description file _target.mk_ does not require any changes + as long as your project does not depend on libraries other than Qt. + +# At _libports/run/qt5_quicktest.run_, you can find a suitable template for + a Genode run script. You might need to adapt at least the program name + and the RAM quota. + + +File systems based on FUSE +========================== + +Besides a few special-purpose file systems (e.g., a RAM based file system and +the TAR archive file system), up to now, Genode supports merely one +general-purpose file system, namely FAT32, which is severely limited, + +As one possible step to provide support for serious general purpose file +systems in Genode, we took a look at the FUSE API, created an experimental +implementation thereof, and ported _fuse-exfat_ and _fuse-ext2_ to Genode. +Since FUSE file systems implement file operations by using the actual path +as an argument, it was easy to write a wrapper of these operations in form of a +libc plugin called _libc_fuse_. By combing our FUSE implementation with the +original 'FUSE' file-system server code, accessing the file system in question +from a program has become possible. As Genode does not use the normal mount +mechanism employed by the original FUSE implementation, the start-up code, +normally implemented in the main routine of a FUSE file-system server, needs +to be provided by the FUSE file system port itself. The FUSE file system +uses the new _libc_block_ plugin described in Section +[New C-runtime plugin for accessing block devices] to direct requests +referring to a virtual Unix block device to a Genode block session. + +For quickly trying out the new FUSE-based file systems, there are ready-to-use +run scripts located at _libports/run/libc_fuse_ext2.run_ and +_libc_fuse_exfat.run_ respectively. Before using those run scripts, make sure +to prepare the packages "exfat" and "fuse-ext2" as found in the _libports_ +repository. + + +Launchpad +========= + +The launchpad demo application has been updated to use a more modern +configuration syntax and has its built-in default configuration removed. +Furthermore, launch entries have become able to supply configurations +to sub systems, either via a '' sub node or a '' +declaration. + + +DosBox +====== + +DosBox is a DOS emulator, which is primarily focused on running DOS games. +Because it is fun to play with, we used DosBox as a proof of concept to +demonstrate the ease of porting existing software to Genode and of course, we +like to play as well. A step-by-step tutorial of how to approach such porting +work will be published soon. For trying out DosBox on Genode, we added a +simple-to-use run script at _ports/run/dosbox.run_. + + +Mesa / EGL driver +================= + +Our work on QML required us to improve our EGL driver for Mesa. Among other +changes, we enabled the driver to let Mesa render into a user-provided buffer +instead of the screen. This can be achieved with the +'eglCreateWindowSurface()' function, which takes a buffer description as third +argument. + + +libSDL improvements +=================== + +For porting DosBox, we had to extend and improve our libSDL port. The former +SDL_Timer implementation was too coarse. So we had to change it to a more +fine granular one. In addition, we enabled the SDL_CDROM dummy code and ported +SDL_net, which is needed for network support in DosBox. + + +Runtime environments +#################### + +GNU Debugger +============ + +We improved the GDB support on Genode to cover advanced debugging features, +namely the 'set var' and 'call' commands. As those commands require the +debugger to modify the content of CPU registers of threads, we enhanced +Genode's 'Cpu_session::state' function such that it can do both obtain and +modify the state of a thread while the thread is paused. + +To make the use of GDB for debugging Genode subsystems more comfortable, we +added a new command named "gdb" to Genode's command-line interface +(cli_monitor). This command works similarly to the "start" command, but +instead of starting the subsystem binary directly, an _init_ subsystem gets +started, which then starts _terminal_crosslink_, Noux, GDB and GDB monitor, +which, in turn, starts the application binary as its target. The "gdb" command +supports the following command line options: + +:--ram: the initial RAM quota provided to the whole subsystem + (including the GDB-related components) +:--ram-limit: limit for expanding RAM quota +:--gdb-ram-preserve: the RAM quota that GDB monitor should preserve + for itself + +For the "gdb" command to work, _terminal_crosslink_, _noux_, _gdb_monitor_ and +the file _gdb_command_config_ are expected to be present as ROM modules. The +GDB client needs to get mounted at _/bin_ in Noux and the target binaries +need to be available as ROM modules (loaded by GDB monitor) and also mounted +at _/gdb_ in Noux (loaded by the GDB client). Additionally, the source code of +the target application can be provided at _/gdb/src/_ to Noux to enable +source-level debugging. How the Noux mountings get established can be +configured in the _gdb_command_config_ file. The default configuration in +_os/src/server/cli_monitor/gdb_command_config_ mounts GDB from a tar archive +named _gdb.tar_, the GDB target binaries come from a tar archive named +'gdb_target.tar', and the target source code comes from a tar archive named +'gdb_target-src.tar'. + +To simplify the assembly of such a scenario, the _ports/run/noux_gdb.inc_ +include file provides functions that help to create the needed tar files: + +:'create_gdb_tar': creates a tar archive for the GDB client +:'create_binary_tar': creates a tar archive for the target application +:'create_source_tar': creates a tar archive for the source code of + the target application +:'create_binary_and_source_tars': is a convenience wrapper for the previous + two functions + +The integration of GDB with cli_monitor is exemplified by the run script +at _ports/run/noux_gdb_dynamic.run_. + + +Virtualization on NOVA +====================== + +To ease the creation of custom virtual machine monitors on top of NOVA, we +added a set of generic utilities to the public include location +_ports/include/vmm_. As a nice side effect, this change simplifies the +Genode-specific code of the Seoul VMM. In the future, the VMM utilities +will ease the realization of alternative virtual-machine monitors on the NOVA +hypervisor. + +The Seoul VMM facilitates the co-location of the VMM and the guest OS within +the same protection domain to keep context-switching costs as low as possible. +However, this approach requires tight control over the virtual address space +of the VMM. In particular, the VMM must not interfere with the guest-physical +memory, which is always mapped at the lower part of the virtual address space. +Maintaining this condition is complicated. Relaxing this rigid scheme would +make our life much easier and would furthermore enable the use of shared +libraries for the VMM. For this reason, we explored the option of running the +guest OS in a distinct protection domain. This change does not only simplify +the VMM, but it also nearly doubles the maximum configurable guest-memory size +for a 32bit Genode/NOVA host system. The price for those benefits may be a +degraded performance. However, using basic benchmarks (e.g., compiling the +Linux kernel in a VM), we could hardly see negative effects. Co-location and +non-co-location can be set by Seoul's new 'colocate' configuration attribute. + + +ARM TrustZone +============= + +In Genode's release 12.11, we added experimental support for ARM TrustZone +technology, which principally enabled the execution of Genode in the so-called +secure world of TrustZone while executing Linux in the so-called normal world. +These experiments were conducted on ARM's Versatile express platform. With the +current release, we have added the i.MX53 SoC as additional platform to +experiment with its TrustZone aware devices. If you want to try out a very +basic example, starting a Linux instance running on the non-secure world +beside Genode running in the secure one, you can test the 'tz_vmm.run' run +script on Versatile Express, i.MX53 Quickstart board, or i.MX53 SABRE tablet. + + +Platforms +######### + +Execution on bare hardware (base-hw) +==================================== + +The development of Genode's custom kernel platform for the ARM architecture +goes full-steam ahead. The new version introduces a new way of using the +kernel's asynchronous notification mechanism to deliver page faults and +interrupts to the user land, yielding vastly simplified code - compared to the +classical (L4) approach of providing a page-fault protocol via synchronous +IPC. To better support real-time workloads, we enhanced the scheduler with +static priorities. Finally, we complemented the base-hw platform with the code +needed for process destruction. This way, we can execute dynamic workloads +such as Noux. Base-hw is still at an experimental stage and very much in flux, +but the pace of the current development illustrates well that it is making its +way to become a fully-fledged base platform for Genode soon. + + +NOVA microhypervisor +==================== + +After enabling multi-processor support and the destruction of kernel objects +in the previous releases, the new implemented life-time management of kernel +capability selectors represents the final piece of the puzzle to use NOVA as a +fully-supported kernel platform for Genode. + +A kernel capability selector on NOVA is similar in nature to a file descriptor +on Unix, as it refers to a kernel object. In contrast to a file descriptor, +however, the kernel object is not a file but the identity of an user-level +object. Another difference is the way how a capability selector appears in a +process. On Unix, most file descriptors are directly created by a process via +system calls like _open_ or _socket_. On NOVA, processes obtain capability +selectors via IPC messages from other processes. Both file descriptors and +capability selectors have in common that they should be closed when they are +no longer needed. Otherwise, the name space of file descriptors or capability +selectors gets polluted by stale kernel objects and long-running processes +bear the risk of resource leakage. + +Because Genode is implemented in C++, the framework is able to manage the +lifetime of capability selectors by modelling the 'Capability' type as a +reference-counting smart pointer. After we first implemented this idea for the +Fiasco.OC base platform, we have now successfully applied the same approach to +the NOVA platform. + + +Fiasco.OC kernel and L4Linux +============================ + +To enable the advanced features of the GNU debugger as described in Section +[GNU Debugger], we extended Genode's facility to access thread states such +that the register contents of a paused thread (which is not currently +executing a syscall) can get modified by the 'Cpu_session::state()' function. + +L4Linux is a paravirtualized Linux kernel that can be used on top of the +Fiasco.OC kernel. To experiment with use cases where many instances of L4Linux +are executed on a single machine, and as a way to exercise the new +resource-balancing mechanisms described in Section [Dynamic resource balancing], +we extended L4Linux with a custom ballooning mechanism, which is similar to +solutions like Xen's balloon driver. For this, the new parent interface +extensions for requesting and yielding resources are used. L4Linux registers a +yield signal context at its parent. Whenever the parent triggers a yield, the +balloon driver "inflates its balloon", which means that it requests all pages +available, and then frees the corresponding memory dataspaces. + +The ballooning driver is enabled by default. From Genode's perspective, +L4Linux behaves like a regular Genode subsystem that positively responds to +resource-yield requests. + + +OKL4 kernel +=========== + +We removed the support for the paravirtualized OKLinux kernel for OKL4. +For running Linux on top of Genode, there exist several alternatives +such as the Seoul VMM on NOVA, the ARM TrustZone support in base-hw, or +the paravirtualized L4Linux kernel running on top of Fiasco.OC. +