diff --git a/doc/news.txt b/doc/news.txt index fd000ba87..61932419a 100644 --- a/doc/news.txt +++ b/doc/news.txt @@ -3,6 +3,20 @@ Genode News =========== +Guide for porting 3rd-party software to Genode | 2014-01-28 +########################################################### + +| Our new porting guide provides step-by-step instructions on how to +| port existing libraries, applications, and device drivers to Genode. + +The comprehensive documentation is written in the style of a tutorial, taking +DosBox as a practical example to motivate the steps needed to port an existing +application to Genode. Furthermore, it gives valuable insights into +time-tested methodologies of porting libraries and device drivers. + +[http://genode.org/documentation/developer-resources/porting - Genode Porting Guide...] + + Road Map 2014 | 2014-01-17 ########################## diff --git a/doc/porting_guide.txt b/doc/porting_guide.txt new file mode 100644 index 000000000..162101d85 --- /dev/null +++ b/doc/porting_guide.txt @@ -0,0 +1,1476 @@ + ==================== + Genode Porting Guide + ==================== + + Genode Labs GmbH + + +Overview +######## + +This document describes the basic workflows for porting applications, libraries, +and device drivers to the Genode framework. It consists of the following +sections: + +:[http:porting_applications - Porting third-party code to Genode]: + Overview of the general steps needed to use 3rd-party code on Genode. + +:[http:porting_dosbox - Porting a program to natively run on Genode]: + Step-by-step description of applying the steps described in the first + section to port an application, using DosBox as an example. + +:[http:porting_libraries - Native Genode port of a library]: + Many 3rd-party applications have library dependencies. This section shows + how to port a library using SDL_net (needed by DosBox) as an example. + +:[http:porting_noux_packages - Porting an application to Genode's Noux runtime]: + On Genode, there exists an environment specially tailored to execute + command-line based Unix software, the so-called Noux runtime. This section + demonstrates how to port and execute the tar program within Noux. + +:[http:porting_device_drivers - Porting devices drivers]: + This chapter describes the concepts of how to port a device driver to the + Genode framework. It requires the basic knowledge introduced in the previous + chapters and should be read last. Before reading this guide, it is strongly + advised to read the "The Genode Build System" documentation. + + +Porting third-party code to Genode +################################## + +Porting an existing program or library to Genode is for the most part a +straight-forward task and depends mainly on the complexity of the program +itself. Genode provides a fairly complete libc based on FreeBSD's libc whose +functionality can be extended by so-called libc plugins. If the program one +wants to port solely uses standard libc functions, porting becomes easy. Every +porting task involves usually the same steps which are outlined below. + + +Steps in porting applications to Genode +======================================= + +# Check requirements/dependencies (e.g. on Linux) + + The first step is gathering information about the application, + e.g. what functionality needs to be provided by the target system and + which libraries does it use. + +# Create a port Makefile + + Prepare the source code of the application for the use within Genode. The + Genode build-system infrastructure uses fetch rules, so called port-files, + which describe how the source is obtained, what patches are applied to the + source code, if needed, and where the source code will be stored and + configured. + +# Check platform dependent code and create stub code + + This step may require changes to the original source code + of the application to be compilable to Genode. At this point, it + is not necessary to provide a working implementation for required + functions. Just creating stubs of the various functions is fine. + +# Create build Makefile + + To compile the application we need build rules. Within these rules + we also declare all dependencies (e.g. libraries) that are needed + by it. The location of these rules depends on the type + of the application. Normal programs on one hand use a _target.mk_ file, + which is located in the program directory (e.g. _src/app/foobar_) + within a given Genode repository. Libraries on the other hand use + one or more _.mk_ files that are placed in the _lib/mk_ + directory of a Genode repository. In addition, libraries have to + provide _import-.mk_ files. Amongst other things, these + files are used by applications to find the associated header files + of a library. The import files are placed in the _lib/import_ + directory. + +# Create a run script to ease testing + + To ease the testing of applications, it is reasonable to write a run script + that creates a test scenario for the application. This run script is used + to automatically build all components of the Genode OS framework that are + needed to run the application as well as the application itself. Testing + the application on any of the kernels supported by Genode becomes just a + matter of executing the run script. + +# Compile the application + + The ported application is compiled from within the respective build + directory like any other application or component of Genode. The build + system of Genode uses the build rules created in the fourth step. + +# Run the application + + While porting an application, easy testing is crucial. By using the run script + that was written in the fifth step we reduce the effort. + +# Debug the application + + In most cases, a ported application does not work right away. We have to + debug misbehaviour and implement certain functionality in the platform-depending + parts of the application so that is can run on Genode. There are + several facilities available on Genode that help in the process. These are + different on each Genode platform but basically break down to using either a + kernel debugger (e.g., JDB on Fiasco.OC) or 'gdb(1)'. The reader of this guide + is advised to take a look at the "User-level debugging on Genode via GDB" + documentation. + +_The order of step 1-4 is not mandatory but is somewhat natural._ + + +Porting a program to natively run on Genode +########################################### + +As an example on how to create a native port of a program for Genode, we will +describe the porting of DosBox more closely. Hereby, each of the steps +outlined in the previous section will be discussed in detail. + + +Check requirements/dependencies +=============================== + +In the first step, we build DosBox for Linux/x86 to obtain needed information. +Nowadays, most applications use a build-tool like Autotools or something +similar that will generate certain files (e.g., _config.h_). These files are +needed to successfully compile the program. Naturally they are required on +Genode as well. Since Genode does not use the original build tool of the +program for native ports, it is appropriate to copy those generated files +and adjust them later on to match Genode's settings. + +We start by checking out the source code of DosBox from its subversion repository: + +! $ svn export http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk@3837 dosbox-svn-3837 +! $ cd dosbox-svn-3837 + +At this point, it is helpful to disable certain options that are not +available or used on Genode just to keep the noise down: + +! $ ./configure --disable-opengl +! $ make > build.log 2>&1 + +After the DosBox binary is successfully built, we have a log file +(build.log) of the whole build process at our disposal. This log file will +be helpful later on when the _target.mk_ file needs to be created. In +addition, we will inspect the DosBox binary: + +! $ readelf -d -t src/dosbox|grep NEEDED +! 0x0000000000000001 (NEEDED) Shared library: [libasound.so.2] +! 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] +! 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] +! 0x0000000000000001 (NEEDED) Shared library: [libSDL-1.2.so.0] +! 0x0000000000000001 (NEEDED) Shared library: [libpng12.so.0] +! 0x0000000000000001 (NEEDED) Shared library: [libz.so.1] +! 0x0000000000000001 (NEEDED) Shared library: [libSDL_net-1.2.so.0] +! 0x0000000000000001 (NEEDED) Shared library: [libX11.so.6] +! 0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6] +! 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] +! 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1] +! 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] + +Using _readelf_ on the binary shows all direct dependencies. We now know +that at least libSDL, libSDL_net, libstdc++, libpng, libz, and +libm are required by DosBox. The remaining libraries are mostly +mandatory on Linux and do not matter on Genode. Luckily all of these +libraries are already available on Genode. For now all we have to do is to +keep them in mind. + + +Creating the port Makefile +========================== + +Since DosBox is an application, which depends on several ported +libraries (e.g., libSDL), the 'ports' repository within the Genode +source tree is a natural fit. On that account, the port Makefile +_ports/ports/dosbox.mk_ is created. It is often reasonable to also +create a corresponding _ports/ports/dosbox.inc_ file that contains +the used version of the program which is included by _dosbox.mk_ as +well as by the associated build Makefile. Through this approach it is +easier to update ports whose structures stays the same but varies only +in its version string. + +For DosBox the _dosbox.inc_ looks as follows: + +! DOSBOX_REV := 3837 +! DOSBOX_VERSION := svn-$(DOSBOX_REV) +! DOSBOX := dosbox-$(DOSBOX_VERSION) + +In addition, the corresponding _dosbox.mk_ contains: + +! include ports/dosbox.inc +! +! DOSBOX_SVN_URL = http://svn.code.sf.net/p/dosbox/code-0/dosbox/trunk +! +! # +! # Interface to top-level prepare Makefile +! # +! PORTS += $(DOSBOX) +! +! prepare:: $(CONTRIB_DIR)/$(DOSBOX) +! +! # +! # Port-specific local rules +! # +! $(CONTRIB_DIR)/$(DOSBOX): +! $(ECHO) "checking out 'dosbox rev. $(DOSBOX_REV)' to '$@'" +! $(VERBOSE)svn export $(DOSBOX_SVN_URL)@$(DOSBOX_REV) $@ + +_The variable_ 'CONTRIB_DIR' _always contains the path to the directory_ +_that holds the contributed source code relatively to the particular_ +_Genode repository (e.g., ports/contrib)._ + +The DosBox port can now be prepared by executing + +! $ make PKG=dosbox prepare + +from within Genode's 'ports' repository. The 'prepare::' rule is evaluated +and the checkout will be triggered. + + +Check platform-dependent code +============================= + +At this point, it is important to spot platform dependent source files or +rather certain functions that are not yet available on Genode. These source +files should be omitted. Of course they may be used as a guidance when +implementing the functionality for Genode later on, when creating the +_target.mk_ file. In particular the various 'cdrom_ioctl_*.cpp' files are such +candidates in this example. + + +Creating the build Makefile +=========================== + +Now it is time to write the build rules into the _target.mk_, which will be +placed in _ports/src/app/dosbox_. + +Armed with the _build.log_ that we created while building DosBox on Linux, +we assemble a list of needed source files. If an application just +uses a simple Makefile and not a build tool, it might be easier to just +reuse the contents of this Makefile instead. + +First we include _dosbox.inc_ and for convenience set the source directory +of DosBox: + +! include $(REP_DIR)/ports/dosbox.inc +! DOSBOX_DIR = $(REP_DIR)/contrib/$(DOSBOX) + +Examining the log file leaves us with the following list of source files: + +! SRC_CC_cpu = $(notdir $(wildcard $(DOSBOX_DIR)/src/cpu/*.cpp)) +! SRC_CC_debug = $(notdir $(wildcard $(DOSBOX_DIR)/src/debug/*.cpp)) +! FILTER_OUT_dos = cdrom_aspi_win32.cpp cdrom_ioctl_linux.cpp cdrom_ioctl_os2.cpp \ +! cdrom_ioctl_win32.cpp +! SRC_CC_dos = $(filter-out $(FILTER_OUT_dos), \ +! $(notdir $(wildcard $(DOSBOX_DIR)/src/*.cpp))) +! […] +! SRC_CC = $(DOSBOX_DIR)/src/dosbox.cpp +! SRC_CC += $(SRC_CC_cpu) $(SRC_CC_debug) $(SRC_CC_dos) $(SRC_CC_fpu) \ +! $(SRC_CC_gui) $(SRC_CC_hw) $(SRC_CC_hw_ser) $(SRC_CC_ints) \ +! $(SRC_CC_ints) $(SRC_CC_misc) $(SRC_CC_shell) +! +! vpath %.cpp $(DOSBOX_DIR)/src +! vpath %.cpp $(DOSBOX_DIR)/src/cpu +! […] + +_The only variable here that is actually evaluated by Genode's build-system is_ +'SRC_CC' _. The rest of the variables are little helpers that make our_ +_life more comfortable._ + +In this case, it is mandatory to use GNUMake's 'notdir' file name function +because otherwise the compiled object files would be stored within +the _contrib_ directories. Genode runs on multiple platforms with varying +architectures and mixing object files is considered harmful, which can happen +easily if the application is build from the original source directory. That's +why you have to use a build directory for each platform. The Genode build +system will create the needed directory hierarchy within the build directory +automatically. By combining GNUMake's 'notdir' and 'wildcard' function, we +can assemble a list of all needed source files without much effort. We then +use 'vpath' to point GNUMake to the right source file within the _contrib_ +directory. + +The remaining thing to do now is setting the right include directories and proper +compiler flags: + +! INC_DIR += $(PRG_DIR) +! INC_DIR += $(DOSBOX_DIR)/include +! INC_DIR += $(addprefix $(DOSBOX_DIR)/src, cpu debug dos fpu gui hardware \ +! hardware/serialport ints misc shell) + +'PRG_DIR' _is a special variable of Genode's build-system_ +_and its value is always the absolute path to the directory containing_ +_the 'target.mk' file._ + +We copy the _config.h_ file, which was generated in the first step to this +directory and change certain parts of it to better match Genode's +environment. Below is a skimmed diff of these changes: + +! --- config.h.orig 2013-10-21 15:27:45.185719517 +0200 +! +++ config.h 2013-10-21 15:36:48.525727975 +0200 +! @@ -25,7 +25,8 @@ +! /* #undef AC_APPLE_UNIVERSAL_BUILD */ +! +! /* Compiling on BSD */ +! -/* #undef BSD */ +! +/* Genode's libc is based on FreeBSD 8.2 */ +! +#define BSD 1 +! +! […] +! +! /* The type of cpu this target has */ +! -#define C_TARGETCPU X86_64 +! +/* we define it ourself */ +! +/* #undef C_TARGETCPU */ +! +! […] + +Thereafter we specify the compiler flags: + +! CC_OPT = -DHAVE_CONFIG_H -D_GNU_SOURCE=1 -D_REENTRANT +! ifeq ($(filter-out $(SPECS),x86_32),) +! INC_DIR += $(PRG_DIR)/x86_32 +! CC_OPT += -DC_TARGETCPU=X86 +! else ifeq ($(filter-out $(SPECS),x86_64),) +! INC_DIR += $(PRG_DIR)/x86_64 +! CC_OPT += -DC_TARGETCPU=X86_64 +! endif +! +! CC_WARN = -Wall +! #CC_WARN += -Wno-unused-variable -Wno-unused-function -Wno-switch \ +! -Wunused-value -Wno-unused-but-set-variable + +As noted in the commentary seen in the diff we define 'C_TARGETCPU' +and adjust the include directories ourselves according to the target +architecture. + +While debugging compiler warnings for 3rd-party code are really helpful but +tend to be annoying after the porting work is finished, we can +remove the hashmark to keep the compiler from complaining too +much. + +Lastly, we need to add the required libraries, which we acquired in step 1: + +! LIBS += libc libm libpng sdl stdcxx zlib +! LIBS += libc_log libc_fs libc_lwip_nic_dhcp config_args + +In addition to the required libraries, a few Genode specific +libraries are also needed. These libraries implement certain +functions in the libc via the libc's plugin mechanism. +libc_log, for example, is used to print message on stdout via +Genode's LOG service. + + +Creating the run script +======================= + +To ease compiling, running and debugging DosBox, we create a run script +at _ports/run/dosbox.run_. + +First, we specify the components that need to be built + +! set build_components { +! core init drivers/audio_out drivers/framebuffer drivers/input +! drivers/pci drivers/timer server/tar_fs app/dosbox +! } +! build $build_components + +and instruct _tool/run_ to create the boot directory that hosts +all binaries and files which belong to the DosBox scenario. + +As the name 'build_components' suggests, you only have to declare +the components of Genode, which are needed in this scenario. All +dependencies of DosBox (e.g. libSDL) will be built before DosBox +itself. + +Nextm we provide the scenario's configuration 'config': + +! append config { +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! } +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! } +! install_config $config + +The _config_ file will be used by the init program to start all +components and application of the scenario, including DosBox. + +Thereafter we declare all boot modules: + +! set boot_modules { +! core init timer audio_out_drv fb_drv ps2_drv ld.lib.so +! libc_fs.lib.so libc.lib.so libc_log.lib.so libm.lib.so +! lwip.lib.so libpng.lib.so stdcxx.lib.so sdl.lib.so +! pthread.lib.so zlib.lib.so tar_fs dosbox dosbox.tar +! } +! build_boot_image $boot_modules + +The boot modules comprise all binaries and other files like +the tar archive that contains DosBox' configuration file _dosbox.conf_ +that are needed for this scenario to run sucessfully. + +Finally, we set certain options, which are used when Genode is executed +in Qemu and instruct _tool/run_ to keep the scenario executing as long +as it is not manually stopped: + +! append qemu_args " -m 256 -soundhw ac97 " +! run_genode_until forever + +_It is reasonable to write the run script in a way that makes it possible_ +_to use it for multiple Genode platforms. Debugging is often done on_ +_Genode/Linux or on another Genode platform running in Qemu but testing_ +_is normally done using actual hardware._ + + +Compiling the program +===================== + +To compile DosBox and all libraries it depends on, we execute + +! $ make app/dosbox + +from within Genode's build directory. + +_We could also use the run script that we created in the previous step but_ +_that would build all components that are needed to actually run_ DosBox +_and at this point our goal is just to get_ DosBox _compiled._ + +At the first attempt, the compilation stopped because g++ could not find +the header file _sys/timeb.h_: + +! /src/genode/ports/contrib/dosbox-svn-3837/src/ints/bios.cpp:35:23: fatal error: +! sys/timeb.h: No such file or directory + +This header is part of the libc but until now there was no program, which +actually used this header. So nobody noticed that it was missing. This +can happen all time when porting a new application to Genode because most +functionality is enabled or rather added on demand. Someone who is +porting applications to Genode has to be aware of the fact that it might be +necessary to extend Genode functionality by enabling so far disabled +bits or implementing certain functionality needed by the +application that is ported. + +Since 'ftime(3)' is a deprecated function anyway we change the code of +DosBox to use 'gettimeofday(2)'. + +After this was fixed we face another problem: + +! /src/genode/ports/contrib/dosbox-svn-3837/src/ints/int10_vesa.cpp:48:33: error: +! unable to find string literal operator ‘operator"" VERSION’ + +The fix is quite simple and the compile error was due to the fact +that Genode uses C++11 by now. It often happens that 3rd party code +is not well tested with a C++11 enabled compiler. In any case a patch file +should be created which will be applied when executing the fetch-rules: + +! $(CONTRIB_DIR)/$(DOSBOX): +! […] +! $(VERBOSE)patch -N -p0 < src/app/dosbox/int10_vesa.patch + +Furthermore it would be reasonable to report the bug to the DosBox +developers so it can be fixed upstream. We can then get rid of our +local patch. + +The next show stoppers are missing symbols in Genode's SDL library port. +As it turns out, we never actually compiled and linked in the cdrom dummy +code which is provided by SDL. + + +Running the application +======================= + +DosBox was compiled successfully. Now it is time to execute the binary +on Genode. Hence we use the run script we created in step 5: + +! $ make run/dosbox + +This may take some time because all other components of the Genode OS +Framework that are needed for this scenario have to be built. + + +Debugging the application +========================= + +DosBox was successfully compiled but unfortunately it did not run. +To be honest that was expected and here the fun begins. + +At this point there are several options to chose from. By running +Genode/Fiasco.OC within Qemu we can use the kernel debugger (JDB) +to take a deeper look at what went wrong (e.g. backtraces of the +running processes, memory dumps of the faulted DosBox process etc.). +Doing this can be quite taxing but fortunately Genode runs on multiple +kernels and often problems on one kernel can be reproduced on another +kernel. For this reason we choose Genode/Linux where we can use all +the normal debugging tools like 'gdb(1)', 'valgrind(1)' and so on. Luckily +for us, DosBox also fails to run on Genode/Linux. The debugging steps +are naturally dependent on the ported software. In the case of DosBox, +the remaining stumbling blocks were a few places where DosBox assumed +Linux as a host platform. + +For the sake of completeness here is a list of all files that were created by +porting DosBox to Genode: + +! ports/ports/dosbox.inc +! ports/ports/dosbox.mk +! ports/run/dosbox.run +! ports/src/app/dosbox/config.h +! ports/src/app/dosbox/patches/bios.patch +! ports/src/app/dosbox/patches/int10_vesa.patch +! ports/src/app/dosbox/target.mk +! ports/src/app/dosbox/x86_32/size_defs.h +! ports/src/app/dosbox/x86_64/size_defs.h + +[image dosbox] + DosBox ported to Genode + + +Native Genode port of a library +############################### + +Porting a library to be used natively on Genode is similar to porting +an application to run natively on Genode. The source codes has to be +obtained and, if needed, patched to run on Genode. +As an example on how to port a library to natively run on Genode, we +will describe the porting of SDL_net in more detail. Ported libraries +are placed in the _libports_ repository of Genode. + + +Checking requirements/dependencies +================================== + +We will proceed as we did when we ported DosBox to run natively on Genode. +First we build SDL_net on Linux to obtain a log file of the whole build +process: + +! $ wget http://www.libsdl.org/projects/SDL_net/release/SDL_net-1.2.8.tar.gz +! $ tar xvzf SDL_net-1.2.8.tar.gz +! $ cd SDL_net-1.2.8 +! $ ./configure +! $ make > build.log 2>&1 + + +Creating the port Makefile +========================== + +We start by creating 'libports/ports/sdl_net.inc': + +! SDL_NET_VERSION = 1.2.8 +! SDL_NET = SDL_net-$(SDL_NET_VERSION) + +Following this we create the actual fetch-rules +in _libports/ports/sdl_net.mk_ + +! include ports/sdl_net.inc +! +! SDL_NET_TGZ = $(SDL_NET).tar.gz +! SDL_NET_URL = http://www.libsdl.org/projects/SDL_net/release/$(SDL_NET_TGZ) +! +! # +! # Interface to top-level prepare Makefile +! # +! # Register SDL_net port as lower case to be consistent with the +! # other libraries. +! # +! PORTS += sdl_net-$(SDL_NET_VERSION) +! +! prepare-sdl_net: $(CONTRIB_DIR)/$(SDL_NET) include/SDL/SDL_net.h +! +! $(CONTRIB_DIR)/$(SDL_NET): clean-sdl_net + +While extracting the archive, we omit certain directories and files that we +do not need and that otherwise would only pollute the 'contrib' directory: + +! # +! # Port-specific local rules +! # +! $(DOWNLOAD_DIR)/$(SDL_NET_TGZ): +! $(VERBOSE)wget -c -P $(DOWNLOAD_DIR) $(SDL_NET_URL) && touch $@ +! +! $(CONTRIB_DIR)/$(SDL_NET): $(DOWNLOAD_DIR)/$(SDL_NET_TGZ) +! $(VERBOSE)tar xfz $< \ +! --exclude Xcode --exclude VisualC --exclude Xcode-iOS \ +! --exclude VisualCE --exclude Watcom-OS2.zip --exclude debian \ +! -C $(CONTRIB_DIR) && touch $@ +! $(VERBOSE)patch -N -p0 < src/lib/sdl_net/SDLnet.patch + +Applications that want to use SDL_net have to include the 'SDL_net.h' header +file. Hence it is necessary to make file visible to applications. This is +done by creating a symbolic link from the original location to +_libports/include/SDL/SDL_net.h_ + +! # +! # Install SDL_net headers +! # +! include/SDL/SDL_net.h: +! $(VERBOSE)mkdir -p $(dir $@) +! $(VERBOSE)ln -fs ../../$(CONTRIB_DIR)/$(SDL_NET)/SDL_net.h include/SDL/ + +Since we created a symbolic link that is placed in Genode's _libports_ +repository, +we have to provide a 'clean' rule. This rule is responsible for removing all +extracted files and the link we created: + +! clean-sdl_net: +! $(VERBOSE)rm -rf include/SDL/SDL_net.h +! $(VERBOSE)rmdir include/SDL 2>/dev/null || true +! $(VERBOSE)rm -rf $(CONTRIB_DIR)/$(SDL_NET) + +_It is important to write the_ 'clean' _rule in way that does not remove_ +_files that are still needed by other components of Genode or ported_ +_applications. In this example we only delete include/SDL if it is_ +_empty. Otherwise_ 'rmdir(1)' _will not remove the directory and we_ +_simply ignore it._ + +Creating the build Makefile +=========================== + +We create the build rules in _libports/lib/mk/sdl_net.mk_: + +! include $(REP_DIR)/ports/sdl_net.inc +! SDL_NET_DIR = $(REP_DIR)/contrib/$(SDL_NET) +! +! SRC_C = $(notdir $(wildcard $(SDL_NET_DIR)/SDLnet*.c)) +! +! vpath %.c $(SDL_NET_DIR) +! +! INC_DIR += $(SDL_NET_DIR) +! +! LIBS += libc sdl + +'SDL_net' should be used as shared library. To achieve this, we +have to add the following statement to the 'mk' file: + +! SHARED_LIB = yes + +_If we omit this statement, Genode's build-system will automatically_ +_build SDL_net as a static library called_ 'sdl_net.lib.a' _that_ +_is linked directly into the application._ + +It is a reasonable to create a dummy application that uses the +library because it is only possible to build libraries automatically +as a dependency of an application. + +Therefore we create +_libports/src/test/libports/sdl_net/target.mk_ with the following content: + +! TARGET = test-sdl_net +! LIBS = libc sdl_net +! SRC_CC = main.cc + +! vpath main.cc $(PRG_DIR)/.. + +At this point we normally would also create _lib/import/import-sdl_net.mk_ +with the following content: + +! REP_INC_DIR += include/SDL + +However in this case this is not necessary because _SDL_net_ depends on +libSDL which already provides its own _import-sdl.mk_ file with exactly +the same content. While preparing SDL_net we placed a symbolic link in +just this directory and therefore every user of SDL_net already knows +about this directory. + + +Compiling the library +===================== + +We compile the SDL_net library as a side effect of building our dummy test +program by executing + +! $ make test/libports/sdl_net + +All source files are compiled fine but unfortunately the linking of the +library does not succeed: + +! /src/genodebuild/foc_x86_32/var/libcache/sdl_net/sdl_net.lib.so: +! undefined reference to `gethostbyaddr' + +The symbol 'gethostbyaddr' is missing, which is often a clear sign +of a missing dependency. In this case however 'gethostbyaddr(3)' is +missing because this function does not exist in Genode's libc _(*)_. +But 'getaddrinfo(3)' exists. We are now facing the choice of implementing +'gethostbyaddr(3)' or changing the code of SDL_net to use 'getaddrinfo(3)'. +Porting applications or libraries to Genode always may involve this kind of +choice. Which way is the best has to be decided by closely examining the +matter at hand. Sometimes it is better to implement the missing functions +and sometimes it is more beneficial to change the contributed code. +In this case we opt for changing SDL_net because the former function is +obsolete anyway and implementing 'gethostbyaddr(3)' involves changes to +several libraries in Genode, namely libc and the network related +libc plugin. Although we have to keep in mind that is likely to encounter +another application or library that also uses this function in the future. + +With this change in place SDL_net compiles fine. + +_(*) Actually this function is implemented in the Genode's_ libc _but is_ +_only available by using libc_resolv which we did not do for the sake of_ +_this example._ + + +Testing the library +=================== + +The freshly ported library is best tested with the application, which was the +reason the library was ported in the first place, since it is unlikely that +we port a library just for fun and profit. Therefore, it is not necessary to +write a run script for a library alone. + +For the records, here is a list of all files that were created by +porting SDL_net to Genode: + +! libports/lib/mk/sdl_net.mk +! libports/ports/sdl_net.inc +! libports/ports/sdl_net.mk +! libports/src/lib/sdl_net/SDLnet.patch +! libports/test/libports/sdl_net/target.mk + + +Porting an application to Genode's Noux runtime +############################################### + +Porting an application to Genode's Noux runtime is basically the same as +porting a program to natively run on Genode. The source code has to be +prepared and, if needed, patched to run in Noux. However in contrast to +this there are Noux build rules (_ports/mk/noux.mk_) that enable us to use +the original build-tool if it is based upon Autotools. Building the +application is done within a cross-compile environment. In this environment +all needed variables like 'CC', 'LD', 'CFLAGS' and so on are set to their +proper values. In addition to these precautions, using _noux.mk_ simplifies certain things. +The system-call handling/functions is/are implemented in the libc plugin +_libc_noux_ (the source code is found in _ports/src/lib/libc_noux_). All +application running in Noux have to be linked against this library which is +done implicitly by using the build rules of Noux. + +As an example on how to port an application to Genode's Noux runtime, we +will describe the porting of GNU's 'tar' tool in more detail. A ported +application is normally referred to as a Noux package. + +Checking requirements/dependencies +================================== + +As usual, we first build GNU tar on Linux/x86 and capture the build +process: + +! $ wget http://ftp.gnu.org/gnu/tar/tar-1.27.tar.xz +! $ tar xJf tar-1.27.tar.xz +! $ cd tar-1.27 +! $ ./configure +! $ make > build.log 2>&1 + + +Creating the port Makefile +========================== + +We start by creating the port Makefile _ports/ports/tar.mk_: + +! GNUTAR = tar-1.27 +! GNUTAR_TXZ = $(GNUTAR).tar.xz +! GNUTAR_SIG = $(GNUTAR_TXZ).sig +! GNUTAR_URL = http://ftp.gnu.org/gnu/tar +! GNUTAR_KEY = GNU + +_As of release 13.08 Genode includes integrity checks for downloaded_ +_3rd-party software. The signature of the downloaded archive will be_ +_checked prior to extraction. New ports to Genode should always use_ +_this checks if it is possible. Unfortunately not all projects provide signature_ +_files though._ + +The remaining part of the port Makefile looks like this: + +! # +! # Interface to top-level prepare Makefile +! # +! PORTS += $(GNUTAR) +! +! prepare:: $(CONTRIB_DIR)/$(GNUTAR) +! +! # +! # Port-specific local rules +! # +! $(DOWNLOAD_DIR)/$(GNUTAR_TXZ): +! $(VERBOSE)wget -c -P $(DOWNLOAD_DIR) $(GNUTAR_URL)/$(GNUTAR_TXZ) && touch $@ +! +! $(DOWNLOAD_DIR)/$(GNUTAR_SIG): +! $(VERBOSE)wget -c -P $(DOWNLOAD_DIR) $(GNUTAR_URL)/$(GNUTAR_SIG) && touch $@ +! +! $(CONTRIB_DIR)/$(GNUTAR): $(DOWNLOAD_DIR)/$(GNUTAR_TXZ).verified +! $(VERBOSE)tar xfJ $(<:.verified=) -C $(CONTRIB_DIR) && touch $@ +! +! $(DOWNLOAD_DIR)/$(GNUTAR_TXZ).verified: $(DOWNLOAD_DIR)/$(GNUTAR_TXZ) \ +! $(DOWNLOAD_DIR)/$(GNUTAR_SIG) +! $(VERBOSE)$(SIGVERIFIER) $(DOWNLOAD_DIR)/$(GNUTAR_TXZ) \ +! $(DOWNLOAD_DIR)/$(GNUTAR_SIG) $(GNUTAR_KEY) +! $(VERBOSE)touch $@ + +Its almost the same as the previous shown port Makefile but adds rules for +verifying the archive signature. + + +Creating the build rule +======================= + +Build rules for Noux packages are located in _ports/src/noux-pkgs_. + +The _tar/target.mk_ corresponding to GNU tar looks like this: + +! NOUX_CONFIGURE_ARGS = --bindir=/bin \ +! --libexecdir=/libexec +! +! include $(REP_DIR)/mk/noux.mk + +The variable 'NOUX_CONFIGURE_ARGS' contains the options that are +passed on to Autoconf's configure script. The Noux specific build +rules in _noux.mk_ always have to be included last. + +The build rules for GNU tar are quite short and therefore at the end +of this chapter we take a look at a much more extensive example. + + +Creating a run script +===================== + +Creating a run script to test Noux packages is the same as it is +with running natively ported applications. Therefore we will only focus +on the Noux-specific parts of the run script and omit the rest. + +First, we add the desired Noux packages to 'build_components': + +! set noux_pkgs "bash coreutils tar" +! +! foreach pkg $noux_pkgs { +! lappend_if [expr ![file exists bin/$pkg]] build_components noux-pkg/$pkg } +! +! build $build_components + +Since each Noux package is, like every other Genode binary, installed to the +_/bin_ directory, we create a tar archive of each package from +each directory: + +! foreach pkg $noux_pkgs { +! exec tar cfv bin/$pkg.tar -h -C bin/$pkg . } + +_Using noux.mk makes sure that each package is always installed to_ +_/bin/._ + +Later on, we will use these tar archives to assemble the file system +hierarchy within Noux. + +Most applications ported to Noux want to read and write files. On that +matter, it is reasonable to provide a file-system service and the easiest +way to do this is to use the ram_fs server. This server provides a RAM-backed +file system, which is perfect for testing Noux applications. With +the help of the session label we can route multiple directories to the +file system in Noux: + +! append config { +! +! […] +! +! +! +! +! +! +! +! +! +! +! +! +! +! […] + +The file system Noux presents to the running applications is constructed +out of several stacked file systems. These file systems have to be +registered in the 'fstab' node in the configuration node of Noux: + +! +! +! +! } + +Each Noux package is added + +! foreach pkg $noux_pkgs { +! append config { +! " }} + +and the routes to the ram_fs file system are configured: + +! append config { +! +! +! +! +! +! +! +! +! } + +In this example we save the run script as _ports/run/noux_tar.run_. + + +Compiling the Noux package +========================== + +Now we can trigger the compilation of tar by executing + +! $ make VERBOSE= noux-pkg/tar + +_At least on the first compilation attempt, it is wise to unset_ 'VERBOSE' +_because it enables us to see the whole output of the_ 'configure' _process._ + +If configure is not able to find a particular header file the first place +to look is generally _libports/include/libc_. This directory is populated by +the rules in _libports/ports/libc.mk_. So if a header file is indeed +missing, adding the relevant header there is necessary. + +By now Genode provides almost all libc header files that are used by +typical POSIX programs. In most cases it is rather a matter of enabling +the right definitions and compilation flags. It might be worth to take a +look at FreeBSD's ports tree because Genode's libc is based upon the one +of FreeBSD 8.2.0 and if certain changes to the contributed code are needed, +they are normally already done in the ports tree. + +The script _noux_env.sh_ that is used to create the cross-compile +environment as well as the famous _config.log_ are found +in _/noux-pkg/_. + + +Running the Noux package +======================== + +We use the previously written run script to start the scenario, in which we +can execute and test the Noux package by issuing: + +! $ make run/noux_tar + +After the system has booted and Noux is running, we first create some test +files from within the running bash process: + +! bash-4.1$ mkdir /tmp/foo +! bash-4.1$ echo 'foobar' > /tmp/foo/bar + +Following this we try to create a ".tar" archive of the directory _/tmp/foo_ + +! bash-4.1$ cd /tmp +! bash-4.1$ tar cvf foo.tar foo/ +! tar: /tmp/foo: Cannot stat: Function not implemented +! tar: Exiting with failure status due to previous errors +! bash-4.1$ + +Well, this does not look too good but at least we have a useful error message +that leads (hopefully) us into the right direction. + + +Debugging an application that uses the Noux runtime +=================================================== + +Since the Noux service is basically the kernel part of our POSIX runtime +environment, we can ask Noux to show us the system calls executed by tar. +We change its configuration in the run script to trace all system calls: + +! […] +! +! +! […] + +We start the runscript again, create the test files and try to create a +".tar" archive. It still fails but now we have a trace of all system calls +and know at least what is going in Noux itself: + +! […] +! [init -> noux] PID 0 -> SYSCALL FORK +! [init -> noux] PID 0 -> SYSCALL WAIT4 +! [init -> noux] PID 5 -> SYSCALL STAT +! [init -> noux] PID 5 -> SYSCALL EXECVE +! [init -> noux] PID 5 -> SYSCALL STAT +! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY +! [init -> noux] PID 5 -> SYSCALL STAT +! [init -> noux] PID 5 -> SYSCALL OPEN +! [init -> noux] PID 5 -> SYSCALL FTRUNCATE +! [init -> noux] PID 5 -> SYSCALL FSTAT +! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY +! [init -> noux] PID 5 -> SYSCALL FCNTL +! [init -> noux] PID 5 -> SYSCALL WRITE +! [init -> noux -> /bin/tar] DUMMY fstatat(): fstatat called, not implemented +! [init -> noux] PID 5 -> SYSCALL FCNTL +! [init -> noux] PID 5 -> SYSCALL FCNTL +! [init -> noux] PID 5 -> SYSCALL WRITE +! [init -> noux] PID 5 -> SYSCALL FCNTL +! [init -> noux] PID 5 -> SYSCALL WRITE +! [init -> noux] PID 5 -> SYSCALL GETTIMEOFDAY +! [init -> noux] PID 5 -> SYSCALL CLOSE +! [init -> noux] PID 5 -> SYSCALL FCNTL +! [init -> noux] PID 5 -> SYSCALL WRITE +! [init -> noux] PID 5 -> SYSCALL CLOSE +! [init -> noux] child /bin/tar exited with exit value 2 +! […] + +_The trace log was shortened to only contain the important information._ + +We now see at which point something went wrong. To be honest, we see the +'DUMMY' message even without enabling the tracing of system calls. But +there are situations where a application is actually stuck in a (blocking) +system call and it is difficult to see in which. + +Anyhow, 'fstatat' is not properly implemented. At this point, we either have +to add this function to the Genode's libc or rather add it to libc_noux. +If we add it to the libc not only applications running in Noux will +benefit but all applications using the libc. Implementing it in +libc_noux is the preferred way if there are special circumstances because +we have to treat the function differently when used in Noux (e.g. 'fork'). + +For the sake of completeness here is a list of all files that were created by +porting GNU tar to Genode's Noux runtime: + +! ports/ports/tar.mk +! ports/run/noux_tar.run +! ports/src/noux-pkg/tar/target.mk + +Extensive build rules example +============================= + +The build rules for OpenSSH are much more extensive than the ones in +the previous example. Let us take a quick look at those build rules to +get a better understanding of possible challenges one may encounter while +porting a program to Noux: + +! # This prefix 'magic' is needed because OpenSSH uses $exec_prefix +! # while compiling (e.g. -DSSH_PATH) and in the end the $prefix and +! # $exec_prefix path differ. +! +! NOUX_CONFIGURE_ARGS += --disable-ip6 \ +! […] +! --exec-prefix= \ +! --bindir=/bin \ +! --sbindir=/bin \ +! --libexecdir=/bin + +In addition to the normal configure options we have to also define the +path prefixes. The OpenSSH build system embeds certain paths in the +ssh binary, which need to be changed for Noux. + +! NOUX_INSTALL_TARGET = install + +Normally the Noux build rules (_noux.mk_) execute 'make install-strip' to +explicitly install binaries that are stripped of their debug symbols. The +generated Makefile of OpenSSH does not use this target. It automatically +strips the binaries when executing 'make install'. Therefore, we set the +variable 'NOUX_INSTALL_TARGET' to override the default behaviour of the +Noux build rules. + +! LIBS += libcrypto libssl zlib libc_resolv + +As OpenSSH depends on several libraries we need to include these in the +build Makefile. These libraries are runtime dependencies and need to be +present when running OpenSSH in Noux. + +Sometimes it is needed to patch the original build system. One way to do +this is by applying a patch while preparing the source code. The other +way is to do it before building the Noux package: + +! noux_built.tag: Makefile Makefile_patch +! +! Makefile_patch: Makefile +! @# +! @# Our $(LDFLAGS) contain options which are usable by gcc(1) +! @# only. So instead of using ld(1) to link the binary, we have +! @# to use gcc(1). +! @# +! $(VERBOSE)sed -i 's|^LD=.*|LD=$(CC)|' Makefile +! @# +! @# We do not want to generate host-keys because we are crosscompiling +! @# and we can not run Genode binaries on the build system. +! @# +! $(VERBOSE)sed -i 's|^install:.*||' Makefile +! $(VERBOSE)sed -i 's|^install-nokeys:|install:|' Makefile +! @# +! @# The path of ssh(1) is hardcoded to $(bindir)/ssh which in our +! @# case is insufficient. +! @# +! $(VERBOSE)sed -i 's|^SSH_PROGRAM=.*|SSH_PROGRAM=/bin/ssh|' Makefile + +The target _noux_built.tag_ is a special target defined by the Noux build +rules. It will be used by the build rules when building the Noux package. +We add the 'Makefile_patch' target as a dependency to it. So after configure +is executed the generated Makefile will be patched. + +Autoconf's configure script checks if all requirements are fulfilled and +therefore, tests if all required libraries are installed on the host system. +This is done by linking a small test program against the particular library. +Since these libraries are only build-time dependencies, we fool the configure +script by providing dummy libraries: + +! # +! # Make the zlib linking test succeed +! # +! Makefile: dummy_libs +! +! NOUX_LDFLAGS += -L$(PWD) +! +! dummy_libs: libz.a libcrypto.a libssl.a +! +! libcrypto.a: +! $(VERBOSE)$(AR) -rc $@ +! libssl.a: +! $(VERBOSE)$(AR) -rc $@ +! libz.a: +! $(VERBOSE)$(AR) -rc $@ + +Porting devices drivers +####################### + +Even though Genode encourages writing native device drivers, this task sometimes +becomes infeasible. Especially if there is no documentation available for a +certain device or if there are not enough programming resources at hand to +implement a fully fledged driver. Examples of ported drivers can be found in +the 'dde_linux', 'dde_oss', and 'dde_ipxe' repositories. + +In this chapter we will exemplary discuss how to port a Linux driver for an ARM +based SoC to Genode. The goal is to execute driver code in user land directly on +Genode while making the driver believe it is running within the Linux kernel. +Traditionally there have been two approaches to reach this goal in Genode. In +the past, Genode provided a Linux environment, called 'dde_linux26', with the +purpose to offer just enough infrastructure to easily port drivers. However, +after adding more drivers it became clear that this repository grew extensively, +making it hard to maintain. Also updating the environment to support newer +Linux-kernel versions became a huge effort which let the repository to be neglected +over time. + +Therefore we choose the path to write a customized environment for each driver, +which provides a specially tailored infrastructure. We found that the +support code usually is not larger than a couple of thousand lines of code, +while upgrading to newer driver versions, as we did with the USB drivers, is +feasible. + + +Basic driver structure +====================== + +The first step in porting a driver is to identify the driver code that has to be +ported. Once the code is located, we usually create a new Genode repository and +write a Makefile that downloads and extracts the code to a directory called +_contrib_ (see 'dde_linux/Makefile') thus implementing the 'make prepare' +command for the repository. Having the source code ready, there are three main +tasks the environment must implement. The first is the driver back end, which is +responsible for raw device access using Genode primitives, the actual +environment that emulates Linux function calls the driver code is using, and the +front end, which exposes for example some Genode-session interface (like NIC or +block session) that client applications can connect to. + + +Further preparations +==================== + +Having the code ready, the next step is to create an _*.mk_ file that actually +compiles the code. For a driver library _lib/mk/.mk_ has to be +created and for a stand-alone program _src//target.mk_ is created +within the repository. With the _*.mk_ file in place, we can now start the +actual compilation. Of course this will cause a whole lot of errors and +warnings. Most of the messages will deal with implicit declarations of functions +and unknown data types. What we have to do now is to go through each warning and +error message and either add the header file containing the desired function or +data type to the list of files that will be extracted to the _contrib_ directory +or create our own prototype or data definition. + +When creating our own prototypes, we put them in a file called _lx_emul.h_. To +actually get this file included in all driver files we use the following code in +the _*.mk_ file: + +! CC_C_OPT += -include $(INC_DIR)/lx_emul.h + +where 'INC_DIR' points to the include path of _lx_emul.h_. + +The hard part is to decide which of the two ways to go for a specific function +or data type, since adding header files also adds more dependencies and often +more errors and warnings. As a rule of thumb, try adding as few headers as +possible. + +The compiler will also complain about a lot of missing header files. Since we do +not want to create all these header files, we use a trick in our _*.mk_ file that +extracts all header file includes from the driver code and creates symbolic +links that correspond to the file name and links to _lx_emul.h_. You can put the +following code snippet in your _*.mk_ file which does the trick: + +!# +!# Determine the header files included by the contrib code. For each +!# of these header files we create a symlink to _lx_emul.h_. +!# +!GEN_INCLUDES := $(shell grep -rh "^\#include .*\/" $(CONTRIB_DIR) |\ +! sed "s/^\#include [^<\"]*[<\"]\([^>\"]*\)[>\"].*/\1/" | \ +! sort | uniq) +! +!# +!# Filter out original Linux headers that exist in the contrib directory +!# +!NO_GEN_INCLUDES := $(shell cd $(CONTRIB_DIR); find -name "*.h" | sed "s/.\///" | \ +! sed "s/.*include\///") +!GEN_INCLUDES := $(filter-out $(NO_GEN_INCLUDES),$(GEN_INCLUDES)) +! +!# +!# Put Linux headers in 'GEN_INC' dir, since some include use "../../" paths use +!# three level include hierarchy +!# +!GEN_INC := $(shell pwd)/include/include/include +! +!$(shell mkdir -p $(GEN_INC)) +! +!GEN_INCLUDES := $(addprefix $(GEN_INC)/,$(GEN_INCLUDES)) +!INC_DIR += $(GEN_INC) +! +!# +!# Make sure to create the header symlinks prior building +!# +!$(SRC_C:.c=.o) $(SRC_CC:.cc=.o): $(GEN_INCLUDES) +! +!$(GEN_INCLUDES): +! $(VERBOSE)mkdir -p $(dir $@) +! $(VERBOSE)ln -s $(LX_INC_DIR)/lx_emul.h $@ + +Make sure 'LX_INC_DIR' is the directory containing the _lx_emul.h_ file. Note +that 'GEN_INC' is added to your 'INC_DIR' variable. + +The process of function definition and type declaration continues until the code +compiles. This process can be quite tiresome. When the driver code finally compiles, the +next stage is linking. This will of course lead to another whole set of errors +that complain about undefined references. To actually obtain a linked binary we +create a _dummies.cc_ file. To ease things up we suggest to create a macro called +'DUMMY' and implement functions as in the example below: + +! /* +! * Do not include 'lx_emul.h', since the implementation will most likely clash +! * with the prototype +! */ +! +!#define DUMMY(retval, name) \ +! DUMMY name(void) { \ +! PDBG( #name " called (from %p) not implemented", __builtin_return_address(0)); \ +! return retval; \ +!} +! +! DUMMY(-1, kmalloc) +! DUMMY(-1, memcpy) +! ... + +Create a 'DUMMY' for each undefined reference until the binary links. We now +have a linked binary with a dummy environment. + + +Debugging +========= + +From here on, we will actually start executing code, but before we do that, let us +have a look at the debugging options for device drivers. Since drivers have to +be tested on the target platform, there are not as many debugging options +available as for higher level applications, like running applications on the +Linux version of Genode while using GDB for debugging. Having these +restrictions, debugging is almost completely performed over the serial line and +on rare occasions with an hardware debugger using JTAG. + +For basic Linux driver debugging it is useful to implement the 'printk' +function (use 'dde_kit_printf' or something similar) first. This way, the driver +code can output something and additions for debugging can be made. The +'__builtin_return_address' function is also useful in order to determine where a +specific function was called from. 'printk' may become a problem with devices +that require certain time constrains because serial line output is very slow. This is +why we port most drivers by running them on top of the Fiasco.OC version of +Genode. There you can take advantage of Fiasco's debugger (JDB) and trace buffer +facility. + +The trace buffer can be used to log data and is much faster than 'printk' over +serial line. Please inspect the 'ktrace.h' file (at +_base-foc/contrib/l4/pkg/l4sys/include/ARCH-*/ktrace.h_) +that describes the complete interface. A very handy function there is + +!fiasco_tbuf_log_3val("My message", variable1, variable2, variable3); + +which stores a message and three variables in the trace buffer. The trace buffer +can be inspected from within JDB by pressing 'T'. + +JDB can be accessed at any time by pressing the 'ESC' key. It can be used to +inspect the state of all running threads and address spaces on the system. There +is no recent JDB documentation available, but + +:Fiasco kernel debugger manual: + + [http://os.inf.tu-dresden.de/fiasco/doc/jdb.pdf] + +should be a good starting point. It is also possible to enter the debugger at +any time from your program calling the 'enter_kdebug("My breakpoint")' function +from within your code. The complete JDB interface can be found in +_base-foc/contrib/l4/pkg/l4sys/include/ARCH-*/kdebug.h_. + +Note that the backtrace ('bt') command does not work out of the box on ARM +platforms. We have a small patch for that in our Fiasco.OC development branch +located at GitHub: [http://github.com/ssumpf/foc/tree/dev] + + +The back end +============ + +To ease up the porting of drivers and interfacing Genode from C code, Genode offers a +library called DDE kit. DDE kit provides access to common functions required +by drivers like device memory, virtual memory with physical-address lookup, +interrupt handling, timers, etc. Please inspect _os/include/dde_kit_ to see the +complete interface description. You can also use 'grep -r dde_kit_ *' to see +usage of the interface in Genode. + +As an example for using DDE kit we implement the 'kmalloc' call: + +!void *kmalloc(size_t size, gfp_t flags) +!{ +! return dde_kit_simple_malloc(size); +!} + +It is also possible to directly use Genode primitives from C++ files, the +functions only have to be declared as 'extern "C"' so they can be called from C +code. + + +The environment +=============== + +Having a dummy environment we may now begin to actually execute driver code. + +Driver initialization +~~~~~~~~~~~~~~~~~~~~~ + +Most Linux drivers will have an initialization routine to register itself within +the Linux kernel and do other initializations if necessary. In order to be +initialized, the driver will register a function using the 'module_init' call. +This registered function must be called before the driver is actually used. To +be able to call the registered function from Genode, we define the 'module_init' +macro in _lx_emul.h_ as follows: + +! #define module_init(fn) void module_##fn(void) { fn(); } + +when a driver now registers a function like + +! module_init(ehci_hcd_init); + +we would have to call + +! module_ehci_hcd_init(); + +during driver startup. Having implemented the above, it is now time to start our +ported driver on the target platform and check if the initialization function is +successful. Any important dummy functions that are called must be implemented +now. A dummy function that does not do device related things, like Linux book +keeping, may not be implemented. Sometimes Linux checks the return values of +functions we might not want to implement, in this case it is sufficient to simply +adjust the return value of the affected function. + +Device probing +~~~~~~~~~~~~~~ +Having the driver initialized, we will give the driver access to the device +resources. This is performed in two steps. In the case of ARM SoC's we have to +check in which state the boot loader (usually U-Boot) left the device. Sometimes +devices are already setup by the boot loader and only a simple device reset is +necessary to proceed. If the boot loader did not touch the device, we most +likely have to check and setup all the necessary clocks on the platform and may +have to perform other low level initializations like PHY setup. + +If the device is successfully (low level) initialized, we can hand it over to +the driver by calling the 'probe' function of the driver. For ARM platforms the +'probe' function takes a 'struct platform_device' as an argument and all +important fields, like device resources and interrupt numbers, should be set to +the correct values before calling 'probe'. During 'probe' the driver will most +likely map and access device memory, request interrupts, and reset the device. +All dummy functions that are related to these tasks should be implemented or +ported at this point. + +When 'probe' returns successful, you may either test other driver functions by +hand or start building the front-end. + + +The front-end +============= + +An important design question is how the front end is attached to the driver. In +some cases the front end may not use the driver directly, but other Linux +subsystems that are ported or emulated by the environment. For example, the USB +storage driver implements parts of the SCSI subsystem, which in turn is used +by the front end. The whole decision depends on the kind of driver that is +ported and on how much additional infrastructure is needed to actually make use +of the data. Again an USB example: For USB HID, we needed to port the USB controller +driver, the hub driver, the USB HID driver, and the generic HID driver in order +to retrieve keyboard and mouse events from the HID driver. + +The last step in porting a device driver is to make it accessible to other +Genode applications. Typically this is achieved by implementing one of Genode's +session interfaces, like a NIC session for network adapters or a block session for +block devices. You may also define your own session interfaces. The +implementation of the session interface will most likely trigger driver calls, +so you have to have to keep an eye on the dummy functions. Also make sure that calls to the +driver actually do what they are supposed to, for example, some wrong return value +of a dummy function may cause a function to return without performing any work. + + +Notes on synchronization +======================== + +After some experiences with Linux drivers and multi-threading, we lately +choose to have all Linux driver code executed by a single thread only. This way no Linux +synchronization primitives have to be implemented and we simply don't have to +worry about subtle pre- and postconditions of many functions (like "this +function has to be called with lock 'x' being held"). + +Unfortunately we cannot get rid of all threads within a device-driver server, +there is at least one waiting for interrupts and one for the entry point that +waits for client session requests. In order to synchronize these threads, we use +Genode's signalling framework. So when, for example, the IRQ thread receives an +interrupt it will send a signal. The Linux driver thread will at certain points +wait for these signals (e.g., functions like 'schedule_timeout' or +'wait_for_completion') and execute the right code depending on the kind of +signal delivered or firmly speaking the signal context. For this to work, we use +a class called 'Signal_dispatcher' (_base/include/base/signal.h_) which inherits +from 'Signal_context'. More than one dispatcher can be bound to a signal +receiver, while each dispatcher might do different work, like calling the +Linux interrupt handler in the IRQ example. + +