The build/real prefix handling using sed breaks if build != real and the
standard include / lib directories are used ($prefix/include and $prefix/lib).
E.G.
prefix_build="/usr", libdir="$prefix/lib", includedir="$prefix/include".
If this gets installed with make DESTDIR="/foo" install, then we end up with
prefix_real = prefix = "/foo/usr" as expected, but
includedir="/foo/foo/usr/include" and libdir="/foo/foo/usr/lib" because of
the double sed invocation (prefix is already expanded). Work around it by
ensuring we only match the beginning of the string.
Submitted upstream: http://bugs.python.org/issue22907
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As investigated in bug #7646, wcsftime() doesn't work properly with
uClibc. Until it gets fixed in uClibc, let's tell Python 3 to not use
it. Python 3 will fall back to strftime(), which works properly.
[Peter: fix typo in comment]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Beyonlo <beyonlo@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since the removal of py/pyc files is Python-specific, this commit
moves the logic removing those files to python.mk and python3.mk
respectively.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The __pycache__ directories are only needed on PEP3147-enabled
installations, which is not the choice Buildroot has made. However,
during the build process of the target Python, the separation between
the target and host Python is not entirely complete, and starting the
host Python to byte-compile the target Python modules triggers the
compilation of a bunch of __pycache__ directories by the host Python,
which uses a PEP 3147 installation.
For now, simply get rid of those useless directories after the
installation of Python is completed. In the long term, it would be
good to achieve a better isolation between the host and target build
to avoid this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python 3 has a new standard for installing .pyc file, called PEP
3147. Unfortunately, this standard requires both the .py and .pyc
files to be installed for a Python module to be found. This is quite
annoying on space-constrained embedded systems, since the .py file is
technically not required for execution.
For the Python standard library, our Python 3 package already contains
a patch named python3-004-old-stdlib-cache.patch, which allows to
disable the PEP 3147 installation.
But that leaves the distutils/setuptools package an unsolved
problem. This patch therefore adds a new patch to Python, which makes
distutils package use the traditional installation path when byte
compiling, rather than the PEP 3147 installation path. Since
setuptools relies on distutils internally, it also fixes setuptools
based packages.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
To be consistent with the recent change of FOO_MAKE_OPT into FOO_MAKE_OPTS,
make the same change for FOO_CONF_OPT.
Sed command used:
find * -type f | xargs sed -i 's#_CONF_OPT\>#&S#g'
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The copy_scripts() method in distutils copies the scripts listed
in the setup file and adjusts the first line to refer to the
current Python interpreter. When cross-compiling, this means that
the adjusted shebang refers to the host Python interpreter.
As a consequence, we add a patch for python and python3 that
force copy_scripts() to preserve the shebang when
cross-compilation is detected.
Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
libffi depends on pthreads.
Python depends on libffi (it can provide a builtin libffi, but also depends on
pthreads). Thus this patch also disable Python support if toolchain is compiled
w/o treads support.
Fixes http://sysmic.org/~jezz/results/204099dd:
../src/closures.c:119:21: fatal error: pthread.h: No such file or directory
[Peter: fix comment dependencies, add python3]
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit makes the decimal module support optional: since Python
3.3 it relies on an external library. The source code of this library
is available built-in as part of the Python sources, but in Buildroot
we generally prefer to use the external library when possible.
To achieve this, this commit adds a patch to Python that is similar to
the one we use for expat support, but this time for the
libmpdec/mpdecimal library.
As a consequence, since mpdecimal now builds properly even when
<fenv.h> is not available (on i386), this commit fixes:
http://autobuild.buildroot.org/results/b64/b64d5c941a7cac00619da3a0696939f86a8eafc2/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This patch adds support for time.tzset() in Python 2.7 and Python 3.
The Python configure script sets ac_cv_working_tzset to 'no' when
cross compiling, although it should be available and work with all
toolchains supported by Buildroot (?).
The patch below pre-sets the variable so that time.tzset() works.
[Thomas: adjust commit log.]
Signed-off-by: Henry Margies <henry.margies@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit mainly fixes bug #7268 by improving the isolation of
the build environment by making sure some host environment header
paths do not leak into the target build. The investigation was done by
David <buildroot-2014@inbox.com>. This is done by not calling the
add_multiarch_paths() function of setup.py when we're cross-compiling,
a change made in the newly introduced
python3-012-dont-add-multiarch-path.patch.
In addition to this, another patch is added to make sure the build is
aborted when one of the Python module fails to build. This is done in
python3-013-abort-on-failed-modules.patch. Without this, the Python
setup.py script simply logs which module failed to build, but doesn't
abort, so it's hard to notice when there is a problem.
[Peter: slightly reword commit message]
Cc: David <buildroot-2014@inbox.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A few minor updates on the patches were needed to resolve some
conflicts on the main Makefile.pre.in file, in the list of modules to
build/install.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The cross-compilation improvements integrated in Python rely on the
compiler exposing a line starting with LIBRARY_PATH when called with
-E -v. This is used by Python setup.py to find the installation
locations of libraries.
However, this LIBRARY_PATH line is not shown by very old compilers,
such as the gcc 4.2.x compiler used on the AVR32 architecture. This
causes libraries installed in the sysroot, such as libffi, to not be
detected by the setup.py script.
To fix this problem, this patch adds addtional logic to setup.py,
which consists in deriving the library paths from the sysroot
location, if no LIBRARY_PATH field was found.
Fixes:
http://autobuild.buildroot.org/results/7a6/7a65e381cc04bf8f74fd63a6dcda502f3c26aeef/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 6251ccf032 ('python3: fix the
value of PYTHON3_PATH') tried to fix the value of PYTHON3_PATH, but
did it incorrectly: it changed PYTHON_VERSION_MAJOR to
PYTHON_VERSION3_MAJOR, while it should have been
PYTHON3_VERSION_MAJOR.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The PYTHON3_PATH was incorrectly referencing the site-packages of
Python 2 packages, due to the usage of PYTHON_VERSION_MAJOR, instead
of PYTHON3_VERSION_MAJOR. This commit fixes that by using the correct
variable, which in our testing fixed the build of python-pyasn against
python3.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This patch reworks the way python3 and python3-config symlink are
installed.
Buildroot wants to control these symlinks' installation:
* the python3 symlink should be unconditionally installed in the target
tree, and the python3-config symlink in the staging tree, since it is
the only python package built and installed in the target tree if the
user selected it;
* the python3 and python3-config symlinks should only be installed in
the host tree when python3 is the selection of the user for the
target.
[Thomas: fix comment as suggested by Yann.]
Signed-off-by: Samuel Martin <s.martin49@gmail.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During installation, host-python and host-python3 run the freshly built
python executable. This is done with a proper LD_LIBRARY_PATH to make
sure it picks up the libpython in the build directory. However, the
python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
Therefore, if libpython exists there, it will be used instead.
If the install step is run in parallel, it is possible that libpython
is already partially copied to $(HOST_DIR)/usr/lib when python is run.
This gives an error like:
python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short
The fix is simple: use RUNPATH instead of RPATH, which allows
LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
build directory is always used. RUNPATH is enabled by passing
--enable-new-dtags to the linker.
Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A confusion was made when applying commit
c3d539b53f ('host python/python3:
conditionally disable unicodedata') and then commit
73293e88c8 ('python, python3: enable
unicodedata for host-python, needed by setuptools').
The first commit added the enabling of unicodedata in the host Python
when the target Python is configured with unicodedata, which is
necessary because building the target unicodedata Python module
requires having unicodedata support in the host Python.
The second commit enabled the unicodedata unconditionally in the host
Python, because it is needed by host-setuptools.
However, when the second commit was applied, the logic that consisted
in disabling unicodedate in the host Python when not needed was kept,
which resulted in breaking the build of host-setuptools due to the
lack of unicodedata support in the host python.
This commit fixes that by only leaving the unconditional
--enable-unicodedata in the host Python configuration options, and
therefore removing the problematic conditional --disable-unicodedata.
Fixes:
http://autobuild.buildroot.org/results/b62/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The python and python3 builds mark libpython as read-only which
prevents it from being stripped out correctly for the target.
Signed-off-by: Przemyslaw Wrzos <przemyslaw.wrzos@calyptech.com>
Acked-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Tested-by: Thomas De Schampheleire <thomas.de_schampheleire@alcatel-lucent.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
As we are going to bump setuptools to a much newer version, the host
python needs to be built with support for unicodedata.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The Python package infrastructure will need the Python 3 package to
provide a PYTHON3_PATH environment variable in order to build
third-party Python modules.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit bumps the Python3 package to use Python 3.4.0rc1.
About the patches:
* The patches below 100 are significantly changed, because like for
Python 2.x, a good number of improvements have been made in the
upstream Python for cross-compilation. Therefore, almost all of
these patches have been modified.
* All the patches above 100 are simply updated for Python 3.4.0, with
a small refactoring for the handling of test modules.
The details of the python3.mk changes are:
* --without-ensurepip to tell Python to not use PIP at build time.
* Many environment variables are no longer passed, they were specific
to our cross-compilation patches
* The fixup of the LIBDIR in the Python Makefile is no longer needed
since Python has switched to _sysconfigdata.py for distutils
configuration instead of parsing the Makefile.
* A new post patch hooks touches the two files generated by pgen to
make sure they are newer than the pgen sources, which ensures pgen
is not built/executed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some parts of python3.mk were hardcoding the 3.3 version as the major
version, which does not work for Python 3.4 and other future
versions. Instead, use the existing PYTHON3_VERSION_MAJOR.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The host python always had --disable-unicodedata, regardless of the
corresponding configuration option BR2_PACKAGE_PYTHON_UNICODEDATA.
Since the host python is used to byte-compile python modules, this meant
that such modules could not contain unicode strings. For example, following
statement in a python module:
print u"\N{SOLIDUS}"
would cause the byte-compilation to fail with message:
SyntaxError: ("(unicode error) \\N escapes not supported (can't load
unicodedata module)",
Instead, conditionally disable unicodedata based on
BR2_PACKAGE_PYTHON_UNICODEDATA, also for the host python.
This fixes bug #6542 (https://bugs.busybox.net/show_bug.cgi?id=6542)
Reported-by: Gernot Vormayr <gvormayr@gmail.com>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The target python3 depends on host-python3, but most of the scripts
call "python", so we need to ensure that $(HOST_DIR)/usr/bin/python
exists. This patch achieves this by creating a python -> python3
symbolic link in $(HOST_DIR), just like we are already doing for the
target Python 3.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In Buildroot, we do not support installing both Python 2.x and Python
3.x on the target.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a package A depends on config option B and toolchain option C, then
the comment that is given when C is not fulfilled should also depend on B.
For example:
config BR2_PACKAGE_A
depends on BR2_B
depends on BR2_LARGEFILE
depends on BR2_WCHAR
comment "A needs a toolchain w/ largefile, wchar"
depends on !BR2_LARGEFILE || !BR2_WCHAR
This comment should actually be:
comment "A needs a toolchain w/ largefile, wchar"
depends on BR2_B
depends on !BR2_LARGEFILE || !BR2_WCHAR
or if possible (typically when B is a package config option declared in that
same Config.in file):
if BR2_B
comment "A needs a toolchain w/ largefile, wchar"
depends on !BR2_LARGEFILE || !BR2_WCHAR
[other config options depending on B]
endif
Otherwise, the comment would be visible even though the other dependencies
are not met.
This patch adds such missing dependencies, and changes existing such
dependencies from
depends on BR2_BASE_DEP && !BR2_TOOLCHAIN_USES_GLIBC
to
depends on BR2_BASE_DEP
depends on !BR2_TOOLCHAIN_USES_GLIBC
so that (positive) base dependencies are separate from the (negative)
toolchain dependencies. This strategy makes it easier to write such comments
(because one can simply copy the base dependency from the actual package
config option), but also avoids complex and long boolean expressions.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(untested)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch lines up the comments in Config.in files that clarify which
toolchain options the package depends on.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since 97c687000 (pkg-autotools.mk: default host AUTORECONF{,_OPT} to the
target values) we automatically enable autoreconf for host builds if it
is enabled for the target, so these can go.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
[Peter: leave change xz tarball format to not end up with circular deps]
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
This finally removes the BR2_HAVE_DEVFILES option, that was used to
install/keep development files on target. With the recent migration of
the internal backend to the package infrastructure, we had anyway lost
the ability to build gcc for the target, and install the uClibc
development files on the target.
[Peter: also remove support/scripts/copy.sh]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Some of the python2 patches were left behind when doing the python3
package. This was because the python build system can now autodetect
what packages can be built in the system.
However, some of these patches are actually useful to reduce drastically
the size of the interpreter.
This patch ports the relevant patches to the python3 package, and adds a
new patch to remove the idle3 IDE as well from the interpreter.
Fixes#5696
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Python3 changes the pyc lookup strategy, ignoring the
__pycache__ directory if the .py file is missing. Change
install location to enable use of .pyc without their parent .py
See http://www.python.org/dev/peps/pep-3147
[Peter: add patch header]
Signed-off-by: Daniel Nelson <daniel@sigpwr.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout at mind.be>
(untested)
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Fixes#5690
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
Fixes#5702
Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
