Let user choose how many jobs the build will use #1558
Labels
Comments
6 tasks
opacam
added a commit
to opacam/python-for-android
that referenced
this issue
Dec 27, 2018
This should be implemented properly as described in kivy#1558, but for now we will avoid issues
KeyWeeUsr
pushed a commit
that referenced
this issue
Jan 9, 2019
…1537) * Refactor hostpython3's recipe into a new base class HostPythonRecipe To make it use as a base class for hostpython2 and hostpython3 recipes in a near future. * Refactor python3's recipe into a new base class GuestPythonRecipe To make it use as a base class for python2 and python3 recipes in a near future. * Move hostpython2 and python2 recipes to a new name hostpython2legacy and python2legacy. This recipes will be useless until properly fixed or maybe it will be removed... * Add the new hostpython2/python2 recipes (v2.7.15) Those new recipes will use the same build method than python3. Also added a set of patches, some of them are enabled by default because they are necessary to perform a successful build. The others are added because maybe we will need them. * Adapt sdl2's stuff to the new python2's build system * Adapt pythonforandroid's core files to the new python2's build system Cause we share the same build system for python2 and python3 recipes, we can safely remove some specific python2's code in favour of the python3's code, otherwise we will surely find troubles in our builds. * Adapt python2's test apps to the new python2's build system The python2's test with openssl and sqlite it will probably success, but without the libs until those are enabled. The pygame's test app will fail until properly fixed. * Enable sqlite support for python recipes * Add ability to compile recipes with clang The android ndk has been migrated his default compiler from gcc to clang and soon will be removed from the ndk. Here we prepare p4a to compile recipes with clang, which will allow us to successfully compile an updated openssl libs. Without this openssl upgrade we will not be able to grant support to python3's _ssl.so module. Note: The default p4a compiler still is gcc...so...no changes on the compile behaviour unless explicitly requested References: #1478 * Update OpenSSL to 1.1.1 (uses clang for compilation) This openssl version is an LTS, supported until 11th September 2023. Important: There is now a distinction between the version we use for downloading, and the version used for creating the library: OpenSSL 1.1 now have prefix with "1.1.so". This explains why others recipes that depends on it require the "library" version. Those recipes, still not are modified to support the new version. References: #1478 * Enhance openssl recipe and remove lib_version Several actions done: - The openssl recipe has been enhanced by introducing a couple of methods (include_flags and link_flags) which should help us to link other recipes with the openssl libs. - Remove lib_version introduced in the last openssl version update, so we don't have to modify all dependant recipes - Add a new variable `url_version`, used to download our recipe in the subclassed method versioned_url (this replaces the removed lib_version) - Add D__ANDROID_API__ to the configure command (needed to successfully build the libraries and link them with python) - Add documentation * Enable openssl support for python recipes This commit grants openssl support for both python versions (2 and 3). The python2 recipe needs some extra work to make it link with python, we need to patch the setup.py file in order to that the python's build system recognises the libraries. This is not the case for the python3 recipe which works like a charm, except for the fact that we have to modify our static class variable ` config_args ` for the python's recipe, i found this to be necessary in my tests, otherwise the openssl libs aren't detected. * Fix hardcoded python2 entry for Arch * Fix hardcoded python2 entries for BootstrapNDKRecipes * Fix duplicated key in options for python2 test * Fix numpy's recipe and adds python3's compatibility. Also: - add numpy to CI tests for python3: this module is some kind of special and probably a dependency for a lot of python packages so better make sure that we not break anything...as we already do with python2 - Add numpy to python2 and python3 test apps, cause already have some buttons that allows to test it from the running app. * Add python3 test for sqlite and openssl and add python3's compatibility to dependant recipes To be able to run successfully the new test we must add python3 compatibility to some recipes that is why we touch them here. * Make that travis test python3 with sqlite3 and openssl * Update android ndk to r17c for docker images * Add one more dependency to our docker images Or the ci tests where libffi is involved will fail. * Update documentation about the new python core * Prevent crash on create_python_bundle When building an apk for service_only's bootstrap the build will crash if we don't add some pure python packages, because the directory "python-install/<distribution directory>" will not be ever created, so...this will make p4a crash when we try to enter into this directory. * Replace exit for BuildInterruptingException ¡¡¡Thanks @AndreMiras!!! * Remove note of quickstart.rst ¡¡¡Thanks @inclement!!! * Redaction corrections ¡¡¡Thanks @inclement!!! * Remove tuple of python versions Because all 3 are automatically added by PythonRecipe. ¡¡¡Thanks @inclement!!! * Replace backslashes To be more readable and pythonic Thanks @KeyWeeUsr!!! * Fix hardcoded toolchain version and arch for python recipe * Fix hardcoded toolchain version and arch for clang * Fix android host for python recipes To allow us to build python 2 or 3 for any arch supported for p4a * Fix hardcoded target for python recipes and clang * Add missing includes for clang Or we will get errors complaining about missing includes related with asm * Fix hardcoded arch for numpy * Fix openssl build for arch x86_64 * Fix openssl build for arch arm64-v8a * Drop commented code from python2legacy recipe * Replace backslashes for openssl recipe To enhance the readability of the sourcecode ¡¡¡Thanks KeyWeeUsr!!! * Fix hardcoded arch flags for libffi recipe To fix libffi build for any arch * Remove unneeded warning for numpy * Fix libffi build for all archs Adds some flags lost (removed when were fixed the hardcoded arch entries) * Remove lines from openssl recipe that seems not needed anymore The removed code was intended to check if the library were build with the right symbols, and if not, clean the build and trigger a rebuild with a pause of 3 seconds (to show a message warning about the rebuild). It seems that new version of the openssl libs doesn't need this anymore because the symbols checked with the removed source code seems to be build without problems. So we remove those lines in order the clean up a little the code and to be reintroduced again if needed (but properly documented) References: #1537 * Replace sum with append for cflags list * Remove unneeded message for build.py ¡¡¡Thanks @inclement!!! * Remove commented line ¡¡¡Thanks @inclement!!! * Remove unneeded files for the project ¡¡¡Thanks @inclement!!! * Remove some unneeded lines for some of the test apps ¡¡¡Thanks @AndreMiras!!! * Remove more unneeded lines for test app and shortens line * Fix initsite for python2's site module * Enhance python2 patches - Move all locale related code from fix-locale.patch to fix-api-minor-than-21.patch, because for api >= 21 localeconv is properly defined in android - Remove fix-locale.patch and the remaining code moved into new patches (and enhanced thanks to some cpython's patches I found in bug tracker), to be neatest - Enhance definitions of fix-api-minor-than-21.patch to be more suitable for a potential upstream merge into cpython source (¡¡¡Thanks @Jonast!!!) References: https://bugs.python.org/issue20306 and https://bugs.python.org/issue26863, * Remove unused patches * Some fixes/enhancements for python2legacy In case we decide to make it work and maintain it ¡¡¡Thanks @AndreMiras!!! Note: At this point the python2legacy recipe will not work, it would require some more work to make it successfully build: - Fix conflicts between python recipes - Add python2legacy compatibility to some basic recipes (For sdl2, at least we need: android, pyjnius, sdl2, setuptools and six) - Add python2legacy compatibility to sdl2 bootstrap (at least) - Add python2legacy compatibility to some core files (bootstrap.py, build.py and recipe.py) * Change ndk downloads page To be safe for the foreseeable future ¡¡¡Thanks @inclement!!! * Fix clang path for crystax * Python2legacy: fix conflicts between python recipes * Python2legacy: provisional fix to the hardcoded job count This should be implemented properly as described in #1558, but for now we will avoid issues * Python2legacy: make work some basic recipes with python2legacy To make it work the python2legacy recipe with the sdl2's bootstrap * Python2legacy: make work sdl2 bootstrap with python2legacy * Python2legacy: fix get_site_packages_dir for python2legacy * Python2legacy: fix strip_libraries for python2legacy * Python2legacy: make the necessary changes to recipe.py to make it work the python2legacy recipe Note: With this commit, we will be able to successfully build using the python2legacy recipe, but be aware that you may need to add python2legacy compatibility to some recipes to make it work with your app, because only some minimal set of recipes has been touched (the ones needed to make work the python2legacy with the sdl2 bootstrap and kivy) * Python2legacy: make it work the openssl libs with python2legacy and python3crystax Also this should fix the the openssl build issues reported when using crystax. Notice that this changes will leave us with two versions of openssl libs. We need the most updated one for our python2 and python3 recipes, and the legacy version for python2legacy and python3crystax. The python2legacy version is too old to support the openssl version 1.1+ and python3crystax is not building fine with the new openssl libs so to fix the crystax issues we force to use the legacy version, which previously has been proved to work. * Python2legacy: add a test app for python2legacy (with openssl and sqlite3) To make easier for us to test if python2legacy is working * Python2legacy: update README file To inform about the python2legacy recipe * Move libraries from LDFLAGS to LIBS Because this is how you are supposed to do it, you must use LDFLAGS for linker flags and LDLIBS (or the equivalent LOADLIBES) for the libraries * Fix linkage problems with python's versioned library The Java method called to load the libraries only supports a .so suffix, so we must remove the version for our python libraries, or we will get linkage errors Resolves: #1501 * Revert "Fix linkage problems with python's versioned library" This reverts commit 99ee28b
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently there are some recipes with hardcoded job count which will probably cause issues if the build is run on a toaster-like machines (such as one of my old machines' N270 + 1GB RAM). While the solution with
multiprocessing.cpu_countis nice, it'd be much better if we had an option for the user to specify the jobs and then fallback tocpu_countvalue by default. Otherwise if I run it like this, my session resources' might be just hijacked by p4a and on that particular CPU it's really bad if I want e.g. to run e.g. latest firefox with multiple tabs while building some app.https://github.com/kivy/python-for-android/search?q=-j4&unscoped_q=-j4&type=Code
https://github.com/kivy/python-for-android/search?q=-j5&unscoped_q=-j5&type=Code
The text was updated successfully, but these errors were encountered: