Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

outdated recipes #2277

Closed
obfusk opened this issue Aug 3, 2020 · 11 comments
Closed

outdated recipes #2277

obfusk opened this issue Aug 3, 2020 · 11 comments

Comments

@obfusk
Copy link
Contributor

obfusk commented Aug 3, 2020

Because many of the recipes I use have outdated versions (which may also have security implications), I'm currently building the following recipes with a patch to use the latest version (as of a week or so ago):
flask hostpython3 libffi openssl pyjnius python3 setuptools six sqlite3.

So far everything works for me without any problems (other than #2264).

I'd be happy to make PRs for those, but the latest versions may break for some users (and build environments).
What's the best/preferred way to handle this?
@AndreMiras
Copy link
Member

Yes please go-ahead with the update. The best practice is for users to pin recipe version in their buildozer.spec if they want to be sure it doesn't get updated 😄

@obfusk obfusk changed the title oudated recipes outdated recipes Aug 4, 2020
@obfusk
Copy link
Contributor Author

obfusk commented Aug 4, 2020

The best practice is for users to pin recipe version in their buildozer.spec if they want to be sure it doesn't get updated

I don't recall seeing an option to pin recipes. How does one do that?

@AndreMiras
Copy link
Member

The same way you pip install a pinned package. So basically users should pin versions in their requirements list if they want to, e.g. https://github.com/AndreMiras/EtherollApp/blob/v2020.0322/buildozer.spec#L40

@obfusk
Copy link
Contributor Author

obfusk commented Aug 5, 2020

The same way you pip install a pinned package.

Ah. I mistakenly assumed that only worked for pip dependencies, not for p4a recipes. Good to know!

@obfusk
Copy link
Contributor Author

obfusk commented Aug 11, 2020

I've started working on a script to check for updates in e.g. p4a recipes.
It currently outputs this:

$ cd python-for-android/pythonforandroid/recipes
$ checkupdates p4a-recipes
[USES HTTP] cdecimal
[USES HTTP] freetype
[USES HTTP] harfbuzz
[USES HTTP] libmysqlclient
[USES HTTP] libogg
[USES HTTP] libpq
[USES HTTP] libvorbis
[USES HTTP] libx264
[USES HTTP] libxml2
[USES HTTP] libxslt
[USES HTTP] zope
[CHECK FAILED] cdecimal
[CHECK FAILED] ethash
[CHECK FAILED] ffpyplayer_codecs
[CHECK FAILED] fontconfig
[CHECK FAILED] freetype
[CHECK FAILED] harfbuzz
[CHECK FAILED] icu
[CHECK FAILED] ifaddrs
[CHECK FAILED] libbz2
[CHECK FAILED] libcurl
[CHECK FAILED] libglob
[CHECK FAILED] libiconv
[CHECK FAILED] liblzma
[CHECK FAILED] libmysqlclient
[CHECK FAILED] libogg
[CHECK FAILED] libpq
[CHECK FAILED] librt
[CHECK FAILED] libsecp256k1
[CHECK FAILED] libtribler
[CHECK FAILED] libvorbis
[CHECK FAILED] libx264
[CHECK FAILED] matplotlib
[CHECK FAILED] netifaces
[CHECK FAILED] openal
[CHECK FAILED] preppy
[CHECK FAILED] pyicu
[CHECK FAILED] pyleveldb
[CHECK FAILED] pyogg
[CHECK FAILED] pyopenal
[CHECK FAILED] pyrxp
[CHECK FAILED] pysdl2
[CHECK FAILED] reportlab
[CHECK FAILED] scrypt
[CHECK FAILED] sdl2_image
[CHECK FAILED] sdl2_mixer
[CHECK FAILED] sdl2_ttf
[CHECK FAILED] storm
[CHECK FAILED] vlc
[POSSIBLY OUTDATED] Pillow = 7.0.0 -> 7.2.0
[POSSIBLY OUTDATED] apsw = 3.15.0-r1 -> 3.32.2-r1
[POSSIBLY OUTDATED] atom = 0.3.10 -> 0.4.1
[POSSIBLY OUTDATED] babel = 2.2.0 -> 2.8.0
[POSSIBLY OUTDATED] bcrypt = 3.1.7 -> 1.1.1
[POSSIBLY OUTDATED] boost = 1.69.0 -> 1.73.0
[POSSIBLY OUTDATED] cffi = 1.13.2 -> 1.14.1
[POSSIBLY OUTDATED] coverage = 4.1 -> 5.2.1
[POSSIBLY OUTDATED] cppy = 4e0b956 -> 1.1.0
[POSSIBLY OUTDATED] cryptography = 2.8 -> 3.0
[POSSIBLY OUTDATED] cython = 0.29.15 -> 0.29.21
[POSSIBLY OUTDATED] dateutil = 2.6.0 -> 2.8.1
[POSSIBLY OUTDATED] decorator = 4.2.1 -> 4.4.2
[POSSIBLY OUTDATED] enaml = 0.9.8 -> 0.10.2
[POSSIBLY OUTDATED] evdev = 0.4.7 -> 0.7.0
[POSSIBLY OUTDATED] feedparser = 5.2.1 -> 5.1.3
[POSSIBLY OUTDATED] ffmpeg = 007e03348dbd8d3de3eb09022d72c734a8608144 -> n2.8.5
[POSSIBLY OUTDATED] ffpyplayer = c99913f2317bf3840eeacf1c1c3db3b3d1f78007 -> 4.3.1
[POSSIBLY OUTDATED] gevent = 1.4.0 -> 20.6.2
[POSSIBLY OUTDATED] greenlet = 0.4.15 -> 0.4.16
[POSSIBLY OUTDATED] groestlcoin_hash = 1.0.1 -> 1.0.3
[POSSIBLY OUTDATED] jedi = 0.9.0 -> 0.17.2
[POSSIBLY OUTDATED] jpeg = 2.0.1 -> 2.0.5
[POSSIBLY OUTDATED] kiwisolver = 0846189 -> 1.2.0
[POSSIBLY OUTDATED] libexpat = master -> R_2_2_9
[POSSIBLY OUTDATED] libgeos = 3.7.1 -> 3.8.1
[POSSIBLY OUTDATED] libshine = c72aba9031bde18a0995e7c01c9b53f2e08a0e46 -> 3.1.1
[POSSIBLY OUTDATED] libsodium = 1.0.16 -> 1.0.18-RELEASE
[POSSIBLY OUTDATED] libtorrent = 1_2_1 -> 1.2.8
[POSSIBLY OUTDATED] libxml2 = 2.9.8 -> 2.9.10
[POSSIBLY OUTDATED] libxslt = 1.1.32 -> 1.1.34
[POSSIBLY OUTDATED] lxml = 4.2.5 -> 4.5.2
[POSSIBLY OUTDATED] m2crypto = 0.30.1 -> 0.36.0
[POSSIBLY OUTDATED] msgpack-python = 0.4.7 -> 0.5.6
[POSSIBLY OUTDATED] numpy = 1.18.1 -> 1.19.1
[POSSIBLY OUTDATED] omemo = 0.11.0 -> 0.12.0
[POSSIBLY OUTDATED] omemo-backend-signal = 0.2.5 -> 0.2.6
[POSSIBLY OUTDATED] opencv = 4.0.1 -> 4.4.0
[POSSIBLY OUTDATED] opencv_extras = 4.0.1 -> 4.4.0
[POSSIBLY OUTDATED] pandas = 1.0.3 -> 1.1.0
[POSSIBLY OUTDATED] png = 1.6.37 -> 1.6.35
[POSSIBLY OUTDATED] protobuf_cpp = 3.6.1 -> 3.12.4
[POSSIBLY OUTDATED] pyaml = 15.8.2 -> 20.4.0
[POSSIBLY OUTDATED] pycparser = 2.14 -> 2.20
[POSSIBLY OUTDATED] pycrypto = 2.7a1 -> 2.6.1
[POSSIBLY OUTDATED] pycryptodome = 3.6.3 -> 3.9.8
[POSSIBLY OUTDATED] pygame = 2.0.0-dev7 -> 1.9.6
[POSSIBLY OUTDATED] pymunk = 5.5.0 -> 5.6.0
[POSSIBLY OUTDATED] pynacl = 1.3.0 -> 1.4.0
[POSSIBLY OUTDATED] pyopenssl = 19.0.0 -> 19.1.0
[POSSIBLY OUTDATED] pyproj = 1.9.6 -> 2.6.1rel
[POSSIBLY OUTDATED] pytz = 2019.3 -> 2020.1
[POSSIBLY OUTDATED] pyusb = 1.0.0b1 -> 1.0.2
[POSSIBLY OUTDATED] pyzbar = 0.1.7 -> 0.1.8
[POSSIBLY OUTDATED] pyzmq = master -> 14.1.1
[POSSIBLY OUTDATED] regex = 2019.06.08 -> 2020.7.14
[POSSIBLY OUTDATED] ruamel.yaml = 0.15.77 -> 0.16.10
[POSSIBLY OUTDATED] sdl2 = 2.0.9 (f2ecfba915c54f7200f504d8b48a5dfe) -> 2.0.12
[POSSIBLY OUTDATED] setuptools = 49.2.1 -> 49.3.1
[POSSIBLY OUTDATED] shapely = 1.7a1 -> 1.7.0
[POSSIBLY OUTDATED] snappy = 1.1.7 -> 1.1.8
[POSSIBLY OUTDATED] sqlalchemy = 1.3.3 -> 1.3.18
[POSSIBLY OUTDATED] sympy = 1.1.1 -> 1.6.2
[POSSIBLY OUTDATED] ujson = 1.35 -> 3.1.0
[POSSIBLY OUTDATED] vispy = 0.4.0 -> 0.6.4
[POSSIBLY OUTDATED] websocket-client = 0.40.0 -> 0.57.0
[POSSIBLY OUTDATED] x3dh = 0.5.3 -> 0.5.9
[POSSIBLY OUTDATED] xeddsa = 0.4.4 -> 0.6.0
[POSSIBLY OUTDATED] zbarlight = 2.1 -> 3.0
[POSSIBLY OUTDATED] zeroconf = 0.24.5 -> 0.28.0
[POSSIBLY OUTDATED] zope = 4.1.3 -> 5.1.0
[POSSIBLY OUTDATED] zope_interface = 4.1.3 -> 5.1.0
24 ok, 72 outdated, 38 failed

@obfusk
Copy link
Contributor Author

obfusk commented Jan 2, 2021

I'd be happy to make some PRs for newer versions.
But maybe not everything 😅.
Anything considered important that I should prioritise?

Currently outputs:

[USES HTTP] cdecimal
[USES HTTP] freetype
[USES HTTP] harfbuzz
[USES HTTP] libmysqlclient
[USES HTTP] libogg
[USES HTTP] libpq
[USES HTTP] libvorbis
[USES HTTP] libx264
[USES HTTP] libxml2
[USES HTTP] libxslt
[CHECK FAILED] audiostream
[CHECK FAILED] cdecimal
[CHECK FAILED] ethash
[CHECK FAILED] ffpyplayer_codecs
[CHECK FAILED] fontconfig
[CHECK FAILED] freetype
[CHECK FAILED] harfbuzz
[CHECK FAILED] icu
[CHECK FAILED] ifaddrs
[CHECK FAILED] libbz2
[CHECK FAILED] libcurl
[CHECK FAILED] libglob
[CHECK FAILED] libiconv
[CHECK FAILED] liblzma
[CHECK FAILED] libmysqlclient
[CHECK FAILED] libogg
[CHECK FAILED] libpq
[CHECK FAILED] librt
[CHECK FAILED] libsecp256k1
[CHECK FAILED] libtribler
[CHECK FAILED] libvorbis
[CHECK FAILED] libwebp
[CHECK FAILED] libx264
[CHECK FAILED] matplotlib
[CHECK FAILED] netifaces
[CHECK FAILED] openal
[CHECK FAILED] preppy
[CHECK FAILED] pyicu
[CHECK FAILED] pyleveldb
[CHECK FAILED] pyogg
[CHECK FAILED] pyopenal
[CHECK FAILED] pyrxp
[CHECK FAILED] pysdl2
[CHECK FAILED] reportlab
[CHECK FAILED] scrypt
[CHECK FAILED] sdl2_image
[CHECK FAILED] sdl2_mixer
[CHECK FAILED] sdl2_ttf
[CHECK FAILED] storm
[CHECK FAILED] vlc
[POSSIBLY OUTDATED] Pillow = 7.0.0 -> 8.0.1
[POSSIBLY OUTDATED] aiohttp = 3.6.2 -> 3.7.3
[POSSIBLY OUTDATED] apsw = 3.15.0-r1 -> 3.34.0-r1
[POSSIBLY OUTDATED] atom = 0.3.10 -> 0.6.0
[POSSIBLY OUTDATED] babel = 2.2.0 -> 2.9.0
[POSSIBLY OUTDATED] bcrypt = 3.1.7 -> 1.1.1
[POSSIBLY OUTDATED] boost = 1.69.0 -> 1.75.0
[POSSIBLY OUTDATED] cffi = 1.13.2 -> 1.14.4
[POSSIBLY OUTDATED] coverage = 4.1 -> 5.3.1
[POSSIBLY OUTDATED] cppy = 4e0b956 -> 1.1.0
[POSSIBLY OUTDATED] cryptography = 2.8 -> 3.3.1
[POSSIBLY OUTDATED] cython = 0.29.15 -> 0.29.21
[POSSIBLY OUTDATED] dateutil = 2.6.0 -> 2.8.1
[POSSIBLY OUTDATED] decorator = 4.2.1 -> 4.4.2
[POSSIBLY OUTDATED] enaml = 0.9.8 -> 0.12.0
[POSSIBLY OUTDATED] evdev = 0.4.7 -> 0.7.0
[POSSIBLY OUTDATED] feedparser = 5.2.1 -> 5.1.3
[POSSIBLY OUTDATED] ffmpeg = n4.3.1 -> n2.8.5
[POSSIBLY OUTDATED] gevent = 1.4.0 -> 20.12.1
[POSSIBLY OUTDATED] greenlet = 0.4.15 -> 0.4.17
[POSSIBLY OUTDATED] groestlcoin_hash = 1.0.1 -> 1.0.3
[POSSIBLY OUTDATED] hostpython3 = 3.8.5 -> 3.9.1
[POSSIBLY OUTDATED] jedi = 0.9.0 -> 0.18.0
[POSSIBLY OUTDATED] jpeg = 2.0.1 -> 2.0.90
[POSSIBLY OUTDATED] kiwisolver = 0846189 -> 1.3.1
[POSSIBLY OUTDATED] libexpat = master -> R_2_2_10
[POSSIBLY OUTDATED] libgeos = 3.7.1 -> 3.9.0
[POSSIBLY OUTDATED] libshine = c72aba9031bde18a0995e7c01c9b53f2e08a0e46 -> 3.1.1
[POSSIBLY OUTDATED] libsodium = 1.0.16 -> 1.0.18-RELEASE
[POSSIBLY OUTDATED] libtorrent = 1_2_1 -> 2.0.1
[POSSIBLY OUTDATED] libxml2 = 2.9.8 -> 2.9.10
[POSSIBLY OUTDATED] libxslt = 1.1.32 -> 1.1.34
[POSSIBLY OUTDATED] libzmq = 4.3.2 -> 4.3.3
[POSSIBLY OUTDATED] lxml = 4.2.5 -> 4.6.2
[POSSIBLY OUTDATED] m2crypto = 0.30.1 -> 0.37.1
[POSSIBLY OUTDATED] msgpack-python = 0.4.7 -> 0.5.6
[POSSIBLY OUTDATED] numpy = 1.18.1 -> 1.19.4
[POSSIBLY OUTDATED] omemo = 0.11.0 -> 0.12.0
[POSSIBLY OUTDATED] omemo-backend-signal = 0.2.5 -> 0.2.6
[POSSIBLY OUTDATED] opencv = 4.0.1 -> 4.5.1
[POSSIBLY OUTDATED] opencv_extras = 4.0.1 -> 4.5.1
[POSSIBLY OUTDATED] openssl = 1.1.1g -> 1.1.1i
[POSSIBLY OUTDATED] pandas = 1.0.3 -> 1.2.0
[POSSIBLY OUTDATED] png = 1.6.37 -> 1.6.35
[POSSIBLY OUTDATED] protobuf_cpp = 3.6.1 -> 3.14.0
[POSSIBLY OUTDATED] psycopg2 = 2.8.5 -> 2.8.6
[POSSIBLY OUTDATED] pyaml = 15.8.2 -> 20.4.0
[POSSIBLY OUTDATED] pycparser = 2.14 -> 2.20
[POSSIBLY OUTDATED] pycrypto = 2.7a1 -> 2.6.1
[POSSIBLY OUTDATED] pycryptodome = 3.6.3 -> 3.9.8
[POSSIBLY OUTDATED] pygame = 2.0.0-dev7 -> 2.0.1
[POSSIBLY OUTDATED] pynacl = 1.3.0 -> 1.4.0
[POSSIBLY OUTDATED] pyopenssl = 19.0.0 -> 20.0.1
[POSSIBLY OUTDATED] pyproj = 1.9.6 -> 3.0.0.post1
[POSSIBLY OUTDATED] python3 = 3.8.5 -> 3.9.1
[POSSIBLY OUTDATED] pytz = 2019.3 -> 2020.5
[POSSIBLY OUTDATED] pyusb = 1.0.0b1 -> 1.1.0
[POSSIBLY OUTDATED] pyzbar = 0.1.7 -> 0.1.8
[POSSIBLY OUTDATED] pyzmq = master -> 14.1.1
[POSSIBLY OUTDATED] regex = 2019.06.08 -> 2020.11.13
[POSSIBLY OUTDATED] ruamel.yaml = 0.15.77 -> 0.16.12
[POSSIBLY OUTDATED] sdl2 = 2.0.9 (f2ecfba915c54f7200f504d8b48a5dfe) -> 2.0.14
[POSSIBLY OUTDATED] setuptools = 49.2.1 -> 51.1.1
[POSSIBLY OUTDATED] shapely = 1.7a1 -> 1.7.1
[POSSIBLY OUTDATED] snappy = 1.1.7 -> 1.1.8
[POSSIBLY OUTDATED] sqlalchemy = 1.3.3 -> 1.3.22
[POSSIBLY OUTDATED] sqlite3 = 3.32.3 -> 3.34.0
[POSSIBLY OUTDATED] sympy = 1.1.1 -> 1.7.1
[POSSIBLY OUTDATED] ujson = 1.35 -> 4.0.1
[POSSIBLY OUTDATED] vispy = 0.4.0 -> 0.6.6
[POSSIBLY OUTDATED] websocket-client = 0.40.0 -> 0.57.0
[POSSIBLY OUTDATED] wsaccel = 0.6.2 -> 0.6.3
[POSSIBLY OUTDATED] x3dh = 0.5.3 -> 0.5.9
[POSSIBLY OUTDATED] xeddsa = 0.4.4 -> 0.6.0
[POSSIBLY OUTDATED] zbarlight = 2.1 -> 3.0
[POSSIBLY OUTDATED] zeroconf = 0.24.5 -> 0.28.7
[POSSIBLY OUTDATED] zope = 4.1.3 -> 5.2.0
[POSSIBLY OUTDATED] zope_interface = 4.1.3 -> 5.2.0
18 ok, 78 outdated, 40 failed

@AndreMiras
Copy link
Member

Sweet! Can't think of any specific priority.
However try to not make it into a single pull request because our CI.
I'm asking because our CI will try to build updated recipes only and it has at least the following two limitations:

  1. doesn't deal with two conflicting recipes in the same PR
  2. would timeout after 30m or so

@obfusk
Copy link
Contributor Author

obfusk commented Jan 4, 2021

Sweet! Can't think of any specific priority.

Any way to know which recipes are used a lot?
I've sent PRs for the ones I'm using myself.

@obfusk
Copy link
Contributor Author

obfusk commented Jan 4, 2021

I've also added a command to check for updates to (pinned) recipes (and pypi packages without recipes) in a buildozer.spec:

$ checkupdates buildozer-spec -r my-local-p4a-recipes -r /path/to/python-for-android/pythonforandroid/recipes 
[POSSIBLY OUTDATED] setuptools = 51.0.0 -> 51.1.1
16 ok, 1 outdated, 0 failed

Unfortunately it might be a while before I have time to properly document & package the script so others can use it easily.
But it is available for anyone to use.

@AndreMiras
Copy link
Member

Sweet! Can't think of any specific priority.

Any way to know which recipes are used a lot?
I've sent PRs for the ones I'm using myself.

We don't have a stats on used recipes yet, so I also think updating the ones you use yourself is a good approach.
Also very nice that you shared your script and improve it over time

@Julian-O
Copy link
Contributor

Closing - there is good discussion here, but no action required by python-for-android team.

@Julian-O Julian-O closed this as not planned Won't fix, can't repro, duplicate, stale Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AndreMiras @Julian-O @obfusk