Skip to content

A pile of python modules used by Red Hat release engineering to build their tools.

License

LGPL-2.1, Unknown licenses found

Licenses found

LGPL-2.1
LICENSE
Unknown
COPYING
Notifications You must be signed in to change notification settings

querti/kobo

This branch is 11 commits behind release-engineering/kobo:master.

Folders and files

NameName
Last commit message
Last commit date
Dec 6, 2023
Jun 24, 2020
Jun 27, 2024
Jun 27, 2024
Aug 6, 2010
Mar 15, 2022
Apr 4, 2013
Jun 18, 2009
Jun 18, 2009
Jun 18, 2009
Oct 9, 2018
Oct 9, 2018
May 2, 2024
Oct 9, 2018
Apr 4, 2013
May 2, 2024
May 24, 2023
Dec 6, 2023

Repository files navigation

kobo

A collection of Python utilities.

Build Status Coverage Status

Development

Use a virtualenv for development. For example, install kobo and dependencies in editable mode:

virtualenv ~/kobo-dev
. ~/kobo-dev/bin/activate
pip install --editable ~/src/kobo
pip install -rtest-requirements.txt

To run the test suite:

  • Run py.test to run tests against installed versions of python and dependencies, or...
  • Install and run tox to run test suite against a matrix of supported python and Django versions (more thorough, slower).

Please submit pull requests against https://github.com/release-engineering/kobo.

Changelog

kobo 0.36.2

BUG FIXES

  • worker: fixed another logging deadlock introduced in kobo 0.36.0

kobo 0.36.1

BUG FIXES

  • hub: fixed display of permissions in user details template
  • worker: fixed a logging deadlock introduced in kobo 0.36.0
  • worker: fixed a locked TaskManager sometimes opening tasks

kobo 0.36.0

FEATURES & IMPROVEMENTS

  • worker: on SIGHUP, will now gracefully await task completion and re-exec

BUG FIXES

  • client: the --hub option to watch-log command is no longer ignored
  • worker: logging thread is more robust, recovers from many errors
  • hub: minor fixes to worker detail UI
  • hub: respond with 404 error on missing logs, rather than 500
  • other minor fixes

kobo 0.35.1

BUG FIXES

  • client: fixed support for proxies (https_proxy) in XML-RPC client

kobo 0.35.0

FEATURES & IMPROVEMENTS

  • Support Django 5.0

BUG FIXES

  • hub: restricted login redirects to same site only
  • hub: fixed assignment of tasks to workers with incompatible arch
  • hub: fixed display of arches/channels on worker detail page
  • hub: avoid SELinux relabelling issue during file upload
  • Minor cleanups of obsolete or deprecated code

kobo 0.34.0

FEATURES & IMPROVEMENTS

  • hub: the / character may now be used within usernames

BUG FIXES

  • hub: uploads of the same name but different content are no longer considered duplicates

kobo 0.33.0

FEATURES & IMPROVEMENTS

  • hub: added a page for listing failed tasks, alongside the existing Running/Finished task lists.

kobo 0.32.0

FEATURES & IMPROVEMENTS

  • hub: introduced inherit_worker argument to create_subtask API, allowing workers to ensure created subtasks are assigned to themselves

kobo 0.31.0

FEATURES & IMPROVEMENTS

  • hub: introduced settings.USERS_ACL_PERMISSION to adjust access to user list

BUG FIXES

  • hub: minor fixes to page titles

kobo 0.30.1

FEATURES & IMPROVEMENTS

  • hub: deleting a worker no longer deletes associated tasks

kobo 0.30.0

BUG FIXES

  • hub: decorate Task.save with transaction.atomic
  • hub: do not call save twice in Task.save()
  • hub: fix Task.cancel_subtasks
  • client: allow file upload size to be sent a string
  • django: use BigIntegerField for file upload sizes
  • hub: return error 404 for non-existent logs
  • hub: move filename resolution to _streamed_log_response
  • worker: do not repeatedly remove already awaited tasks from subtask list
  • django: reimplement django_version_ge in pure Python
  • hub: do not use workers without ids in a filter
  • hub: replace pipes with shlex

FEATURES & IMPROVEMENTS

  • Make xmlrpc timeout configurable
  • django: remove support for Django 1.10 and older
  • hub: update Worker.update_worker documentation

kobo 0.29.1

BUG FIXES

  • Fixed XML-RPC authentication regression introduced in kobo 0.29.0

kobo 0.29.0

FEATURES & IMPROVEMENTS

  • Introduced create-worker command
  • Improved compatibility with Django 4.2
  • Improved output from admin commands on XML-RPC errors
  • Cleaned up unused code

kobo 0.28.0

BUG FIXES

  • Renamed deprecated get_query_set to get_queryset
  • Changed repr to properly deserialize tracebacks sent over XML-RPC
  • Fixed remaining Django 3 deprecations
  • Fixed reporting of failed uploads
  • Changed behavior that log decoding failure doesn't raise an error
  • Made it possible to interrupt assigned tasks

FEATURES & IMPROVEMENTS

  • Retired python 2 support
  • Added Client Credentials Flow OIDC authentication

kobo 0.27.0

FEATURES & IMPROVEMENTS

  • Added logging for authN/authZ events
  • Added --hub option to client

kobo 0.26.0

BUG FIXES

  • Fixed calculating the number of task's subtasks in new Django versions
  • Fixed cli/help_RST compatibility with Python 3
  • fixed encoding errors by forcing the usage of UTF-8 in str and bytes functions

FEATURES & IMPROVEMENTS

  • Added an option to make subtask inherit parents priority and make it configurable

kobo 0.25.0

FEATURES & IMPROVEMENTS

  • Improved compatibility with Django 4.1
  • Improved compatibility with python 3

kobo 0.24.2

BUG FIXES

  • Fixed a crash from Django dumpdata command when using Django 2 and later.

kobo 0.24.1

BUG FIXES

  • Fixed an encoding error when using a python 2 XML-RPC client against a python 3 kobo hub.

kobo 0.24.0

FEATURES & IMPROVEMENTS

  • It is now possible to provide a custom task manager implementation into the kobo worker's main loop. This can be used to alter the worker's task scheduling or execution behavior.

kobo 0.23.0

FEATURES & IMPROVEMENTS

  • Introduced AUTH_METHOD=oidc for XML-RPC clients. If this authentication method is enabled, the OIDC endpoint introduced by kobo 0.22.0 will be used.
  • HubProxy now respects the system configuration for kerberos realm when KRB_REALM has not been set.

kobo 0.22.0

FEATURES & IMPROVEMENTS

  • hub: added an endpoint for OIDC login. Note that kobo itself does not implement any OIDC workflow, which may instead be implemented by the hosting web server.
  • HubProxy will now retry on additional cases of XML-RPC error.

kobo 0.21.0

BUG FIXES

  • Fixed a crash bug in LoggingThread when mixing bytes and text.

FEATURES & IMPROVEMENTS

  • Added methods to HubProxy for explicitly logging into hub.

kobo 0.20.3

BUG FIXES

  • Many compatibility fixes for modern Python and Django versions.
  • Fixed loss of error details when LoggingThread fails.

kobo 0.20.2

FEATURES & IMPROVEMENTS

  • Improved Django 4.x compatibility

kobo 0.20.1

BUG FIXES

  • kobo.rpmlib.get_keys_from_header now correctly handles RPMs using header-only signing. Also fixes pkgset.SimpleRpmWrapper.signature.

kobo 0.20.0

FEATURES & IMPROVEMENTS

  • kobo workers now maintain a last_seen field which may be used to monitor the availability of workers.

kobo 0.19.0

FEATURES & IMPROVEMENTS

  • The resubmit-tasks now supports adjusting the priority of the resubmitted task. The client and server must both upgrade to kobo 0.19.0 in order to use this feature.

BUG FIXES

  • Fixed backwards-incompatible change in kobo 0.18.0: if a kobo-derived client provided its own --profile argument, this could clash with the new argument in kobo.

kobo 0.18.0

FEATURES & IMPROVEMENTS

  • The kobo client now supports a --profile argument to select an alternative configuration file. To enable this functionality, users of kobo should provide a default_profile and configuration_directory when creating a CommandOptionParser.

BUG FIXES

  • Fixed a python3 compatibility issue in kobo.http.
  • Fixed a crash bug for projects using the default XMLRPC_TEMPLATE for the XML-RPC help page with Django > 1.

kobo 0.17.0

FEATURES & IMPROVEMENTS

  • Introduced the concept of global variables to kobo.conf.PyConfigParser. This feature allows a variable to be declared in one file and overridden in a subsequently imported file.

CHANGES

  • The client.get_worker_info XML-RPC method no longer requires authentication. It was formerly restricted to superusers only.

kobo 0.16.0

REMOVED

  • The HubProxy auto_logout feature was removed. This feature was unreliable and could occasionally result in deadlocks, due to the usage of finalizers. It is recommended to call logout() from a finally block in cases where it's important to ensure a logout.

kobo 0.15.1

BUG FIXES

  • Fixed a Django 1.x compatibility issue introduced in kobo 0.15.0

kobo 0.15.0

FEATURES & IMPROVEMENTS

  • Improved Django 3.x compatibility

kobo 0.14.0

FEATURES & IMPROVEMENTS

  • Introduced gssapi login method, for improved Python3 compatibility. This login method requires the hub's krb5login endpoint to be enabled. The default login method is unchanged.

kobo 0.13.0

BUG FIXES

  • Fixed a Python3 compatibility issue in hub migrations
  • Improved Django 1.11 compatibility

kobo 0.12.0

BUG FIXES

  • Improved Django 1.11 and 2.x compatibility

kobo 0.11.0

FEATURES & IMPROVEMENTS

  • The resubmit-tasks command now accepts a --nowait argument

BUG FIXES

  • Fixed usage of shortcuts.run with text mode in Python 3 (#133)
  • Fixed decoding crashes in shortcuts.run when an incomplete multibyte sequence is read (#119)
  • Fixed rpmlib attempting to decode binary data in RPM headers
  • Fixed various Python 3 compatibility issues

kobo 0.10.0

FEATURES & IMPROVEMENTS

  • Improved Python 3 compatibility, particularly in xmlrpc and rpmlib modules
  • Improved Django 1.11 and 2.x compatibility

REMOVED

  • Testing of Python 2.6 has been dropped; some kobo functionality still works with Python 2.6, but may break without warning in later releases
  • Support for Django 1.6 has been removed

kobo 0.9.0

BUG FIXES

  • Fixed LoggingThread on Python 3 (#66)
  • Fixed kobo.django.xmlrpc migrations for Django 2.x
  • Fixed some exceptions discarded without logging (#32)
  • Fixed some reliability issues in kobo.xmlrpc

kobo 0.8.0

FEATURES & IMPROVEMENTS

  • Improved Python 3 compatibility
  • Improved Django 2.0 compatibility
  • Improved tests coverage
  • Header produced by kobo.shortcuts.run(show_cmd=True) is now limited to 79 characters length

BUG FIXES

  • Fixed handling of string SERVER_PORT in wsgi requests
  • Fixed Worker.timeout_task wrongly setting subtasks to INTERRUPTED (#72)
  • Fixed Worker.set_task_weight always crashing (#75)

kobo 0.7.0

FEATURES & IMPROVEMENTS

  • Improved Python 3 compatibility
  • Improved tests coverage

kobo 0.6.0

FEATURES & IMPROVEMENTS

  • kobo worker name no longer needs to match host FQDN
  • improved error reporting when loading configuration
  • improved error reporting from kobo.shortcuts.run
  • reduced memory usage when handling large log files
  • models now respect settings.AUTH_USER_MODEL

BUG FIXES

  • fixed crash on xml-rpc client in python <= 2.7.9
  • fixed spurious whitespace from kobo.shortcuts.run (#40)
  • fixed missing migration for User model

kobo 0.5.0

FEATURES & IMPROVEMENTS

  • kobo.shortcuts.run now supports all Popen keyword arguments
  • resubmit-tasks has a --force argument to resubmit successful tasks
  • new watch-log command for watching a log from CLI
  • admin UI now covers user model

BUG FIXES

  • kobo.shortcuts.run now resumes on interrupted system calls
  • worker load no longer includes assigned but unstarted tasks

kobo 0.4.0

FEATURES & IMPROVEMENTS

  • pkgset.SimpleRpmWrapper has now checksum_type member
  • threads.run_in_threads helper function

Django 1.5 update

Django part of kobo was udpated to be compatible with 1.5 release. Lower version are no more supported. As a side-effect, only python 2.6+ is supported by django part.

From same reason there will be no builds of kobo-django package for RHEL 5 and lower as it lacks required python version.

  • RemoteUserMiddleware is now used for KrbV authentication
  • LimitedRemoreUserMiddleware can be used to authenticate only on entry pages.
  • LongnameUser model is used for auth backend. It allows 255 characters long user names (as they come from KrbV)
  • Class-based generic views ExtraListView and ExtraDetailView. object_list helper stays for compatibility, but it is deprecated now and will be removed in future.

kobo 0.3.0

FEATURES & IMPROVEMENTS

  • State machine implementation - StateEnum, db field, form fields
  • Brand new HTML template, media, views, urls and menu
  • Menu supports django.root
  • Task logs, javascript log watcher, threaded worker stdout logger
  • XML-RPC help pages display list of contents
  • Kerberos support in CookieTransport
  • JSONField to store dicts and lists in database
  • Add relative_path() and split_path() functions to shortcuts

kobo.pkgset.FileWrapper

file_name attribute renamed to file_path. file_name is now a property which returns actual file name.

Action: Change file_name to file_path in your code.

Username hack

Username hack is enabled by default now (when kobo.django.auth is used). It changes username to 255 characters and also overrides validation RE.

Action: On postgresql run: ALTER TABLE auth_user ALTER username TYPE VARCHAR(255); Sqlite users have to use db_update-0.2.0-0.3.0 script.

Worker FQDN checking

Each worker's name must match it's FQDN now. This prevents cut&paste configuration errors when tasks end in INTERRUPTED state.

Action: Change worker names to FQDN. Change related usernames as well.

Changes in kobo.plugins

Removed 'lower_case' attribute. Plugins are now subclassed when a container is created. Each plugin now contains 'container' attribute.

Action: Remove 'lower_case' attribute from plugins, do whatever is necessary in 'normalize_name() class method instead.

  • Improve PluginContainer inheritance. Also add 'container' attribute to each plugin class obtained from a container instance. (Daniel Mach)
  • Plugins are now subclassed when a container is created. (Daniel Mach)

kobo.hub.models.Task refactoring

Field 'traceback' moved to a file (traceback.log). Field 'result' content dumped to a file (stdout.log), it is supposed to contain actual task result. Field 'args' changed to JSONField and data is directly available without any conversion.

Action: Run db_update-0.2.0-0.3.0 script.

Configuration handling in kobo.client and kobo.worker

Configuration no longer uses os.environ to get config file path. ClientCommandContainer, HubProxy and TaskManager constructor has a new mandatory 'conf' argument.

Action: config_file = os.environ.get("<PROJECT_NAME>_CONFIG_FILE", "/etc/<project_name>.conf") conf = kobo.conf.PyConfigParser() conf.load_from_file(config_file) ... and pass conf to ClientCommandContainer, HubProxy or TaskManager

Configuration passed to tasks

TaskBase constructor has now a mandatory argument 'conf', which is automatically set in TaskManager.

Action: Add 'conf' argument to tasks classes with custom constructor.

About

A pile of python modules used by Red Hat release engineering to build their tools.

Resources

License

LGPL-2.1, Unknown licenses found

Licenses found

LGPL-2.1
LICENSE
Unknown
COPYING

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 96.8%
  • HTML 2.2%
  • Other 1.0%