[Feature] Generate release_date property in Indexer

[HebaruSan]

This is an outgrowth of KSP-CKAN/CKAN#2789 (comment). I assume that the initial functionality of the queue-based bot will be identical to NetKAN-bot, so ideas for further improvements should be submitted here for better tracking.

Motivation

KSP-CKAN/CKAN#1155 requests a release date column in CKAN. Currently we do not have the ability to generate the metadata to populate such a column. It should correspond roughly to when a given version was indexed. netkan.exe can't generate this value by itself, because it requires knowing when the bot originally found a version, which can't be determined from checking the .netkan file or the .zip download. Some interaction with the CKAN-meta repo over time is necessary.

Suggestion

The Indexer component could add and/or update this property between receipt of inflated metadata and saving it to disk. Sample code here:

https://gist.github.com/HebaruSan/665599c5c8fb9733c23c8aa64f29bc48

• If the file exists, pull the release_date property forward unchanged
• If the file exists but has no release_date property, set it to the file creation date in git
• If the file is brand new, set the release_date property to now

Code notes

• object_pairs_hook=collections.OrderedDict preserves the ordering of the properties
• .isoformat(timespec='seconds') formats the datetime according to ISO8601, the format most commonly used by internet programmers and widely supported by languages and runtimes
• git log --format='%aI' -- filename prints the dates of all commits for a file in reverse chronological order, so the oldest is at the end of the list

Note that the git log command will only work with a full (non-shallow) clone of the repo, otherwise the full history of the file won't be present.

[HebaruSan]

Replaced by KSP-CKAN/CKAN#2916.

EDIT: Oops, this was the issue, not the pull request.