Skip to content

Commit

Permalink
Add very crude parsing of "repomd.xml"
Browse files Browse the repository at this point in the history
This gives us more accurate version scraping that's not reliant on the cache of the HTML file listing (which instead matches how `yum`/`dnf` query these version numbers, so should stay in sync better).

This should fix our issues with CI sometimes returning a mismatch in 8.0 versions.

**However**, this *causes* a mismatch in 5.7 versions because the latest 5.7 release (that we were not picking up before due to the aforementioned HTML caching but are now with this updated code) is apparently not built/released for Debian (https://dev.mysql.com/downloads/mysql/), so we also need to decide whether we're going to introduce Debian/Oracle version skew or perhaps deprecate/remove the Debian variants of 5.7 (which is ostensibly EOL in ~3 months).
  • Loading branch information
tianon committed Jul 24, 2023
1 parent 2baf92d commit 18afafd
Showing 1 changed file with 26 additions and 1 deletion.
27 changes: 26 additions & 1 deletion versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,28 @@ fetch_rpm_versions() {
local oracleVersion="$1"; shift
local package="$1"; shift

curl -fsSL "$repo/$arch/" 2>/dev/null \
local baseurl="$repo/$arch"

# *technically*, we should parse "repodata/repomd.xml", look for <data type="primary">, and use the <location href="..."> value out of it, but parsing XML is not trivial with only basic tools, it turns out, so instead we rely on MySQL's use of "*-primary.xml.*" as the filename we're after 👀
local primaryLocation
primaryLocation="$(
# 2>/dev/null in case "$arch" doesn't exist in "$repo" 🙈
curl -fsSL "$baseurl/repodata/repomd.xml" 2>/dev/null \
| grep -oE 'href="[^"]+-primary[.]xml([.]gz)?"' \
| cut -d'"' -f2
)" || return 1
[ -n "$primaryLocation" ] || return 1

local decompressor='cat'
case "$primaryLocation" in
*.gz) decompressor='gunzip' ;;
*.xml) ;;
*) echo >&2 "error: unknown compression (from '$baseurl'): $primaryLocation"; exit 1 ;;
esac

# again, *technically* we should properly parse XML here, but y'know, it's complicated
curl -fsSL "$baseurl/$primaryLocation" \
| "$decompressor" \
| grep -oE '"'"$package"'-[0-9][^"]+[.]el'"$oracleVersion"'[.]'"$arch"'[.]rpm"' \
| sed -r 's/^"'"$package-|[.]$arch[.]rpm"'"$//g' \
| sort -rV
Expand Down Expand Up @@ -111,6 +132,10 @@ for version in "${versions[@]}"; do
export bashbrewArch
doc="$(jq <<<"$doc" -c '.oracle.architectures = (.oracle.architectures + [ env.bashbrewArch ] | sort)')"
done
if [ -z "$rpmVersion" ]; then
echo >&2 "error: missing version for '$version'"
exit 1
fi
baseVersion="$(jq <<<"$doc" -r '.version // ""')"
# example 8.0.22-1.el7 => 8.0.22
oracleBaseVersion="${rpmVersion%-*}"
Expand Down

0 comments on commit 18afafd

Please sign in to comment.