Add script to be able to find missing file for blob

[obeyler]

Hi,
I found the way to upload K8S component and add them to the blobs with

export VERSION=1.18.8
make update

this will create entries for K8S compoenent inside the ./config/blobs.yml
but some others component are missing and I don't find any script who downdload and add blod for them
such as :

ipvsadm/ipvsadm-1.30.tar.gz:
ipvsadm/libnl-3.2.25.tar.gz:
ipvsadm/popt-1.16.tar.gz:
jq/jq:
lb/haproxy-1.8.22.tar.gz:
lb/keepalived-2.0.16.tar.gz:
nfs-c7-rpms/gssproxy-0.7.0-26.el7.x86_64.rpm:
nfs-c7-rpms/keyutils-1.5.8-3.el7.x86_64.rpm:
nfs-c7-rpms/libbasicobjects-0.1.1-32.el7.x86_64.rpm:
nfs-c7-rpms/libcollection-0.7.0-32.el7.x86_64.rpm:
nfs-c7-rpms/libevent-2.0.21-4.el7.x86_64.rpm:
nfs-c7-rpms/libini_config-1.3.1-32.el7.x86_64.rpm:
nfs-c7-rpms/libnfsidmap-0.25-19.el7.x86_64.rpm:
nfs-c7-rpms/libpath_utils-0.2.1-32.el7.x86_64.rpm:
nfs-c7-rpms/libref_array-0.1.5-32.el7.x86_64.rpm:
nfs-c7-rpms/libverto-libevent-0.2.5-4.el7.x86_64.rpm:
nfs-c7-rpms/nfs-utils-1.3.0-0.65.el7.x86_64.rpm:
nfs-c7-rpms/quota-4.01-19.el7.x86_64.rpm:
nfs-c7-rpms/rpcbind-0.2.0-48.el7.x86_64.rpm:
nfs-trusty-debs/libevent-2.0-5_2.0.21-stable-1ubuntu1.14.04.2_amd64.deb:
nfs-trusty-debs/libkeyutils1_1.5.6-1_amd64.deb:
nfs-trusty-debs/libnfsidmap2_0.25-5_amd64.deb:
nfs-trusty-debs/nfs-common_1.2.8-6ubuntu1.2_amd64.deb:
nfs-trusty-debs/nfs-kernel-server_1.2.8-6ubuntu1.2_amd64.deb:
nfs-trusty-debs/rpcbind_0.2.1-2ubuntu2.2_amd64.deb:
nfs-xenial-debs/keyutils_1.5.9-8ubuntu1_amd64.deb:
nfs-xenial-debs/libevent-2.0-5_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb:
nfs-xenial-debs/libnfsidmap2_0.25-5_amd64.deb:
nfs-xenial-debs/nfs-common_1.2.8-9ubuntu12_amd64.deb:
nfs-xenial-debs/nfs-kernel-server_1.2.8-9ubuntu12_amd64.deb:
nfs-xenial-debs/rpcbind_0.2.3-0.2_amd64.deb:
socat/socat-1.7.3.3.tar.bz2:

[obeyler]

Do you think that we need to add the missing file at the end of utils/update-from-upstream ? of make another file for that?
there is utils/getrpm for nfs-c7-rpms/* but it doesn't do add blob and nothing for *.deb
nor

socat/socat-1.7.3.3.tar.bz2
ipvsadm/ipvsadm-1.30.tar.gz
ipvsadm/libnl-3.2.25.tar.gz
ipvsadm/popt-1.16.tar.gz
jq/jq
lb/haproxy-1.8.22.tar.gz
lb/keepalived-2.0.16.tar.gz

[jhunt]

What are you trying to accomplish?

[obeyler]

On my bosh release, I have a addblob.sh to downlod and add the blob. I think it's a good thing when a bosh release is done to know from where each file is coming. I want to be able to rebuild every thing to test for example a new version or if a CVE is found anywhere. You've also done for some file this but not for all. Is there any reason about that ?

[jhunt]

If you're just trying to get the blobs, bosh sync-blobs should download them and populate the local working directory with the tarballs.

[obeyler]

I already do that but with this I'm not able to know from where the blob came. by example the socat file came from I don't know and so on I'm not able to check if a new version exist or not. You think it's a bad practice to do that ?

[jhunt]

Okay, so the answer to

What are you trying to accomplish?

is "check for new versions of blobs"

The reason those blobs are not in the update-blobs script is that they change so infrequently, and I only built that script because I got tired of manually chasing down new versions of things. I have been working on another project that unifies all the messy logic of "what is the latest version of X available", a surprisingly intractable problem that stymies a whole host of release engineering activities. When I finish that up, I plan to rework the packaging in this BOSH release (if BOSH is still a thing by then) to take advantage of that.