They Shoot Horses, Don't They?

[fxcoudert]

The problem: old unbuildable formulas

• We current have 565 formulas that have a Mavericks bottle, meaning they haven't had any change in over 4 years.
• Of those 62 formulas currently don't build on Big Sur (Intel). Here they are, along with number of monthly installs:

(Edit: the list was updated 2020-12-20)

325 winexe
244 connect
125 vera++
102 zssh
101 cabocha
73 shntool
73 pcre++
73 hardlink-osx
62 git-hooks
51 mpack
41 makeicns
15 git-cal
0 mdxmini

They need to be fixed, or removed. Some of them are already heavily patched. Because they haven't been updated in 4 years, they're unlikely to be actively maintained.

Suggested solution

We have limited volunteer availability, so I would suggest:

• All those that have less than 20 monthly installs are removed
• If someone wants to add them back, they are welcome! They will be treated as new formulas
• This issue can become a help-wanted issue to fix the others, if we can… (or remove them on a per-case basis, if they can't be fixed)

What do you think @Homebrew/core ?

[claui]

• All those that have less than 20 monthly installs are removed

Fixed that for you.

• If someone wants to add them back, they are welcome! They will be treated as new formulas

Sounds reasonable.

[carlocab]

I agree with removing them all, though I don't mind the initial suggestion of keeping some of them too.

If someone wants to add them back, they are welcome! They will be treated as new formulas

Can we document this in the PR where they are removed? (Or I suppose a link to this issue in that PR would suffice.) It seems the current message someone who tries to install a removed formula would see is something like

==> Searching for a previously deleted formula (in the last month)...
pgplot was deleted from homebrew/core in commit 246cfef514:
  pgplot: delete due to license issues (https://github.com/Homebrew/homebrew-core/issues/64332)

To show the formula before removal run:
  git -C "$(brew --repo homebrew/core)" show 246cfef514^:Formula/pgplot.rb

If you still use this formula consider creating your own tap:
  https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

[mitchblank]

A couple of those are on the list of bottling issues I've been looking at:

• lha -- pgformatter 1.6 #9197 should fix
• pcre++ -- not sure why it didn't rebottle already; compile/test worked fine for me locally on Big Sur. Maybe a dependency had been broken but is now fixed?
• md -- as you're aware we have md: fix build under Xcode 12 #66347 ready to go to disable it

A couple others I was planning to take a quick look at tonight (hfsutils, ipv6toolkit)

[mitchblank]

hfsutils and ipv6toolkit were just broken due to stale download URLs; both build OK on Big Sur once that is fixed (#66371, #66372 respectively)

[MikeMcQuaid]

• All those that have less than 20 monthly installs are removed

s/removed/deprecated or disabled

• If someone wants to add them back, they are welcome! They will be treated as new formulas

👍🏻

[fxcoudert]

deprecated or disabled

If a formula is not building, not maintained, what's the point of disabled? To keep it in our history? That's why we use git. (Otherwise these disabled formulas accumulate for no reason)

[MikeMcQuaid]

If a formula is not building, not maintained, what's the point of disabled? To keep it in our history? That's why we use git. (Otherwise these disabled formulas accumulate for no reason)

We can remove disabled formulae once they've been disabled for a while. They are a significantly nicer experience for end-users to extract/modify/fix than formulae that are removed.

[SMillerDev]

Disabled will tell users it's not supported while removed is a surprise on the next attempted install.

[fxcoudert]

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

[scpeters]

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

yeah, you can add a date to deprecate! and disable! calls. Before the deprecate! date, no errors are given, and afterwards, deprecation warnings are given when installing a formula. I believe that adding a disable! date will deprecate the formula before the date and disable it afterwards. I think it's reasonable to add disable! statements with the date of Big Sur's release and then remove them outright after a fixed period of time. I would hope that disabled formulae are just ignored by CI, so that the cost of not deleting them is low.

[SMillerDev]

I would hope that disabled formulae are just ignored by CI, so that the cost of not deleting them is low.

They are ignored, I'm fine with keeping disabled formula for a year. We just haven't had the feature long enough to get to that yet.

[fxcoudert]

Could we also agree that disable! and deprecate! should systematically have a date, so we can start the clock and not keep them forever?

[SMillerDev]

I agree they should have a date. And I think I should be able to write an audit that checks it they should be removed.

[fxcoudert]

An audit to mandate dates should be good. Checking for expiration of dates seems likely to show up in unrelated PRs, causing trouble, so probably not wanted. Also, I can image there would be cases where we have good reasons to remove earlier or later?

[MikeMcQuaid]

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

Makes sense to me. I don't have strong feelings on what this date is.

Could we also agree that disable! and deprecate! should systematically have a date, so we can start the clock and not keep them forever?

The date is when they kick in, not when they are removed.

An audit to mandate dates should be good. Checking for expiration of dates seems likely to show up in unrelated PRs, causing trouble, so probably not wanted. Also, I can image there would be cases where we have good reasons to remove earlier or later?

The best fix for me (albeit a slightly more involved one) is having a GitHub Actions job which deletes these formulae (or opens a PR to do so) when they hit the relevant date.

[fxcoudert]

The date is when they kick in

Yes, but having a date systematically (which is not always the case currently), even if it's today or in the past, means we know how long they've been broken/disabled.

[MikeMcQuaid]

Yes, but having a date systematically (which is not always the case currently), even if it's today or in the past, means we know how long they've been broken/disabled.

So if we want to deprecate/disable immediately you'd see setting this to e.g. today's date? This would make sense to me. If so, I'd suggest we consider deprecating the non-date format.

[SMillerDev]

If so, I'd suggest we consider deprecating the non-date format.

I think that's a good idea.

[fxcoudert]

So if we want to deprecate/disable immediately you'd see setting this to e.g. today's date?

This is what I've been doing, yes.

[MikeMcQuaid]

I think that's a good idea.

Cool, done (but can't merge until next minor release): Homebrew/brew#9478

[fxcoudert]

Dates being added to all formulas where they were missing: #66549

[fxcoudert]

Update: remaining problematic formulas

cabocha cdrtools connect git-cal hardlink-osx mdxmini mpack pcre++ shntool vcprompt vera++ winexe zssh

[mitchblank]

remaining problematic formulas

The ones on that list I've previously looked at:

• pcre++ -- I tired to see what the problem is but it build and tested fine for me locally on Big Sur. Not sure why it didn't rebottle correctly.
• zssh -- zssh: re-run autoreconf #66397 works for me, not sure what else needs to be done. Just an argument about testing at this point
• cdrtools -- That one could be ugly. It looks like it could be amenable to the just-rerun-autoconf fix (most of the early failures seem to be tracked to things fixed years ago there) However, it has a unique build system and a lot of custom m4 going on so even getting it to accept a new autoconf run could take a lot of work

[fxcoudert]

cdrtools: their "latest version" link downloads cdrtools-3.02a09, so maybe we could consider that stable (it was released in 2017)?

[fxcoudert]

pcre++ fails to download on CI, but works for me:

==> FAILED
==> Downloading https://www.daemon.de/idisk/Apps/pcre++/pcre++-0.9.5.tar.gz
curl: (7) Failed to connect to www.daemon.de port 443: Operation timed out

[EricFromCanada]

The author of cdrtools releases snapshots of it and several others as part of the schilytools package, so it appears that what's listed under the cdrtools "alpha" folder is what's presented as a "release".

[mitchblank]

cdrtools-3.02a09, so maybe we could consider that stable (it was released in 2017)?

Just tried it... still getting:

        ==> COMPILING "cdda2wav.o"
In file included from cdda2wav.c:102:
./mytype.h:29:1: error: unknown type name 'error'
error need an integer type with 32 bits, but do not know one!
^

This is very often autoconf related and it definitely has xcode12-related breakages there:

$ grep -c -e '-Werror,-Wimplicit-function-declaration' $( find . -name config.log )
./incs/i386-darwin-clang/config.log:159
./cdda2wav/OBJ/i386-darwin-clang/config.log:0

I'll try seeing if some $CFLAGS massage helps...

[mitchblank]

OK, opened #66661 for cdrtools. Builds/tests for me but we'll see what CI thinks of it

[mitchblank]

Status of a few others from the "updated" list:

• connect - opened connect: switch to github #67268 to fix
• cabocha - has a depends_on crf++ so maybe will progress after crf++: find a new URL for this old package #67265 lands?
• hardlink-osx -- doesn't work on APFS; maybe remove at this point? hardlink in APFS macOS High Sierra 10.13 selkhateeb/hardlink#31
• git-cal -- looks like the perl MakeMaker PREFIX thing; I'll try to sweep through those soon -- feedgnuplot: change install location inside keg #66367 #66370 (comment)
• vcprompt -- is this still a problem, @fxcoudert ? your own vcprompt: update URL #65374 seems to have already triggered a Big Sur bottle to exist

[fxcoudert]

@mitchblank I've updated the list. Re hardlink-osx I'm 👍 for disabling it, especially as the thread you linked raises questions of data loss!

[mitchblank]

I've updated the list.

You still have dupseek on the list, but we've had a Big Sur bottle for that as of 12 days ago: #66509

[mitchblank]

Took a quick look at mpack; it just needed a new download UR: #67290

[chenrui333]

looks like I completely missed this discussion, do we have some guidelines of how to use deprecate! vs disable!? One tricky thing is the repo got removed (I remember there are some utilities suffered from that issue).

[bayandin]

My general understanding is:

• deprecate!: the latest version we can build is unmaintained (examples: most of the versioned formulae after EOL, infer)
• disable!: we can't build the version we have
• repo got removed — sounds like software is not OSS anymore, probably we should delete the formula

[SMillerDev]

In my idea it's deprecate! for software that SHOULD no longer be used, upstream depreciation or archival is the clearest example.

And disable! for anything that either MUST not be used anymore due to license issues or prolonged depreciation, or CAN'T be used anymore due to build issues.

[Rylan12]

I opened a PR with some documentation about this in Homebrew/brew#10144. Feedback welcome!

[fxcoudert]

I'm closing this issue as we've made great progress on this list, thanks everyone