HOMEBREW_NO_BOTTLE_SOURCE_FALLBACK enabled by default?

[jimtut]

I'm using homebrew on an M1 Mac. For the first few weeks these Macs were out, "brew install ..." would work for a LOT of stuff. I'm new to homebrew, but it seemed to build from source all the time, since there were no bottles.

Starting this week or so, something like "brew install hexedit" gives:

Error: hexedit: no bottle available!
You can try to install from source with e.g.
  brew install --build-from-source hexedit

Running brew install --build-from-source hexedit works.

Why won't brew automatically fall back to source if there is no bottle? I thought HOMEBREW_NO_BOTTLE_SOURCE_FALLBACK would force this no-fallback behaviour, but I haven't set that, so why won't brew fallback to source?

[carlocab]

Not sure about your specific question, but to save you the trouble of typing --build-from-source every time:

brew install -s hexedit

[jimtut]

Thanks! Isn't there a way to use a bottle if it exists, and fallback to building from source? I was under the impression that that was the default behaviour if HOMEBREW_NO_BOTTLE_SOURCE_FALLBACK was NOT set.

[carlocab]

I am under that impression as well, hence the first part of my answer. 😅

[Rylan12]

I think this was an intentional change, but I can understand why it's confusing. Hopefully, this explanation will help a little bit.

First, you mentioned you're new to Homebrew (welcome!) so I'll mention that on Intel macs, we distribute pretty much all of our software with bottles. That's the supported way of installing anything with Homebrew, so it's atypical for a user to need to build from source.

For Intel macs, we've already gone through and built bottles for all the formulae, but we haven't yet had the opportunity to do so for ARM. However, we're just getting ready to start doing so! Once we officially support ARM macs, most software will be installable from the bottles which will save users lots of time and pain. However, we have to start from scratch with this, which means that most formulae won't have these bottles for quite some time. We can't build bottles until all of the dependencies also have working bottles, so it's quite a process.

Because installing from a bottle is the typical "Homebrew way to install things," it will be surprising to many users if things need to build from source. This is especially annoying for those formulae that will fail when building from source after hours of waiting. Previously, brew install would try to install from a bottle first. If that failed (or there were no bottles available), it would fall-back to building from source (since very few ARM bottles exist, this is what you're experiencing). We made a change recently (I think it's in Homebrew/brew#9518) that, instead, has brew install fail with the error message you saw.

That way, once more formulae have bottles, it will be immediately apparent to users whether or not the formula they're trying to install works on ARM. If it does, it will likely have a bottle. If not, it's a warning that the formula they're trying to install likely won't work. This will save users lots of time and effort because they won't have to wait for long builds to fail just at the end.

So, to summarize: yes, this is slightly more inconvenient at the moment (although as Carlo mentioned, -s is much nicer than --build-from-source) but in the long run, we think this will be more helpful to users (and maintainers).

Hope this helps give some context

[carlocab]

You could open a PR to suggest changes in the docs.

The brew maintainers have a lot on their plates. Plus, I think documentation written from the perspective of a user could turn out to be even better than something a maintainer would write.

[rgaufman]

How about a "Warning, no pre-compiled bottle is available. Brew will attempt to build this recipe from source in 5 seconds, please press Ctrl+C if you wish to cancel..."

[Rylan12]

I think that's unlikely to be accepted. We generally don't have user interactivity in Homebrew, and that doesn't quite fit with our design philosophy (although, I've been surprised by the things we've done before so it's not impossible).

The best thing to do is probably to update the docs and add the new env variable if needed. I haven't had a chance to take a look and probably won't for quite some time as I've got an ever-increasing to-do list of Homebrew things that need fixing/modification. I'll happily review a PR if someone else opens one. Feel free to ping if you do.

[vheon]

That way, once more formulae have bottles, it will be immediately apparent to users whether or not the formula they're trying to install works on ARM. If it does, it will likely have a bottle. If not, it's a warning that the formula they're trying to install likely won't work. This will save users lots of time and effort because they won't have to wait for long builds to fail just at the end.

I personally see 2 problems with this change, especially when using https://github.com/Homebrew/homebrew-bundle:

• That change it is shared with Linuxbrew for which not all formulae have bottles (for example https://formulae.brew.sh/formula/fennel#default). Before this change I think I would get brew to try to install through a bottle but then fallback to install from source since the bottle is not available.
• Installing with --HEAD IMHO assumes that you want to build from source (or at least that was the behaviour we had before) and not it isn't.

Is there a setting to force the fallback when a bottle is missing?

[Rylan12]

• That change it is shared with Linuxbrew for which not all formulae have bottles (for example https://formulae.brew.sh/formula/fennel#default). Before this change I think I would get brew to try to install through a bottle but then fallback to install from source since the bottle is not available.

That does seem reasonable to me. I don't use Homebrew on Linux, though, so I'd need to ask @Homebrew/linux about this. Should Homebrew on Linux default to falling back on building from source? If not, the HOMEBREW_BOTTLE_SOURCE_FALLBACK option that I'm adding (see below) should cover this.

• Installing with --HEAD IMHO assumes that you want to build from source (or at least that was the behaviour we had before) and not it isn't.

Yes, I think this is a bug. Installing with --HEAD should imply building from source so this shouldn't be an issue. The PR I've linked below will fix this.

Is there a setting to force the fallback when a bottle is missing?

Not currently.

I've opened Homebrew/brew#10155 to do two things:

1. replace HOMEBREW_NO_BOTTLE_SOURCE_FALLBACK env variable with HOMEBREW_BOTTLE_SOURCE_FALLBACK. If set, it will re-allow falling back on building from source.
2. fix brew install --HEAD so that it won't require HOMEBREW_BOTTLE_SOURCE_FALLBACK to be set to be used without --build-from-source.