Download does not start immediately, a lot of network traffic (file is downloaded twice?)

[marcel-idana]

brew doctor output

Your system is ready to brew.

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.3.18
ORIGIN: https://github.com/Homebrew/brew
HEAD: abc0584215659c5a8716d0798ea2801388d01368
Last commit: 5 days ago
Core tap JSON: 30 Aug 22:32 UTC
Core cask tap JSON: 30 Aug 22:32 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nano
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 3.3.4 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.4_1/bin/ruby
CPU: dodeca-core 64-bit arm_blizzard_avalanche
Clang: 15.0.0 build 1500
Git: 2.46.0 => /opt/homebrew/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 14.6.1-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: N/A
Rosetta 2: false

What were you trying to do (and why)?

Install any homebrew cask. For example rsudio:

brew install --cask rstudio --verbose --debug

This occurred also with appcleaner, webstorm, goland and more casks.

What happened (include all command output)?

Installing a cask takes a lot of time, the download bar is not shown immediately and the installation causes a lot of network traffic. Maybe issue 10528 is related? Also maybe I have some odd OS state that triggers this.

In the shown output below a long pause with network activity occurs. ### LONG PAUSE WITH NETWORK ACTIVITY ### is not part of the output was inserted by me.

.
.
.
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.18\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --request GET https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
### LONG PAUSE WITH NETWORK ACTIVITY ###
==> Downloading https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
.
.
.

Full unmodified output:

/opt/homebrew/Library/Homebrew/brew.rb (Cask::CaskLoader::FromAPILoader): loading rstudio
==> Cask::Installer#install
/opt/homebrew/Library/Homebrew/brew.rb (Cask::CaskLoader::FromAPILoader): loading rstudio@daily
==> Printing caveats
==> Caveats
rstudio depends on R. The R Project provides official binaries:

  brew install --cask r

Alternatively, the Homebrew-compiled version of R omits the GUI app:

  brew install r

==> Cask::Installer#fetch
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.18\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.18\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --request GET https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
==> Downloading https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.18\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --fail --retry 3 --remote-time --output /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg.incomplete --location https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  633M  100  633M    0     0  4781k      0  0:02:15  0:02:15 --:--:-- 8639k
==> Checking quarantine support
/usr/bin/env /usr/bin/xattr -h
/usr/bin/env /Library/Developer/CommandLineTools/usr/bin/swift -target arm64-apple-macosx14 /opt/homebrew/Library/Homebrew/cask/utils/quarantine.swift
==> Quarantine is available.
==> Verifying Gatekeeper status of /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env /usr/bin/xattr -p com.apple.quarantine /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
==> /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg is not quarantined
==> Quarantining /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env /Library/Developer/CommandLineTools/usr/bin/swift -target arm64-apple-macosx14 /opt/homebrew/Library/Homebrew/cask/utils/quarantine.swift /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg https://download1.rstudio.org/electron/macos/RStudio-2024.04.2-764.dmg https://posit.co/products/open-source/rstudio/
==> Verifying checksum for 'f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg'
/usr/bin/env hdiutil imageinfo -format /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
==> Installing Cask rstudio
==> Cask::Installer#stage
==> Extracting primary container
==> Using container class UnpackStrategy::Dmg for /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env hdiutil attach -plist -nobrowse -readonly -mountrandom /private/tmp/homebrew-dmg20240831-74270-a6qqrj /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env find . -print0
/usr/bin/env mkbom -s -i /private/tmp/20240831-74270-v561h8.list -- /private/tmp/20240831-74270-4cmry6.bom
/usr/bin/env ditto --bom /private/tmp/20240831-74270-4cmry6.bom -- /private/tmp/homebrew-dmg20240831-74270-a6qqrj/dmg.aBXMuD /private/tmp/homebrew-unpack20240831-74270-aotcow
/usr/bin/env diskutil info -plist /private/tmp/homebrew-dmg20240831-74270-a6qqrj/dmg.aBXMuD
/usr/bin/env diskutil eject disk6s1
/usr/bin/env cp -pR /private/tmp/homebrew-unpack20240831-74270-aotcow/RStudio.app/. /opt/homebrew/Caskroom/rstudio/2024.04.2,764/RStudio.app
==> Verifying Gatekeeper status of /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env /usr/bin/xattr -p com.apple.quarantine /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
==> /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg is quarantined
==> Propagating quarantine from /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg to /opt/homebrew/Caskroom/rstudio/2024.04.2,764
/usr/bin/env /usr/bin/xattr -p com.apple.quarantine /Users/marcel-idana/Library/Caches/Homebrew/downloads/f7312e36afbfbb0468f24fd43c24192ac61e0fc1d8c72ddac3a67d182190ab15--RStudio-2024.04.2-764.dmg
/usr/bin/env /usr/bin/xargs -0 -- /bin/chmod -h u+w
/usr/bin/env /usr/bin/xargs -0 -- /usr/bin/xattr -w com.apple.quarantine 0381\;66d249c5\;\;F8F482A8-F919-4D20-8531-B4F232B834CD
==> Creating metadata directory: /opt/homebrew/Caskroom/rstudio/.metadata/2024.04.2,764/20240830223820.469
==> Creating metadata subdirectory: /opt/homebrew/Caskroom/rstudio/.metadata/2024.04.2,764/20240830223820.469/Casks
==> Installing artifacts
==> Installing artifact of class Cask::Artifact::App
==> Moving App 'RStudio.app' to '/Applications/RStudio.app'
🍺  rstudio was successfully installed!

What did you expect to happen?

Cask installation takes less time and the download bar is shown immediately.

Step-by-step reproduction instructions (by running brew commands)

brew install --cask rstudio --verbose --debug

[MikeMcQuaid]

@marcel-idana Do you have a proxy/VPN? Does brew fetch exhibit the same behaviour? Do you see this on most casks or just the ones mentioned? Any formulae? Thanks!

[marcel-idana]

Thanks for your quick response!

Weeks ago I tried to download files with curl with a proxy. But I only used command line flags. My .curlrc is empty and environment variables http_proxy and https_proxy are unset. Also no enabled proxy settings found in macOS.

I have two user accounts on my machine and changed user permissions with sudo chown -R $(whoami) /opt/homebrew to use Homebrew not with the account that installed Homebrew.

It also happen with the casks firefox and chromium.

brew fetch has the same behaviour.

Also the non cask formulae rust (brew install rust) behaves like this.

I also reinstalled Homebrew with no result.

I still believe that my OS is in some odd state and i would be sorry if i wasted your time ✌🏼

[cho-m]

I think this happens when upstream has no content-disposition header. This causes brew to fallback on HTTP GET to verify this information which doesn't get cached:

brew/Library/Homebrew/utils/curl.rb

Line 232 in fafa618

[[], ["--request", "GET"]].each do |request_args|

Thus, essentially a double download.

For example, in top 5 casks:

1. visual-studio-code - content-disposition: attachment; filename=VSCode-darwin-arm64.zip; filename*=UTF-8''VSCode-darwin-arm64.zip
2. iterm2 - no
3. android-commandlinetools - content-disposition: attachment
4. docker - no
5. google-chrome - content-disposition: attachment

So far, only Microsoft and Google software.

---

Anyway, should try to at least cache the GET to get ~2x speed up (maybe slightly less if any non-brew cache speeds up 2nd download).

[tdtgit]

I also have same issue for some specific package, like microsoft-edge. It take a while, maybe more 30s, from Upgrading... to Downloading...

[xkia]

I also encountered a similar issue, through network monitoring, there is download traffic, but the progress bar takes a very long time to appear, and the progress bar starts from 0.

/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.21\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head https://download.calibre-ebook.com/7.18.0/calibre-7.18.0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.21\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --request GET https://download.calibre-ebook.com/7.18.0/calibre-7.18.0.dmg
==> Downloading https://download.calibre-ebook.com/7.18.0/calibre-7.18.0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.21\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --retry 3 --fail --location --silent --head --location https://download.calibre-ebook.com/7.18.0/calibre-7.18.0.dmg
/usr/bin/env /opt/homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/4.3.21\ \(Macintosh\;\ arm64\ Mac\ OS\ X\ 14.6.1\)\ curl/8.7.1 --header Accept-Language:\ en --fail --progress-bar --retry 3 --remote-time --output /Users/user/Library/Caches/Homebrew/downloads/38f63421d13e4582a2acb67634a1b892eb1eb16a551788e05deddf0ed5bf72d6--calibre-7.18.0.dmg.incomplete --continue-at - --location https://download.calibre-ebook.com/7.18.0/calibre-7.18.0.dmg