Slow incremental builds using xcodebuild

[wojciech-kulik]

Recently, I discovered that adding CODE_SIGNING_ALLOWED=NO" to xcodebuild commands significantly speeds up incremental builds. However, the problem is that you can't run the app built this way, because obviously it's unsigned.

I suspect that xcodebuild signing process is for some reason very slow, so if we could figure out a way to sign in afterwards, maybe we could speed up builds. Unfortunately, I haven't yet found a way to sign the app.

So far I tried:

codesign --force --deep --sign "Apple Development: John Snow (XYZ)" /Users/wkulik/Library/Developer/Xcode/DerivedData/XYZ-hblnhsksxjrctzekqmlevcflnsji/Build/Products/Debug-iphonesimulator/XYZ.app

and also signing all frameworks and bundles before signing the whole app, but it doesn't work - the app can't be launched.

[wojciech-kulik]

Root Cause

It turns out that the issue is caused by the way the xcodebuild tool handles signing. It performs multiple requests to developerservices2.apple.com API which take over 20 seconds each time you run a build 🤦‍♂️.

The difference between xcodebuild and Xcode is that Xcode runs those requests on startup and stores the responses, while xcodebuild, on the other hand, calls them every time you invoke it.

On top of that, xcodebuild performs sequentially developerservices2.apple.com/services/v1/capabilities request per each target you have, so the more targets you have, the more significant the difference between Xcode and xcodebuild is going to be. The funny thing is that each request is exactly the same, so it's not even necessary to call them all 🙈.

What's Next

It's a very inefficient behavior, but unfortunately, I don't think it's possible to fix or configure it on our side. We need to wait for a fix provided by Apple, possibly to cache responses and optimize the signing.

I have already created a ticket in Feedback Assistant: FB15478221. You can also follow the thread here. Initially, I assumed the local signing process was slow for some reason, but the discussion in that thread revealed that network requests were responsible for the delay.

However, there is also good news, you can speed up builds by temporarily blocking that domain. In my case, it reduces an incremental build from 30 seconds down to 10 seconds. Below you will find details on how to do that.

Caution

Blocking that domain will most likely result in disabling some Xcode functions like automatic registering devices, capabilities, etc. Therefore, it's best to block it when you are working on code and you don't need any modifications related to signing or capabilities.

Workaround 1

Enable workaround:

sudo bash -c "echo '127.0.0.1 developerservices2.apple.com' >>/etc/hosts"

Disable workaround:

sudo sed -i '' '/developerservices2\.apple\.com/d' /etc/hosts

Workaround 2

If you use some tool to sniff network traffic like Proxyman or Charles Proxy, you can block requests to https://developerservices2.apple.com/* and automatically return some error like 999 status code. It will prevent xcodebuild from further calls.

Workaround 3 - plugin integration

Now, xcodebuild.nvim plugin provides automatic integration with this workaround. It blocks the Apple server while xcodebuild is running and unblocks it when the process is finished. Here is the instruction on how to enable it: read more. This feature has been introduced in #202.

Still slower than Xcode 🥺

This workaround improves build time significantly, but there is still some overhead introduced by xcodebuild command.

In my case, incremental build using Xcode takes ~4s, and incremental build using xcodebuild takes ~10s.

Based on the response from Apple (#106) and my profiling, it looks like in the case of my project it takes an additional ~0.3s to initialize the tool, ~3s to load the project, and around ~2s of some extra time needed by xcodebuild.

It seems like xcodebuild is lacking some performance improvements like a project cache. I suspect that the tool was designed more for CI/CD & clean builds rather than incremental builds.

If this issue is important to you I recommend creating a ticket in Feedback Assistant and referencing my ticket FB15478221. The more people report this issue, the greater the chance that it will be fixed.

[Almaz5200]

Yeah, definetly helped me too, but the change is not as drastic. It went from 27 seconds on average to 18 seconds. I see that for you it makes four of each requests, however I only see one of them. Which Xcode version do you use? Could it be that they improved it at a later version? Mine is 16.0

[wojciech-kulik]

I use Xcode 16.0. It looks like it might be connected with the number of targets in the project. I've got the main target + 3 extensions. The funny thing is that each request is exactly the same... very poor optimization.

Update:
I tried a project with a single target and there is only one request then.

Update 2:
Also requests to developerservices2.apple.com/services/v1/capabilities are not parallelized, so the more targets you have the more significant the difference between Xcode and xcodebuild is going to be.

[Almaz5200]

Yeah, and that would explain the fact that you've seen more improvement than I did as well. The project I've tested on doesn't have any extensions

[rudrankriyam]

I found it mentioned in the developer forums and the difference is evident when you turn off the internet and see it does not hang on GatherProvisioningInputs anymore

[marcusziade]

can confirm

[wojciech-kulik]

I'm working to integrate an automatic script to disable developerservices2.apple.com domain during build based on @rudrankriyam's code: https://x.com/rudrankriyam/status/1847734299740811675 to make it simpler to use without the need to manually turn on & off the workaround.

[wojciech-kulik]

The script to automate applying the workaround has been integrated with xcodebuild.nvim.

Here is the instruction explaining how to enable this feature:
https://github.com/wojciech-kulik/xcodebuild.nvim/wiki/Tips-&-Tricks#%EF%B8%8Fimprove-build-time

Don't forget to update the plugin first :)!

[LionWY]

follow

[kambala-decapitator]

Unfortunately, I haven't yet found a way to sign the app

Looks like your codesign command is incorrect - you forgot entitlements for the app. Also, don't pass --deep!

Correct sign workflow is:

1. For watch app and all extensions: first sign all frameworks, then sign the executable (or its bundle) with respective entitlements
2. Sign all app frameworks
3. Sign the main app with respective entitlements

You can also check the codesign command that xcodebuild issues in Xcode build log or in the command line invocation (just don't pass -quiet for the latter).

[simpers]

Does tuist also suffer from this? As far as I can see, running tuist build ... sets up an xcodebuild process so unless they also have a workaround for this, it affects them too.

Is this a new Xcode 16.x thing or has it always been like this? I haven't paid enough attention to build times using tuist only as I've mostly used it for cold builds.

Since this project is the first time I tried to work outside of Xcode – where remote work over SSH is the real benefit for me – I haven't been able to assess these things until now.

[wojciech-kulik]

@simpers If tuist uses xcodebuild without any hacks then most likely it affects them too.

I’m not sure if it was introduced at some point, but for sure Xcode 15 & 16 are affected.