Editable in manifest mode

[Ikar-Rahl]

Proposed Workflow ("Editable Mode")

I propose a formal "Editable" workflow integrated into Manifest Mode.

This is an ongoing draft of my initial experimentations and is not (yet) a functional proposal.

1. Setup

I am working on MyProj defined by vcpkg.json.

2. Materialization

I signal to vcpkg that zlib should be editable (via a command, environment variable, or configuration).

Expected outcome:

• First, vcpkg automatically resolves the correct version of zlib based on the current manifest, fetches the source (preferably via git clone to preserve history, or source extraction), and places it in a local user-writable directory (e.g., ./vcpkg_edits/zlib).
• Then, vcpkg automatically treats this new local directory as the source for the zlib port, effectively creating a temporary local overlay.

3. Iteration

1. I modify the source in ./vcpkg_edits/zlib.
2. I run the standard build command for MyProj (e.g., cmake --build .).

Crucial Change (The "Dirty Build"):

• For the Port: Vcpkg detects this port is currently "Editable". Instead of the standard behavior (hash check -> clean -> rebuild -> repackage), it performs a Dirty Build (Incremental). It invokes the underlying build system (CMake, Ninja, Make) directly on the existing build artifacts in the editable directory.
• For the Consumer: Simultaneously, the ABI hash of the editable port is updated (via a "dirty bit" or salted hash). This signals to the consumer (MyProj) that a dependency has changed, triggering a non-standard incremental build/re-link of the consumer project against the updated binary.

4. Completion

I validate my changes. I can then commit the changes (if git history was preserved or added) or generate a patch. I disable the editable state, and vcpkg returns to strict manifest compliance.

The Golden Rule: "Whatever happens in editable mode, stays in editable mode." No impact on CI, registry, shared code. Local only.

[Ikar-Rahl]

@microsoft-github-policy-service agree company="Adobe"

[Ikar-Rahl]

Working Proof of Concept (POC)

Here is a working POC demonstrating the proposed workflow.
Sister PR in vcpkg repo: microsoft/vcpkg#49403

By adding the following to vcpkg-configuration.json:

"editable-ports": {
  "path": "editable-ports",
  "ports": [
    "libpng"
  ]
}

You get a directory in editable-ports/libpng that contains 3 subfolders:

• The port itself: Used exactly like an overlay port.
• Src dir: Currently supports vcpkg_from_git, vcpkg_from_github, and vcpkg_from_gitlab (restricted to ports with exactly one call to these functions).
• Build dir: Similar structure to the one in buildtrees.

Internal Logic

We defined 3 new internal variables:

• Editable: Is the current port being built editable?
• Editable-Mode: Is the current vcpkg install process, globally, running in an editable mode? (This allows incremental build and install of downstream dependencies).
• IncrementalInstall: Allows installing only updated files and removing deleted ones.

This is a POC to drive future discussions and enhancements, but it proves the feasibility of this approach and outlines what is needed.

Capabilities

• Source Editing: Direct editing of port sources.
• Content-Based ABI: ABI Hash is calculated based on the content of the src folder (disables cache uploading when in editable mode).
• Incremental Builds: Skips configuration step (relying on the underlying build system to retrigger if needed) and performs incremental build/link.
• Clean Package Dir: The internal package directory is fully cleaned before install (as some ports rely on it being empty).
• Incremental Install: Updates vcpkg_installed based on file content changes.
• Downstream propagation: Incremental build/link/install for all ports depending on the editable port.

Current Limitations

• Dirty Install State: Building a port while having installed files from its previous version in the vcpkg_installed folder may cause issues with legacy CMake (2.x) or exotic generators. FIXED
• Fetcher Support: Only vcpkg_from_git/github/gitlab are currently supported.
• Single Source: Only 1 vcpkg_from_* call is supported; an error is thrown if a second is encountered. FIXED
• Build System: Only vcpkg_cmake_configure is supported (no support for meson, etc., yet).
• Source Updates: Modifications to the vcpkg_from_* arguments after the initial clone are ignored until the user deletes the src folder manually. (This behavior might be desirable).
• Global Flag: Editable-Mode is currently a global flag that applies to all ports, not just those dependent on an editable port. FIXED
• Testing: Not thoroughly tested; edge cases likely exist.

All of these cases can be enhanced, but the priority was to demonstrate that this is a viable technical direction.

[Ikar-Rahl]

Updates

• Wildcard Support: Added wildcard support for the editable-port field. Patterns like "boost-*" or "*" now work as expected.
• Updated Schema: Reflects the new configuration options.
• Staged Install Mechanism: Switched from keeping files in the vcpkg_installed folder to staging them in a temporary directory and restoring them with the correct timestamp if not updated. This allows a port to build with its files properly removed and unavailable during the build process, providing a much more robust approach.
• Editable Subtree Scope: Switched from a global "Editable Mode" flag (which applied to all ports) to an "Editable Subtree" logic. The mode now applies only if a port is explicitly editable or has a direct/indirect dependency that is editable. This correctly limits the incremental build behavior to the affected downstream dependencies.
• ABI Salting: The ABI hash is now salted on the initial editable run to ensure it does not reuse the previously installed non-editable port, forcing a proper initial rebuild.
• General Cleanup: Multiple cleanups from previous iterations (though some artifacts may still remain).

This fixes the Dirty Install State and Global Flag point described before.

[Ikar-Rahl]

Updates

• Added support for multi-source ports: sources/src1, sources/src2 etc. (Single Source limitation fixed)
• Redirected package folder to the editable port (now it contain: port, sources, build and package).

Limitations left:

• Fetcher Support: Only vcpkg_from_git/github/gitlab are currently supported. The rest still build though.
• Build System: Only vcpkg_cmake_configure is supported (no support for meson, etc., yet). They other should still build though.
• Source Updates: Modifications to the vcpkg_from_* arguments after the initial clone are ignored until the user deletes the src folder manually. This will be tackled next.
• Testing: Not thoroughly tested; edge cases likely exist.