Skip to content

Commit

Permalink
Refactors pmm -> corepack
Browse files Browse the repository at this point in the history
  • Loading branch information
arcanis committed Sep 26, 2020
1 parent 8c494c6 commit 9809b8f
Show file tree
Hide file tree
Showing 102 changed files with 2,774 additions and 1,091 deletions.
5 changes: 5 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module.exports = {
extends: [
`@yarnpkg`,
],
};
12 changes: 6 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,20 @@ jobs:
with:
node-version: 14.x

- name: 'Build Node with pmm master'
- name: 'Build Node with corepack master'
run: |
yarn pack
git clone -b mael/pmm --depth=1 https://github.com/arcanis/node.git node && cd node
git config user.name 'John Doe'
git config user.email '[email protected]'
rm -rf deps/pmm && tar xvf ../package.tgz && mv package deps/pmm
git add . && git commit -m 'Updates pmm'
rm -rf deps/corepack && tar xvf ../package.tgz && mv package deps/corepack
git add . && git commit -m 'Updates corepack'
./configure
DISTTYPE=nightly DATESTRING=YYYY-MM-DD COMMIT=XXXX make binary -j8
mv node-v15.0.0-nightlyYYYY-MM-DDXXXX-*.tar.gz node-pmm-${{matrix.platform[0]}}-x64.tar.gz
mv node-v15.0.0-nightlyYYYY-MM-DDXXXX-*.tar.gz node-corepack-${{matrix.platform[0]}}-x64.tar.gz
- name: Upload build artifacts
uses: actions/upload-artifact@v2
with:
name: node-pmm-${{matrix.platform[0]}}-x64
path: node/node-pmm-${{matrix.platform[0]}}-x64.tar.gz
name: node-corepack-${{matrix.platform[0]}}-x64
path: node/node-corepack-${{matrix.platform[0]}}-x64.tar.gz
1,012 changes: 907 additions & 105 deletions .pnp.js

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"recommendations": [
"arcanis.vscode-zipfs",
"dbaeumer.vscode-eslint"
]
}
15 changes: 0 additions & 15 deletions .vscode/pnpify/typescript/bin/tsc

This file was deleted.

15 changes: 0 additions & 15 deletions .vscode/pnpify/typescript/bin/tsserver

This file was deleted.

15 changes: 0 additions & 15 deletions .vscode/pnpify/typescript/lib/tsc.js

This file was deleted.

15 changes: 0 additions & 15 deletions .vscode/pnpify/typescript/lib/tsserver.js

This file was deleted.

15 changes: 0 additions & 15 deletions .vscode/pnpify/typescript/lib/typescript.js

This file was deleted.

6 changes: 0 additions & 6 deletions .vscode/pnpify/typescript/package.json

This file was deleted.

8 changes: 7 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
{
"typescript.tsdk": ".vscode/pnpify/typescript/lib"
"typescript.tsdk": "/Users/mael.nison/pmpm/.yarn/sdks/typescript/lib",
"search.exclude": {
"**/.yarn": true,
"**/.pnp.*": true
},
"eslint.nodePath": ".yarn/sdks",
"typescript.enablePromptUseWorkspaceTsdk": true
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
77 changes: 77 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
## What problem does it solve?

Various problems arise from npm being the only package manager shipped by default:

- Projects using popular package management solutions other than npm (particularly Yarn and pnpm) require additional installation step that must often be repeated when switching between Node versions. This lead to a significant part of the Node userbase effectively being a second-class citizen, which sounds unfortunate.

- Because one package manager currently holds a special treatment, users are more likely to pick it even if they would choose another solution should they have the choice (it really depends on how they balance the tradeoffs, but sometimes they value simplicity over purely technical factors). This artificial barrier hurts our community by making it harder to pick the right tool for the job.

- Having a single official package manager means that all the keys belong to a single player which can do whatever it pleases with it (even the Node project only has a limited influence over it, since removing the unique package manager would be poorly accepted by the community). Spreading these responsibilities over multiple projects gives less power to each, ensuring that everyone behave well.

Discussion thread: https://github.com/nodejs/node/issues/15244

## Envisioned workflow

1. Users would install Node as usual.

2. Node would be distributed slightly differently:

- Pmm would be included by Node out of the box.

- The full npm package wouldn't be included out of the box anymore (this might be an incremental move, with first a major version shipping pmm + npm, and the next one discarding npm).

- **However**, the Node distribution would include jump binaries for all three main package managers (`yarn`, `npm`, and `pnpm`) that would simply delegate to `pmm <package manager name>`. Pmm would then handle the install logic by following the logic described in later sections.

- Pmm could potentially be distributed as a Node subcommand rather than a standalone binary. In this case, commands in this document (such as `pmm install <name@version>`) would be replaced by `node --pmm install <name@version>` (or any other variant).

3. Regular users would keep using the `yarn` / `npm` / `pnpm` global binaries just like they are used to. The one difference is that the package manager implementations would be lazily downloaded, without having to be manually installed (because the global jumpers would be included in the Node distribution, cf previous point).

- Projects that don't list the `engines.pm` field would allow any package manager, and Pmm would install them based on predefined versions. Those versions will be frozen in time within Pmm itself to "known good values". For example, the default npm version could be 6.14.5, and the default Yarn one 1.22.4. Users that would want to upgrade to higher versions would just have to update the `engines.pm` field (cf next section).

4. Project authors would most of the time only have to care about the binaries as well, but they would be able to upgrade package manager versions simply by changing the versions set in the `engines.pm` field.

- Pmm could reasonably provide some kind of basic CLI interface to select a version to upgrade to in a few keystrokes (similar to what `emsdk` does for the [emscripten toolchain](https://github.com/emscripten-core/emsdk#how-do-i-check-for-updates-to-the-emscripten-sdk), or what [nvm](https://github.com/nvm-sh/nvm) does for Node releases).

5. Docker users would follow a similar workflow to other users; the default image would run network queries to install the right package manager for the project being installed.

- However, users with strong offline requirements would be able to run the `pmm install <name@version>` command when preparing their images. It would ensure that the requested package manager is made available for later use.

- Network access could be disabled entirely by setting `PMM_ENABLE_NETWORK=0` in the environmen - Pmm would then only use the package managers that got installed by prior `pmm install` calls.

6. Package manager maintainers would submit a PR to the Node repository each time they wish for a new version to be made available through Pmm (can be easily automated using a GitHub Action on each of our repositories). Merging the PR would instantly make the new version available to Node users (once they upgrade).

## How does it work?

When any of the embed binaries are called (whether it's `yarn`, `npm`, or `pnpm`), the tool will find the closest ancestor `package.json` for the current directory. It will then extract the `engines.pm` key, configured as such:

```json
{
"engines": {
"pm": "yarn@^2.0.0"
}
}
```

The tool will then check whether it got called via the right binary endpoint (`npm` or `npx` when the package manager is configured for npm, `yarn` when configured for Yarn, etc), and will report an error otherwise. This ensures that we can't accidentally call, say, pnpm on an npm project (which would otherwise lead to diverging environments since the lockfiles and features wouldn't be the same depending on the interpreting package managers).

If the check succeeded, the tool will check whether a compatible package manager has been installed (they're all stored on the disk in the local user's home folder). If not, it will install the latest matching release (based on the information dynamically retrieved from [`versions.json`](/versions.json)). Once it has ensured that a version exists, it'll forward the call to it.

## Frequently asked questions

**Why not just ask the user which package manager they want to use when installing Node?**

Whether to use npm or Yarn or pnpm isn't up to the user but to each individual project. Different projects leverage different features from different package managers. For example one project might rely on the Yarn workspaces, whereas another has setup their repository with pnpm in mind.

**How would things work with global packages?**

Nothing would change in the context of this particular proposal. Npm would keep installing its globals alongside Node, and Yarn would keep installing them into the user's home directory.

**Why not just keep only npm?**

While npm is favored by the majority of the ecosystem, a significant portion decided to use different tools. Their use cases deserve to be heard rather than be discarded simply because a slightly higher percentage of users happens not to directly benefit from it. Additionally, keeping powers balanced is important - even more so given that npm is a corporate entity with little oversight.

From the npm perspective, a project such as Pmm would also have its benefits: projects regularly break when upgrading from one Node version to another because of npm being upgraded as well. By pinning the package manager version, they would ensure that their users only upgrade when they are ready to, decreasing accidental frustration.

## Known issues

- The `pnpx` and `npx` binaries can only be called from within pnpm and npm projects, respectively. This is because otherwise we cannot infer the package manager version from the local manifest, as it would list another package manager instead. Fixing that is possible if we include "global installs" features inside pmm (so that we would fallback to the global `npx` in those circumstances). It seemed out of scope for the initial prototype, but we certainly can discuss it in an issue.
Loading

0 comments on commit 9809b8f

Please sign in to comment.