PNPM prune fails to read 'specifiers' #97

insanity54 · 2024-07-15T09:12:02Z

Hi thanks for isolate-package. I'm interested in this project, I think it could help me build docker images from my pnpm monorepo. I followed the prerequisites as best I could, but I ran into an error when I tried to isolate one of my packages.

npx isolate-package isolate                                                                        ✔  ▼ 
info Using isolate-package version 1.18.0
Error: Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
    at generatePnpmLockfile (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:175:11)
    at processLockfile (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)
    at isolate (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/isolate.ts:175:29)
    at run (file:///home/cj/.npm/_npx/27834e9986fbcd8f/node_modules/isolate-package/src/isolate-bin.ts:9:3)

I'm not sure what the 'specifiers' thing is. I searched isolate-package source code for that string and it looks like it's maybe the package version specifier in package.json?

example: "@futurenet/scout": "workspace:^",

Curiously, isolate-package did generate a bunch of output in the isolate directory. I'm not sure if it completed it's job, though, since I haven't seen what a successful run of isolate looks like.

Not sure if it's something I did wrong with my project configuration but I figured I'd document what I came across. I think my next move is to double check that my package build process is correct. Right now, all of my packages have a build step. I read a bunch of articles and docs pages about isolate-package and I'm learning about internal packages for the first time. Maybe I will try using internal packages instead.

Another thing I'm suspicious of, maybe it's causing conflicts, is how Temporal loads it's workflows. My project integrates with Temporal and Temporal insists on using webpack to load workflow files. Normally I wouldn't use webpack at all, I'd use tsc instead. I'm not sure how that ties into everything but maybe what I'll do is see if I can re-create this error on a simpler project with fewer dependencies.

Edit 1

I made a simplified git repo and successfully used isolate-package to isolate the taco package. taco package depends on meat package and isolate-package ran without error. This points to the issue being with my code.

The text was updated successfully, but these errors were encountered:

0x80 · 2024-07-15T10:48:26Z

@insanity54 I have just released an update that improves error logging by re-throwing the original error so you should get a full stack trace. Hope it helps.

If you can't find the problem in your codebase, and you are willing to give me access, then I could take a look.

moonrockfamily · 2024-07-26T17:36:11Z

@insanity54 I was getting the same error and created a small app to try to reproduce without success; it worked just fine! I then tried to replicate with the same scope of dependencies on a fresh directory and it still worked as expected... Hmm, very strange! The following does not identify the root cause, but helped me get unstuck. I hope it helps you.

Error output with debug logging

PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Cleaned the existing isolate output directory
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903       
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Pruning the lockfile
error Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
TypeError: Cannot read properties of undefined (reading 'specifiers')
    at pruneLockfile (C:\Users\thiles\gitrepo\nodered\node_modules\.pnpm\@pnpm+prune-lockfile@6.1.4\node_modules\@pnpm\prune-lockfile\src\index.ts:56:56)
    at generatePnpmLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:145:15)
    at processLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)       
    at isolate (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/isolate.ts:175:29)
    at run (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/isolate-bin.ts:9:3)
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Cleaned the existing isolate output directory
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903       
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Pruning the lockfile
error Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')
TypeError: Cannot read properties of undefined (reading 'specifiers')
    at pruneLockfile (C:\Users\thiles\gitrepo\nodered\node_modules\.pnpm\@pnpm+prune-lockfile@6.1.4\node_modules\@pnpm\prune-lockfile\src\index.ts:56:56)
    at generatePnpmLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/lib/lockfile/helpers/generate-pnpm-lockfile.ts:145:15)
    at processLockfile (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/lib/lockfile/process-lockfile.ts:69:7)       
    at isolate (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/isolate.ts:175:29)
    at run (file:///C:/Users/thiles/gitrepo/nodered/node_modules/.pnpm/isolate-package@1.19.0/node_modules/isolate-package/src/isolate-bin.ts:9:3)

Cleanup pnpm-lock files

The error message related to pruning the lock file gave me the hunch to remove all existing pnpm-lock.yaml files as follows:

removed all pnpm-lock.yaml files in all subdirectories from the root monorepo
removed node_modules recursively
pnpm install
npx isolate

It now produced the expected isolate output without the error!
Conclusion/assumption: An old lock file was being processed

Output after Cleanup with debug logging

PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> pnpm install
Scope: all 2 workspace projects
 WARN  7 deprecated subdependencies found: are-we-there-yet@2.0.0, gauge@3.0.2, glob@7.2.3, inflight@1.0.6, ldapjs@2.3.3, npmlog@5.0.1, rimraf@3.0.2
../..                                    | +631 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
../..                                    | Progress: resolved 631, reused 631, downloaded 0, added 631, done
../../node_modules/.pnpm/bcrypt@5.1.1_encoding@0.1.13/node_modules/bcrypt: Running install script, done in 593ms
../../node_modules/.pnpm/bcrypt@5.1.0_encoding@0.1.13/node_modules/bcrypt: Running install script, done in 604ms

dependencies:
+ @thiles/node-red-contrib-amqp 2.4.4
+ do-red 0.3.5 (1.0.1 is available)
+ moment 2.30.1
+ node-red 3.1.11 (4.0.2 is available)
+ node-red-contrib-activedirectory 0.3.2
+ node-red-contrib-better-ldap-servicely 0.1.7
+ node-red-contrib-countula 1.1.9
+ node-red-contrib-cron-plus 1.5.7 (2.1.0 is available)
+ node-red-contrib-flogger 1.2.2
+ node-red-contrib-flowtest 0.1.2
+ node-red-contrib-moment 4.0.0 (5.0.0 is available)
+ node-red-contrib-mongodb3 2.0.1
+ node-red-contrib-oauth2 4.0.3 (6.2.1 is available)
+ node-red-contrib-objectid 1.1.0
+ node-red-contrib-oracledb-2 0.3.3
+ node-red-contrib-powershell 0.0.3 (0.0.5 is available)
+ node-red-contrib-reusable-flows 1.0.0
+ node-red-contrib-salesforce-fix 0.8.5
+ node-red-contrib-semaphore 1.0.1
+ node-red-node-daemon 0.4.0 (0.6.0 is available)
+ oracledb 5.5.0 (6.6.0 is available)
+ xml2js 0.4.23 (0.6.2 is available)

Done in 9.3s
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration> npx isolate 
info Using isolate-package version 1.19.0
debug Using buildDirName from config: .
debug Workspace root resolved to C:\Users\thiles\gitrepo\nodered
debug Isolate target package (root)\app\sfUserIntegration
debug Isolate output directory (root)\app\sfUserIntegration\isolate
debug Detected package manager pnpm 9.5.0+sha512.140036830124618d624a2187b50d04289d5a087f326c9edfc0ccd733d76c4f52c3a313d4fc148794a2a9d81553016004e6742e8cf850670268a7387fc220c903       
debug Use PNPM pack instead of NPM pack
debug Detected pnpm packages globs: [ 'app/sfUserIntegration' ]
debug Registering package app\sfUserIntegration
debug Packed (temp)/sfUserIntegration-1.0.0.tgz
debug Generating PNPM lockfile...
debug Relevant importer ids: [ 'app/sfUserIntegration' ]
debug Setting target package importer on root
debug Pruning the lockfile
debug Created lockfile at C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration\isolate\pnpm-lock.yaml
debug Copied .npmrc file to the isolate output
debug Deleting temp directory (root)\app\sfUserIntegration\isolate\__tmp
info Isolate completed at C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration\isolate
PS C:\Users\thiles\gitrepo\nodered\app\sfUserIntegration>

isolate.config.json

{
    "logLevel": "debug",
    "buildDirName": "."
}

app\sfUserIntegration\package.json with Catalog: scheme pnpm version specifiers

{
  "description": "Nodered - Flows for provisioning Georgian Users; eg. sensing Salesforce Contact's Admission Group needs",
  "name": "sfUserIntegration",
  "version": "1.0.0",
  "scripts": {
    "prestart": "nvs auto",
    "start": "node --max-old-space-size=8196 --env-file .env node_modules/node-red/red.js --userDir=. --port=1884"
  },
  "files": [
    "flows.json",
    "flows_cred.json",
    "settings.js",
    "Microsoft.PowerShell_profile.ps1",
    "pnpm-lock.yaml"
  ],
  "dependencies": {
    "@thiles/node-red-contrib-amqp": "catalog:messaging",
    "do-red": "~0.3.5",
    "moment": "catalog:essential",
    "node-red": "catalog:essential",
    "node-red-contrib-activedirectory": "^0.3.2",
    "node-red-contrib-better-ldap-servicely": "~0.1.7",
    "node-red-contrib-countula": "catalog:essential",
    "node-red-contrib-cron-plus": "catalog:essential",
    "node-red-contrib-flogger": "catalog:essential",
    "node-red-contrib-flowtest": "~0.1.2",
    "node-red-contrib-moment": "catalog:essential",
    "node-red-contrib-mongodb3": "catalog:db",
    "node-red-contrib-oauth2": "catalog:api",
    "node-red-contrib-objectid": "~1.1.0",
    "node-red-contrib-oracledb-2": "catalog:db",
    "node-red-contrib-powershell": "0.0.3",
    "node-red-contrib-reusable-flows": "^1.0.0",
    "node-red-contrib-salesforce-fix": "catalog:api",
    "node-red-contrib-semaphore": "~1.0.1",
    "node-red-node-daemon": "~0.4.0",
    "oracledb": "catalog:db",
    "xml2js": "^0.4.23"
  }
}

0x80 · 2024-07-26T19:32:29Z

@moonrockfamily did you have lockfiles in packages folders (outside of the isolate directories)? I think in a monorepo setup there should be only one ever, in the root.

Maybe there's some sort of version mismatch going on. I can imagine that if the lockfile was created using an older version, and isolate-package detects a newer pnpm version, it might invoke the newer prune-lockfile version without respecting the version recorded in the lockfile itself.

But I'm just speculating here :)

0x80 · 2024-08-04T10:47:12Z

@moonrockfamily @insanity54 If you still have a way to reproduce your error, please give 1.20.0-0 a try and let me know if it solved things. You can install it with @next

This version now respects/preserves the actual lockfile version when pruning, instead of going by the declared version in the packageManager field of the manifest.

moonrockfamily · 2024-08-06T18:15:11Z

Thanks @0x80!

0x80 · 2024-08-07T06:26:21Z

@moonrockfamily have you been able to test it on your initial problem?

I have been using 12.0.0-0 in my own projects and it seems at least it didn't break anything so I will release it soon. Wish I had some proper tests to cover this 👀

moonrockfamily · 2024-08-07T14:43:40Z

I tried to reproduce with 1.19 but couldn't find the recipe to make a reproducible failure!

insanity54 · 2024-08-07T15:12:15Z

I can still reproduce the error on 1.19.0 and 1.20.0-0. I made a reproduction repo

https://github.com/insanity54/isolate-issue-97

0x80 · 2024-08-07T20:51:45Z

Thanks @insanity54 I will get back to it in a week or two to see if I can find the real cause then

linear bot changed the title ~~Error: Failed to generate lockfile: Cannot read properties of undefined (reading 'specifiers')~~ PNPM prune fails to read 'specifiers' Aug 4, 2024

0x80 linked a pull request Aug 4, 2024 that will close this issue

Preserve pnpm lockfile version when pruning #101

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PNPM prune fails to read 'specifiers' #97

PNPM prune fails to read 'specifiers' #97

insanity54 commented Jul 15, 2024 •

edited

Loading

0x80 commented Jul 15, 2024

moonrockfamily commented Jul 26, 2024 •

edited

Loading

0x80 commented Jul 26, 2024 •

edited

Loading

0x80 commented Aug 4, 2024 •

edited by linear bot

Loading

moonrockfamily commented Aug 6, 2024

0x80 commented Aug 7, 2024

moonrockfamily commented Aug 7, 2024

insanity54 commented Aug 7, 2024 •

edited

Loading

0x80 commented Aug 7, 2024

PNPM prune fails to read 'specifiers' #97

PNPM prune fails to read 'specifiers' #97

Comments

insanity54 commented Jul 15, 2024 • edited Loading

0x80 commented Jul 15, 2024

moonrockfamily commented Jul 26, 2024 • edited Loading

Error output with debug logging

Cleanup pnpm-lock files

Output after Cleanup with debug logging

0x80 commented Jul 26, 2024 • edited Loading

0x80 commented Aug 4, 2024 • edited by linear bot Loading

moonrockfamily commented Aug 6, 2024

0x80 commented Aug 7, 2024

moonrockfamily commented Aug 7, 2024

insanity54 commented Aug 7, 2024 • edited Loading

0x80 commented Aug 7, 2024

insanity54 commented Jul 15, 2024 •

edited

Loading

moonrockfamily commented Jul 26, 2024 •

edited

Loading

0x80 commented Jul 26, 2024 •

edited

Loading

0x80 commented Aug 4, 2024 •

edited by linear bot

Loading

insanity54 commented Aug 7, 2024 •

edited

Loading