Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate options for registry cleanup #3017

Open
3 tasks done
arboleya opened this issue Aug 23, 2024 · 16 comments · May be fixed by #3670
Open
3 tasks done

Investigate options for registry cleanup #3017

arboleya opened this issue Aug 23, 2024 · 16 comments · May be fixed by #3670
Assignees
Labels
chore Issue is a chore

Comments

@arboleya
Copy link
Member

arboleya commented Aug 23, 2024

This is a follow-up issue after #2377

When someone runs npm info fuels, the output is far from ideal:

Click for screenshot

npm-info-fuels

The solution to this would include two steps:

  • Contact NPM support to remove all irrelevant tags
  • Deprecate irrelevant tags
  • Stop publishing temporary tags to NPM

The ability to publish temporary PRs is helpful, but it'd be great if we could do that separately from the NPM registry.

Could we use GitHub packages to resolve the second point?

Ideally, we'd still be able to publish temp tags while keeping our packages clean and tidy.

@arboleya arboleya added the chore Issue is a chore label Aug 23, 2024
@petertonysmith94
Copy link
Contributor

@danielbate I think the second point here is super valuable, as it'd enable us to run the UI integration tests by default, and not only on merge to master.

@arboleya arboleya added the temp:notion label Sep 8, 2024 — with Linear
@arboleya arboleya added the temp-linear label Sep 8, 2024 — with Linear
@danielbate danielbate self-assigned this Dec 4, 2024
@arboleya
Copy link
Member Author

arboleya commented Dec 5, 2024

@danielbate The reference script/workflow I mentioned:

@arboleya
Copy link
Member Author

arboleya commented Dec 5, 2024

Also, here's the NPM policy:
https://docs.npmjs.com/policies/unpublish

@danielbate
Copy link
Member

I have contacted support to remove the following versions, let's see how it goes. Worst case scenario we can alter the script and deprecate these versions.

@arboleya
Copy link
Member Author

Easy, just 21k+ versions. 😅

@danielbate
Copy link
Member

Okay so unfortunately support cannot remove our versions... Our weekly download count is too high and we have too many dependents.

I'm going to make an adjustment to the script so that it deprecates these versions rather than attempting to un-publish. I'd also favour having it run on a cron whilst we go through this process.

I'll also look to move future published PR versions to a seperate registry.

@danielbate
Copy link
Member

Deprecation will clean up info alot, this is the result for nextjs who have more published versions than us, but deprecate.

Screenshot 2024-12-11 at 11 03 02

@danielbate
Copy link
Member

@arboleya I've investigated using a private registry and it certainly is possible. However one reason we use PR versions is for testing with consumers. Using a private/separate registry will require the consumer to configure the .npmrc file to load from the registry.

I think this problem could be mitigated if we just correctly and regularly deprecate versions. Once #3465 is through I'll make a change so we can pass singular versions, and maybe even a cron that cleans up versions every few days (but I'm not sold on the latter as consumers take time to respond). Do you think this would suffice, or still prefer a separate registry?

@arboleya
Copy link
Member Author

@danielbate Could we deprecate a specific tag when the related PR goes to master?

Can custom registries be specified as a CLI option or just as a .npmrc config?

npm install abc --registry=xyz

Either way, it's perfectly fine to move in small increments here.

We can start with deprecation and still consider possible future improvements.

@danielbate
Copy link
Member

Could we deprecate a specific tag when the related PR goes to master?

Yes exactly my thoughts. So on a feature PR merge, we could deprecate any associated pr release tags. On a release PR merge, we deprecate any previous next release tags. I definitely think there's room for automated deprecations to manage this.

AFAIK it's only via the npmrc for a custom registry on install, but I will see.

Copy link
Member Author

@danielbate We will stop publishing non-tag versions to NPM, effective immediately. For temporary and non-tagged releases (pr-*, next), we must use another registry like GitHub. Bumping priority on this.

@danielbate danielbate removed their assignment Jan 31, 2025
@danielbate
Copy link
Member

Makes sense. Given the increase in priority I've unassigned my self for now. Hopefully it can be picked up sooner by either @petertonysmith94 or @nedsalk?

@petertonysmith94
Copy link
Contributor

petertonysmith94 commented Jan 31, 2025

@arboleya I've been exploring using GitHub packages as a registry option (#3639).

To enable publishing, we must make many changes to our package names in package.json, changesets, etc., to include the organisation scope (FuelsLab/) — see the scope here.

A better option might be using AWS CodeArtifact, which should reduce this complexity and be more in-sync with the actual release process to NPM.

Note

It does appear now I have managed to publish to GitHub Packages
https://github.com/FuelLabs/fuels-ts/pkgs/npm/fuels

@arboleya
Copy link
Member Author

arboleya commented Jan 31, 2025

Great job, man! Record time. 👏

I don't see anything wrong with Github - do you?

I think the flow would look something like this:

---
title: Release Flow
---
flowchart
  PR([PR #4876]) --> Master(master)
  PR --> |‎ ‎ pr-4876‎‎ ‎ | GH
  Master --> IsRelease[[is this a release?]]
  IsRelease --> Yes(yes)
  IsRelease --> No(no)
  Yes --> Wait[/manual approval/]
  Wait --> |‎ ‎ latest‎‎ ‎ | Npm{NPM}
  No --> |‎ ‎ next‎‎ ‎ | GH(((GitHub)))
Loading

@petertonysmith94
Copy link
Contributor

Great job, man! Record time. 👏

I don't see anything wrong with Github - do you?

I think the flow would look something like this:

I'm happy with GitHub for now.

I've implemented a workflow based on your diagram in #3639. The manual approval will still be required at the moment - we will need to separate the workflow into two jobs:

  • Release + publishing to GitHub
  • Publishing to NPM

@petertonysmith94
Copy link
Contributor

petertonysmith94 commented Feb 6, 2025

We have managed to deprecate all our 0.0.0-x versions on the following action:

We deprecated all our dist-tags on the following action:

Using the following PR:


We've have a PR to integration GitHub packages:

I've created an issue to enhance this workflow further:

@petertonysmith94 petertonysmith94 linked a pull request Feb 7, 2025 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Issue is a chore
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants