Spacelpa for develop

[JAremko]

@syl20bnr I have a cron job (based on GitHub workflows) that builds daily resale of Spacemacs deps for PR tests (to speed them up). The config is pretty simple. I'm thinking. Maybe we can reuse it for making Spacelpa for develop branch?

Considering how to verify the archive. Automatic signing with a private key isn't the best way:

• It is unsafe, especially if the signing is done in the same environment where Emacs downloads and installs 3-d party packages to make the bundle. The last part can be solved by splitting the process but it complicates CI a lot.
• It seems that verifying signature on different platforms doesn't always work and require extra binaries.
• If we created signed archive and then realized that it contains "bad" stuff we can't take it back by "un-signing"
• Everyone who has write access to the CI repo can steal the private key.

So what if instead of doing this we go with hashes? We can use git to verify them. We'll have a white list of valid spacelpa releases and their hashes that can be part of Spacemacs repo and bot will keep it up to date by checking the spacelpa release list via https.

[JAremko]

Also it seems that there is a problem with github release throttling. Sometimes downloads become extremely slow. Not sure why. It still livable but unpleasant. I think this is some kind of client based quota so not a deal breaker. In the long run, we might need a different place to store release files 🤔 Maybe we can push them into ipfs with something like this it also support pinata that can store 1G for free (so we can guaranty that the latest releases are always available).

Then a release can be downloaded:

• via ipfs or public proxy.
• Automatically in two ways:
  • (if ipfs intalled) via local proxy where the repo link is something like http://127.0.0.1:8080/ipfs/<hash>/ <- works pretty well.
  • via cloudflare proxy https://cloudflare-ipfs.com/ipfs/<hash> <- didn't test, not sure what their quota if any.

Additional benefit is that we don't have to pack files since ipfs can store folders. So we'll have to allocate extra space only for changed files and this free gigabyte will cover a lot of releases while all the super old releases will be still available if someone has them (and uses ipfs)

[lebensterben]

We should not need to pack anything. We store GitHub hash only. Each commit is uniquely identifiable by the hash so quelpa only needs to download that specific commit.

[JAremko]

Yeah simply cloning the repo could be the best way if we go with hashes. Releases only better in the way that they are smaller if you download the whole thing. Then the user will be able to fetch updates using git (it can be integrated into Spacemacs)

We can even download specific commit via https https://github.com/syl20bnr/spacemacs/archive/c8b914c8de7a844f2aebbba465e8569751bad479.zip but it's better to use shallow clone with git.

But another problem here is that we'll have to store entire spacelpa at GitHub as repo if we want to use git sha and clone. And it has soft limit 5 GB (after this github will start to send angry emails) we could solve it by rewriting history and squashing old commits but it'll kill all the git integration (and sha verification). If instead ppl will download releases we can go testelpa-develop way and store only build script. Or keep whole repo with packages and squash history when the repo grows too big (it won't affect users).