feat(next): codegenDir and update .astro paths

[florian-lefebvre]

Changes

• Moves dts from .astro/astro to .astro. My original plan in 4.14 was to move stuff from .astro to .astro/astro. I did that partially back then (for content, env and actions) but I revert it in this PR (see comments)
• Adds codegenDir to astro:config:setup. Points to .astro/integrations/<normalized_integration_name>. This is convenient for integrations authors to be able to put files in a folder where they're sure they can't override files generated by Astro (eg. .astro/types.d.ts), nor files generated by another integration. In 4.14, we introduced injectTypes which writes files to this codegenDir directory already, so we just expose it now

Testing

Should still pass

Docs

Changeset, integration API docs. Will be done after this PR is approved

[changeset-bot]

🦋 Changeset detected

Latest commit: 0c30ceb

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

This PR is blocked because it contains a major changeset. A reviewer will merge this at the next release if approved.

[Princesseuh]

For language-tools this mean that we need to load from multiple paths, here https://github.com/withastro/language-tools/blob/b79ff829224838492fcdfa273a03316863e724da/packages/ts-plugin/src/index.ts#L12 and here https://github.com/withastro/language-tools/blob/b79ff829224838492fcdfa273a03316863e724da/packages/language-server/src/nodeServer.ts#L45

[florian-lefebvre]

Done in withastro/language-tools#951

[bluwy]

Why are we moving all the files one-level down? Don't we control .astro so we are free to structure it however we want? Is this making way for other integrations to put stuff in .astro without naming collisions? If that's true, I don't think we should be explicitly catering it as they should be using a unique enough name (like .astro/my-integration-special-data.json) to prevent collision chances.

[florian-lefebvre]

ATM we can't control where people put stuff. I'm in favor of creating somle kind of convention (.astro/astro for core) there. At least that was the intent when doing injectTypes in 4.14. To protect ourselves even more, we'd need a injectAsset (probably bad name) that would do basically the same as injectTypes BUT for any file and without creating references.

[bluwy]

Do we want to say that it's safe to put your stuff in .astro? There's always a risk borrowing a folder you don't own and putting stuff there, and I think it should apply here too.

Astro owns the .astro folder so I don't think we need to protect ourselves from integrations tampering it. If an integration caused a name conflict in the .astro folder or something else that caused Astro to crash, the integration should be fixing it, not us.

For injectTypes, I see it more as a way for integrations to easily add types and sync with our command. And deduplicates the same amount of work to construct and write types.

[florian-lefebvre]

There's always a risk borrowing a folder you don't own and putting stuff there, and I think it should apply here too

It makes sense to me, as an integration author, to want to put codegen files in there since that's where astro does it. otherwise, that means you need to create your own .folder, add it to gitignore etc

---

As for the rest, I get your point, I guess we could move things outside of .astro/astro instead. I'd like to at least expose the directory .astro/integrations/<normalized> to integrations authors so that they can put stuff in there without conflicting with astro. wdyt?

[bluwy]

I think it's fine for integrations to put stuff into .astro, just that we can't guarantee to not break your integration if we decide to add certain file names that could collide with yours. So if integrations want to avoid that risk, the names that they put in there should be unique enough.

For .astro/integrations/, I'm fine with that if it's only documentation that you're encouraged to put your assets there if needed. I'm not so sure about codifying the guarantee or like helping generate the folder. Maybe others having different thoughts on this.

[florian-lefebvre]

Doesn't have to be in the beta so not urgent to answer. @Princesseuh when you have some time I'd like your opinion on that

[Princesseuh]

I don't see a big need for the PR to be done now, as it seems very unlikely for integrations to conflict with our files. Perhaps in the future if we see a lot of code gen being done. I think this can also somewhat be done in a non-breaking way? It's only the JSON schemas that are breaking

I'm personally in favour of a dedicated .astro/integrations/{integration name} folder with an API for codegen, I think it makes it nicer for both end users and integration authors, since you don't need to manage .gitignore or folder creation and the structure is previsible for debugging (ah, all the files for {integration} are in this folder)

[florian-lefebvre]

Alright then if you're both okay with that, I'll do the following:

• close the language tools PR
• move dts from .astro/astro to .astro
• expose the integration dir as URL in astro:config:setup

But yeah it doesn't have to be a 5.0 thing

[ematipico]

To be honest, with the little knowledge I have of injectTypes when I read this changeset, I failed to understand what codegenDir is for and how it is related to injectTypes.

For example, the following phrase doesn't say anything to me. I don't understand what space you're referring to, and how other integrations are involved in this feature.

It allows you to have space without risking being overriden by another integration

I would try to help with the review, but the poor description of the PR and the changeset don't help.

Can I ask you to revisit the changeset, and make a ELI5 version of it?

[florian-lefebvre]

@ematipico I clarified both PR desc and changeset, thanks for the feedback!

[ematipico]

Here's a suggestion. In the changeset you highlight the fact that integrations now don't conflict with each other, and the hook provide the reserved directory to do so.

However, nothing blocks users from doing new URL("../other-integration", codegenDir), and that's how integrations can tamper with each other. Sure, integrations shouldn't do that.

Instead, I think we should be stricter and provide callbacks so we have finer control, for example:

const integration = {
    name: 'my-integration',
    hooks: {
        'astro:config:setup': ({ writeToCodegenDir, purgeCodegenDir, readFromCodegeDir }) => {
						purgeCodegenDir();
						writeToCodegenDir(() => {
							return { filename: "file.ts", contents: "{}" }
						});
						const map = await readFromCodegeDir();
						for (const [filePath, contents] of map) {

						}
        }
    }
}

No need to block