[RFC] Major Change To How Paths Are Generated from Pages Folders: `app/projects/pages/projects/index.ts` ➡️ `app/projects/pages/index.ts` #1694

flybayer · 2021-01-11T22:42:22Z

flybayer
Jan 11, 2021
Maintainer

Background

In Blitz.js, a folder named pages is a "magic" folder that turns folders and files inside it into a route URL. The first level inside any pages folder is the root url of /.

Problem: Deeply nested file paths because url base path is always `/`

Since we can have pages folders nested anywhere inside app/, this results in a lot of folder and file nesting and duplicate words in the full file path.

app/test-results/pages/test-results/[testResultId].tsx

=> URL: /test-results/[testResultId]

Recently we've had an increase in the number of people asking questions about this. And folks seem to intuitively expect it to work as the proposal shown below instead of the current functionality.

Proposal

Part 1: Change default base path to be path of the `pages` folder itself

-app/test-results/pages/test-results/[testResultId].tsx
+app/test-results/pages/[testResultId].tsx

=> URL: /test-results/[testResultId]

Part 2: Add new `_pages` folder which always has base path of root `/`

This makes sense because we already have _app.tsx and _document.tsx which mean "root app" and "root document".

app/orders/_pages/admin/orders/latest.tsx

=> URL: /admin/orders/latest

Part 3: Add ability to prefix `pages` with `^` to escape upward one path segment

This solves a case that part 2 doesn't solve.

-app/project/tasks/_pages/projects/[projectId]/tasks/new.tsx
+app/projects/tasks/^pages/[projectId]/tasks/new.tsx

=> /projects/[projectId]/tasks/new

Optional: allow multiple escape characters such that ^^pages would escape upward two path segments

Example

├── app
    ├── account
    │   └── pages
    │       ├── forgot-password.tsx
    │       ├── login.tsx
    │       └── signup.tsx
    ├── pages
    │   ├── about.js
    │   ├── index.js
    │   └── pricing.js
    └── projects
        ├── _pages
        │   └── admin
        │       └── projects
        │           └── index.js
        ├── pages
        │   ├── [projectId]
        │   │   └── edit.js
        │   ├── [projectId].js
        │   ├── index.js
        │   └── new.js
        └── tasks
            └── ^pages
                └── [projectId]
                    └── tasks
                        ├── [taskId]
                        │   └── edit.js
                        ├── [taskId].js
                        ├── index.js
                        └── new.js

=> URLs:
 /
 /about
 /pricing
 /account/forgot-password
 /account/login
 /account/signup
 /projects
 /projects/new
 /projects/[projectId]
 /projects/[projectId]/edit
 /projects/[projectId]/tasks
 /projects/[projectId]/tasks/new
 /projects/[projectId]/tasks/[taskId]
 /projects/[projectId]/tasks/[taskId]/edit
 /admin/projects

Considerations

Significant breaking change - how to mitigate it?

This is very significant breaking change. One that, if not handled gracefully, could make folks really regret starting to use Blitz in early alpha. So we'd have to do everything we can to make this as easy as possible.

Codemod CLI tool

We can make a CLI codemod tool that would automatically rename all pages folders inside app/ to _pages and update the corresponding import paths. This should be straightforward to do with minimal edge cases.

Legacy config option

Another thing we can do is add a config option in blitz.config.js to preserve the current behavior without any auto nesting. This isn't ideal because it makes the blitz internal code more complicated, and we'd still want to remove this code at some point, maybe at 1.0.

But this is certainly a temporary option if you all want it.

API Routes

I think any changes we make to pages folders should also be made to api folders

Becomes more difficult to immediately see the actual URL of a file

We already have a blitz routes CLI command that prints the exact URL of all your files. As your apps get larger and more complex, it gets harder to grasp all your URLs no matter what the framework does. And in frameworks like Rails and Laravel, the CLI routes command is basically a necessity since they don't use file based routing. Therefore I think this can actually be a sign we are moving in the right direction toward a more powerful, mature framework.

Some folks nest most files an extra layer deeper inside `app`

One thing we've seen in realworld Blitz apps is that you end up with a lot of files and folders directly inside app/*. At least some people's solution to this is move all model/resource files a layer deeper into something like app/modules/* or app/data/*.

The problem with that is it totally wrecks the new proposed base path logic. The possible breaking change mitigation discussed above would address this.

As part of the discussion in #1183, we sort of agreed that the best solution to this problem of too many files in app/* is to move all non-model/resource files/folders into their own folder like app/other/* or app/mixed/*. This reduces nesting for most of your files, which are resource files, keeping them at the top level inside app/

Next.js compatibility

We will still maintain Next.js compatibility because the root pages/ folder will continue to function like it always has

Your Feedback

This is something we considered but dismissed all the way back before the initial alpha launch, but has repeatedly come up as a confusion/frustration.

So please share your honest feedback on this!

We need to make a decision on this asap, ideally within 1-2 weeks or less so we can make the change before official beta launch, if we decide to :)

👉 Follow Up Alternate Proposal (Jan 12)

Many of the responses are not in favor of this proposal for the same reason I initially dismissed it back before alpha launch.

This indicates that maybe the core issue here is not our current API but rather defaults.

Alternate Proposal

Change our default pages structure to put all pages for all models inside app/pages/ instead of scattered around the various model folders in app/. "Defaults" here means the page structure of new apps and for all blitz generate commands.

This solves the confusion and frustration of deep nesting for most people
It preserves the simplicity of our current pages API without additional complexity of _pages and ^pages
It allows advanced users/projects to still opt-in to organizing pages like we have been (nested inside various domain/model folders)

Default structure would be organized like this:

app
├── pages
│   ├── index.tsx
│   ├── about.tsx
│   ├── pricing.tsx
│   ├── account
│   │   ├── forgot-password.tsx
│   │   ├── login.tsx
│   │   └── signup.tsx
│   ├── projects
│   │   ├── index.tsx
│   │   ├── new.tsx
│   │   └── [projectId]
│   │       ├── index.tsx
│   │       ├── edit.tsx
│   │       └── tasks
│   │           ├── index.tsx
│   │           ├── new.tsx
│   │           └── [taskId]
│   │               ├── index.tsx
│   │               └── edit.tsx
│   └── admin
│       └── projects.tsx
├── auth
│   ├── mutations
│   └── components
├── projects
│   ├── queries
│   ├── mutations
│   └── components
└── tasks
    ├── queries
    ├── mutations
    └── components

Answered by Skn0tt

Jan 12, 2021

I am strongly in favor of the alternate proposal. It sets the right defaults for beginners and allows us to introduce "splitting by domain" to anyone who might need it. It also brings the default structure more in line with how Next.js works, which is good for adoption.

View full answer

subimage · 2021-01-12T03:24:08Z

subimage
Jan 12, 2021

@flybayer first, let me commend you for the robust and mature method of suggesting changes with this RFC process. Bravo!

This appears to be an improvement on the current system; and one that I think makes a lot of sense if you're married to this approach.

That said; if you're considering other approaches…

Blitz' magic "pages" directory is interesting for smaller apps; as you can get up and running quickly…but IMHO this will become messy and difficult to maintain for larger applications when mixed with API Routes.

The way Rails actually handles this is with the config/routes.rb file; which is a DSL that can be easily read in addition to the 'rake routes' command to list all generated paths. The only other things that can be accessed in Rails via direct path are the items inside the public directory; which is treated essentially like a static webserver would serve resources.

In a Rails route file, typing something like resources :users, only: %i(show update index destroy) transforms into these routes…

Not everything has to follow this model, but in the Blitz approach it's less clear to me what HTTP methods are available from the paged routes. Perhaps that's not a concern of Blitz, or is solved some other way I'm unaware of by reading through the docs?

1 reply

flybayer Jan 12, 2021
Maintainer Author

Yeah this is quite a bit different than Rails :)

There's already an open discussion for config-based routing which is something else we can consider: #1001

And fyi here's what the blitz routes command outputs:

┌──────┬───────────────────────────────────────────────┬──────────────────────────────────────────────┬──────┐
│ HTTP │ Source File                                   │ URI                                          │ Type │
├──────┼───────────────────────────────────────────────┼──────────────────────────────────────────────┼──────┤
│ GET  │ app/pages/index.tsx                           │ /                                            │ PAGE │
│ GET  │ app/account/auth/pages/forgot-password.tsx    │ /forgot-password                             │ PAGE │
│ GET  │ app/account/auth/pages/login.tsx              │ /login                                       │ PAGE │
│ GET  │ app/account/auth/pages/signup.tsx             │ /signup                                      │ PAGE │
│ GET  │ app/account/pages/account/index.tsx           │ /account                                     │ PAGE │
│  *   │ app/api/_health.ts                            │ /api/_health                                 │ API  │
│  *   │ app/billing/api/webhooks/stripe.ts            │ /api/webhooks/stripe                         │ API  │
│ POST │ app/account/auth/mutations/forgotPassword.ts  │ /api/account/auth/mutations/forgotPassword   │ RPC  │

MrLeebo · 2021-01-12T07:42:23Z

MrLeebo
Jan 12, 2021
Maintainer

I'm not convinced that this change should occur. Deeply nested routes are inherently going to be more complex, but it isn't just the routes themselves, the ways the user views and interacts with these concepts is also connected. I'm afraid that adding a new syntax could muddy a developer's ability to perceive the existence of those relationships when viewing the source code.

If the user is expected to navigate through "projects" in order to reach "tasks," and cannot jump to "tasks" directly from "home," then the organizational structure should reveal that relationship. The "magic folders" are fine for hierarchies that are wide, shallow, and loosely connected, but you shouldn't try to shoehorn nested relationships into them because it obfuscates the connection.

This may just be my opinion, but I'm turned off by the ^path and ^^path proposals, and even if they existed I think I'd prefer to organize the deeply nested routes in the most plain way possible, a single root "pages" folder.

app
├── pages
│   ├── index.tsx
│   ├── about.tsx
│   ├── pricing.tsx
│   ├── account
│   │   ├── forgot-password.tsx
│   │   ├── login.tsx
│   │   └── signup.tsx
│   ├── projects
│   │   ├── index.tsx
│   │   ├── new.tsx
│   │   └── [projectId]
│   │       ├── index.tsx
│   │       ├── edit.tsx
│   │       └── tasks
│   │           ├── index.tsx
│   │           ├── new.tsx
│   │           └── [taskId]
│   │               ├── index.tsx
│   │               └── edit.tsx
│   └── admin
│       └── projects.tsx
├── auth
│   ├── mutations
│   └── components
├── projects
│   ├── queries
│   ├── mutations
│   └── components
└── tasks
    ├── queries
    ├── mutations
    └── components

0 replies

andregoldstein · 2021-01-12T09:17:19Z

andregoldstein
Jan 12, 2021

Thanks so much for putting this up for the proposal. I'm really impressed with the way you interact with the community, whatever it ultimately decides to approve!

I think, a little like @MrLeebo above, I don't have a problem with pages living directly inside the pages folder eg : pages/about.js or pages/about/index.js it feels simple and perhaps aligning oneself with the default Next routing is no bad thing here. If pages are to remain inside a "module" folder as @flybayer suggests, to me at least, his proposal of projects/pages/index.js is a lot more appealing than projects/pages/projects/index.js

I appreciate however that some people would prefer to keep their module-specific pages alongside the queries/resolvers. For me, simple is king but I appreciate that other developers will have different reasonings.

0 replies

jxe · 2021-01-12T10:48:19Z

jxe
Jan 12, 2021

Like the others who've commented, I think introducing a little language within the pathnames is a step in the wrong (complexity-increasing) direction.

My dream framework would make the routes even more obvious than next or blitz does: I guess I'd have a flat directory where even nested routes are not nested files.

app
|-- pages
|   |-- index.tsx
|   |-- projects%[projectId].tsx
|   |-- projects%[projectId]%tasks%[taskId].tsx

(Or perhaps a unicode slash-like character instead of '%'. Maybe '╱'.)

The way next does it reminds me of the way .java files' package names are tied to their directory location, and the proposal makes this more extreme.

0 replies

malkomalko · 2021-01-12T12:47:07Z

malkomalko
Jan 12, 2021

A lot of great thought and comments from everybody already 👏🏻. Some additional thoughts:

We're building on top of next, but it doesn't mean we have to follow their approach to mapping a folder structure to a url structure.
It can be hard to follow (as already discussed) what the current url of a given nested page is.
Adding additional magic in file/folder structure leads to an increase in complexity for somebody picking up blitz.

As another approach, I'd like to suggest adding the url into the page with a new api. Let's call it BlitzPage.route just like BlitzPage.getLayout.

const SomePage: BlitzPage = () => {
  return (...)
}

SomePage.route = "/some/route/:id"
SomePage.getLayout = (page) => <Layout title="SomePage">{page}</Layout>

This has the added benefit of keeping the url as close to the page as possible, while allowing you to organize in any way possible.

While the route method is a simple string above, we might want to think about a few different signatures to allow for things like route middleware, etc.

2 replies

flybayer Jan 12, 2021
Maintainer Author

Yeah something like this has crossed my mind. Maybe you can share this in this discussion on config-based routing?

👉 #1001

mandado Jan 14, 2021

I like that approach define route on page, because that way we can keeping the current structure of "contexts" that exists.

Skn0tt · 2021-01-12T14:04:16Z

Skn0tt
Jan 12, 2021
Maintainer

Therefore I think this can actually be a sign we are moving in the right direction toward a more powerful, mature framework.

To my mind, it'd be a move in the wrong direction. "mature" != "similar to other frameworks".

Similarly to @MrLeebo, I'd like to advocate for keeping it as is, specifically because it to decouple your domain dissection and path logic.

Imagine I build something like a Instagram, where there are a lot of distinct features that all center around users: /[userId]/bio, /[userId]/stories and /[userId]/followers.
These three features are part of three different domains, and the current path structure allows me to split that logic into three different folders that are all centered around a single domain.
With the proposed routing logic, that won't be possible anymore (at least without implementing the complicated path-syntax).

When I originally saw our current routing logic, I didn't get it. It's not obvious. But when one of you explained to me the Why, I started to like it, and I imagine it's the same for other people.
I think we should counter this through education, not adaptation.

1 reply

jxe Jan 12, 2021

Agreed! The "active resource" URL style around which the proposal is geared is obsolete anyways. URLs are an API for users, not for mirroring some abstract entity diagram.

Just look at your browser's URL bar: does it say github.com/users/blitz-js/repositories/blitz/discussions/1694? No, it does not.

flybayer · 2021-01-12T17:45:49Z

flybayer
Jan 12, 2021
Maintainer Author

Thank you for your input everyone!

I just added an alternative follow up proposal at the end of the original post. Please review and comment :)

0 replies

chrisdunnbirch · 2021-01-12T17:50:26Z

chrisdunnbirch
Jan 12, 2021

Firstly a shout out to Norfolk Developers group. It was via your online talk that I got in to Blitz @flybayer 👍

I'm not convined that this would be an improvement as it adds a considerable amount of cognitive complexity to understand the resulting URL for any given directory.

This topic is something that I have been actively thinking about over the past couple of weeks born out of frustration from the current limitations of how the pages directory works in both Next and Blitz.

Ideally we would be able to locate assets, components, containers, helpers, hook, mutations, queries, etc contextually (with root-level global equivalents where necessary) to prevent duplication of directory structure for different content types. Something like this, with nested pages/ directories from which URLs are generated from 1 level beneath, would work really well:

app
├── assets
│   └── ...
├── components
│   └── ...
├── containers
│   └── ...
├── helpers
│   └── ...
├── hooks
│   └── ...
├── mutations
│   └── ...
├── queries
│   └── ...
└── user
    ├── assets
    │   └── default-avatar.jpg
    ├── components
    │   └── Avatar
    │       └── index.tsx
    └── pages
        └── [userId]
            ├── index.tsx // URL: site.com/user/1
            ├── containers
            │   └── UserDetails
            │       └── index.tsx
            │   └── UserActivity
            │       └── index.tsx
            ├── mutations
            │   └── ...
            ├── queries
            │   └── ...
            └── pages
                └── settings
                    ├── index.tsx // URL: site.com/user/1/settings
                    ├── mutations
                    │   └── ...
                    ├── queries
                    │   └── ...
                    └── pages
                        ├── password
                        │   ├── index.tsx // URL: site.com/user/1/settings/password
                        │   ├── components
                        │   │   └── PasswordField
                        │   │       └── index.tsx
                        │   ├── containers
                        │   │   └── UpdatePasswordForm
                        │   │       └── index.tsx
                        │   └── mutations
                        │       └── ...
                        └── email
                            ├── index.tsx // URL: site.com/user/1/settings/email
                            ├── components
                            │   └── EmailField
                            │       └── index.tsx
                            ├── containers
                            │   └── UpdateEmailForm
                            │       └── index.tsx
                            ├── helpers
                            │   └── validatePassword
                            │       └── index.ts
                            └── mutations
                                └── ...

How achievable this is I honestly don't know but this would be the utopia of directory structure in Next/Blitz apps from my experience.

2 replies

flybayer Jan 12, 2021
Maintainer Author

Hmm, yeah that's a bit complex too because the first level, app/user/pages, is generated from 1 level beneath but subsequent levels, app/user/pages/[userId]/pages/settings/ is generated from it's own level, not 1 beneath.

But aside from that it looks like the main thing in your example here is the ability to put non-page folders/files inside pages

chrisdunnbirch Jan 12, 2021

But aside from that it looks like the main thing in your example here is the ability to put non-page folders/files inside pages

Yeah exactly this, I like to contextualise my various content and only abstract/hoist it up towards global level when necessary. My current way of working results in duplication of the directory structure for sections/pages of the app across directories like app/pages/..., app/components/..., app/containers/... etc like so:

app
├── assets
│   └── user
│       └── default-avatar.jpg
├── components
│   └── user
│       ├── Avatar
│       │   └── index.tsx
│       └── profile
│           └── settings
│               ├── password
│               │   └── PasswordField
│               │       └── index.tsx
│               └── email
│                   └── EmailField
│                       └── index.tsx
├── containers
│   └── user
│       └── profile
│           └── UserDetails
│           │   └── index.tsx
│           └── UserActivity
│           │   └── index.tsx
│           └── settings
│               ├── password
│               │   └── UpdatePasswordForm
│               │       └── index.tsx
│               └── email
│                   └── UpdateEmailForm
│                       └── index.tsx
├── helpers
│   └── user
│       └── profile
│           └── settings
│               └── email
│                   └── validateEmail
│                       └── index.ts
├── hooks
│   └── ...
├── mutations
│   └── ...
├── queries
│   └── ...
└── user
    └── pages
        └── [userId]
            ├── index.tsx // URL: site.com/user/1
            └── settings
                ├── index.tsx // URL: site.com/user/1/settings
                ├── password
                │   ├── index.tsx // URL: site.com/user/1/settings/password
                └── email
                    └── index.tsx // URL: site.com/user/1/settings/email

Add more sections other than user and this becomes very repetitive very quickly.

Skn0tt · 2021-01-12T19:23:43Z

Skn0tt
Jan 12, 2021
Maintainer

I am strongly in favor of the alternate proposal. It sets the right defaults for beginners and allows us to introduce "splitting by domain" to anyone who might need it. It also brings the default structure more in line with how Next.js works, which is good for adoption.

6 replies

flybayer Jan 13, 2021
Maintainer Author

Awesome, I think this is the clear winner! We'll go with this

flybayer · 2021-01-14T17:29:17Z

flybayer
Jan 14, 2021
Maintainer Author

I've implemented the alternate proposal here: #1719

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Major Change To How Paths Are Generated from Pages Folders: `app/projects/pages/projects/index.ts` ➡️ `app/projects/pages/index.ts` #1694

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 10 comments 12 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

[RFC] Major Change To How Paths Are Generated from Pages Folders: app/projects/pages/projects/index.ts ➡️ app/projects/pages/index.ts #1694

flybayer Jan 11, 2021 Maintainer

Background

Problem: Deeply nested file paths because url base path is always /

Proposal

Part 1: Change default base path to be path of the pages folder itself

Part 2: Add new _pages folder which always has base path of root /

Part 3: Add ability to prefix pages with ^ to escape upward one path segment

Example

Considerations

Significant breaking change - how to mitigate it?

Codemod CLI tool

Legacy config option

API Routes

Becomes more difficult to immediately see the actual URL of a file

Some folks nest most files an extra layer deeper inside app

Next.js compatibility

Your Feedback

👉 Follow Up Alternate Proposal (Jan 12)

Alternate Proposal

Replies: 10 comments · 12 replies

flybayer Jan 12, 2021 Maintainer Author

MrLeebo Jan 12, 2021 Maintainer

flybayer Jan 12, 2021 Maintainer Author

Skn0tt Jan 12, 2021 Maintainer

flybayer Jan 12, 2021 Maintainer Author

flybayer Jan 12, 2021 Maintainer Author

Skn0tt Jan 12, 2021 Maintainer

flybayer Jan 13, 2021 Maintainer Author

flybayer Jan 14, 2021 Maintainer Author

[RFC] Major Change To How Paths Are Generated from Pages Folders: `app/projects/pages/projects/index.ts` ➡️ `app/projects/pages/index.ts` #1694

flybayer
Jan 11, 2021
Maintainer

Problem: Deeply nested file paths because url base path is always `/`

Part 1: Change default base path to be path of the `pages` folder itself

Part 2: Add new `_pages` folder which always has base path of root `/`

Part 3: Add ability to prefix `pages` with `^` to escape upward one path segment

Some folks nest most files an extra layer deeper inside `app`

Replies: 10 comments 12 replies

flybayer Jan 12, 2021
Maintainer Author

MrLeebo
Jan 12, 2021
Maintainer

flybayer Jan 12, 2021
Maintainer Author

Skn0tt
Jan 12, 2021
Maintainer

flybayer
Jan 12, 2021
Maintainer Author

flybayer Jan 12, 2021
Maintainer Author

Skn0tt
Jan 12, 2021
Maintainer

flybayer Jan 13, 2021
Maintainer Author

flybayer
Jan 14, 2021
Maintainer Author