I can build my app but it throws error during preview testing (pnpm run preview:worker)

[shashankboosi]

Hello,

Thanks for your work on supporting Next.js with Cloudflare workers.

Here is some information about my application, I am using Next.js 14.2.3 but the API routes are handled by Hono.js and my authentication also happens on the server side.The application works fine while developing with next dev and next build.

I followed all the steps from here on an already existing app to see if cf workers could be an option for deployment but the preview testing(pnpm run preview:worker) fails when I try to redirect my application from an auth provider after authentication back into my dashboard. Here is the error that I get

✘ [ERROR] t [KindeSDKError]: Attempting to commit invalid access_token token <access_token_placeholder> to memory

      at new t
  (file:///Users/shashankreddyboosi/Documents/Projects/Darwin/.worker-next/.next/standalone/.next/server/app/api/[[...route]]/route.js:1:257792)
      at null.<anonymous>
  (file:///Users/shashankreddyboosi/Documents/Projects/Darwin/.worker-next/.next/standalone/.next/server/app/api/[[...route]]/route.js:1:259689)
      at null.<anonymous>
  (file:///Users/shashankreddyboosi/Documents/Projects/Darwin/.worker-next/.next/standalone/.next/server/app/api/[[...route]]/route.js:1:259174)
      at Object.throw
  (file:///Users/shashankreddyboosi/Documents/Projects/Darwin/.worker-next/.next/standalone/.next/server/app/api/[[...route]]/route.js:1:259278)
      at a
  (file:///Users/shashankreddyboosi/Documents/Projects/Darwin/.worker-next/.next/standalone/.next/server/app/api/[[...route]]/route.js:1:258049)
  {
    errorCode: 'INVALID_TOKEN_MEMORY_COMMIT'
  } 

Here is the api routes(route.ts) entry point where hono.js is used instead of next.js
Path: app/api/[[...route]]/route.ts

import { Hono } from "hono";
import { handle } from "hono/vercel";
import { logger } from "hono/logger";
import { authRoute } from "./auth";

const app = new Hono().basePath("/api");

const routes = app
  .route("/", authRoute);

app.use("*", logger());

app.get("/hello", (c) => {
  return c.json({
    message: "Hello Next.js!",
  });
});

export const GET = handle(app);
export const POST = handle(app);

export type AppType = typeof routes; // Using this to take advantage of Hono RPC

I am using Kinde as my auth provider. Here is the Kinde config
Path: app/api/[[...route]]/kinde.ts

import {
  createKindeServerClient,
  GrantType,
  type ACClientOptions,
  type SessionManager,
  type UserType,
} from "@kinde-oss/kinde-typescript-sdk";
import { type Context } from "hono";
import { getCookie, setCookie, deleteCookie } from "hono/cookie";
import { createMiddleware } from "hono/factory";

export const kindeClient = createKindeServerClient(
  GrantType.AUTHORIZATION_CODE,
  {
    authDomain: process.env.KINDE_DOMAIN,
    clientId: process.env.KINDE_CLIENT_ID,
    clientSecret: process.env.KINDE_CLIENT_SECRET,
    redirectURL: process.env.KINDE_REDIRECT_URI,
    logoutRedirectURL: process.env.KINDE_LOGOUT_REDIRECT_URI,
  } as ACClientOptions
);

export const sessionManager = (c: Context): SessionManager => ({
  async getSessionItem(key: string) {
    const result = getCookie(c, key);
    return result;
  },
  async setSessionItem(key: string, value: unknown) {
    const cookieOptions = {
      secure: true,
      httpOnly: true,
      sameSite: "Lax",
    } as const;
    if (typeof value === "string") {
      setCookie(c, key, value, cookieOptions);
    } else {
      setCookie(c, key, JSON.stringify(value), cookieOptions);
    }
  },
  async removeSessionItem(key: string) {
    deleteCookie(c, key);
  },
  async destroySession() {
    ["id_token", "access_token", "user", "refresh_token"].forEach((key) => {
      deleteCookie(c, key);
    });
  },
});

and I have configured my logic on the authRoute
Path: app/api/[[...route]]/auth.ts

import { Hono } from "hono";
import { kindeClient, sessionManager } from "./kinde";

export const authRoute = new Hono()
  .get("/login", async (ctx) => {
    const manager = sessionManager(ctx);
    const loginUrl = await kindeClient.login(manager);
    return ctx.redirect(loginUrl.toString());
  })
  .get("/callback", async (ctx) => {
    // called every time we login or register
    const url = new URL(ctx.req.url);
    console.log("callback url", ctx.req.url);
    await kindeClient.handleRedirectToApp(sessionManager(ctx), url);
    return ctx.redirect("/dashboard");
  });

The error mainly occurs in the callback get endpoint and it throws 500 Internal Server error with the error mentioned on the top.

One weird thing that I noticed is that the callback url on the browser is different to when I printed it on the server in the endpoint. It shows as follows http://n/api/callback?code=<code_placeholder>&scope=openid%20profile%20email%20offline&state=ceee3598cfc87428e886232a574a. The localhost:8771 becomes n on the server log, I hope that helps.

This is my wrangler.toml:

main = ".worker-next/index.mjs"
name = "darwin-app"
compatibility_date = "2024-09-23"
compatibility_flags = ["nodejs_compat"]
assets = { directory = ".worker-next/assets", binding = "ASSETS" }

[[kv_namespaces]]
binding = "NEXT_CACHE_WORKERS_KV"
id = <id_placeholder>

As I said, this error happened when I started using cloudflare workers preview to see if it is viable for deploying the application. If you can help me find the root cause of this. That would be great as I was potentially considering Cloudflare for my app deployment.

Thanks :)

[shashankboosi]

Any update on this @vicb? Any insight would be greatly appreciated. So that I can decide if this is the direction to go or not. Thanks :)

[vicb]

Hey @shashankboosi ,

As of today opennextjs-cloudflare is more a proof of concept than something you can really rely on for a prod app.

However the team is dedicated and hard at work to make the cloudflare adapter more reliable. We are working on improving the cloudflare adapter itself, the compatibility with Node.js, and the tooling. That's a lot of work but we will get there.

On the adapter side, @mhart was able to get PPR to work on workers. We also want to re-use the awesome work @conico974 , @khuezy, and others have put into opennextjs-aws. For example we plan to re-use their solution to run middleware & routing (I believe this will solve your issue).

Sharing code with other members of OpenNEXT will allow us to catch up with their level of support. It also means that a fix or a feature contributed to the common code will benefit all of us. That's the power of open source!

Integrating more deeply with OpenNEXT will also enable users to deploy part of their apps to different platforms. For example you will be able to deploy most of your app to cloudfare workers and a the few routes needing Node.js to AWS or any other serverless platform.

The future is exciting but we need a little bit of time ;)

[shashankboosi]

Hey @vicb,

Thanks for getting back on this!

I am glad you guys are pushing hard on the adapter, and deeply integrating with OpenNext. I am looking forward to see all of this come to life as it will help a lot more developers to integrate with Cloudflare.

I would have happily contributed to this if I had more context on the exact problem and places to look at on the AWS side of OpenNEXT to integrate it on the Cloudflare side but I guess you guys might be more equipped for this :)

I am hoping to reach MVP for the project by the end of year, so fingers crossed on where I might deploy ;)

[vicb]

I would have happily contributed to this if I had more context on the exact problem and places to look at on the AWS side of OpenNEXT to integrate it on the Cloudflare side but I guess you guys might be more equipped for this :)

We would be delighted to get some contributors. We will share what we have as soon as possible (days) so that contributors can help us - we'll probably create "Help Wanted" issues for contributors to pick at that point.

[shashankboosi]

Hey @vicb,

I had a bit of time today and I decided to investigate the codebase to understand and also try to see if a fix for the above issue can be found :)

I believe the reason for the error lies in the fact that, currently all the requests in the workers go through packages/cloudflare/src/cli/templates/worker.ts where the fetchHandler defined, takes care of all the requests but looks like the response is transformed to only cater for a few use cases and sets the content-encoding to identity for all requests(see the end of file).

The response doesn't handle cookies as there is a bit of a difference in the way cookies are handled on cf workers as seen here.

I checked my headers and the encodings are compressed. I checked the aws repo for it and they do something like that for their lambda env as seen at packages/open-next/src/wrappers/aws-lambda-streaming.ts but in Cloudflare's case we might have to use the CompressionStream provided by workers

I believe if the compression streams are implemented and handled within that fetch handler along with cookies. The issue that I am facing might be fixed.

Posting this so that it might help you guys as I believe there is still a bit of a knowledge gap for me to implement it myself given that I had zero idea about this work till last week. I hope this helps you guys or anyone working on the fetch handler.

Correct me if I am wrong on any of this hehe

Thanks :)