Add "How Clerk works" and "Client handshake" to initial concepts section

docs/concepts/client-handshake.mdx

@@ -0,0 +1,70 @@
+---
+title: Client handshake
+description: The client handshake is a mechanism that is used to resolve a request's authentication state from “unknown” to definitively signed in or signed out.
+type: conceptual
+---
+
+# The client handshake
+
+<Callout type="warning">
+  This flow applies only to website browser traffic and not native applications.
+</Callout>
+
+Clerk uses a mechanism called "the client handshake" to resolve a request's authentication state from unknown (`handshake`) to `signed-in` or `signed-out`. Clerk's session management architecture relies on a short-lived session JWT to validate requests, along with a long-lived session that keeps the session JWT fresh by interacting with the Frontend API (FAPI) while the web application is active. The long-lived session token is stored in an HttpOnly cookie associated with the Frontend API domain. If a short-lived session JWT is expired on a request to an application's backend, the SDK doesn't know if the session has ended, or if a new short-lived JWT needs to be issued. When an SDK gets into this state, it triggers the handshake.
+
+With the handshake, we can resolve authentication state on the backend and ensure the request is properly handled as signed in or out, instead of being in a potentially “unknown” state. The handshake flow relies on redirects to exchange session information between FAPI and the application, ensuring the resolution of unknown authentication states minimizes performance impact and behaves consistently across different framework and language implementations.
+
+The resolution is blazing fast and usually happens at the Edge.
+
+## Handshake flow
+
+The handshake mechanism operates by way of a redirect from the host application to a FAPI endpoint:
+
+1. A request is made to an application using Clerk.
+2. Clerk SDK determines the authentication state of the request (`signed-in`, `signed-out`, or `handshake`).
+3. If the authentication state is `handshake`, Clerk SDK responds with a 307 redirect to the FAPI handshake endpoint: `https://clerk.your-domain.com/v1/client/handshake`.
+4. The handshake endpoint gets information about the current session and returns a handshake payload. The encoded handshake payload contains a list of `Set-Cookie` header directives to be passed along with the final response.
+    - If the session is active, a fresh session JWT cookie is returned.
+    - If the session is inactive, the session JWT cookie is wiped and the request will be treated as signed out.
+5. The handshake endpoint redirects back to the host application along with the handshake payload, encoded either in the URL (development instances) or as a cookie (production instances).
+6. The handshake payload is parsed by Clerk SDK and `Set-Cookie` headers are set on the response.
+7. If an updated session JWT is returned, the JWT is verified and then, if verification is successful, the request is treated as signed in. If an updated session JWT is not returned, the request is treated as signed out.
+
+## Scenarios that trigger a handshake
+
+<Callout type="warning">
+  A handshake is only triggered if a request is determined to be a document request (detected by the presence of the `Sec-Fetch-Dest: document` header, or `Accept: text/html`). Otherwise, the request is treated as `signed-out`.
+</Callout>
+
+- Development instance and a dev browser is detected in URL 
+[(URL-based session sync)](https://clerk.com/docs/upgrade-guides/url-based-session-syncing#url-based-session-syncing)
+- Production instance and request is being made to a satellite application
+- The current browser has a session token (`__session`), but no active client is detected (`__client_uat` = 0 or missing)
+- The current browser has an active client (`__client_uat` > 0) but no session token
+- The `client_uat` cookie indicating the latest timestamp a client was updated, is more recent than current session token issued at timestamp (`__client_uat` > `sessionToken.iat`)
+- The `__session` token cookie exists but is expired
+- The `__session` token cookie exists but is not active yet
+
+## Handshake architecture
+
+The handshake flow is composed of two main pieces:
+
+1. The FAPI [`/client/handshake` endpoint](https://clerk.com/docs/reference/frontend-api/tag/Client#operation/handshakeClient)
+2. The `@clerk/backend` `authenticateRequest()` method. The isomorphic `@clerk/backend` SDK is used by all framework specific Clerk SDKs such as `@clerk/nextjs`.
+
+### Handshake payload for `/client/handshake` endpoint
+
+The handshake payload is a signed JWT that contains an array of `set-cookie` header directives. This allows us to transfer cookies from the FAPI domain to the application's domain across environments without additional cookie setting logic embedded into our SDKs. In development, the payload is transported in the URL in a `__clerk_handshake` query parameter. In production, the payload is transported in a `__clerk_handshake` cookie. On returning from the handshake endpoint, the handshake payload is parsed and the cookie directives are set on the final response.
+
+Below is an example of a handshake payload:
+
+```json
+  "handshake": [
+    "__client_uat=; Path=/; Expires=Thu, 01 Jan 1970 00:00:00 GMT",
+    "__clerk_handshake=; Path=/; Domain=example.com; Expires=Thu, 01 Jan 1970 00:00:00 GMT",
+    "__client_uat=1706822359; Path=/; Domain=example.com; Max-Age=315360000; Secure; SameSite=Lax",
+    "__session=<...CLERK_SESSION_JWT...>; Path=/; Expires=Wed, 05 Feb 2025 09:08:53 GMT; Secure"
+  ]
+```
+
+To verify the handshake payload JWT, use the instance [public RSA key](https://clerk.com/docs/reference/frontend-api/tag/Well-Known#operation/GetJWKS).

docs/concepts/how-clerk-works.mdx

@@ -0,0 +1,120 @@
+---
+title: How Clerk works
+description: Learn about the main concepts and objects that power Clerk's powerful authentication and user management system.
+type: conceptual
+---
+
+# How Clerk works
+
+Clerk is a powerful authentication and user management system that allows you to add secure authentication to your application in minutes. This guide will discuss some of the main concepts and objects that power Clerk.
+
+<Callout type="info">
+  Did you know that Clerk has a blog? You can learn more in-depth about how Clerk works by reading our ["How We Roll"](https://clerk.com/blog/how-we-roll-roundup) blog series.
+</Callout>
+
+## Development and production instances
+
+Clerk provices both development and production environments for your app. Production instances are used for production, pre-production, or staging environments with fixed domains. Production instances offer the strongest security and performance guarantees. Development instances contain certain trade-offs meant to support ease of active development. 
+
+<Callout type="warning">
+Never use a development instance in a production environment. Development instances don't have strong security guarantees.
+</Callout>
+
+## Main objects
+
+Clerk uses five main objects:
+
+### [Client](/docs/references/javascript/client#client)
+
+A client represents the current device or software accessing an application such as your web browser, native application for Android or iOS, Chrome Extension, or Electron app.
+
+### [Session](/docs/references/javascript/session)
+
+A session is a secure representation of the authentication state of the current user. Each client can hold multiple sessions on the same device. This is identical to how Gmail works in a browser.
+
+### [User](/docs/users/overview)
+
+A user represents the current user of the session. The object holds all the basic user information such as the user's name, email addresses, and phone numbers, and including their public, private, and unsafe metadata.
+
+### [Organization](/docs/organizations/roles-permissions)
+
+An organization represents the current organization of the session. Users can belong to many organizations. One of them will be the current organization of the session. 
+
+### [Roles](/docs/organizations/roles-permissions)
+
+A user belongs to an organization with a role that defines their permissions. Currently, Clerk supports two roles, `org:admin` and `org:member`.
+
+## Clerk Frontend API (FAPI)
+
+Every Clerk development and production instance has a dedicated [Frontend API](https://clerk.com/docs/reference/frontend-api). This is the authentication, session, user & organization management API you or your users will interact with within the web browser, native application, Chrome extension, or Electron app.
+
+Clerk assigns a unique random API URL such as `https://delicate-wahoo-73.accounts.dev/sign-in` to each development instance. For production instances, the Frontend API lives at `https://clerk.[your-domain.com]`.
+
+### How Clerk's Frontend API works
+
+Clerk's Frontend API is responsible for maintaining the authentication state of the current client (browser or native application). Based on the authentication's state (`signed-in`, `signed-out`, `handshake`) it mints a short-lived session [JSON Web Token (JWT)](/docs/backend-requests/making/jwt-templates) for the current user and session to be used by the host application that is secured by Clerk. The session JWT is stored in a cookie or can be retrieved at any time via the [`getToken()`](/docs/references/javascript/session#get-token) method.
+
+### Clerk cookies
+
+Clerk's Frontend API uses two cookies for session management in production instances:
+
+* `__client` cookie: a long-lived session that is used to keep the the session JWT fresh by interacting with the Frontend API.
+* `__session` cookie: a short-lived session JWT to validate requests on your application or your API. 
+
+#### The `__client` cookie
+
+The `__client` cookie is set on Clerk Frontend API (for example `https://clerk.<example.com>`) for each instance. It's HTTP-only, first-party, and secure. It contains a long-lived client JWT that lasts 7 days by default. The duration is configurable in the Clerk Dashboard's  [Sessions page](https://dashboard.clerk.com/last-active?path=sessions). The JWT identifies the current client (browser, native application, or chrome extension) and sets the current active sessions.
+
+#### The `__session` cookie
+
+The `__session` cookie is set on the host application that is secured by Clerk (for example `https://dashboard.<example.com>`). It's a JS cookie, it's secure, and it contains a short-lived session JWT that lasts 60 seconds. This JWT contains the current session, user, and organization identifiers and must be sent to the application API to authenticate API requests. In SSR, the `__session` cookie travels automatically to the server. ClerkJS ensures that the `__session` cookie is automatically refreshed in Client Side Rendering (CSR) and Server Side Rendering (SSR), ensuring a fresh session is always available.
+
+However, if a `__session` cookie is expired on a request to an application's backend, the SDK doesn't know if the session has ended, or if a new short-lived JWT needs to be issued. When an SDK gets into this state, it triggers Clerk's client handshake.
+
+#### The `__client_uat` cookie
+
+The `__client_uat` cookie is set on the host application that is secured by Clerk and hints Clerk SDKs about the timestamp of the latest authentication update of the current client.
+
+### The client handshake
+
+Clerk uses a client handshake mechanism to resolve a request's authentication state from unknown (`handshake`) to `signed-in` or `signed-out`. 
+
+With this handshake, Clerk can resolve authentication state on the backend and ensure the request is properly handled as signed in or out, instead of being in a potentially unknown state. The handshake flow relies on redirects to exchange session information between FAPI and the application, ensuring the resolution of unknown authentication states minimizes performance impact and behaves consistently across different framework and language implementations.
+
+Learn more about [how Clerk's client handshake works.](/docs/concepts/client-handshake).
+
+### Client piggybacking
+
+Every Frontend API request returns a response payload that contains two top level keys: 
+
+```json
+{
+  "client": ...,
+  "response": ...
+}
+```
+
+<Callout type="info">
+  The `/v1/client/sessions/:id/tokens` endpoint refreshes the Clerk session tokens and is an exception to this format.
+</Callout>
+
+The `response` key contains the resource or resources of each endpoint. The `client` key contains the client piggybacking payload. The `client` object identifies the current device and contains all sessions, users, and organization data for the current device (see [Main Objects](#main-objects) section). 
+
+This is effectively the necessary frontend state for ClerkJS and powers the JS SDKs, the [React hooks](/docs/references/react/use-user), and [Clerk components](/docs/components/overview). It is a best-effort way for the backend to update the frontend state on every Frontend API request. 
+
+### Clerk Frontend API for non-standard web browsers
+
+Clerk's [Frontend API](https://clerk.com/docs/reference/frontend-api) offers a seamless authentication solution for non-standard web browser environments such as React Native applications, Chrome Extensions, and hybrid apps in platforms like Capacitor.js or Electron.
+
+These platforms treat cookies differently than web browsers. As a result, ClerkJS should use the HTTP Authorization header instead of the `__client` cookie for secure communication with the Clerk Frontend API. To achieve session persistence, the `__client` JWT should be stored in a secure storage provided by each platform and then it should be injected into every Clerk Frontend API request. 
+
+It's not recommended to use Client piggybacking because of the following issues:
+
+1. It's a very large chunk of a tree-like state and doesn't change often.
+1. It's hard to compute on the backend as it requires a lot of data from the database. The computation takes place on every request, adding a significant overhead.
+1. It's best effort.
+1. It's not a two-way communication channel.
+
+## Clerk Backend API (BAPI)
+
+Clerk's [Backend API](https://clerk.com/docs/reference/backend-api) is a restful CRUD API for the server side. It allows a sudo-level management of all Clerk objects.

docs/manifest.json

@@ -23,6 +23,17 @@
       ["Fastify", "/quickstarts/fastify"]
     ]
   ],
+  [
+    {
+      "title": "Concepts",
+      "icon": "lightbulb",
+      "root": "concepts"
+    },
+    [
+      ["How Clerk works", "/concepts/how-clerk-works"],
+      ["Client handshake", "/concepts/client-handshake"]
+    ]
+  ],
   [
     {
       "title": "Guides",