planning: Jan Architecture supports both Mobile and Desktop

[dan-menlo]

Goal

• Jan's architecture can be "packaged" to different platforms
  • Mobile and Desktop: Single User
  • Hosted (i.e. k8s): Multi-user
• Jan can support multi-user
  • Threads only show up
  • User has specific data fields (e.g. ActiveDirectory or SAML)
  • [Later] Extensions might also have user-specific fields

Resources

Goal is for Jan's architecture to be scalable from Mobile to Desktop to Server

• e.g. janhq/internal#158
• e.g. planning: Jan and Cortex's Extension Framework #3773

[louis-jan]

Jan on Mobile architecture

Background

Jan's mobile app was first built on React Native, which brought our first inference UI to the world. We maintained it for a while, but the problem is that it's scoped at mobile UI only and not really designed to be scaled to other platforms such as Desktop.

We built Jan Desktop, an Electron application with extensibility. Let's take a deep look at how we built it.

Extensible Jan Architecture

Electron is good, but not for mobile

Electron is built on two major upstream projects: Chromium and Node.js. Neither of those is mobile-friendly, especially within the Apple ecosystem.

To make Electron work on mobile, we have to deal with both tough challenges simultaneously. I stopped looking for a solution that sticks to Electron. I don’t want to join a two-on-one fight.

Capacitor is a popular candidate

I have read about various projects, and they have chosen Capacitor to port their apps to mobile, including Obsidian. There have been a lot of discussions about this, and all the conclusions seem positive. I tried to initialize a Capacitor about an hour ago, and sadly I realized that… Capacitor can only run web-compliant code, not Node.js code.

It can just run, but it's not scalable. We can port our extensive code base to run on the web and native code, but that's a lot of work. And it's really platform-specific, which isn't what we're looking for.

Capacitor Structure
Capacitor is good for driving stuff behind a web app only

Let's take a deeper look at how capacitors work. It gets webDir as the main entry. Which mean it’s very web app driven specific. Thus, it creates a bridge somewhere under the hood, between JS and Native APIs, which is a great approach, one that React Native and others are also taking.

import { CapacitorConfig } from '@capacitor/cli';

const config: CapacitorConfig = {
  appId: 'com.company.appname',
  appName: 'My Capacitor App',
  webDir: 'www',
};

export default config;

However, a browser's JS runtime is not the solution we are looking for.

There is no alternative to Electron on Mobile?

I tried searching the intranet for an alternative to Electron that can run a NodeJS backend on mobile and communicate with a native browser process but with no luck.

NodeJS is not mobile-friendly!!!

There is still a great candidate left

Tauri - the one we love, but it has not joined our game yet.

How cool is it that Tauri recently announced beta support for mobile. I knew it would happen, but I didn’t expect it to be that good.

1. Tauri is designed to be pretty close to Electron. It uses Rust at its core backend and a native webview for the renderer process.
2. Tauri supports native plugins!!! WOW. We can build Tauri plugins with native modules using Kotlin, or Swift. WUT?

Those two are very important because:

First, we designed our architecture to be API-friendly, replaceable, and extensible.

1. Native API friendly: we build on top of a high-performance inference core (inference engines). It would be very hard to embed these in the browser's JS runtime, which is actually not possible, except in a tricky way.
2. Replaceable: We built Jan Backend as an abstraction layer, so the renderer process does not know which backend is plugged in. We are not limited by language, framework or platform.
3. Extensible: We built the Jan web app to be very extensible, with all the logic provided by extensions. We can replace any extension that does not perform well on any design, architecture, or platform.

Rust has a very minimal, no-setup-required runtime and can be run anywhere.

Second, mobile is a different game; it's very native-specific. Using a native webview and modules is a great idea.

Tauri supports plugins:  Plugins are reusable extensions to the Tauri API that solve common problems.

It also supports mobile plugins!

Plugins can run native mobile code written in Kotlin (or Java) and Swift. The default plugin template includes an Android library project using Kotlin and a Swift package including an example mobile command showing how to trigger its execution from Rust code.

I tested in an hour and got into it.

Tauri with native mobile plugin
Tauri Mobile Plugin - With Native Package Manager Support - Native Frameworks and Language Support

I felt like I was developing a mini iOS app inside Jan.

Back to our architecture to see how it fits

The same as Electron, which uses NodeJS. We replace the backend handlers for exposed APIs with frontend ones (e.g. FileSystem). But there is one more layer added to grant us full control over the platform-specific frameworks: Native Plugin.

Jan Architecture
Only one extra layer is needed for full Native API control

Another look at the code structure

Modules Structure

Test results

This is just a test on a small Tauri project on Mobile to test the possibility of:

1. Reusing our entire webapp code base (Jan NextJS App)
2. Communication between components, from web app to Rust backend, to native mobile API
3. Embed an inference engine, such as Llama.cpp.

Let’s goooo!!!

Try to run a Next JS app on Mobile

I actually tested it on desktop a couple of months ago. Now it’s time to run it on mobile.

Build, plugin and communicate with a Native Plugin

One plugin can provide three modules that work with one interface. The native API can be invoked from Rust or Webapp.

Embed an inference engine

As the screenshot above, we can add llama.cpp swift package and use it

This file is from https://github.com/ggerganov/llama.cpp

The example plugin:

You can invoke the plugin functions directly from the web app.

The greet function (row 11 → 15) is an example of:

• Invoke a backend function and get result
• Invoke a native plugin function and get result

There is more

Updating...

[dan-menlo]

Dan's Feedback:

• Desktop and Mobile Architecture should be very similar to each other
  • Should import cortex.cpp in its entirety
  • Use cortex.cpp to detect hardware and then install llama.cpp (or whatever mobile-compatible engine)
• Ideally, all of the extensions should be able to run on Mobile

[dan-menlo]

As discussed, we will be solving for Mobile and Desktop architecture by doing the following:

• Jan to be a UI and packaging layer (e.g. Tauri for Desktop and Mobile)
• Cortex to be a full API Platform

We will be refactoring all APIs and data persistence to the Cortex layer, which will simplify Mobile support in Jan to Tauri and making the UI responsive.

[ilsubyeega]

This issue appears to be closed. Is there a follow-up issue that will be tracked of this?

Edit: my bad, probably #3905