Dynamic Configuration Variables

[bmoellers]

I was wondering if there is a way for the SSR, since it appears to be node js to insert and build the configuration values correctly.

Basically Next JS seems to have a system to use their server to inject env variables in to the build that is serves as SSR which allows a build once - deploy many using the same docker image: https://nextjs.org/docs/pages/building-your-application/configuring/environment-variables#runtime-environment-variables.

Basically we are struggling with deployment of the application across many different lines for testing and production because the pipeline that was built seems to expect the build once - deploy many feature for servers. Another team handles building deployment tools and we are able to deploy and see our application but we cannot adjust values for each line using their system. It use HELM charts and values to control ENV variables and it seems to work just fine with Next JS.

We decided to go with Modern JS because it handles our MFE framework with Module Federation much better. It also might make it more of an issue because we expose components instead of pages for inclusion in our other applications so not sure if this is easily handled.

Hoping I get a response from someone because right now this is a show stopper for us.

Thanks,
Brian

[botagar]

Forgive my ignorance as I've not delved into SSR yet, but does reading process.env not work?
I'm assuming you're talking about having your server read ENV vars from the deployed-and-running container which affects the layout/content of the subsequently served pages and NOT about reading ENV vars during the container build stage.

[bmoellers]

So SSR and process.env works when either using globalVars (see https://modernjs.dev/en/guides/basic-features/env-vars.html#any-other-names) or using the source.define. This allows you to load .env even in SSR.

However that still does string replacement at the time of modern build based on the .env file you use. This means that the SSR bundle and CSR bundle are both set at the time of build and you cannot change the values of process.env after the build. This is pretty standard for React but is a "Build Many - Deploy Many" approach.

What Next JS seems to do is use their node JS server to regenerate SSR and CSR files so that it uses the .env file when it is served not built. So basically when you do node index.js on next it then seems to be able to do a dotenv.config() to load the config and regenerate the SSR and CSR so that the .env file during the serve (aka modern serve not modern build) allowing it to be deployed to a UKS pod, the pod loaded with a .env file, and then it deploys using the ENV for that pod. So a "Build once - Deploy Many". Again this is not the standard way to do react apps but seems to be possible for Next JS using server side static props to pass values down instead of string replacement.

So I was just wondering if Modern JS has a similar feature as our deployment team is struggling with the "Build Many - Deploy Many" approach but then again it does seem to be something they will eventually need to solve. This is not our only React app in the pipeline but the others are with Next JS or may not be deployed yet with our UKS solution.

[botagar]

I had completely missed that section of the framework and have been pulling all my env vars straight from process.env 😅
Whilst I'm not using SSR, I have been using BFF with the following pattern to good success

const ServicesConfig = () => {
  ....
  const { SERVICE_LB_LISTENER_DNS_NAME } = process.env;

  return {
    ...,
    entrypointUrl: SERVICE_LB_LISTENER_DNS_NAME ?? 'http://localhost:8080',
    authServiceBaseUrl: SERVICE_LB_LISTENER_DNS_NAME ?? 'http://localhost:8081',
    mapServiceBaseUrl: SERVICE_LB_LISTENER_DNS_NAME ?? 'http://localhost:8082',
    userServiceBaseUrl: SERVICE_LB_LISTENER_DNS_NAME ?? 'http://localhost:8083',
  };
};

It's in my backlog to tidy this up a little but but it "works" for now 😆

Don't know if this helps at all or not.

My Dockerfile too if you're curious at all

# Prepare base
FROM node:20 AS base
WORKDIR /app
ENV HUSKY=0
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable pnpm && corepack install -g pnpm@latest-9
COPY . .
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile
RUN pnpm run build
RUN pnpm run deploy pruned

# Final stage
FROM node:20-slim AS final
WORKDIR /app
ARG run_args=""
ENV RUN_ARGS=$run_args
ENV NODE_ENV=production
ENV HUSKY=0
COPY --from=base /app/.output/ ./
EXPOSE 80
ENTRYPOINT ["sh", "-c", "node index.js $RUN_ARGS"]

This is deployed on AWS ECS where I set those env vars in the Tasks Env.

[bmoellers]

The problem is that our team doesn't get to control the build pipeline so we don't control how the docker image is created. I am looking to see what I can control but that is the problem sometimes with working in a large company. What works for others may not work based on decisions made for the whole organization.

[zllkjc]

Actually the Modern.js can also load .env file when the project is served. https://modernjs.dev/en/guides/basic-features/env-vars.html#via-env-file

In my understanding, both build-time injection and runtime injection need to exist.

For example, to inject build information at build time like version, buildType, you need to keep SSR and CSR consistent, then we can use globalVars or source.define.

And if the values ​​of RUNTIME_TYPE are different for testing, development, and A/B testing. we can use .env file.
Example of deploying Modern.js with Node.js. if we want inject the variable for test. we can create a file named .env.test and run MODERN_ENV=test PORT=3000 node .output/index, then modernjs server will auto load the env file. So we can use process.env.xxx in SSR(maybe in Data Loader).

maybe I can give a demo for you tomorrow. Or is there something unsatisfactory about this approach?

[bmoellers]

I tried your example and it works for the SSR but again the issue is that CSR then replaces it. So I did your steps in your example (except I had to remove Helmet because of an error). What I see is that the final render on the server is the process.env.API so if this was the button using that API it would use the wrong API. Yes I just crudely printed out the API. If I try to include the process.env.SERVER_API it will break because it is not available in the client package.

Everything prints fine in the console but it is the client code that I am concerned with not the server code.

[bmoellers]

So there is a way to do this at least as a page but I have to see if I can do this with our MFE setup. Your example showed me how it might be possible.

So I modified your page.data.ts to the following and passed the API in to the page:

export const loader = () => {
  console.log('API in loader: ', process.env.SERVER_API || process.env.API);
  return {
    name: 'modern.js',
    api: process.env.SERVER_API,
  };
};

Then in page.tsx (just a snippet) I added using the page loader data:

import { useLoaderData } from '@modern-js/runtime/router';
import './index.css';

const Index = () => {
  if (process.env.MODERN_TARGET === 'browser') {
    console.log('API in browser: ', process.env.API);
  } else {
    console.log('API in server: ', process.env.SERVER_API || process.env.API);
  }

  const data = useLoaderData() as any;
  console.log(data);

So this results in the console log of the data (using screenshot):

Notice that the data.api is the process.env.SERVER_API from the MODERN_TARGET=test node .output/index which is what I need. So now the server based ENV can be added to the page more dynamically. This is basically almost the same as Next JS passing it using their getServerSideProps just a different mechanic.

However will this translate over the Module Federation and MFE sharing because we are not officially loading a page and may not have access to that data but it should be a pretty simple test.

[bmoellers]

This doesn't seem to work with how we architected our MFE where we expose clients. Basically when running yarn dev so we can stand up our sandbox as well, get the error:

Uncaught Error: useLoaderData must be used within a data router.  See https://reactrouter.com/routers/picking-a-router.

Totally makes sense since we are exposing the routes in the microfront from components not routes. Not sure yet if our MFE can expose routes instead of components and still use them as components.

If I do do the modern deploy && node .output/index.js the routes work fine but there seem to be an issue with the module federation sharing of the react libraries.

The host is throwing the following error:

The remote is also throwing an error (one of many):

Not sure if that is even solvable. I don't know that I can simplify the ENV sharing for an MFE and see if I can find what is happening here using the modern deploy && node model but it doesn't look like it will work with module federation.

Note several people have tried to share how they deploy but please realize that I work for a very large company and have no control over the deployment pipeline. So this hampers me but I am also working with that team to help decide the path moving forward on their end as well.

[bmoellers]

So I fixed this issue but the problem now is that you cannot use page.data.ts and useLoaderData for micro frontend approach. It works in the routes just fine in our application, so http:localhost/health will allow the data to be loaded and injected when running via node env. However using the Module Federation MFE approach it never runs the routes, even if you share at the route level, so the loader never runs.

So love the fact it is possible if we were using the standard deployment but not going to work for us. Basically we need to do something weird like what environmentPlugin for RS Pack does pre build, post build.

So modern deploy && "process_env_replace.sh" && cp .env .output/.env && node .output/index. Or something like that. I would be nice to do this in the build process but still working out how to do it.

[zllkjc]

OK, I now fully understand your issue. You need Modern.js to inject different environment variables into the MF bundle when hosting it in different environments. This problem is actually not related to SSR/CSR.

sorry, I took a long vacation and didn't pay attention to this issue

[lanmingle]

根据我的经验，如果框架不被支持，如纯 nginx + html 或其它等等，在我以往使用 docker 以及 k8s 中，我建议这样做。

1、创建一个 json 文件，程序中读取这个 json 文件（非编译级别，基于动态加载， 如fetch等等）
2、创建一个 docker-entrypoint.sh 读取环境变量，动态写入这个 json 文件（可以采用 jq 等命令，可以参考我以下的配置）

```shell
#!/bin/bash

set -e

config_origin="/etc/nginx/conf.c"
config_target="/etc/nginx/conf.d"

# export RESOLVER_KUBERNETES_DNS="resolver kube-dns.kube-system valid=10s;"

mkdir -p $config_origin
mkdir -p $config_target

for conf in `ls ${config_origin} | grep .conf` ; do
    cp -r ${config_origin}/${conf} ${config_target}
    # envsubst '${RESOLVER_KUBERNETES_DNS}' < ${config_origin}/${conf} > ${config_target}/${conf}
    # envsubst < ${config_origin}/${conf} > ${config_target}/${conf}
done

composition_origin="/etc/nginx/conf.j"
composition_target="/usr/share/nginx/scanComposition"

mkdir -p $composition_origin
mkdir -p $composition_target

for json in `ls ${composition_origin} | grep .json` ; do
    envsubst <  ${composition_origin}/${json} > ${composition_target}/${json}
done

system_environment=$(env | jq -nR 'def parse: capture("(?<key>[^=]*)=(?<value>.*)");reduce inputs as $line ({}; ($line | parse) as $p | .[$p.key] = ($p.value) )')

echo ${system_environment} > /usr/share/nginx/html/system.environment.json

cat > /usr/share/nginx/html/system.environment.js <<EOF
(function(win){
win.systemEnvironment=${system_environment};
})(window);
EOF

exec "$@"

[bmoellers]

Because of the sensitive nature of what we are working with, we are not allowed to write the dynamic variables to the window because a bad actor could then more easily create a man in the middle attack and steal data of our end user. So taking the .env and process.env.SENSITIVE_API and using window.SENSITIVE_API is not going to be allowed from our security review.

[lanmingle]

If there is no way to solve it, you can use encryption to achieve it.

[zllkjc]

This problem seems difficult to solve alone from the framework, mainly need to container to work. In fact, the environment variables and String Replace in server, I think the @lanmingle provided the good method..

However, it seems that you cannot fully control the entire deployment process. It may have been possible to provide a Hook for developers to choose to pass static resources through and process them, but it is not currently available.

I don't know how you host the MF Bundle, if you use modern.js to host it, you can try:

1. host the js as a server route, not a static file
   first, copy the MF bundle to some dir. And then create a modern.js plugin, modify the server route ModifyServerRoutes, the entryPath should be target to the copy file
2. use middleware to modify the string in MF bundle.

[zllkjc]

If you use CDN to host the MF Bundle, it's no way..