Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Module federation #4334

Draft
wants to merge 30 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
17d28c1
WIP - first draft
GiladShoham May 18, 2021
8047bc2
first draft of federated button
GiladShoham May 20, 2021
267df71
do no use library var
GiladShoham May 20, 2021
0481059
change library to uniqeName
GiladShoham May 20, 2021
02091b0
more updates
GiladShoham May 20, 2021
f65eaf4
Merge branch 'master' into module-federation
GiladShoham May 24, 2021
eb8b917
Merge branch 'master' into module-federation
GiladShoham May 25, 2021
95ac060
first working version of component level bundling
GiladShoham May 25, 2021
7e2dd15
Merge branch 'master' of https://github.com/teambit/bit into module-f…
GiladShoham Jun 14, 2021
a5d50ca
WIP - bit start with MF
GiladShoham Jun 14, 2021
a855644
dev server is start working
GiladShoham Jun 14, 2021
a4e6ae6
Merge branch 'master' into module-federation
GiladShoham Jun 21, 2021
e3bf9a6
Merge branch 'master' into module-federation
GiladShoham Jun 23, 2021
6ea5f54
fixing merge issues
GiladShoham Jun 23, 2021
b295410
start making the overview/composition using the MF modules
GiladShoham Jun 23, 2021
02473ef
overview and composition tabs are working with MF
GiladShoham Jun 27, 2021
1dd581f
Merge branch 'master' of https://github.com/teambit/bit into module-f…
GiladShoham Jun 28, 2021
3df4db5
linting
GiladShoham Jun 29, 2021
15c500b
creating a task to bundle the link files/env itself
GiladShoham Jun 29, 2021
2e6bd67
Merge branch 'master' into module-federation
GiladShoham Jul 15, 2021
46eefa6
Merge branch 'master' into module-federation
GiladShoham Jul 18, 2021
d9ca320
compile fix
GiladShoham Jul 18, 2021
6de14e9
merge fixes
GiladShoham Jul 18, 2021
5dd119d
Merge branch 'master' into module-federation
GiladShoham Jul 19, 2021
02d0d14
Merge branch 'master' into module-federation
GiladShoham Jul 19, 2021
dddb183
Merge branch 'master' into module-federation
GiladShoham Jul 21, 2021
580d41b
split ui root file into 2 different files - core aspects and host asp…
GiladShoham Jul 21, 2021
9d6bd31
Merge branch 'master' into module-federation
GiladShoham Jul 26, 2021
e0dc3b6
import core root from file
GiladShoham Jul 26, 2021
c102480
moving create of preview server entries to the preview aspect
GiladShoham Jul 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions scopes/compilation/bundler/browser-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ import { ExecutionContext } from '@teambit/envs';

export type BrowserRuntime = {
entry: (context: ExecutionContext) => Promise<string[]>;
exposes?: (context: ExecutionContext) => Promise<Record<string, string>>;
};
9 changes: 5 additions & 4 deletions scopes/compilation/bundler/component-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,6 @@ export class ComponentServer {
// why is this here
errors?: Error[];
constructor(
/**
* browser runtime slot
*/
private pubsub: PubsubMain,

/**
Expand Down Expand Up @@ -48,9 +45,13 @@ export class ComponentServer {
return this._port;
}

set port(port: number) {
this._port = port;
}

_port: number;
async listen() {
const port = await selectPort(this.portRange);
const port = this.port ?? (await selectPort(this.portRange));
this._port = port;
const server = await this.devServer.listen(port);
const address = server.address();
Expand Down
14 changes: 14 additions & 0 deletions scopes/compilation/bundler/dev-server-context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,16 @@ export type Target = {
* output path of the target
*/
outputPath: string;

/**
* module federation namespace name
*/
mfName?: string;

/**
* module federation exposed module
*/
mfExposes?: Record<string, string>;
};

export interface BundlerContext extends BuildContext {
Expand Down Expand Up @@ -45,4 +55,8 @@ export interface DevServerContext extends ExecutionContext {
* title of the page.
*/
title?: string;

port?: number;

exposes?: Record<string, string>;
}
31 changes: 25 additions & 6 deletions scopes/compilation/bundler/dev-server.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import { ComponentServer } from './component-server';
import { DevServer } from './dev-server';
import { DevServerContext } from './dev-server-context';
import { getEntry } from './get-entry';
import { getExposes } from './get-exposes';
import { selectPort } from './select-port';

export type DevServerServiceOptions = { dedicatedEnvDevServers?: string[] };

Expand Down Expand Up @@ -56,16 +58,25 @@ export class DevServerService implements EnvService<ComponentServer> {
acc[envId] = [context];
return acc;
}, {});
const portRange = [3300, 3400];
const usedPorts: number[] = [];

const servers = await Promise.all(
Object.entries(byOriginalEnv).map(async ([id, contextList]) => {
let mainContext = contextList.find((context) => context.envDefinition.id === id);
if (!mainContext) mainContext = contextList[0];
const additionalContexts = contextList.filter((context) => context.envDefinition.id !== id);
const devServerContext = await this.buildContext(mainContext, additionalContexts);
const devServer: DevServer = await devServerContext.envRuntime.env.getDevServer(devServerContext);
this.enrichContextWithComponentsAndRelatedContext(mainContext, additionalContexts);
const envDevServerContext = await this.buildEnvServerContext(mainContext);
const envDevServer: DevServer = envDevServerContext.envRuntime.env.getDevServer(envDevServerContext);
const port = await selectPort(portRange, usedPorts);
usedPorts.push(port);
envDevServerContext.port = port;

return new ComponentServer(this.pubsub, devServerContext, [3300, 3400], devServer);
// TODO: consider change this to a new class called EnvServer
const componentServer = new ComponentServer(this.pubsub, envDevServerContext, portRange, envDevServer);
componentServer.port = port;
return componentServer;
})
);

Expand All @@ -85,18 +96,26 @@ export class DevServerService implements EnvService<ComponentServer> {
/**
* builds the execution context for the dev server.
*/
private async buildContext(
private enrichContextWithComponentsAndRelatedContext(
context: ExecutionContext,
additionalContexts: ExecutionContext[] = []
): Promise<DevServerContext> {
): void {
context.relatedContexts = additionalContexts.map((ctx) => ctx.envDefinition.id);
context.components = context.components.concat(this.getComponentsFromContexts(additionalContexts));
}

/**
* builds the execution context for the dev server.
*/
private async buildEnvServerContext(context: ExecutionContext): Promise<DevServerContext> {
const entry = await getEntry(context, this.runtimeSlot);
const exposes = await getExposes(context, this.runtimeSlot);
return Object.assign(context, {
entry: await getEntry(context, this.runtimeSlot),
entry,
// don't start with a leading "/" because it generates errors on Windows
rootPath: `preview/${context.envRuntime.id}`,
publicPath: `/public`,
exposes,
});
}
}
93 changes: 93 additions & 0 deletions scopes/compilation/bundler/env-server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import { Component } from '@teambit/component';
import { ExecutionContext } from '@teambit/envs';
import { PubsubMain } from '@teambit/pubsub';

import { AddressInfo } from 'net';

import { DevServer } from './dev-server';
import { BindError } from './exceptions';
import { EnvsServerStartedEvent } from './events';
import { BundlerAspect } from './bundler.aspect';
import { selectPort } from './select-port';

export class EnvServer {
// why is this here
errors?: Error[];
constructor(
/**
* browser runtime slot
*/
private pubsub: PubsubMain,

/**
* components contained in the existing component server.
*/
readonly context: ExecutionContext,

/**
* port range of the component server.
*/
readonly portRange: number[],

/**
* env dev server.
*/
readonly devServer: DevServer
) {}

hostname: string | undefined;

/**
* determine whether component server contains a component.
*/
hasComponent(component: Component) {
return this.context.components.find((contextComponent) => contextComponent.equals(component));
}

get port() {
return this._port;
}

_port: number;
async listen() {
const port = await selectPort(this.portRange);
this._port = port;
const server = await this.devServer.listen(port);
const address = server.address();
const hostname = this.getHostname(address);
if (!address) throw new BindError();
this.hostname = hostname;

this.pubsub.pub(BundlerAspect.id, this.cresateEnvServerStartedEvent(server, this.context, hostname, port));
}

private getHostname(address: string | AddressInfo | null) {
if (address === null) throw new BindError();
if (typeof address === 'string') return address;

let hostname = address.address;
if (hostname === '::') {
hostname = 'localhost';
}

return hostname;
}

private onChange() {}

private cresateEnvServerStartedEvent: (DevServer, ExecutionContext, string, number) => EnvsServerStartedEvent = (
envServer,
context,
hostname,
port
) => {
return new EnvsServerStartedEvent(Date.now(), envServer, context, hostname, port);
};

/**
* get the url of the component server.
*/
get url() {
return `/preview/${this.context.envRuntime.id}`;
}
}
20 changes: 20 additions & 0 deletions scopes/compilation/bundler/events/envs-server-started-event.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/* eslint-disable max-classes-per-file */

import { BitBaseEvent } from '@teambit/pubsub';

class EnvsServerStartedEventData {
constructor(readonly EnvsServer, readonly context, readonly hostname, readonly port) {}
}

export class EnvsServerStartedEvent extends BitBaseEvent<EnvsServerStartedEventData> {
static readonly TYPE = 'components-server-started';

constructor(readonly timestamp, readonly envsServer, readonly context, readonly hostname, readonly port) {
super(
EnvsServerStartedEvent.TYPE,
'0.0.1',
timestamp,
new EnvsServerStartedEventData(envsServer, context, hostname, port)
);
}
}
1 change: 1 addition & 0 deletions scopes/compilation/bundler/events/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export * from './components-server-started-event';
export * from './envs-server-started-event';
24 changes: 24 additions & 0 deletions scopes/compilation/bundler/get-exposes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { ExecutionContext } from '@teambit/envs';
import { BrowserRuntimeSlot } from './bundler.main.runtime';

/**
* computes the bundler entry.
*/
export async function getExposes(
context: ExecutionContext,
runtimeSlot: BrowserRuntimeSlot
): Promise<Record<string, string>> {
// TODO: refactor this away from here and use computePaths instead
const slotEntries = await Promise.all(
runtimeSlot.values().map(async (browserRuntime) => browserRuntime.exposes?.(context))
);

const exposes = slotEntries.reduce((acc, current) => {
if (current) {
acc = Object.assign(acc, current);
}
return acc;
}, {});

return exposes || {};
}
4 changes: 2 additions & 2 deletions scopes/compilation/bundler/select-port.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ import { Port } from '@teambit/toolbox.network.get-port';
/**
* get an available port between range 3000 to 3200 or from port range
*/
export async function selectPort(range: number[] | number): Promise<number> {
return Port.getPortFromRange(range);
export async function selectPort(range: number[] | number, usedPorts?: number[]): Promise<number> {
return Port.getPortFromRange(range, usedPorts);
}
5 changes: 5 additions & 0 deletions scopes/component/component/component-factory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ export interface ComponentFactory {
*/
hasIdNested(componentId: ComponentID, includeCache?: boolean): Promise<boolean>;

/**
* Get temp dir for the host
*/
getTempDir(id: string): string;

/**
* determine whether host should be the prior one in case multiple hosts persist.
*/
Expand Down
14 changes: 10 additions & 4 deletions scopes/compositions/compositions/compositions.preview.runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,26 @@ export class CompositionsPreview {
private preview: PreviewPreview
) {}

render(componentId: string, modules: PreviewModule, otherPreviewDefs, context: RenderingContext) {
async render(componentId: string, modules: PreviewModule, otherPreviewDefs, context: RenderingContext) {
console.log('im in render of composition');
if (!modules.componentMap[componentId]) return;

const compositions = this.selectPreviewModel(componentId, modules);
const compositions = await this.selectPreviewModel(componentId, modules);
const active = this.getActiveComposition(compositions);

modules.mainModule.default(active, context);
}

/** gets relevant information for this preview to render */
selectPreviewModel(componentId: string, previewModule: PreviewModule) {
const files = previewModule.componentMap[componentId] || [];
async selectPreviewModel(componentId: string, previewModule: PreviewModule) {
// const files = (await previewModule.componentMap[componentId]()) || [];
const allFunc = previewModule.componentMap[componentId];
const promises = allFunc.map((func) => func());
const files = await Promise.all(promises);
console.log('selectPreviewModel', files);

// allow compositions to come from many files. It is assumed they will have unique named
// const combined = Object.assign({}, ...files);
const combined = Object.assign({}, ...files);
return combined;
}
Expand Down
9 changes: 6 additions & 3 deletions scopes/docs/docs/docs.preview.runtime.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,17 @@ export class DocsPreview {
private preview: PreviewPreview
) {}

render = (componentId: string, modules: PreviewModule, [compositions]: [any], context: RenderingContext) => {
render = async (componentId: string, modules: PreviewModule, [compositions]: [any], context: RenderingContext) => {
const docsModule = this.selectPreviewModel(componentId, modules);

modules.mainModule.default(NoopProvider, componentId, docsModule, compositions, context);
};

selectPreviewModel(componentId: string, modules: PreviewModule) {
const relevant = modules.componentMap[componentId];
async selectPreviewModel(componentId: string, modules: PreviewModule) {
// const relevant = modules.componentMap[componentId];
const allFunc = modules.componentMap[componentId];
const relevant = await allFunc[0]();

if (!relevant) return undefined;

// only one doc file is supported.
Expand Down
15 changes: 15 additions & 0 deletions scopes/envs/envs/environments.main.runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { Component, ComponentAspect, ComponentMain, ComponentID, AspectData } fr
import { GraphqlAspect, GraphqlMain } from '@teambit/graphql';
import { Harmony, Slot, SlotRegistry } from '@teambit/harmony';
import { Logger, LoggerAspect, LoggerMain } from '@teambit/logger';
import { flatten } from 'lodash';
import { ExtensionDataList, ExtensionDataEntry } from '@teambit/legacy/dist/consumer/config/extension-data';
import findDuplications from '@teambit/legacy/dist/utils/array/find-duplications';
import { EnvService } from './services';
Expand Down Expand Up @@ -81,6 +82,20 @@ export class EnvsMain {
return this.createRuntime(components);
}

/**
* list all registered envs.
*/
listEnvs(): Environment[] {
return flatten(this.envSlot.values());
}

/**
* list all registered envs ids.
*/
listEnvsIds(): string[] {
return this.envSlot.toArray().map(([envId]) => envId);
}

/**
* get the configured default env.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,10 @@ export default class DuplicateDependencies implements Insight {
return formatted;
}

private getDependents(
priorVersions: VersionSubgraph[]
): { totalOutdatedDependents: number; dependentsByVersion: VersionWithDependents[] } {
private getDependents(priorVersions: VersionSubgraph[]): {
totalOutdatedDependents: number;
dependentsByVersion: VersionWithDependents[];
} {
let totalOutdatedDependents = 0;
const dependentsByVersion: VersionWithDependents[] = [];
priorVersions.forEach((pVersion: VersionSubgraph) => {
Expand Down
Loading