Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: dynamic plugin imports #1383

Open
wants to merge 26 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
9d967c8
dynamic import plugins
ChristopherTrimboli Dec 22, 2024
cb945d2
Merge branch 'develop' into chore/dynamic-plugins
ChristopherTrimboli Dec 22, 2024
8e86d66
loadPlugin abstracted function
ChristopherTrimboli Dec 23, 2024
0747012
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 23, 2024
e510db6
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 23, 2024
2efee64
change to callback import param
ChristopherTrimboli Dec 23, 2024
c5d4dc4
edit dev.sh plugin instructions
ChristopherTrimboli Dec 23, 2024
af11d77
format lint
ChristopherTrimboli Dec 23, 2024
feba542
strict Plugin type imports
ChristopherTrimboli Dec 23, 2024
70c207a
fix default plugin goat usage
ChristopherTrimboli Dec 23, 2024
23cb080
loadPlugins, DynamicPlugin type, core/plugins helpers
ChristopherTrimboli Dec 23, 2024
12d8537
nuke Goat plugin
ChristopherTrimboli Dec 23, 2024
7b2a150
tune up forgetten plugins
ChristopherTrimboli Dec 23, 2024
2199949
remove JS docs
ChristopherTrimboli Dec 23, 2024
0726718
promise all loadPlugins
ChristopherTrimboli Dec 23, 2024
b243fb4
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 24, 2024
98c6a89
remove TEEMode import
ChristopherTrimboli Dec 24, 2024
32c7e5a
import cache to solve multiple imports
ChristopherTrimboli Dec 26, 2024
fb8dda4
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 26, 2024
217754b
remove junk function
ChristopherTrimboli Dec 26, 2024
5896700
add abstract plugin back, regen lockfile
ChristopherTrimboli Dec 26, 2024
430dab1
consolidate same secret coinbase plugins
ChristopherTrimboli Dec 26, 2024
4f90663
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 26, 2024
d22a710
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 26, 2024
d77f058
add chronos plugin back
ChristopherTrimboli Dec 26, 2024
df634eb
Merge remote-tracking branch 'upstream/develop' into chore/dynamic-pl…
ChristopherTrimboli Dec 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 133 additions & 105 deletions agent/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,36 +25,14 @@ import {
stringToUuid,
validateCharacterConfig,
CacheStore,
getSecret,
loadPlugins,
} from "@elizaos/core";
import { RedisClient } from "@elizaos/adapter-redis";
import { zgPlugin } from "@elizaos/plugin-0g";
import { bootstrapPlugin } from "@elizaos/plugin-bootstrap";
import createGoatPlugin from "@elizaos/plugin-goat";
// import { intifacePlugin } from "@elizaos/plugin-intiface";
import { DirectClient } from "@elizaos/client-direct";
import { aptosPlugin } from "@elizaos/plugin-aptos";
import {
advancedTradePlugin,
coinbaseCommercePlugin,
coinbaseMassPaymentsPlugin,
tokenContractPlugin,
tradePlugin,
webhookPlugin,
} from "@elizaos/plugin-coinbase";
import { confluxPlugin } from "@elizaos/plugin-conflux";
import { evmPlugin } from "@elizaos/plugin-evm";
import { storyPlugin } from "@elizaos/plugin-story";
import { flowPlugin } from "@elizaos/plugin-flow";
import { imageGenerationPlugin } from "@elizaos/plugin-image-generation";
import { multiversxPlugin } from "@elizaos/plugin-multiversx";
import { nearPlugin } from "@elizaos/plugin-near";
import { nftGenerationPlugin } from "@elizaos/plugin-nft-generation";
import { createNodePlugin } from "@elizaos/plugin-node";
import { solanaPlugin } from "@elizaos/plugin-solana";
import { suiPlugin } from "@elizaos/plugin-sui";
import { TEEMode, teePlugin } from "@elizaos/plugin-tee";
import { tonPlugin } from "@elizaos/plugin-ton";
import { zksyncEraPlugin } from "@elizaos/plugin-zksync-era";
import { TEEMode } from "@elizaos/plugin-tee";
ChristopherTrimboli marked this conversation as resolved.
Show resolved Hide resolved
import Database from "better-sqlite3";
import fs from "fs";
import path from "path";
Expand Down Expand Up @@ -457,10 +435,6 @@ function isFalsish(input: any): boolean {
return falsishValues.includes(value.trim().toLowerCase());
}

function getSecret(character: Character, secret: string) {
return character.settings?.secrets?.[secret] || process.env[secret];
}

let nodePlugin: any | undefined;

export async function createAgent(
Expand Down Expand Up @@ -488,12 +462,133 @@ export async function createAgent(
throw new Error("Invalid TEE configuration");
}

let goatPlugin: any | undefined;
if (getSecret(character, "ALCHEMY_API_KEY")) {
goatPlugin = await createGoatPlugin((secret) =>
getSecret(character, secret)
);
}
const dynamicPlugins = await loadPlugins(character, [
{
secrets: ["COINBASE_API_KEY", "COINBASE_PRIVATE_KEY"],
importFn: () =>
import("@elizaos/plugin-coinbase").then(
(m) => m.coinbaseMassPaymentsPlugin
),
},
{
secrets: ["COINBASE_API_KEY", "COINBASE_PRIVATE_KEY"],
importFn: () =>
import("@elizaos/plugin-coinbase").then((m) => m.tradePlugin),
},
{
secrets: ["COINBASE_API_KEY", "COINBASE_PRIVATE_KEY"],
importFn: () =>
import("@elizaos/plugin-coinbase").then(
(m) => m.tokenContractPlugin
),
},
{
secrets: ["COINBASE_API_KEY", "COINBASE_PRIVATE_KEY"],
importFn: () =>
import("@elizaos/plugin-coinbase").then(
(m) => m.advancedTradePlugin
),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking a helper function (my idea of load plugins) which is just import('@elizaos/plugin-coinbase').then((mod) => [mod.pluginA, modPluginB, modPluginC]). Then we don't need to import coinbase 3 times.

Copy link
Contributor

@ryanleecode ryanleecode Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And then to add them you can just spread:

plugins: [
...myCoinbasePluginsDynamicallyImportedOrNull
].filter(Boolean)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok this final point I think, taking break, will work on that, we getting close 👍🏻

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solved importCache I believe: 32c7e5a

Main issue was handling unique secrets, I do like the simple statements of secrets and importFn, now if duplicate importFn, it uses a cache, so won't import duplicates, but DX is much nicer else would have big nested ternaries and if/else stuffed in array with secrets spread.

If have better style lemme know.

Copy link
Contributor Author

@ChristopherTrimboli ChristopherTrimboli Dec 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case that secrets are exact same, uses your suggestion, as array to import: 430dab1

But case still stands I think best separated if secrets are unique, for webhook, and others.
https://github.com/elizaOS/eliza/pull/1383/files#diff-935219608f7b5ca6c8b8548cfdce88c7d3cdb6bb6d9f9d8df644b364f6557e4eR437-R462

},
{
secrets: "CONFLUX_CORE_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-conflux").then((m) => m.confluxPlugin),
},
{
secrets: "SOLANA_PUBLIC_KEY",
importFn: () =>
import("@elizaos/plugin-solana").then((m) => m.solanaPlugin),
},
{
secrets: [
"NEAR_ADDRESS",
"NEAR_WALLET_PUBLIC_KEY",
"NEAR_WALLET_SECRET_KEY",
],
importFn: () =>
import("@elizaos/plugin-near").then((m) => m.nearPlugin),
},
{
secrets: "EVM_PUBLIC_KEY",
importFn: () =>
import("@ai16z/plugin-evm").then((m) => m.evmPlugin),
},
{
secrets: [
"SOLANA_PUBLIC_KEY",
"SOLANA_ADMIN_PUBLIC_KEY",
"SOLANA_PRIVATE_KEY",
"SOLANA_ADMIN_PRIVATE_KEY",
],
importFn: () =>
import("@elizaos/plugin-nft-generation").then(
(m) => m.nftGenerationPlugin
),
},
{
secrets: "ZEROG_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-0g").then((m) => m.zgPlugin),
},
{
secrets: "COINBASE_COMMERCE_KEY",
importFn: () =>
import("@elizaos/plugin-coinbase").then(
(m) => m.coinbaseCommercePlugin
),
},
{
secrets: [
"FAL_API_KEY",
"OPENAI_API_KEY",
"VENICE_API_KEY",
"HEURIST_API_KEY",
],
importFn: () =>
import("@elizaos/plugin-image-generation").then(
(m) => m.imageGenerationPlugin
),
},
{
secrets: ["FLOW_ADDRESS"],
importFn: () =>
import("@elizaos/plugin-flow").then((m) => m.flowPlugin),
},
{
secrets: "APTOS_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-aptos").then((m) => m.aptosPlugin),
},
{
secrets: "MVX_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-multiversx").then(
(m) => m.multiversxPlugin
),
},
{
secrets: "ZKSYNC_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-zksync-era").then(
(m) => m.zksyncEraPlugin
),
},
{
secrets: "TON_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-ton").then((m) => m.tonPlugin),
},
{
secrets: "SUI_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-sui").then((m) => m.suiPlugin),
},
{
secrets: "STORY_PRIVATE_KEY",
importFn: () =>
import("@elizaos/plugin-story").then((m) => m.storyPlugin),
},
]);

return new AgentRuntime({
databaseAdapter: db,
Expand All @@ -502,76 +597,9 @@ export async function createAgent(
evaluators: [],
character,
// character.plugins are handled when clients are added
plugins: [
bootstrapPlugin,
getSecret(character, "CONFLUX_CORE_PRIVATE_KEY")
? confluxPlugin
: null,
nodePlugin,
getSecret(character, "SOLANA_PUBLIC_KEY") ||
(getSecret(character, "WALLET_PUBLIC_KEY") &&
!getSecret(character, "WALLET_PUBLIC_KEY")?.startsWith("0x"))
? solanaPlugin
: null,
(getSecret(character, "NEAR_ADDRESS") ||
getSecret(character, "NEAR_WALLET_PUBLIC_KEY")) &&
getSecret(character, "NEAR_WALLET_SECRET_KEY")
? nearPlugin
: null,
getSecret(character, "EVM_PUBLIC_KEY") ||
(getSecret(character, "WALLET_PUBLIC_KEY") &&
getSecret(character, "WALLET_PUBLIC_KEY")?.startsWith("0x"))
? evmPlugin
: null,
(getSecret(character, "SOLANA_PUBLIC_KEY") ||
(getSecret(character, "WALLET_PUBLIC_KEY") &&
!getSecret(character, "WALLET_PUBLIC_KEY")?.startsWith(
"0x"
))) &&
getSecret(character, "SOLANA_ADMIN_PUBLIC_KEY") &&
getSecret(character, "SOLANA_PRIVATE_KEY") &&
getSecret(character, "SOLANA_ADMIN_PRIVATE_KEY")
? nftGenerationPlugin
: null,
getSecret(character, "ZEROG_PRIVATE_KEY") ? zgPlugin : null,
getSecret(character, "COINBASE_COMMERCE_KEY")
? coinbaseCommercePlugin
: null,
getSecret(character, "FAL_API_KEY") ||
getSecret(character, "OPENAI_API_KEY") ||
getSecret(character, "VENICE_API_KEY") ||
getSecret(character, "HEURIST_API_KEY")
? imageGenerationPlugin
: null,
...(getSecret(character, "COINBASE_API_KEY") &&
getSecret(character, "COINBASE_PRIVATE_KEY")
? [
coinbaseMassPaymentsPlugin,
tradePlugin,
tokenContractPlugin,
advancedTradePlugin,
]
: []),
...(teeMode !== TEEMode.OFF && walletSecretSalt
? [teePlugin, solanaPlugin]
: []),
getSecret(character, "COINBASE_API_KEY") &&
getSecret(character, "COINBASE_PRIVATE_KEY") &&
getSecret(character, "COINBASE_NOTIFICATION_URI")
? webhookPlugin
: null,
getSecret(character, "ALCHEMY_API_KEY") ? goatPlugin : null,
getSecret(character, "FLOW_ADDRESS") &&
getSecret(character, "FLOW_PRIVATE_KEY")
? flowPlugin
: null,
getSecret(character, "APTOS_PRIVATE_KEY") ? aptosPlugin : null,
getSecret(character, "MVX_PRIVATE_KEY") ? multiversxPlugin : null,
getSecret(character, "ZKSYNC_PRIVATE_KEY") ? zksyncEraPlugin : null,
getSecret(character, "TON_PRIVATE_KEY") ? tonPlugin : null,
getSecret(character, "SUI_PRIVATE_KEY") ? suiPlugin : null,
getSecret(character, "STORY_PRIVATE_KEY") ? storyPlugin : null,
].filter(Boolean),
plugins: [bootstrapPlugin, nodePlugin, ...dynamicPlugins].filter(
Boolean
),
providers: [],
actions: [],
services: [],
Expand Down
1 change: 1 addition & 0 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ export * from "./parsing.ts";
export * from "./uuid.ts";
export * from "./environment.ts";
export * from "./cache.ts";
export * from "./plugins.ts";
export { default as knowledge } from "./knowledge.ts";
export * from "./utils.ts";
50 changes: 50 additions & 0 deletions packages/core/src/plugins.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import { Character, DynamicPlugin, Plugin } from "./types";

/**
* Gets a secret from the character settings or environment variables
* @param {Character} character - The character object
* @param {string} secret - The secret key
* @returns {string|undefined} The secret value or undefined if not found
*/
ChristopherTrimboli marked this conversation as resolved.
Show resolved Hide resolved
export function getSecret(
character: Character,
secret: string
): string | undefined {
return character.settings?.secrets?.[secret] || process.env[secret];
ChristopherTrimboli marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Loads a plugin if all secrets are available
* @param {string|string[]} keys - The secret keys required for the plugin
* @param {Character} character - The character object
* @param {() => Promise<Plugin>} importFn - The function to import the plugin
* @returns {Promise<Plugin|null>} The plugin or null if secrets are missing
*/
export async function loadPlugin(
keys: string | string[],
character: Character,
importFn: () => Promise<Plugin>
): Promise<Plugin | null> {
const keyArray = Array.isArray(keys) ? keys : [keys];
const hasAllSecrets = keyArray.every((key) => getSecret(character, key));
if (!hasAllSecrets) return null;
return importFn();
}

/**
* Loads multiple plugins if all secrets are available
* @param {Character} character - The character object
* @param {DynamicPlugin[]} definitions - The plugin definitions
* @returns {Promise<Plugin[]>} The loaded plugins
*/
export async function loadPlugins(
character: Character,
definitions: DynamicPlugin[]
): Promise<Plugin[]> {
const plugins: Plugin[] = [];
for (const def of definitions) {
const p = await loadPlugin(def.secrets, character, def.importFn);
if (p) plugins.push(p);
}
return plugins;
ChristopherTrimboli marked this conversation as resolved.
Show resolved Hide resolved
}
5 changes: 5 additions & 0 deletions packages/core/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,11 @@ export type Plugin = {
clients?: Client[];
};

export interface DynamicPlugin {
secrets: string | string[];
importFn: () => Promise<Plugin>;
}

/**
* Available client platforms
*/
Expand Down
24 changes: 11 additions & 13 deletions scripts/dev.sh
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,17 @@ cat << "EOF"
* *
* "@elizaos/your-plugin-name": "workspace:*" *
* *
* 5. Edit the 'index.ts' file in 'agent/src': *
* *
* a. Import your plugin: *
* *
* import yourPlugin from '@elizaos/your-plugin-name'; *
* *
* b. Add your plugin to the `plugins` array: *
* *
* const plugins = [ *
* existingPlugin, *
* yourPlugin, *
* ]; *
* *
* 5. Edit the 'index.ts' file in 'agent/src': *
* *
* a. Use the `loadPlugins` function to dynamically import your plugin: *
* *
* { *
* secrets: ["YOUR_PLUGIN_SECRET_KEY"], *
* importFn: () => import("path/to/your-plugin").then((m) => m.yourPluginExport), *
* }, *
* *
* b. Add this object to the array passed to the `loadPlugins` function. *
* *
* This will ensure that your plugin's development server runs *
* alongside others when you execute this script. *
***********************************************************************
Expand Down
Loading