Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: custom root certificate support #19

Merged
merged 45 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
0badd79
feat: setup basic dataset versions
DaniAkash Apr 8, 2024
e5bced7
feat: setup concurrent annotation upload
DaniAkash Apr 9, 2024
31d1827
feat: setup datasets base class
DaniAkash Apr 9, 2024
3a192c5
feat: setup image classification dataset classes
DaniAkash Apr 10, 2024
ce8e305
feat: setup bulk upload utils
DaniAkash Apr 12, 2024
4ddc80b
feat: setup bulkupload
DaniAkash Apr 12, 2024
1daefde
feat: upload from folder method
DaniAkash Apr 12, 2024
911ea52
feat: upload from csv
DaniAkash Apr 12, 2024
1f3eb9a
fix: search test cases
DaniAkash Apr 15, 2024
bb12468
fix: lint issues
DaniAkash Apr 18, 2024
07c1474
feat: added algorithm to search
DaniAkash Apr 19, 2024
dfa9655
fix: test cases
DaniAkash Apr 22, 2024
cc27f48
feat: setup tests for search
DaniAkash Apr 23, 2024
4a3376e
chore: fix lint issues
DaniAkash Apr 23, 2024
b9b03fa
fix: failing tests
DaniAkash Apr 23, 2024
7916a07
chore: upgrade grpc module
DaniAkash Apr 23, 2024
2f732dc
chore: replace deprecated uuid package
DaniAkash Apr 23, 2024
7d7750d
chore: setup base rag class
DaniAkash Apr 24, 2024
1cad4c6
feat: setup llma powered utilities
DaniAkash Apr 24, 2024
41ffd5e
feat: setup chat method
DaniAkash Apr 24, 2024
0dad81b
fix: lint issues
DaniAkash Apr 24, 2024
23574af
feat: setup rag with tests
DaniAkash Apr 29, 2024
d5a53d6
fix: failing tests with url helper
DaniAkash Apr 29, 2024
c081079
fix: lint issues
DaniAkash Apr 30, 2024
fd1c824
fix: addressed review comments
DaniAkash May 2, 2024
a95725d
feat: use event emitter for bulk upload progress
DaniAkash May 2, 2024
350171a
chore: fix lint issues
DaniAkash May 2, 2024
e1efb8a
Merge branch 'dani/datasets' into dani/rag
DaniAkash May 2, 2024
50c5905
refactor: use array notation for polygon type
DaniAkash May 2, 2024
84ddcd3
Merge branch 'main' of https://github.com/Clarifai/clarifai-nodejs in…
DaniAkash May 2, 2024
f2de443
chore: remove unused server state management code
DaniAkash May 3, 2024
1e59898
fix: increase client state test timeout
DaniAkash May 3, 2024
3948e4e
fix: lint issues
DaniAkash May 3, 2024
575d6fc
fix: based on doc examples
DaniAkash May 13, 2024
5f7ef91
fix: add missing external dependency
DaniAkash May 13, 2024
0ce4489
fix: updated model test cases
DaniAkash May 13, 2024
9a0dd12
test: added more model test case
DaniAkash May 13, 2024
cdd8dab
feat: added test case for multimodal input
DaniAkash May 14, 2024
d87c051
chore: set default search metric to euclidean
DaniAkash May 15, 2024
6384f07
fix: constructors
DaniAkash Jun 20, 2024
904e6b8
Merge branch 'dani/rag' of https://github.com/Clarifai/clarifai-nodej…
DaniAkash Jun 20, 2024
942efde
feat: support custom root certificates path for authconfig
DaniAkash Jun 26, 2024
e58d799
Merge branch 'main' into dani/custom-root-ca
DaniAkash Jul 2, 2024
5a2ce41
fix: change rag baseworkflow
DaniAkash Jul 2, 2024
d6ab393
fix: revert base workflow
DaniAkash Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion src/client/auth/helper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import resources_pb2 from "clarifai-nodejs-grpc/proto/clarifai/api/resources_pb"
import { grpc } from "clarifai-nodejs-grpc";
import { V2Client } from "clarifai-nodejs-grpc/proto/clarifai/api/service_grpc_pb";
import process from "process";
import fs from "fs";

// TypeScript interface for the cache
export interface Cache {
Expand Down Expand Up @@ -75,6 +76,7 @@ export class ClarifaiAuthHelper {
private token: string;
private _base: string;
private _ui: string;
private _rootCertificatesPath: string;

/**
* A helper to get the authorization information needed to make API calls with the grpc
Expand All @@ -97,6 +99,7 @@ export class ClarifaiAuthHelper {
* https://clarifai.com (default), https://host:port, http://host:port,
* host:port (will be treated as http, not https). It's highly recommended to include
* the http:// or https:// otherwise we need to check the endpoint to determine if it has SSL during this __init__.
* @param rootCertificatesPath - path to the root certificates file. This is only used for grpc secure channels.
* @param validate - Whether to validate the inputs. This is useful for overriding vars then validating.
*/
constructor(
Expand All @@ -106,12 +109,14 @@ export class ClarifaiAuthHelper {
token: string = "",
base: string = DEFAULT_BASE,
ui: string = DEFAULT_UI,
rootCertificatesPath: string = "",
validate: boolean = true,
) {
this.userId = userId;
this.appId = appId;
this._pat = pat;
this.token = token;
this._rootCertificatesPath = rootCertificatesPath;
this._base = base;
this._ui = ui;

Expand Down Expand Up @@ -142,6 +147,13 @@ export class ClarifaiAuthHelper {
throw new Error(
"Need 'pat' or 'token' in the query params or use one of the CLARIFAI_PAT or CLARIFAI_SESSION_TOKEN env vars",
);
} else if (
this._rootCertificatesPath &&
!fs.existsSync(this._rootCertificatesPath)
) {
throw new Error(
`Root certificates path ${this._rootCertificatesPath} does not exist`,
);
}
}

Expand All @@ -163,6 +175,8 @@ export class ClarifaiAuthHelper {
const pat = process.env.CLARIFAI_PAT || "";
const base = process.env.CLARIFAI_API_BASE || DEFAULT_BASE;
const ui = process.env.CLARIFAI_UI || DEFAULT_UI;
const rootCertificatesPath =
process.env.CLARIFAI_ROOT_CERTIFICATES_PATH || "";

return new ClarifaiAuthHelper(
userId,
Expand All @@ -171,6 +185,7 @@ export class ClarifaiAuthHelper {
token,
base,
ui,
rootCertificatesPath,
validate,
);
}
Expand Down Expand Up @@ -227,7 +242,16 @@ export class ClarifaiAuthHelper {
let client: V2Client;

if (https) {
client = new V2Client(this._base, grpc.ChannelCredentials.createSsl());
if (this._rootCertificatesPath) {
client = new V2Client(
this._base,
grpc.ChannelCredentials.createSsl(
fs.readFileSync(this._rootCertificatesPath),
),
);
} else {
client = new V2Client(this._base, grpc.ChannelCredentials.createSsl());
}
} else {
let host: string;
let port: number = 80;
Expand Down Expand Up @@ -282,6 +306,13 @@ export class ClarifaiAuthHelper {
this._ui = httpsCache(uiHttpsCache, ui);
}

/**
* Return the root certificates path.
*/
get rootCertificatesPath(): string {
return this._rootCertificatesPath;
}

/**
* Return the base domain for the API.
*/
Expand Down
4 changes: 4 additions & 0 deletions src/client/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ export class BaseClient {
protected pat: string;
protected userAppId: UserAppIDSet;
protected base: string;
protected rootCertificatesPath: string;

/**
* Constructs a new BaseClient instance with specified configuration options.
Expand All @@ -39,6 +40,7 @@ export class BaseClient {
* @param {string} [authConfig.token] An optional token for authentication.
* @param {string} [authConfig.base='https://api.clarifai.com'] The base URL for the API endpoint. Defaults to 'https://api.clarifai.com'.
* @param {string} [authConfig.ui='https://clarifai.com'] The URL for the UI. Defaults to 'https://clarifai.com'.
* @param {string} [authConfig.rootCertificatesPath] Path to the SSL root certificates file, used to establish secure gRPC connections.
*/
constructor(authConfig: AuthConfig = {}) {
const pat = getFromDictOrEnv("pat", "CLARIFAI_PAT", authConfig);
Expand All @@ -52,6 +54,7 @@ export class BaseClient {
authConfig.token,
authConfig.base,
authConfig.ui,
authConfig.rootCertificatesPath,
false,
)
: ClarifaiAuthHelper.fromEnv(false); // The validate parameter is set to false explicitly
Expand All @@ -60,6 +63,7 @@ export class BaseClient {
this.pat = this.authHelper.pat;
this.userAppId = this.authHelper.getUserAppIdProto();
this.base = this.authHelper.base;
this.rootCertificatesPath = this.authHelper.rootCertificatesPath;
}

/**
Expand Down
1 change: 1 addition & 0 deletions src/utils/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ export type AuthConfig =
userId: string;
appId: string;
pat: string;
rootCertificatesPath?: string;
token?: string;
base?: string;
ui?: string;
Expand Down
15 changes: 15 additions & 0 deletions tests/client/auth/helper.unit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,21 @@ describe("ClarifaiAuthHelper", () => {
expect(() => ClarifaiAuthHelper.validateSecretsDict(secrets)).toThrow();
});

it("throws error for invalid root certificates path", () => {
expect(
() =>
new ClarifaiAuthHelper(
"userId",
"appId",
"pat",
"",
"https://customdomain.com",
"https://customdomain.com/ui",
"invalid",
),
).toThrow(`Root certificates path invalid does not exist`);
});

it("getUserAppIdProto returns correct UserAppIDSet proto object", () => {
const helper = new ClarifaiAuthHelper("userId", "appId", "pat");
const userAppIdProto = helper.getUserAppIdProto();
Expand Down
Loading