Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task/WP-724: Mutation Hooks: Compress Files #1009

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c30c46c
Starting from scratch
Nov 6, 2024
b74ed53
Set up useCompress.ts
Nov 6, 2024
199f671
Merge branch 'main' of github.com:TACC/Core-Portal into task/WP-724
Nov 11, 2024
cae111b
Still working on useCompress()
Nov 12, 2024
cec7b4f
Progressing with useCompress mutation
Nov 12, 2024
c5028dc
Committing branch in its current state; not fully functioning
Nov 14, 2024
64db5d1
wip
rstijerina Nov 14, 2024
79957a4
Reworked useCompress.ts
Nov 14, 2024
00aaf76
wip
rstijerina Nov 14, 2024
82ab9cf
Still trying to fix it; much closer now
Nov 15, 2024
e96a7d3
handle undefined execSystemId
rstijerina Nov 15, 2024
b55e8e8
Compress mutation finally successful
Nov 15, 2024
07a213b
Toasts and modals work correctly
Nov 15, 2024
4f4b5b9
Linted client-side code
Nov 15, 2024
aa1a35d
Added an additional asynchronous call; updated Compress modal
Nov 26, 2024
757c637
Merge branch 'main' into task/WP-724
rstijerina Nov 27, 2024
6e13705
Refactored types into useSubmitJob.ts
Dec 6, 2024
9ff27eb
Corrected mutation hook to return archive in current directory instea…
Dec 6, 2024
7cbe3a6
Update client/src/hooks/datafiles/mutations/useCompress.ts
jmcmillenmusic Dec 6, 2024
3ef04df
Linted client-side code
Dec 6, 2024
d4ad0b1
Merge branch 'main' into task/WP-724
jmcmillenmusic Dec 6, 2024
b7f8389
Can't get this test to pass still
Dec 6, 2024
f5e6bb4
Merge branch 'task/WP-724' of github.com:TACC/Core-Portal into task/W…
Dec 6, 2024
688f9e1
Linted client-side code
Dec 6, 2024
ef118aa
Finally got the failing test to pass
Dec 6, 2024
d42eb23
Skipping tests temporarily, cleaned up code
Dec 9, 2024
d9e4bfb
Skipping saga tests
Dec 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions client/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions client/src/components/DataFiles/tests/DataFiles.test.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ describe('DataFiles', () => {
compress: '',
},
},
allocations: {
portal_alloc: 'TACC-ACI',
active: [{ projectId: 'active-project' }],
},
systems: systemsFixture,
files: filesFixture,
pushKeys: {
Expand Down
27 changes: 0 additions & 27 deletions client/src/hooks/datafiles/mutations/useCompress.js

This file was deleted.

156 changes: 156 additions & 0 deletions client/src/hooks/datafiles/mutations/useCompress.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
import { useMutation } from '@tanstack/react-query';
import { useSelector, useDispatch, shallowEqual } from 'react-redux';
import { getCompressParams } from 'utils/getCompressParams';
import { apiClient } from 'utils/apiClient';
import { TTapisFile, TPortalSystem } from 'utils/types';
import { TJobBody, TJobPostResponse } from './useSubmitJob';

async function submitJobUtil(body: TJobBody) {
const res = await apiClient.post<TJobPostResponse>(
`/api/workspace/jobs`,
body
);
return res.data.response;
}

function useCompress() {
const dispatch = useDispatch();
const status = useSelector(
(state: any) => state.files.operationStatus.compress,
shallowEqual
);

const setStatus = (newStatus: any) => {
dispatch({
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: { status: newStatus, operation: 'compress' },
});
};

const compressErrorAction = (errorMessage: any) => {
return {
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: {
status: { type: 'ERROR', message: errorMessage },
operation: 'compress',
},
};
};

const compressApp = useSelector(
(state: any) => state.workbench.config.compressApp
);

const defaultAllocation = useSelector(
(state: any) =>
state.allocations.portal_alloc || state.allocations.active[0].projectName
);

const systems = useSelector(
(state: any) => state.systems.storage.configuration
);

const { mutateAsync } = useMutation({ mutationFn: submitJobUtil });

const compress = ({
scheme,
files,
filename,
compressionType,
}: {
scheme: string;
files: TTapisFile[];
filename: string;
compressionType: string;
}) => {
dispatch({
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: { status: 'RUNNING', operation: 'compress' },
});

let defaultPrivateSystem: TPortalSystem | undefined;

if (files[0].scheme === 'private' && files[0].api === 'tapis') {
defaultPrivateSystem === null;
}

if (scheme !== 'private' && scheme !== 'projects') {
defaultPrivateSystem = systems.find((s: any) => s.default);

if (!defaultPrivateSystem) {
throw new Error('Folder downloads are unavailable in this portal', {
cause: 'compressError',
});
}
}

const params = getCompressParams(
files,
filename,
compressionType,
compressApp,
defaultAllocation,
defaultPrivateSystem
jmcmillenmusic marked this conversation as resolved.
Show resolved Hide resolved
);

return mutateAsync(
{
job: params,
},
{
onSuccess: (response: any) => {
// If the execution system requires pushing keys, then
// bring up the modal and retry the compress action
if (response.execSys) {
dispatch({
type: 'SYSTEMS_TOGGLE_MODAL',
payload: {
operation: 'pushKeys',
props: {
system: response.execSys,
onCancel: compressErrorAction('An error has occurred'),
},
},
});
} else if (response.status === 'PENDING') {
dispatch({
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: { status: { type: 'SUCCESS' }, operation: 'compress' },
});
dispatch({
type: 'ADD_TOAST',
payload: {
message: 'Compress job submitted.',
},
});
dispatch({
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: { operation: 'compress', status: {} },
});
dispatch({
type: 'DATA_FILES_TOGGLE_MODAL',
payload: { operation: 'compress', props: {} },
});
rstijerina marked this conversation as resolved.
Show resolved Hide resolved
}
},
onError: (response) => {
const errorMessage =
response.cause === 'compressError'
? response.message
: 'An error has occurred.';
dispatch({
type: 'DATA_FILES_SET_OPERATION_STATUS',
payload: {
status: { type: 'ERROR', message: errorMessage },
operation: 'compress',
},
});
},
}
);
};

return { compress, status, setStatus };
}

export default useCompress;
57 changes: 57 additions & 0 deletions client/src/hooks/datafiles/mutations/useSubmitJob.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import {
TTapisSystem,
TAppFileInput,
TTapisJob,
TJobArgSpecs,
TJobKeyValuePair,
} from 'utils/types';

export type TJobPostOperations = 'resubmitJob' | 'cancelJob' | 'submitJob';

export type TParameterSetSubmit = {
appArgs?: TJobArgSpecs;
containerArgs?: TJobArgSpecs;
schedulerOptions?: TJobArgSpecs;
envVariables?: TJobKeyValuePair[];
};

export type TConfigurationValues = {
execSystemId?: string;
execSystemLogicalQueue?: string;
maxMinutes?: number;
nodeCount?: number;
coresPerNode?: number;
allocation?: string;
memoryMB?: number;
};

export type TOutputValues = {
name: string;
archiveSystemId?: string;
archiveSystemDir?: string;
};

export interface TJobSubmit extends TConfigurationValues, TOutputValues {
archiveOnAppError?: boolean;
appId: string;
fileInputs?: TAppFileInput[];
parameterSet?: TParameterSetSubmit;
}

export type TJobBody = {
operation?: TJobPostOperations;
uuid?: string;
job: TJobSubmit;
licenseType?: string;
isInteractive?: boolean;
execSystemId?: string;
};

export interface IJobPostResponse extends TTapisJob {
execSys?: TTapisSystem;
}

export type TJobPostResponse = {
response: IJobPostResponse;
status: number;
};
30 changes: 30 additions & 0 deletions client/src/hooks/datafiles/useDataFilesAllocations.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
const dataFilesAllocations = {
hosts: {
'ls6.tacc.utexas.edu': ['TACC-ACI'],
'data.tacc.utexas.edu': ['TACC-ACI'],
'ranch.tacc.utexas.edu': ['TACC-ACI'],
'stampede2.tacc.utexas.edu': ['TACC-ACI'],
'maverick2.tacc.utexas.edu': ['TACC-ACI'],
'frontera.tacc.utexas.edu': ['TACC-ACI'],
},
portal_alloc: 'TACC-ACI',
active: [
{
title: 'TACC-ACI',
projectId: 9192,
projectName: 'TACC-ACI',
systems: [
{
name: 'ls6',
host: 'ls6.tacc.utexas.edu',
},
{
name: 'frontera',
host: 'frontera.tacc.utexas.edu',
},
],
},
],
};

export default dataFilesAllocations;
Loading