-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiple file upload ParseFilePipeBuilder validation #2424
Comments
Need to add an example of using
with a regular expression. Current examples sometimes seem like to validate the file extension rather than the mimetype . And people check |
Any updates? 😕 |
How I solve this problem with multer: ////////////////// image-multer-options.ts
const imageFilter = (req: Request, file: Express.Multer.File, callback: (error: Error, acceptFile: boolean) => void) => {
if (!Boolean(file.mimetype.match(/(jpg|jpeg|png|gif)/))) callback(null, false);
callback(null, true);
}
export const imageOptions: MulterOptions = {
limits: {fileSize: 5242880},
fileFilter: imageFilter
}
////////////////// people.controller.ts
@Post()
@ApiConsumes('multipart/form-data')
@UseInterceptors(FilesInterceptor('images', 100, imageOptions))
async createPerson(@Body(ValidationPipe) dto: CreatePersonDto, @UploadedFiles() images: Express.Multer.File[]) {
const personToCreate = this.mapper.map(dto, CreatePersonDto, Person);
const newPerson = await this.peopleService.create(personToCreate, images, dto.relations);
return newPerson;
} But here we don't have a response notification that some of the files didn't pass the validation, instead incorrect file just won't pass to images array |
Is the Sorry, I just noticed through my custom validator that it must treat as an array too. Otherwise any validation will fail. In this case, default validators won't work and I'll need to use custom implementations? An example of a custom implementation of the import { MaxFileSizeValidator as DefaultMaxFileSizeValidator } from '@nestjs/common';
export class MaxFileSizeValidator extends DefaultMaxFileSizeValidator {
isValid(fileOrFiles: Express.Multer.File | Express.Multer.File[]): boolean {
if (Array.isArray(fileOrFiles)) {
const files = fileOrFiles;
return files.every((file) => super.isValid(file));
}
const file = fileOrFiles;
return super.isValid(file);
}
} |
I'm not using @uploadedfiles decorator. Don't know why I can't catch errors in this. So in Multer, it had fileFilter method and I try using that to catch files right from the request, causing catch file from the request so u can check file size, name, path,..v..v.. is valid or not and threw an error if the file is not validated. This's an example hope it can help. If _.isMatch make u confuse, it's from lodash package.
More detail in: https://www.npmjs.com/package/multer -> Find in fileFilter method |
I wrapped one pipe in another to apply it to all files individually along the lines of: export class ParseFilesPipe implements PipeTransform<Express.Multer.File[]> {
constructor(private readonly pipe: ParseFilePipe) { }
async transform(files: Express.Multer.File[]) {
for (const file of files)
await this.pipe.transform(file);
return files;
}
} The transform throws if the file is invalid. One could also catch and collect the errors to generate per-file messages. |
@brunnerh, thanks for your reply. When you use FileFieldsInterceptor, files parameter type comes be object. For this issue, I updated the code as follows. import { ParseFilePipe, PipeTransform } from '@nestjs/common';
export class ParseFilesPipe implements PipeTransform<Express.Multer.File[]> {
constructor(private readonly pipe: ParseFilePipe) {}
async transform(
files: Express.Multer.File[] | { [key: string]: Express.Multer.File },
) {
if (typeof files === 'object') {
files = Object.values(files);
}
for (const file of files) await this.pipe.transform(file);
return files;
}
} |
One thing also mentioned in multer doc is that you can throw errors in the filefilter function, which will be delegated to the express layer (which will then be delegated to nestjs), by passing an error to the callback function directly.
|
@emircanok , can you give us an example of how you're using your |
I have created a custom file size and type validator to validate multiple files.
import { FileValidator } from '@nestjs/common';
type FileType = Express.Multer.File | Express.Multer.File[] | Record<string, Express.Multer.File[]>;
type Result = { errorFileName?: string; isValid: boolean };
export const runFileValidation = async (args: {
multiple: boolean;
file: FileType;
validator: (file: Express.Multer.File) => Promise<boolean> | boolean;
}): Promise<Result> => {
if (args.multiple) {
const fileFields = Object.keys(args.file);
for (const field of fileFields) {
const fieldFile = args.file[field];
if (Array.isArray(fieldFile)) {
for (const f of fieldFile) {
if (!args.validator(f)) {
return { errorFileName: f.originalname, isValid: false };
}
}
} else {
if (!args.validator(fieldFile)) {
return { errorFileName: fieldFile.originalname, isValid: false };
}
}
}
return { isValid: true };
}
if (Array.isArray(args.file)) {
for (const f of args.file) {
if (!args.validator(f)) {
return { errorFileName: f.originalname, isValid: false };
}
}
return { isValid: true };
}
if (args.validator(args.file as any)) {
return { errorFileName: args.file.originalname as string, isValid: false };
}
return { isValid: true };
};
export class FileSizeValidator extends FileValidator {
private maxSizeBytes: number;
private multiple: boolean;
private errorFileName: string;
constructor(args: { maxSizeBytes: number; multiple: boolean }) {
super({});
this.maxSizeBytes = args.maxSizeBytes;
this.multiple = args.multiple;
}
async isValid(
file?: Express.Multer.File | Express.Multer.File[] | Record<string, Express.Multer.File[]>,
): Promise<boolean> {
const result = await runFileValidation({
file,
multiple: this.multiple,
validator: (f) => f.size < this.maxSizeBytes,
});
this.errorFileName = result.errorFileName;
return result.isValid;
}
buildErrorMessage(file: any): string {
return (
`file ${this.errorFileName || ''} exceeded the size limit ` +
parseFloat((this.maxSizeBytes / 1024 / 1024).toFixed(2)) +
'MB'
);
}
}
export class FileTypeValidator extends FileValidator {
private multiple: boolean;
private errorFileName: string;
private filetype: RegExp | string;
constructor(args: { multiple: boolean; filetype: RegExp | string }) {
super({});
this.multiple = args.multiple;
this.filetype = args.filetype;
}
isMimeTypeValid(file: Express.Multer.File) {
return file.mimetype.search(this.filetype) === 0;
}
async isValid(
file?: Express.Multer.File | Express.Multer.File[] | Record<string, Express.Multer.File[]>,
): Promise<boolean> {
const result = await runFileValidation({
multiple: this.multiple,
file: file,
validator: (f) => this.isMimeTypeValid(f),
});
this.errorFileName = result.errorFileName;
return result.isValid;
}
buildErrorMessage(file: any): string {
return `file ${this.errorFileName || ''} must be of type ${this.filetype}`;
}
}
import { ParseFilePipe, Patch, UploadedFiles, UseInterceptors } from '@nestjs/common';
import { FileFieldsInterceptor } from '@nestjs/platform-express';
import { FileSizeValidator, FileTypeValidator } from './file-validator';
export class ImageController {
@UseInterceptors(
FileFieldsInterceptor([
{ name: 'logo', maxCount: 1 },
{ name: 'background', maxCount: 1 },
]),
)
@Patch('/upload-image')
async updateLandingPage(
@UploadedFiles(
new ParseFilePipe({
validators: [
new FileSizeValidator({
multiple: true,
maxSizeBytes: 5 * 1024 * 1024, // 5MB
}),
new FileTypeValidator({
multiple: true,
filetype: /^image\/(jpeg|png|gif|bmp|webp|tiff)$/i,
}),
],
}),
)
files: {
logo: Express.Multer.File[];
background: Express.Multer.File[];
},
) {
console.log(files);
return 'ok';
}
} |
|
This is how I'm handling multi-file uploads using NestJS: controller: @Public()
@ApiCreatedResponse()
@ApiBadRequestResponse()
@ApiNotFoundResponse()
@ApiTags('Clients', 'Client Booking Process')
@UseInterceptors(AnyFilesInterceptor(), new FilesSizeInterceptor())
@Post(':formId/elements/:formElementId/upload')
uploadImages(
@Req() req: any,
@Body() body: UploadImagesDto,
@UploadedFiles()
files: Express.Multer.File[],
) {
new ImageFileValidationPipe().transform(files);
return this.formsService.uploadImages(
req.params.formId,
req.params.formElementId,
files,
body,
);
} FilesSizeInterceptor: import {
CallHandler,
ExecutionContext,
HttpException,
Injectable,
NestInterceptor,
} from '@nestjs/common';
import { Observable } from 'rxjs';
@Injectable()
export class FilesSizeInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
const request = context.switchToHttp().getRequest();
const files = request.files as Express.Multer.File[];
for (const file of files) {
if (file.size > 5 * 1024 * 1024) {
throw new HttpException('File size too large', 400);
}
}
return next.handle();
}
} ImageFileVlidationPipe: import { BadRequestException, PipeTransform } from '@nestjs/common';
export class ImageFileValidationPipe implements PipeTransform {
transform(files: Express.Multer.File[]): Express.Multer.File[] {
const allowedMimeTypes = ['image/jpeg', 'image/png', 'image/webp'];
for (const file of files) {
if (!allowedMimeTypes.includes(file.mimetype)) {
throw new BadRequestException('Invalid file type.');
}
}
return files;
}
} |
I think you can handle the validate file type into Interceptor too instead of using another pipe |
You're totally right, but I use these different rules in different places. That's the main reason why I have these two validations in two different files. |
That was very helpful and nearly did the trick for me only the reassignment of files changed the output so that the anticipated format was not matching in the controller. import { ParseFilePipe, PipeTransform } from '@nestjs/common';
export class ParseFilesPipe implements PipeTransform<Express.Multer.File[]> {
constructor(private readonly pipe: ParseFilePipe) {}
async transform(
files: Express.Multer.File[] | { [key: string]: Express.Multer.File[] },
) {
for (const file of Object.values(files).flat())
await this.pipe.transform(file);
return files;
}
} |
I had multiple fields, each with multiple files and so had to modify this a bit, but this is the solution that ultimately worked. export class ModifiedMaxFileSizeValidator extends MaxFileSizeValidator {
constructor(options: any) {
super(options);
}
isValid(fileOrFiles: IFile | Array<IFile> | Record<string, Array<IFile>>): boolean {
const isObject = Object.prototype.toString.call(fileOrFiles) === '[object Object]';
if(isObject) {
const allFilesValid = Object.values(fileOrFiles).every((arr) => {
return arr.every((file: IFile) => {
return super.isValid(file);
});
});
return allFilesValid;
}
if(Array.isArray(fileOrFiles)) {
const files = fileOrFiles;
const allFilesValid = files.every((file) => super.isValid(file));
return allFilesValid;
}
const file = fileOrFiles as IFile;
return super.isValid(file);
}
} |
does not work for many file upload
|
For me, that is the best approach. I just made some changes for improve the usage syntax in controller: import { ParseFilePipe, PipeTransform } from '@nestjs/common';
export class ParseFilesPipe
extends ParseFilePipe
implements PipeTransform<Express.Multer.File[]>
{
async transform(
files: Express.Multer.File[] | { [key: string]: Express.Multer.File[] },
) {
for (const file of Object.values(files).flat()) await super.transform(file);
return files;
}
} Controller Example: public async upload(
@UploadedFiles(
new ParseFilesPipe({
validators: [
new MaxFileSizeValidator({ maxSize: 1024 * 1024 * 5 }), // 5MB
new FileTypeValidator({ fileType: /(jpg|jpeg|png|gif)$/ }),
],
errorHttpStatusCode: 422,
fileIsRequired: true,
}),
)
files: any
) {
await this.upload(files);
} |
Is there an existing issue that is already proposing this?
Is your feature request related to a problem? Please describe it
When using
FilesInterceptor
for doing a bulk upload and trying to follow the docs for performing the files validation for max size and mime types, there's no definitive guide available to do that,Describe the solution you'd like
There should be defined way of doing these basic file validations when files are uploaded as an array.
Teachability, documentation, adoption, migration strategy
What is the motivation / use case for changing the behavior?
Right now, not able to perform or reuse the logic for multiple files upload.
The text was updated successfully, but these errors were encountered: