Skip to content

Commit cc4447e

Browse files
FeezyHendrixAbdulHafeez AbdulRaheempre-commit-ci[bot]
authored
feat: classification workflow for imagery upload (#668)
* feat: added multipart initate upload route * feat: added all multipart upload functions * feat(UploadBox): Swapped upload from single to multipart upload using uppy * feat(frontend): add @uppy/drag-drop and @uppy/progress-bar dependencies; update moduleResolution to bundler * feat(frontend): feat: integrate Uppy for image uploads and enhance file handling - Added Uppy instance to the global App component for shared file upload functionality. - Implemented UppyImageUploader component to utilize Uppy for image uploads with AWS S3 integration. - Refactored file handling to extract EXIF data upon file addition and display upload progress. - Updated Vite configuration to target 'esnext' for improved compatibility * feat(image-upload): Refactor S3 integration to use boto3 instead of MinIO - Replaced MinIO client with boto3 for S3 operations in s3.py. - Updated methods for file upload, download, and presigned URL generation to align with boto3 API. - Adjusted error handling to use ClientError from botocore. - Modified user_schemas.py to generate presigned URLs using boto3. - Updated dependencies in pyproject.toml to include boto3 and remove MinIO. - Ensured compatibility with existing code by stripping leading slashes from S3 paths. - Added content type headers for multipart upload requests in the frontend UppyImageUploader component * feat(image-upload-workflow): Refactor code structure for improved readability and maintainability * feat(image-upload-workflow): Implement drone image processing workflow UI with upload, classification, review, and processing steps * feat(image-upload-workflow): Refactor image upload components and enhance upload functionality - Replaced UppyImageUploader with UppyFileUploader for consistency across components. - Improved ImageUpload component to track upload progress and display upload queue. - Added support for staging uploads in UppyFileUploader. - Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability. - Removed unused UppyImageUploader component and its related code BREAKING CHANGE: Switch from minio to boto3 * feat(image-upload-workflow): Refactor image upload components and enhance upload functionality - Replaced UppyImageUploader with UppyFileUploader for consistency across components. - Improved ImageUpload component to track upload progress and display upload queue. - Added support for staging uploads in UppyFileUploader. - Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability. - Removed unused UppyImageUploader component and its related code * fixed with minio * feat(image-upload-workflow): Add project_id and filename to the upload request in UppyFileUploader * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat(image-upload-workflow): feat(migrations): Enhance project_images table migration with conditional enum creation and improved index handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(image-upload-workflow): feat(step-switcher): Implement generic StepSwitcher component for project steps navigation * feat(image-upload-workflow): feat(db-models): Add DbProjectImage model for managing project images with relationships and indexing * feat(image-upload-workflow): Enhance image processing with new database save function and EXIF handling * refactor(image-upload-workflow): Replace deprecated get_presigned_url with generate_presigned_download_url across the codebase; add new migration script for images from S3 * fix(image-upload-workflow): Implement upsert functionality for image migration with conflict handling and improved logging * Added multipart imagery upload capability (#649) * feat: added multipart initate upload route * feat: added all multipart upload functions * feat(UploadBox): Swapped upload from single to multipart upload using uppy * feat(frontend): add @uppy/drag-drop and @uppy/progress-bar dependencies; update moduleResolution to bundler * feat(frontend): feat: integrate Uppy for image uploads and enhance file handling - Added Uppy instance to the global App component for shared file upload functionality. - Implemented UppyImageUploader component to utilize Uppy for image uploads with AWS S3 integration. - Refactored file handling to extract EXIF data upon file addition and display upload progress. - Updated Vite configuration to target 'esnext' for improved compatibility * feat(image-upload): Refactor S3 integration to use boto3 instead of MinIO - Replaced MinIO client with boto3 for S3 operations in s3.py. - Updated methods for file upload, download, and presigned URL generation to align with boto3 API. - Adjusted error handling to use ClientError from botocore. - Modified user_schemas.py to generate presigned URLs using boto3. - Updated dependencies in pyproject.toml to include boto3 and remove MinIO. - Ensured compatibility with existing code by stripping leading slashes from S3 paths. - Added content type headers for multipart upload requests in the frontend UppyImageUploader component * feat(image-upload-workflow): Refactor code structure for improved readability and maintainability * feat(image-upload-workflow): Implement drone image processing workflow UI with upload, classification, review, and processing steps * feat(image-upload-workflow): Refactor image upload components and enhance upload functionality - Replaced UppyImageUploader with UppyFileUploader for consistency across components. - Improved ImageUpload component to track upload progress and display upload queue. - Added support for staging uploads in UppyFileUploader. - Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability. - Removed unused UppyImageUploader component and its related code BREAKING CHANGE: Switch from minio to boto3 * feat(image-upload-workflow): Refactor image upload components and enhance upload functionality - Replaced UppyImageUploader with UppyFileUploader for consistency across components. - Improved ImageUpload component to track upload progress and display upload queue. - Added support for staging uploads in UppyFileUploader. - Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability. - Removed unused UppyImageUploader component and its related code * fixed with minio * feat(image-upload-workflow): Add project_id and filename to the upload request in UppyFileUploader * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat(image-upload-workflow): feat(migrations): Enhance project_images table migration with conditional enum creation and improved index handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(image-upload-workflow): feat(step-switcher): Implement generic StepSwitcher component for project steps navigation * feat(image-upload-workflow): feat(db-models): Add DbProjectImage model for managing project images with relationships and indexing * feat(image-upload-workflow): Enhance image processing with new database save function and EXIF handling * refactor(image-upload-workflow): Replace deprecated get_presigned_url with generate_presigned_download_url across the codebase; add new migration script for images from S3 * fix(image-upload-workflow): Implement upsert functionality for image migration with conflict handling and improved logging * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: AbdulHafeez AbdulRaheem <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat(image-upload-worklow): Implement image classification functionality - Added new hooks for starting classification, fetching batch status, and retrieving batch images in the projects API. - Enhanced the ImageClassification component to handle classification processes, including polling for updates and displaying status. - Updated the DroneImageProcessingWorkflow to manage batch ID and transition between upload and classification steps. - Modified UppyFileUploader to support batch ID generation and pass it upon upload completion. - Created a new migration to add classification-related fields to the project_images table. - Developed new classification routes for starting batch classification and retrieving batch status and images. - Implemented the ImageClassifier class to handle image classification logic, including quality checks and task matching. - Added service functions for interacting with the classification API, including starting classification and fetching batch details * feat(image-upload-workflow): Add sharpness_score field to project_images and implement sharpness calculation in image classification * feat(image-upload-workflow): Enhance image classification with batch processing and status checks * feat(image-upload-workflow): Update image status references from 'uploaded' to 'staged' and enhance upload notifications * feat(image-upload-workflow): Add logging for classification requests and image count checks * feat(image-upload-workflow): feat(image-processing-workflow): Implement image processing workflow state management and update components for classification handling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat(image-upload-workflow): Update migration and enhance image classification component with presigned URL handling * feat(image-upload-workflow): Improve exception handling in _convert_exif_value function * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat(image-upload-workflow): add image detail modal and improve image status display - Introduced a modal to display detailed information about selected images, including status, filename, GPS data, and upload time. - Enhanced status badge styles for better visual distinction. - Updated image list to allow selection and display of the modal. - Improved loading indicators and button states during classification process. - Refactored status color and label functions for better readability and maintainability * fix(image-upload-workflow): enhance loading and empty states in image processing workflow * fix(image-upload-workflow): implement duplicate image handling and status updates * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat(image-upload-workflow): add thumbnail generation and upload functionality - Implemented thumbnail generation for uploaded images using PIL. - Added thumbnail URL to image records in the database. - Updated image processing workflow to handle thumbnail uploads to S3. - Enhanced image classification to utilize thumbnail URLs for display. - Modified frontend components to support thumbnail rendering in the image grid * feat(image-upload-workflow): enhance image review process with batch ID handling and add task group interfaces * feat(image-upload-workflow): Enhance image classification workflow with accept/reject functionality - Updated `useGetBatchImagesQuery` to specify query options for image classification results. - Implemented image acceptance mutation in `ImageClassification` component to handle rejected images. - Added warning for high issue rates in classified images. - Introduced a button to mark rejected images as good in `ImageClassification`. - Enhanced `ImageReview` component with map integration and improved image handling. - Added batch deletion functionality with confirmation dialog in `DroneImageProcessingWorkflow`. - Refactored `Accordion` component to support dynamic content and improved styling. - Updated classification service to include image acceptance and batch deletion methods * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: AbdulHafeez AbdulRaheem <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 2812aa8 commit cc4447e

File tree

29 files changed

+3701
-359
lines changed

29 files changed

+3701
-359
lines changed

compose.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,8 @@ services:
134134
depends_on:
135135
- redis
136136
env_file: .env
137+
extra_hosts:
138+
- "localhost:host-gateway"
137139
networks:
138140
- dtm-network
139141
volumes:

0 commit comments

Comments
 (0)