Commit cc4447e
feat: classification workflow for imagery upload (#668)
* feat: added multipart initate upload route
* feat: added all multipart upload functions
* feat(UploadBox): Swapped upload from single to multipart upload using uppy
* feat(frontend): add @uppy/drag-drop and @uppy/progress-bar dependencies; update moduleResolution to bundler
* feat(frontend): feat: integrate Uppy for image uploads and enhance file handling
- Added Uppy instance to the global App component for shared file upload functionality.
- Implemented UppyImageUploader component to utilize Uppy for image uploads with AWS S3 integration.
- Refactored file handling to extract EXIF data upon file addition and display upload progress.
- Updated Vite configuration to target 'esnext' for improved compatibility
* feat(image-upload): Refactor S3 integration to use boto3 instead of MinIO
- Replaced MinIO client with boto3 for S3 operations in s3.py.
- Updated methods for file upload, download, and presigned URL generation to align with boto3 API.
- Adjusted error handling to use ClientError from botocore.
- Modified user_schemas.py to generate presigned URLs using boto3.
- Updated dependencies in pyproject.toml to include boto3 and remove MinIO.
- Ensured compatibility with existing code by stripping leading slashes from S3 paths.
- Added content type headers for multipart upload requests in the frontend UppyImageUploader component
* feat(image-upload-workflow): Refactor code structure for improved readability and maintainability
* feat(image-upload-workflow): Implement drone image processing workflow UI with upload, classification, review, and processing steps
* feat(image-upload-workflow): Refactor image upload components and enhance upload functionality
- Replaced UppyImageUploader with UppyFileUploader for consistency across components.
- Improved ImageUpload component to track upload progress and display upload queue.
- Added support for staging uploads in UppyFileUploader.
- Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability.
- Removed unused UppyImageUploader component and its related code
BREAKING CHANGE: Switch from minio to boto3
* feat(image-upload-workflow): Refactor image upload components and enhance upload functionality
- Replaced UppyImageUploader with UppyFileUploader for consistency across components.
- Improved ImageUpload component to track upload progress and display upload queue.
- Added support for staging uploads in UppyFileUploader.
- Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability.
- Removed unused UppyImageUploader component and its related code
* fixed with minio
* feat(image-upload-workflow): Add project_id and filename to the upload request in UppyFileUploader
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat(image-upload-workflow): feat(migrations): Enhance project_images table migration with conditional enum creation and improved index handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(image-upload-workflow): feat(step-switcher): Implement generic StepSwitcher component for project steps navigation
* feat(image-upload-workflow): feat(db-models): Add DbProjectImage model for managing project images with relationships and indexing
* feat(image-upload-workflow): Enhance image processing with new database save function and EXIF handling
* refactor(image-upload-workflow): Replace deprecated get_presigned_url with generate_presigned_download_url across the codebase; add new migration script for images from S3
* fix(image-upload-workflow): Implement upsert functionality for image migration with conflict handling and improved logging
* Added multipart imagery upload capability (#649)
* feat: added multipart initate upload route
* feat: added all multipart upload functions
* feat(UploadBox): Swapped upload from single to multipart upload using uppy
* feat(frontend): add @uppy/drag-drop and @uppy/progress-bar dependencies; update moduleResolution to bundler
* feat(frontend): feat: integrate Uppy for image uploads and enhance file handling
- Added Uppy instance to the global App component for shared file upload functionality.
- Implemented UppyImageUploader component to utilize Uppy for image uploads with AWS S3 integration.
- Refactored file handling to extract EXIF data upon file addition and display upload progress.
- Updated Vite configuration to target 'esnext' for improved compatibility
* feat(image-upload): Refactor S3 integration to use boto3 instead of MinIO
- Replaced MinIO client with boto3 for S3 operations in s3.py.
- Updated methods for file upload, download, and presigned URL generation to align with boto3 API.
- Adjusted error handling to use ClientError from botocore.
- Modified user_schemas.py to generate presigned URLs using boto3.
- Updated dependencies in pyproject.toml to include boto3 and remove MinIO.
- Ensured compatibility with existing code by stripping leading slashes from S3 paths.
- Added content type headers for multipart upload requests in the frontend UppyImageUploader component
* feat(image-upload-workflow): Refactor code structure for improved readability and maintainability
* feat(image-upload-workflow): Implement drone image processing workflow UI with upload, classification, review, and processing steps
* feat(image-upload-workflow): Refactor image upload components and enhance upload functionality
- Replaced UppyImageUploader with UppyFileUploader for consistency across components.
- Improved ImageUpload component to track upload progress and display upload queue.
- Added support for staging uploads in UppyFileUploader.
- Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability.
- Removed unused UppyImageUploader component and its related code
BREAKING CHANGE: Switch from minio to boto3
* feat(image-upload-workflow): Refactor image upload components and enhance upload functionality
- Replaced UppyImageUploader with UppyFileUploader for consistency across components.
- Improved ImageUpload component to track upload progress and display upload queue.
- Added support for staging uploads in UppyFileUploader.
- Updated DroneImageProcessingWorkflow to adjust modal dimensions for better usability.
- Removed unused UppyImageUploader component and its related code
* fixed with minio
* feat(image-upload-workflow): Add project_id and filename to the upload request in UppyFileUploader
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat(image-upload-workflow): feat(migrations): Enhance project_images table migration with conditional enum creation and improved index handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(image-upload-workflow): feat(step-switcher): Implement generic StepSwitcher component for project steps navigation
* feat(image-upload-workflow): feat(db-models): Add DbProjectImage model for managing project images with relationships and indexing
* feat(image-upload-workflow): Enhance image processing with new database save function and EXIF handling
* refactor(image-upload-workflow): Replace deprecated get_presigned_url with generate_presigned_download_url across the codebase; add new migration script for images from S3
* fix(image-upload-workflow): Implement upsert functionality for image migration with conflict handling and improved logging
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: AbdulHafeez AbdulRaheem <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(image-upload-worklow): Implement image classification functionality
- Added new hooks for starting classification, fetching batch status, and retrieving batch images in the projects API.
- Enhanced the ImageClassification component to handle classification processes, including polling for updates and displaying status.
- Updated the DroneImageProcessingWorkflow to manage batch ID and transition between upload and classification steps.
- Modified UppyFileUploader to support batch ID generation and pass it upon upload completion.
- Created a new migration to add classification-related fields to the project_images table.
- Developed new classification routes for starting batch classification and retrieving batch status and images.
- Implemented the ImageClassifier class to handle image classification logic, including quality checks and task matching.
- Added service functions for interacting with the classification API, including starting classification and fetching batch details
* feat(image-upload-workflow): Add sharpness_score field to project_images and implement sharpness calculation in image classification
* feat(image-upload-workflow): Enhance image classification with batch processing and status checks
* feat(image-upload-workflow): Update image status references from 'uploaded' to 'staged' and enhance upload notifications
* feat(image-upload-workflow): Add logging for classification requests and image count checks
* feat(image-upload-workflow): feat(image-processing-workflow): Implement image processing workflow state management and update components for classification handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat(image-upload-workflow): Update migration and enhance image classification component with presigned URL handling
* feat(image-upload-workflow): Improve exception handling in _convert_exif_value function
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat(image-upload-workflow): add image detail modal and improve image status display
- Introduced a modal to display detailed information about selected images, including status, filename, GPS data, and upload time.
- Enhanced status badge styles for better visual distinction.
- Updated image list to allow selection and display of the modal.
- Improved loading indicators and button states during classification process.
- Refactored status color and label functions for better readability and maintainability
* fix(image-upload-workflow): enhance loading and empty states in image processing workflow
* fix(image-upload-workflow): implement duplicate image handling and status updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat(image-upload-workflow): add thumbnail generation and upload functionality
- Implemented thumbnail generation for uploaded images using PIL.
- Added thumbnail URL to image records in the database.
- Updated image processing workflow to handle thumbnail uploads to S3.
- Enhanced image classification to utilize thumbnail URLs for display.
- Modified frontend components to support thumbnail rendering in the image grid
* feat(image-upload-workflow): enhance image review process with batch ID handling and add task group interfaces
* feat(image-upload-workflow): Enhance image classification workflow with accept/reject functionality
- Updated `useGetBatchImagesQuery` to specify query options for image classification results.
- Implemented image acceptance mutation in `ImageClassification` component to handle rejected images.
- Added warning for high issue rates in classified images.
- Introduced a button to mark rejected images as good in `ImageClassification`.
- Enhanced `ImageReview` component with map integration and improved image handling.
- Added batch deletion functionality with confirmation dialog in `DroneImageProcessingWorkflow`.
- Refactored `Accordion` component to support dynamic content and improved styling.
- Updated classification service to include image acceptance and batch deletion methods
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: AbdulHafeez AbdulRaheem <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>1 parent 2812aa8 commit cc4447e
File tree
29 files changed
+3701
-359
lines changed- src
- backend
- app
- arq
- db
- images
- migrations/versions
- models
- projects
- frontend
- src
- api
- components
- DroneOperatorTask/DescriptionSection
- DroneImageProcessingWorkflow
- UppyFileUploader
- common/Accordion
- services
- store
- reducers
- slices
29 files changed
+3701
-359
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
134 | 134 | | |
135 | 135 | | |
136 | 136 | | |
| 137 | + | |
| 138 | + | |
137 | 139 | | |
138 | 140 | | |
139 | 141 | | |
| |||
0 commit comments