Skip to content

Latest commit

 

History

History
353 lines (289 loc) · 15.3 KB

README.md

File metadata and controls

353 lines (289 loc) · 15.3 KB
Raw

GeekyRemB: Advanced Background Removal & Image Processing Node for ComfyUI

GeekyRemB Demo Video

GeekyRemB is a sophisticated image processing node that brings professional-grade background removal, blending, and animation capabilities to ComfyUI. It combines AI-powered processing with traditional image manipulation techniques to offer a comprehensive solution for complex image processing tasks.

Table of Contents

  1. User Guide
  2. Developer Documentation
  3. License
  4. Acknowledgements

User Guide

Installation

  1. Install ComfyUI if you haven't already. Follow the ComfyUI installation guide for detailed instructions.
  2. Clone the GeekyRemB repository into your ComfyUI custom nodes directory: 
    git clone https://github.com/YourUsername/ComfyUI-GeekyRemB.git
  3. Install dependencies: 
    pip install -r requirements.txt
  4. Restart ComfyUI to load the new node.

Features

Background Removal

  • AI-Powered Removal Using Multiple Models:
    • u2net: General-purpose, high-quality background removal.
    • u2netp: Faster processing with slight quality trade-off.
    • u2net_human_seg: Optimized for human subjects.
    • u2net_cloth_seg: Specialized for clothing segmentation.
    • silueta: Enhanced edge detection for finer details.
    • isnet-general-use: Balanced performance for various subjects.
    • isnet-anime: Optimized for anime and cartoon-style images.
  • Professional Chroma Keying:
    • Supports multiple chroma key colors (green, blue, red).
    • Adjustable tolerance for precise color selection.
  • Advanced Alpha Matting:
    • Refines edges for seamless background removal.
    • Controls for foreground and background thresholds.

Image Processing

  • Professional Blend Modes:
    • Normal, Multiply, Screen, Overlay, Soft Light, Hard Light, Difference, Exclusion, Color Dodge, Color Burn, Linear Light, Pin Light, and more.
    • Accurate alpha channel management ensuring proper transparency handling.
  • Precise Mask Refinement Tools:
    • Thresholding, edge detection, mask expansion/erosion, blur, and small region removal.
  • Support for Images of Different Dimensions:
    • Automatic scaling and aspect ratio adjustments.
  • Automatic Alpha Channel Management:
    • Ensures consistent image formats across different operations.

Animation

  • Multiple Animation Types:
    • Bounce: Smooth up/down motion.
    • Travel Left/Right: Linear horizontal movement.
    • Rotate: Continuous rotation around the visible center.
    • Spiral: Combines rotation with radial movement.
    • Fade In/Out: Opacity transitions for gradual appearance/disappearance.
    • Zoom In/Out: Scaling transitions for zoom effects.
    • Shake: Quick oscillating movements for dynamic effects.
    • Slide Up/Down: Vertical sliding motions.
    • Flip Horizontal/Vertical: Mirroring effects.
    • Wave: Sinusoidal vertical movement for a waving effect.
    • Pulse: Periodic scaling for a pulsing effect.
    • Swing: Oscillating rotation for a swinging motion.
    • Spin: Continuous spinning around the center.
    • Flash: Rapid brightness changes for flashing effects.
  • Configurable Speed and Frame Count:
    • Control the tempo and smoothness of animations.
  • Position, Scale, and Rotation Control:
    • Fine-tune the placement and transformation of animated elements.
  • Advanced Animation Parameters:
    • Steps: Define multi-step animations for complex movements.
    • Phase Shift: Staggered animations for dynamic compositions.
  • Easing Functions:
    • Linear, Ease In/Out Quad, Ease In/Out Cubic, and more for smooth transitions.

Parameters Guide

Essential Settings

  • enable_background_removal: Toggle background processing on or off.
  • removal_method: Choose between AI-based removal (rembg), color-based removal (chroma_key), or both.
  • model: Select the AI model for rembg method (e.g., u2net, u2netp, etc.).
  • blend_mode: Choose how foreground and background images are combined.
  • opacity: Control the strength of blending (0.0-1.0).

Advanced Settings

  • mask_expansion: Fine-tune mask edges (-100 to 100) to expand or contract the mask.
  • edge_detection: Enable additional edge processing for sharper outlines.
  • edge_thickness: Set the thickness of detected edges (1-10).
  • mask_blur: Smooth mask edges to reduce harsh transitions (0-100).
  • alpha_matting: Enable sophisticated edge refinement using alpha matting.
  • remove_small_regions: Clean up mask artifacts by removing small regions.
  • small_region_size: Define the minimum size of regions to retain in the mask (1-1000).
  • animation_type: Select the type of animation to apply (e.g., spin, fade_in).
  • animation_speed: Control the speed of the animation (0.1-10.0).
  • animation_duration: Set the duration of one animation cycle (0.1-10.0 seconds).
  • repeats: Number of times the animation repeats (1-100).
  • reverse: Reverse the animation direction on every repeat.
  • easing_function: Choose the easing function for smooth transitions.
  • delay: Delay before the animation starts (0.0-5.0 seconds).
  • animation_frames: Total number of output frames for the animation (1-3000).
  • x_position: Initial horizontal position of the element (-1000 to 1000).
  • y_position: Initial vertical position of the element (-1000 to 1000).
  • scale: Scale factor for resizing the element (0.1-5.0).
  • rotation: Initial rotation angle of the element (-360 to 360 degrees).
  • steps: Number of steps in multi-step animations (1-10).
  • phase_shift: Phase shift for staggered animations (0.0-1.0).

Optional Inputs

  • background: Secondary image for composition behind the foreground.
  • additional_mask: Extra mask for complex selections and refinements.
  • invert_additional_mask: Invert the additional mask for varied effects.

Developer Documentation

Technical Implementation

Blend Mode System

GeekyRemB features a sophisticated blending engine that handles complex image compositions with precision and efficiency.

class EnhancedBlendMode:
    @staticmethod
    def _ensure_rgba(img: np.ndarray) -> np.ndarray:
        # Ensures the image has an alpha channel
        ...

    @staticmethod
    def _apply_blend(target: np.ndarray, blend: np.ndarray, operation, opacity: float = 1.0) -> np.ndarray:
        # Core blending logic with alpha handling
        ...

    @classmethod
    def get_blend_modes(cls) -> Dict[str, Callable]:
        # Returns a dictionary of available blend modes
        ...

    @staticmethod
    def multiply(target: np.ndarray, blend: np.ndarray, opacity: float = 1.0) -> np.ndarray:
        # Implementation of Multiply blend mode
        ...

Key Features:

  • Proper Alpha Channel Management: Ensures all images have an alpha channel for consistent blending.
  • Automatic Dimension Handling: Automatically adjusts image dimensions to match during blending operations.
  • Memory-Efficient Processing: Utilizes optimized numpy operations for fast and efficient blending.
  • Numerical Precision Optimization: Maintains high precision in color computations to prevent artifacts.

Animation Engine

The animation system provides smooth, configurable motion for dynamic image compositions.

class EnhancedAnimator:
    @staticmethod
    def animate_element(
        element: Image.Image,
        animation_type: str,
        animation_speed: float,
        frame_number: int,
        total_frames: int,
        x_start: int,
        y_start: int,
        canvas_width: int,
        canvas_height: int,
        scale: float,
        rotation: float,
        easing_func: Callable[[float], float],
        repeat: int,
        reverse: bool,
        delay: float,
        steps: int = 1,
        phase_shift: float = 0.0
    ) -> Tuple[Image.Image, int, int]:
        # Generates precise frame-by-frame animations
        ...

Features:

  • Frame-Accurate Positioning: Ensures that elements move precisely as per frame calculations.
  • Sub-Pixel Interpolation: Handles smooth transitions and movements at sub-pixel levels.
  • Memory Pooling for Frame Sequences: Efficiently manages memory when handling multiple frames.
  • Automatic Boundary Handling: Prevents elements from moving out of the canvas boundaries.

Background Removal

GeekyRemB supports multiple background removal strategies with sophisticated refinement techniques.

class EnhancedGeekyRemB:
    def remove_background_rembg(self, image: Image.Image) -> Tuple[Image.Image, Image.Image]:
        # AI-powered background removal using rembg
        ...

    def remove_background_chroma(self, image: Image.Image) -> Tuple[Image.Image, Image.Image]:
        # Color-based background removal using chroma key
        ...

Highlights:

  • Model-Specific Optimizations: Different AI models are optimized for various types of subjects and use cases.
  • GPU Acceleration: Leverages available GPU resources for faster processing.
  • Advanced Edge Detection: Refines edges to ensure seamless background removal.
  • Artifact Reduction: Minimizes visual artifacts through mask refinement and blending techniques.

Performance Optimizations

GeekyRemB incorporates several performance enhancements to ensure efficient processing:

  • ThreadPoolExecutor for Parallel Processing: Utilizes multi-threading to handle multiple frames concurrently.
  • Efficient Numpy Operations: Employs optimized numpy functions for fast numerical computations.
  • Smart Caching System: Implements an LRU (Least Recently Used) cache to store and reuse processed frames, reducing redundant computations.
  • Memory Management Optimizations: Ensures that memory usage is kept optimal, preventing leaks and overconsumption.

Error Handling

Robust error management is implemented throughout GeekyRemB to ensure reliability:

try:
    result = self.process_frame(frame, background)
except Exception as e:
    logger.error(f"Processing error: {str(e)}")
    return self.fallback_process(frame)

Features:

  • Comprehensive Logging: Errors are logged with detailed messages to aid in debugging.
  • Graceful Degradation: In case of failures, the system falls back to safe defaults to maintain operation.
  • Input Validation: Ensures that all inputs are validated before processing to prevent unexpected errors.

Extending GeekyRemB

Developers can enhance and expand GeekyRemB by adding new features, optimizing existing ones, or integrating additional functionalities. Below are guidelines and best practices to assist in development:

Adding New Animation Types

To introduce a new animation type:

  1. Define the Animation in the AnimationType Enum:

    class AnimationType(Enum):
    NEW_ANIMATION = "new_animation"
    # Existing animations...

  2. Implement the Animation Logic in EnhancedAnimator.animate_element:

    elif animation_type == AnimationType.NEW_ANIMATION.value:
    # Define how the element should animate
    x += int(math.sin(progress * math.pi) * amplitude)
    y += int(math.cos(progress * math.pi) * amplitude)
    # Any additional transformations

  3. Update the INPUT_TYPES to Include the New Animation:

    "animation_type": ([anim.value for anim in AnimationType],),

  4. Document the New Animation in the User Guide under the Features and Parameters Guide sections.

Optimizing Blend Modes

To add or optimize a blend mode:

  1. Implement the Blend Mode Function in EnhancedBlendMode:

    @staticmethod
def new_blend_mode(target: np.ndarray, blend: np.ndarray, opacity: float = 1.0) -> np.ndarray:
    # Define the blend operation
    def blend_op(t, b):
        return (t + b) / 2  # Example operation
    return EnhancedBlendMode._apply_blend(target, blend, blend_op, opacity)

  2. Register the Blend Mode in the get_blend_modes method:

    @classmethod
def get_blend_modes(cls) -> Dict[str, Callable]:
    return {
        "new_blend_mode": cls.new_blend_mode,
        # Existing blend modes...
    }

  3. Update the INPUT_TYPES to include the new blend mode option.

  4. Document the Blend Mode in the User Guide.

Enhancing Mask Processing

To improve mask refinement techniques:

  1. Modify the EnhancedMaskProcessor.refine_mask Method to include new processing steps.
  2. Add Configuration Options in ProcessingConfig for any new parameters.
  3. Ensure Thread-Safety and Optimize Performance when adding new operations.

Notable Technical Achievements

  1. Seamless Handling of Images with Different Dimensions: Automatically adjusts images to ensure compatibility during processing and blending.
  2. Professional-Grade Blend Modes Matching Industry Standards: Implements a wide range of blend modes with accurate color and transparency handling.
  3. Efficient Batch Processing with Memory Optimization: Utilizes multi-threading and caching to process multiple frames efficiently.
  4. Sophisticated Alpha Channel Management: Maintains high-quality transparency handling across all operations.
  5. Frame-Accurate Animation System: Ensures that animations are smooth and precisely timed across all frames.
  6. Robust Error Recovery Mechanisms: Implements comprehensive error handling to maintain reliability during processing.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Note: Some included models may have separate licensing requirements for commercial use.

Acknowledgements

GeekyRemB builds upon several outstanding open-source projects:

  • Rembg by Daniel Gatis: Core background removal capabilities.
  • ComfyUI: The foundation of our node system.
  • WAS Node Suite: Inspiration for layer utility features.

Special thanks to:

  • The ComfyUI Community: For valuable feedback and suggestions.
  • Open-Source Contributors: Who help improve the node continuously.
  • AI Model Creators: Whose work enables our advanced background removal features.

For updates, issues, or contributions, please visit the GitHub repository. We welcome feedback and contributions from the community.