AI Video Analysis & Simulation Framework

This project provides a comprehensive framework for analyzing, simulating, and benchmarking video content using state-of-the-art AI models. It is designed for researchers and developers who want to evaluate, compare, and stress-test video understanding systems under various scenarios.

---

🎯 Main Purpose

• Analyze video content using advanced AI models (e.g., CLIP, Video-ChatGPT, Mobile-VideoGPT, Video-LLaVA).
• Simulate and manipulate videos (visual/audio) to test model robustness and sensitivity.
• Benchmark and compare different models on original and manipulated videos.
• Automate testing and evaluation for both research and practical applications.

---

🧩 Key Components

• Video Analysis:

  • Extracts semantic information from videos using multimodal AI models.
  • Supports frame-level and sequence-level understanding.

• Simulation & Manipulation:

  • Applies various transformations (visual, audio, metadata) to videos.
  • Includes frame selection (e.g., optical flow), organic perturbation, and audio modification.

• Organicity Scenario:

  • Implements the "organicity" scenario, where videos are modified to appear as natural and authentic as possible while still being machine-generated or manipulated.
  • The goal is to test whether AI models and detection systems can distinguish between truly original and highly-organic (but synthetic) videos.
  • See ultimate_organic_perturbation.py for advanced organicity simulation and scoring.

• Testing & Evaluation:

  • Automated scripts to compare model outputs on original vs. manipulated videos.
  • Quantitative and qualitative benchmarking (accuracy, consistency, robustness, etc).

• Training & Finetuning:

  • Scripts and instructions for training or finetuning your own models (see Mobile-VideoGPT/scripts/ and Video-ChatGPT/docs/).

---

🛠️ Installation

Requirements

pip install torch torchvision opencv-python pillow numpy clip

FFmpeg (for video/audio processing)

# macOS
brew install ffmpeg
# Ubuntu
sudo apt install ffmpeg

---

🚀 Usage Overview

• Video Analysis Example:

  • See test_2.py, test.py, or video_chatgpt_test.py for sample scripts.

• Simulation/Manipulation:

  • Use scripts like ultimate_organic_perturbation.py (for organicity scenario) or audio_perturbation.py to modify videos/audio.

• Automated Testing:

  • Run comprehensive tests with comprehensive_test.py, final_ai_detection_test.py, or model-specific test scripts.

• Benchmarking:

  • Use provided scripts in Video-ChatGPT/quantitative_evaluation/ and Mobile-VideoGPT/eval/ for large-scale evaluation.

---

📂 Project Structure

AIVideo/
├── CogVLM/                    # CogVLM model implementation
├── Mobile-VideoGPT/           # Mobile VideoGPT model & evaluation
├── Video-ChatGPT/             # Video ChatGPT implementation & benchmarks
├── test_2.py                  # Example video analysis & manipulation script
├── comprehensive_test.py      # Full pipeline test & evaluation
├── ultimate_organic_perturbation.py # Advanced organicity simulation
├── audio_perturbation.py      # Audio simulation
└── ...                        # Other scripts & utilities

---

📝 License

This project is for research and educational purposes only.

---

🤝 Contributing

1. Fork this repository
2. Create a feature branch (git checkout -b feature/YourFeature)
3. Commit your changes (git commit -m 'Add some feature')
4. Push to the branch (git push origin feature/YourFeature)
5. Open a Pull Request

---

📞 Contact

For questions or suggestions, please open an issue on GitHub.