✨✨VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
-
Updated
Jan 7, 2025 - Python
#
large-multimodal-models
Here are 49 public repositories matching this topic...
[NeurIPS 2024] An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions
gpt sora text-to-video large-language-models chatgpt large-vision-language-models large-multimodal-models gpt-4v large-video-language-models
-
Updated
Oct 9, 2024 - Python
Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
python transformers openai agents process-mining ai-agents process-automation huggingface ultralytics large-language-models anthropic segment-anything ai-agents-framework large-multimodal-models google-gemini large-action-model omniparser gpt4o generative-process-automation computer-use
-
Updated
Jan 2, 2025 - Python
LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
-
Updated
Feb 1, 2024 - Python
A Framework of Small-scale Large Multimodal Models
-
Updated
Dec 18, 2024 - Python
A collection of resources on applications of multi-modal learning in medical imaging.
medical-imaging multimodal-learning visual-question-answering multimodal-deep-learning large-language-models medical-report-generation multimodal-large-language-models large-multimodal-models
-
Updated
Nov 11, 2024
An open-source implementation for training LLaVA-NeXT.
chatbot llama multimodal multi-modality gpt-4 visual-language-learning chatgpt vision-language-model llava large-multimodal-models llama3 gpt4o llava-next
-
Updated
Oct 23, 2024 - Python
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
machine-learning natural-language-processing deep-neural-networks computer-vision deep-learning evaluation question-answering stem multimodality multimodal-learning visual-question-answering multimodal multimodal-deep-learning foundation-models large-language-models llm llms large-multimodal-models
-
Updated
Jan 2, 2025 - Python
[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
chatbot llama multimodal gpt-4 chatgpt vision-language-model vision-language-learning large-multimodal-models
-
Updated
Aug 24, 2024 - Python
Open Platform for Embodied Agents
-
Updated
Dec 23, 2024 - Python
The official evaluation suite and dynamic data release for MixEval.
benchmark evaluation benchmarking-suite evaluation-framework benchmarking-framework foundation-models large-language-models large-language-model llm-inference llm-evaluation large-multimodal-models llm-evaluation-framework benchmark-mixture mixeval
-
Updated
Nov 10, 2024 - Python
A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.
finetuning multimodal vision-language foundation-models instruction-tuning large-language-model llava visual-instruction-tuning multimodal-large-language-models large-multimodal-models qwen-vl llava-next
-
Updated
Dec 23, 2024 - Python
[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions
gpt language-model large-language-models chatgpt instruction-tuning vision-language-model large-vision-language-models gpt4v large-multimodal-models gpt-4v eccv2024
-
Updated
Jul 1, 2024 - Python
Embed arbitrary modalities (images, audio, documents, etc) into large language models.
multimodal multi-modality large-language-models llm vision-language-model llava large-context large-multimodal-models
-
Updated
Mar 27, 2024 - Python
[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"
evaluation multimodality multimodal-learning visual-question-answering multimodal large-language-models llm llms large-vision-language-model large-vision-language-models large-multimodal-models lvlms lvlm
-
Updated
Sep 26, 2024 - Python
A curated list of awesome Multimodal studies.
deep-learning multimodal-learning multimodal multimodal-deep-learning multimodal-data multimodal-dialogue multimodal-large-language-models large-multimodal-models
-
Updated
Jan 6, 2025 - HTML
The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".
chatbot llama multimodal vision-language-model llava vision-language-learning large-multimodal-models gpt-4o
-
Updated
Nov 27, 2024 - Python
[ECCV 2024] BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models
-
Updated
Aug 19, 2024 - Python
[ECCV 2024] API: Attention Prompting on Image for Large Vision-Language Models
visual-prompting prompting vision-language-model large-vision-language-model large-vision-language-models large-multimodal-models vision-language-models
-
Updated
Oct 10, 2024 - Python
Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"
machine-learning natural-language-processing computer-vision deep-learning evaluation question-answering visual-question-answering multimodal multimodal-deep-learning foundation-models large-language-models llm llms mllm multimodal-large-language-models large-multimodal-models
-
Updated
Oct 19, 2024 - Python
Improve this page
Add a description, image, and links to the large-multimodal-models topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the large-multimodal-models topic, visit your repo's landing page and select "manage topics."