multimodal

70 proyectos comparten este topic de GitHub

Las líneas conectan a los miembros que están mediblemente relacionados entre sí. El tamaño de los puntos refleja las estrellas.

🧬 Miembros

Janus-Series: Unified Multimodal Understanding and Generation Models

Visualize, query, and stream to train on multimodal robotics data.

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX…

The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/

notes for software engineers getting up to speed on new AI developments. Serves as datastore for…

Solve Visual Understanding with Reinforced VLMs

Align Anything: Training All-modality Model with Feedback

Fengshenbang-LM

MOSS‑TTS Family is an open‑source speech and sound generation model family from MOSI.AI and the OpenMOSS…

Foundation Architecture for (M)LLMs

streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a…

SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding…

Qwen-VL-Series-Finetune

An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

Get clean data from tricky documents, powered by vision-language models ⚡

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and…

Awesome-Multimodal-Research

A curated list of Multimodal Related Research.

Unifying 3D Mesh Generation with Language Models

alan-sdk-cordova

The Self-Coding System for Your App — Alan AI SDK for Cordova

Codebase for Aria - an Open Multimodal Native MoE

🩺 首个会看胸部X光片的中文多模态医学大模型 | The first Chinese Medical Multimodal Model…

MOVA: Towards Scalable and Synchronized Video–Audio Generation

[ICLR'24 spotlight] Chinese and English Multimodal Large Model Series (Chat and Paint) |…

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One…

⚡ Self-hostable YesCaptcha-compatible captcha solver built with FastAPI, Playwright, and OpenAI-compatible…

A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.

library supporting NLP and CV research on scientific papers

Multimodal-AND-Large-Language-Models

Paper list about multimodal and large language models, only used to record papers I read in the daily arxiv…

Pluralistic-Inpainting

[CVPR 2019]: Pluralistic Image Completion

A Toolbox for MultiModal Recommendation. Integrating 10+ Models...

Awesome-Reasoning-Foundation-Models

✨✨Latest Papers and Benchmarks in Reasoning with Foundation Models

Official implementation of SEED-LLaMA (ICLR 2024).

Project Page For "Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement"

tokenize-anything

[ECCV 2024] Tokenize Anything via Prompting

Multimodal RL training framework for diffusion & omni models

blended-diffusion

Official implementation for "Blended Diffusion for Text-driven Editing of Natural Images" [CVPR 2022]

Create browser automation as if you were teaching a human using GPT-4 Vision.

Official code of "EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model"

Awesome-Multimodal-Modeling

Awesome Multimodal Modeling [Covers MLLM, UMM, and NMM]

VisualWebArena is a benchmark for multimodal agents.

[NeurIPS 2025 Spotlight] Official implementation of Spatial-MLLM: Boosting MLLM Capabilities in Visual-based…

Multi-Modal learning toolkit based on PaddlePaddle and PyTorch, supporting multiple applications such as…

MultimodalRecSys

A curated list of awesome resources about multimodal recommender systems.

[VLDB' 25] ChatTS: LLM for Time Series Understanding and Reasoning

Official skills for the GLM family of models.

Flexible time series feature extraction & processing

Open-AI's DALL-E for large scale training in mesh-tensorflow.

Towards Generalist Biomedical AI

Virtual Sparse Convolution for Multimodal 3D Object Detection

Code for the paper "LLark: A Multimodal Instruction-Following Language Model for Music" by Josh Gardner,…

LLaVA-Interactive-Demo

LLaVA-Interactive-Demo

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models,…

Joint speech-language model - respond directly to audio!

Multimodal-Sentiment-Analysis

多模态情感分析——基于BERT+ResNet的多种融合方法

[ICLR 2026] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

Awesome-Multimodal-Papers

A curated list of awesome Multimodal studies.

Easily convert common crawl to a dataset of caption and document. Image/text Audio/text Video/text, ...

HPT - Open Multimodal LLMs from HyperGAI

《多模态大模型：新一代人工智能技术范式》配套教学资源

Thinking-with-Visual-Primitives

Archived snapshot of Thinking-with-Visual-Primitives

DeepEar ｜顺风耳 An open-source framework for Deep Research and Financial Signal Tracking. …

🔍 OpenSearch-VL provides a fully open recipe for training strong multimodal deep search agents through…

[ICML 2026] Official repo for "DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models"

Multimodal-Recommendation-Library

A Continuously Updated Library for Advanced Models for Multimodal Recommendation

ICLR2026-Guide-CN

不想啃 5000+ 全文？我已经替你和 LLM 啃完了 — ICLR 2026 全景中文导读

GEMS: Agent-Native Multimodal Generation with Memory and Skills

[ICLR 2026 Oral] FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal…

[ICLR 2026] Official code repository for "⚡️VisionTrim: Unified Vision Token Compression for…

🔗 Familias relacionadas

foundation-models 43 multimodal-large-language-models 17 aigc 77 grpo 19 audio 57 natural-language-processing 66 mllm 11 vision-and-language 5

Medido a partir de los temas de GitHub compartidos por ambos proyectos, ponderado por cuán raros son cada uno de los temas.