attention-mechanisms

20 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

PaLM-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture.…

Awesome-Transformer-Attention

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related…

musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in…

toolformer-pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

alphafold3-pytorch

Implementation of Alphafold 3 from Google Deepmind in Pytorch

awesome-attention-mechanism-in-cv

Awesome List of Attention Modules and Plug&Play Modules in Computer Vision

muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI…

meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in…

magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

MEGABYTE-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of…

robotic-transformer-pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch

triplet-attention

Official PyTorch Implementation for "Rotate to Attend: Convolutional Triplet Attention Module." [WACV 2021]

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out…

clinical-calculator-tooluse

Explorations into training LLMs to use clinical calculators from patient history, using open sourced models.…

🔗 Related families

attention-mechanism 54 attention-model 6 text-to-image 33 mesh-generation 4 arxiv 18 long-context 6

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.