attention-mechanism
54 projects share this GitHub topic
Lines connect members that are measurably related to each other. Dot size reflects stars.
🧬 Members
LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
★ 99.5k
vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a…
★ 25.4k
nndl
邱锡鹏《神经网络与深度学习》（蒲公英书）理论书 v2 与通识版
★ 19k
RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT…
★ 14.6k
x-transformers
A concise but complete full-attention transformer with a set of promising experimental features from various…
★ 5.9k
DALLE-pytorch
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
★ 5.6k
Awesome-Transformer-Attention
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related…
★ 5k
RuVector
RuVector is a High Performance, Real-Time, Self-Learning Ai, Vector GNN, Memory DB built in Rust.
★ 4.4k
awesome-speech-recognition-speech-synthesis-papers
Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language…
★ 3.1k
a-PyTorch-Tutorial-to-Image-Captioning
Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
★ 2.9k
whisper-timestamped
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
★ 2.8k
keras-attention
Keras Attention Layer (Luong and Bahdanau scores).
★ 2.8k
pytorch-GAT
My implementation of the original GAT paper (Veličković et al.). I've additionally included the…
★ 2.7k
how-to-train-your-gpt
Build a modern LLM from scratch. Every line commented. Explained like we are five.
★ 2.3k
reformer-pytorch
Reformer, the efficient Transformer, in Pytorch
★ 2.2k
alphafold2
To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the…
★ 1.6k
soundstorm-pytorch
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
★ 1.5k
lambda-networks
Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute
★ 1.5k
llm-internals
Learn LLM internals step by step - from tokenization to attention to inference optimization.
★ 1.3k
Chatbot_CN
★ 1.3k
flamingo-pytorch
Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of…
★ 1.3k
self-attention-cv
Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.
★ 1.2k
CoCa-pytorch
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
★ 1.2k
performer-pytorch
An implementation of Performer, a linear attention-based transformer, in Pytorch
★ 1.2k
EEG-DL
A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.
★ 1.2k
OpenSTL
OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning
★ 1.1k
pytorch-original-transformer
My implementation of the original transformer model (Vaswani et al.). I've additionally included the…
★ 1.1k
GeoTransformer
[CVPR2022] Geometric Transformer for Fast and Robust Point Cloud Registration
★ 966
RETRO-pytorch
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
★ 877
PaLM-pytorch
Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways
★ 825
TimeSformer-pytorch
Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification
★ 729
MIRNet
[ECCV 2020] Learning Enriched Features for Real Image Restoration and Enhancement. SOTA results for image…
★ 723
Transformer-TTS
A Pytorch Implementation of "Neural Speech Synthesis with Transformer Network"
★ 690
bottleneck-transformer-pytorch
Implementation of Bottleneck Transformer in Pytorch
★ 678
memorizing-transformers-pytorch
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of…
★ 646
Deepdive-llama3-from-scratch
Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement…
★ 631
metal-flash-attention
FlashAttention (Metal Port)
★ 612
zeta
Build high-performance AI models with modular building blocks
★ 597
neural_sp
End-to-end ASR/LM implementation with PyTorch
★ 594
RT-2
Democratization of RT-2 "RT-2: New model translates vision and language into action"
★ 582
nuwa-pytorch
Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch
★ 548
parti-pytorch
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
★ 538
MultiModalMamba
A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model.…
★ 473
triplet-attention
Official PyTorch Implementation for "Rotate to Attend: Convolutional Triplet Attention Module." [WACV 2021]
★ 441
Awesome-Attention-Heads
An awesome repository & A comprehensive survey on interpretability of LLM attention heads.
★ 412
Star-Attention
Efficient LLM Inference over Long Sequences
★ 392
FLASH-pytorch
Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"
★ 372
sign-language-translator
Python library & framework to build custom translators for the hearing-impaired and translate between Sign…
★ 359
seq2seq-summarizer
Pointer-generator reinforced seq2seq summarization in PyTorch
★ 358
PALM-E
Implementation of "PaLM-E: An Embodied Multimodal Language Model"
★ 338
seq2seq_chatbot
基于seq2seq模型的简单对话系统的tf实现，具有embedding、attention、beam_search等功能，数…
★ 335
tensorflow_end2end_speech_recognition
End-to-End speech recognition implementation base on TensorFlow (CTC, Attention, and MTL training)
★ 314
nanodl
JAX library for training sub-4B foundation models for edge
★ 306
TeaLeaves
End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer,…
★ 42 · GitHub ↗
🔗 Related families
Measured from GitHub topics shared by both projects, weighted by how rare each topic is.