Popular repositories Loading

1. Sequoia Sequoia Public

   scalable and robust tree-based speculative decoding algorithm

   Python 331 37

2. TriForce TriForce Public

   [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

   Python 236 15

3. MagicPIG MagicPIG Public

   [ICLR2025] MagicPIG: LSH Sampling for Efficient LLM Generation

   Python 181 12

4. MagicDec MagicDec Public

   [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

   Python 105 6

5. UMbreLLa UMbreLLa Public

   LLM Inference on consumer devices

   Python 81 5

6. Sirius Sirius Public

   Sirius, an efficient correction mechanism, which significantly boosts Contextual Sparsity models on reasoning tasks while maintaining its efficiency gain.

   Python 21 4