Sequence Packing

Translation pending

This page has not been translated yet. Please read the Chinese version.

Intuition and Motivation

Method and Formulation

Bin Packing: How to Combine

Cross-contamination: Attention Must Be Isolated

FlashAttention varlen Interface

Comparison with Baselines

Implementation Notes

Tuning and Practical Experience

References