Skip to content

Commit 7f15a7f

Browse files
authored
Removed outdated warning about batch contamination (#4423)
1 parent 8b0a3ce commit 7f15a7f

File tree

1 file changed

+0
-3
lines changed

1 file changed

+0
-3
lines changed

docs/source/reducing_memory_usage.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,6 @@ from trl import SFTConfig
9090
training_args = SFTConfig(..., packing=True, max_length=512)
9191
```
9292

93-
> [!WARNING]
94-
> Packing may cause batch contamination, where adjacent sequences influence one another. This can be problematic for some applications. For more details, see [#1230](https://github.com/huggingface/trl/issues/1230).
95-
9693
## Liger for reducing peak memory usage
9794

9895
> [Liger Kernel](https://github.com/linkedin/Liger-Kernel) is a collection of Triton kernels designed specifically for LLM training. It can effectively increase multi-GPU training throughput by 20% and reduce memory usage by 60%.

0 commit comments

Comments
 (0)