"Corrected CP from 8 to 16 in 3 locations (§0 TL;DR via interruption note, §10.2, §13.1, Q18, Q25)", "Removed 'fp8/bf16 mixed'; replaced with 'BF16 training; FP8 is inference quantization'", "Updated ...

* Pre-train a GPT-2 (~124M-parameter) language model using PyTorch and Hugging Face Transformers. * Distribute training across multiple GPUs with Ray Train with minimal code changes. * Stream training ...