the world’s largest distributed LLM training job on TPU v5e | Google Cloud Blog
cloud.google.com
We used Multislice Training to run the world’s largest LLM distributed training job on a compute cluster of 50,944 Cloud TPU v5e chips.