Google and Meta Open-source TorchTPU to Bring Native PyTorch Support to Google TPUs

8.3 relevance

TorchTPU open-sourcing brings native PyTorch to TPUs, directly relevant to AI/ML infrastructure and cloud compute.

DevTools infoq.com

Answer the board on AI readiness

Arrive with evidence, not opinion. See how your foundation benchmarks against your peers, get a clear read, and leave with a ranked place to start.

Why It Matters

For engineers building AI/ML workloads on GCP, TorchTPU eliminates the friction of migrating from GPUs to TPUs, enabling native PyTorch code to run with minimal changes and unlocking TPU-specific performance gains for large-scale training and inference.

Key Takeaways

Evaluate TorchTPU for existing PyTorch workloads on GCP TPUs by swapping device strings and testing the DeferAndFuse mode for potential 50% speedups over strict eager execution.

Google and Meta Open-source TorchTPU to Bring Native PyTorch Support to Google TPUs

Summary

Google and Meta open-sourced TorchTPU, a native PyTorch backend for Google TPUs that replaces the prior PyTorch/XLA approach. TorchTPU prioritizes eager execution, requires minimal code changes (e.g., replacing `cuda` with `tpu`), and introduces a DeferAndFuse hybrid mode that Google claims can be 50% faster than strict eager. In benchmarks, a 30B-parameter Muse Glimmer model achieved 25.7k token/s inference on a single TPUv7 chip, while Qwen3 MoE 235B scaled to 8,192 TPUv7 chips for training.

Author

Sergio De Simone