PyTorch¶ PyTorch General Mixed Precision Distributed data parallel DDP under the hood DP vs DDP FSDP Tensor Parallelism Pipeline Parallelism Device Mesh