iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

TorchScript

By · Reviewed & last updated · Part of PyTorch Tutorial

TorchScript is PyTorchs intermediate representation, produced by tracing (running with example inputs) or scripting (parsing your Python). The resulting .pt file runs without Python via libtorch — handy for C++/Java/Swift servers, mobile apps, or edge devices. ONNX export is the alternative when you target a non-PyTorch runtime.

Trace, script, save, and run a model

EXAMPLE
import torch
import torch.nn as nn

class TinyNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 32)
        self.fc2 = nn.Linear(32, 2)
    def forward(self, x):
        x = torch.relu(self.fc1(x))
        return self.fc2(x)

model = TinyNet().eval()

# 1) Tracing — fast, but loses control flow that depends on data
example = torch.randn(1, 10)
traced = torch.jit.trace(model, example)
traced.save('tinynet_traced.pt')

# 2) Scripting — slower to write (must be TorchScript-compatible)
#    but preserves if/for/while based on tensor shape
scripted = torch.jit.script(model)
scripted.save('tinynet_scripted.pt')

# 3) Both produce ScriptModule files that load without your model code
loaded = torch.jit.load('tinynet_traced.pt').eval()
print(loaded(example))

# 4) Mixed: trace at the top level, mark a branch as @torch.jit.script
class ConditionalNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(10, 2)
    def forward(self, x: torch.Tensor, use_softmax: bool):
        out = self.fc(x)
        if use_softmax:
            out = torch.softmax(out, dim=-1)
        return out

scripted2 = torch.jit.script(ConditionalNet().eval())
torch.jit.save(scripted2, 'cond.pt')

# 5) Optimise for inference (operator fusion, constant folding)
opt = torch.jit.optimize_for_inference(scripted)
opt.save('tinynet_opt.pt')

# 6) Load and run from C++ (sketch):
# #include <torch/script.h>
# auto module = torch::jit::load('tinynet_traced.pt');
# auto out = module.forward({torch::randn({1, 10})}).toTensor();

# 7) ONNX path — for runtimes that are not libtorch
torch.onnx.export(model, example, 'tinynet.onnx',
                  input_names=['x'], output_names=['logits'],
                  dynamic_axes={'x': {0: 'batch'}, 'logits': {0: 'batch'}},
                  opset_version=17)

Why it matters

Trace when the model is a feedforward graph with no data-dependent control flow; script when there is `if` or `for` over tensor shapes. Mixing the two (script the branchy submodule, trace the rest) gets you the speed of tracing without losing correctness on the parts that need real control flow.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
scripted = torch.jit.script(model)
scripted.save('model.ts')
restored = torch.jit.load('model.ts')
Try it Yourself »

Discussion

Loading…