Unbacked SymInts

Unbacked SymInts

Update: 2023-02-21
Share

Description

This podcast goes over the basics of unbacked SymInts. You might want to listen to this one before listening to https://pytorch-dev-podcast.simplecast.com/episodes/zero-one-specialization Some questions we answer (h/t from Gregory Chanan):

 

- Are unbacked symints only for export?  Because otherwise I could just break / wait for the actual size.  But maybe I can save some retracing / graph breaks perf if I have them too?  So the correct statement is "primarily" for export?

- Why am I looking into the broadcasting code at all?  Naively, I would expect the export graph to be just a list of ATen ops strung together.  Why do I recurse that far down?  Why can't I annotate DONT_TRACE_ME_BRO?

- How does 0/1 specialization fit into this?  I understand we may want to 0/1 specialize in a dynamic shape regime in "eager" mode (is there a better term?), but that doesn't seem to matter for export?

- So far we've mainly been talking about how to handle our own library code.  There is a worry about pushing complicated constraints downstream, similar to torchscript.  What constraints does this actually push?

Comments 
In Channel
Compiler collectives

Compiler collectives

2024-08-0416:33

Higher order operators

Higher order operators

2024-04-2117:10

CUDA graph trees

CUDA graph trees

2024-03-2420:50

Min-cut partitioner

Min-cut partitioner

2024-03-1715:56

AOTInductor

AOTInductor

2024-03-0217:30

Compiled autograd

Compiled autograd

2024-02-1918:07

PT2 extension points

PT2 extension points

2024-02-0515:54

Unsigned integers

Unsigned integers

2024-01-1713:07

Inductor - IR

Inductor - IR

2024-01-1618:00

Unbacked SymInts

Unbacked SymInts

2023-02-2121:31

torchdynamo

torchdynamo

2022-12-0625:35

PyTorch 2.0

PyTorch 2.0

2022-12-0417:51

History of functorch

History of functorch

2022-11-0719:10

loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Unbacked SymInts

Unbacked SymInts

PyTorch