I really dislike that some code becomes much less ergonomic because of the static shape restriction.
What we can actually do is:
- Support non-static ops in eager mode.
- Crate a smarter
jit(). The way torch inductor handles some ops is via graph breaks. I.e. if I were to do something like jit(\(x) mean(unique(x)), then it could factor this out into two exectuables, the eager one that just calls unique() and then the second executable that calls the mean.
Of course, we still suffer from re-compiles, but I currently don't really see how we can easily get around this. IREE is the best option but does not have enough adoption.
I really dislike that some code becomes much less ergonomic because of the static shape restriction.
What we can actually do is:
jit(). The way torch inductor handles some ops is via graph breaks. I.e. if I were to do something likejit(\(x) mean(unique(x)), then it could factor this out into two exectuables, the eager one that just callsunique()and then the second executable that calls the mean.Of course, we still suffer from re-compiles, but I currently don't really see how we can easily get around this. IREE is the best option but does not have enough adoption.