-
Notifications
You must be signed in to change notification settings - Fork 84
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[nvfuser] Don't allow shape only nvfuser region (#1559)
Related: #1251 Microbenchmark ```python import torch import torch.utils.benchmark import thunder # Seen in HF-Qwen2 Model # [t6866] = nvFusion4(t6857) # # lora_res_12 = prims.reshape(t6857, (1, 4096, 16)) # lora_res_12: "cuda:0 bf16[1, 4096, 16]" # # t6866 = prims.reshape(lora_res_12, (4096, 16)) # t6866: "cuda:0 bf16[4096, 16]" def fn(x): x = x.reshape(-1, 4096, 16) x = x.reshape(4096, 16) return x x = torch.randn(4096, 16, device="cuda") jfn = thunder.jit(fn, nv_enable_shape_only_fusion=False) jfn(x) timer = torch.utils.benchmark.Timer("jfn(x)", globals={"jfn": jfn, "x": x}) print(timer.blocked_autorange(min_run_time=1)) # print(thunder.last_traces(jfn)[-1]) ``` Before PR ```python <torch.utils.benchmark.utils.common.Measurement object at 0x7305591c54e0> jfn(x) Median: 36.18 us 3 measurements, 10000 runs per measurement, 1 thread ``` After PR ```python <torch.utils.benchmark.utils.common.Measurement object at 0x7e533962cc10> jfn(x) Median: 18.95 us IQR: 0.02 us (18.93 to 18.96) 6 measurements, 10000 runs per measurement, 1 thread ```
- Loading branch information
1 parent
e37bec2
commit 2d0199e
Showing
4 changed files
with
70 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters