pytorch

pytorch copied to clipboard

Pitch

Enable Linear-Eltwise fusion in NNC.

Description

The code change is similar to https://github.com/pytorch/pytorch/pull/77157 which has enabled conv2d related fusions. This PR adds a fusion pass to fuse Linear with elementwise OP for TE subgraph. This pass will insert prepack and packed run ops for linear and enable fusion of linear with elementwise OPs. The fused packed run ops is implemented via external call in NNC.

The below elementwise fusion with Linear is added in this PR:

• relu
• sigmoid
• tanh
• leaky_relu
• hardtanh
• gelu
• clamp

Code structure

NNC integration of fused linear-eltwise OP via external call is located in:

torch/csrc/jit/tensorexpr/kernel.cpp

torch/csrc/jit/tensorexpr/operators/matmul.h
torch/csrc/jit/tensorexpr/operators/matmul.cpp

torch/csrc/jit/tensorexpr/lowerings.cpp
torch/csrc/jit/tensorexpr/external_functions.cpp

Fused linear OP implementation is done in:

aten/src/ATen/native/mkldnn/LinearPrepack.h
aten/src/ATen/native/mkldnn/LinearPrepack.cpp

This PR has the dependency on https://github.com/pytorch/pytorch/pull/83249. I've separated the changes of https://github.com/pytorch/pytorch/pull/83249 in one commit here https://github.com/pytorch/pytorch/commit/c5b24227d3c9ded70ddb268a622986497cc4182e.

UT

test/test_mkldnn_fusion.py

Aug 23 '22 07:08 chunyuan-w

:link: Helpful links

• :test_tube:  See artifacts and rendered test results at hud.pytorch.org/pr/83896
• :page_facing_up:  Preview Python docs built from this PR
• :page_facing_up:  Preview C++ docs built from this PR
• :question:Need help or want to give feedback on the CI? Visit our office hours

:x: 1 New Failures, 3 Pending

As of commit 04ef186b9a (more details on the Dr. CI page):

2022-08-24T03:34:50.4021048Z processing existing schema:  duration_ns(__torch__.torch.classes.profiling.InstructionStats _0) -> int _0
2022-08-24T03:34:50.4022695Z processing existing schema:  source(__torch__.torch.classes.profiling.SourceStats _0) -> __torch__.torch.classes.profiling.SourceRef _0
2022-08-24T03:34:50.4024967Z processing existing schema:  line_map(__torch__.torch.classes.profiling.SourceStats _0) -> Dict(int, __torch__.torch.classes.profiling.InstructionStats) _0
2022-08-24T03:34:50.4026600Z processing existing schema:  __init__(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-08-24T03:34:50.4027758Z processing existing schema:  enable(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-08-24T03:34:50.4029548Z processing existing schema:  disable(__torch__.torch.classes.profiling._ScriptProfile _0) -> NoneType _0
2022-08-24T03:34:50.4031697Z processing existing schema:  _dump_stats(__torch__.torch.classes.profiling._ScriptProfile _0) -> __torch__.torch.classes.profiling.SourceStats[] _0
2022-08-24T03:34:50.4033397Z processing existing schema:  __init__(__torch__.torch.classes.c10d.ProcessGroup _0, int _1, int _2) -> NoneType _0
2022-08-24T03:34:50.4034778Z processing existing schema:  __init__(__torch__.torch.classes.c10d.Work _0) -> NoneType _0
2022-08-24T03:34:50.4036505Z processing existing schema:  __init__(__torch__.torch.classes.dist_rpc.WorkerInfo _0, str _1, int _2) -> NoneType _0
2022-08-24T03:34:50.4037176Z The PR is introducing backward incompatible changes to the operator library. Please contact PyTorch team to confirm whether this change is wanted or not. 
2022-08-24T03:34:50.4037657Z 
2022-08-24T03:34:50.4037791Z Broken ops: [
2022-08-24T03:34:50.4038353Z 	mkldnn_prepacked::conv2d_prepack(Tensor W, Tensor? B, int[2] stride, int[2] padding, int[2] dilation, int groups, int[4] input_size, str attr) -> __torch__.torch.classes.mkldnn.ConvOpContext
2022-08-24T03:34:50.4038907Z 	__getstate__(__torch__.torch.classes.mkldnn.ConvOpContext _0) -> ((Tensor, Tensor?, int[], int[], int[], int, int[], str) _0)
2022-08-24T03:34:50.4039481Z 	__setstate__(__torch__.torch.classes.mkldnn.ConvOpContext _0, (Tensor, Tensor?, int[], int[], int[], int, int[], str) _1) -> NoneType _0
2022-08-24T03:34:50.4039765Z ]
2022-08-24T03:34:50.6219965Z + sccache_epilogue
2022-08-24T03:34:50.6220643Z + echo '::group::Sccache Compilation Log'
2022-08-24T03:34:50.6222637Z ##[group]Sccache Compilation Log
2022-08-24T03:34:50.6222974Z + echo '=================== sccache compilation log ==================='

Aug 23 '22 07:08 facebook-github-bot

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/83896

• :page_facing_up: Preview Python docs built from this PR
• :page_facing_up: Preview C++ docs built from this PR
• :question: Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 Failures, 2 Pending

As of commit 9c7e51676b37072073c7c3881d9c6002019d99a3:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Sep 21 '22 07:09 pytorch-bot[bot]

/easycla

As part of the transition to the PyTorch Foundation, this project now requires contributions be covered under the new CLA. See #85559 for additional details.

This comment will trigger a new check of this PR. If you are already covered, you will simply see a new "EasyCLA" check that passes. If you are not covered, a bot will leave a new comment with a link to sign.

Oct 04 '22 00:10 facebook-github-bot

• :x: - login: @chunyuan-w / name: Chunyuan WU . The commit (c5b24227d3c9ded70ddb268a622986497cc4182e, b45cdc399fa67f1665438c5fb5faefd5291a59ef, 79c30eb01271cc1608ab71bb8d66f5872cd8f60a, ec781583690f97ae13d85020d579e40674aa0a6a, f94c743090c0547b4edaaa676f63bb8598f67d73, f1c686b22cb23bb695319ccc505ad4a3a6e79b50, 15fd073f2da8da7a3eb4264cbcd6a94b0fd96fe8, a7f8bce8336f200b352c784580ee1967c90ab8d2, 04ef186b9a4105a92a5c6c2533ce67a12c4b1ecf, b08eedc2896210957f2a115b29680069d8ac3d71, 9c7e51676b37072073c7c3881d9c6002019d99a3) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

Oct 04 '22 00:10 linux-foundation-easycla[bot]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

Dec 03 '22 02:12 github-actions[bot]