FBGEMM
FBGEMM copied to clipboard
pytorch
Ability to compile using MinGW build tools
In this pull request I have done some code changes to allow mingw to build FBGEMM, everything builds and links, tests and benchmarks too. fix #2113
This is the build procedure
#run on a msys2 enviroment
mkdir -p build
cd build
cmake .. -G"MSYS Makefiles"
make
make test
I had trouble setting some flags for the tests, so I hardcoded FBGEMM_STATIC under mingw builds, I only did this to prove the point that mingw supports this compilation, and runs tests, however I lack the skills to edit the cmake so that the compilation just works, linkage error only happens when building the tests.
These are the result of the tests
$ make test
Running tests...
Test project C:/path/pytorch-help/FBGEMM/build
Start 1: Bfloat16ConvertTest
1/25 Test #1: Bfloat16ConvertTest .............. Passed 0.09 sec
Start 2: EmbeddingSpMDM8BitTest
2/25 Test #2: EmbeddingSpMDM8BitTest ........... Passed 0.54 sec
Start 3: EmbeddingSpMDMNBitTest
3/25 Test #3: EmbeddingSpMDMNBitTest ........... Passed 0.95 sec
Start 4: EmbeddingSpMDMTest
4/25 Test #4: EmbeddingSpMDMTest ...............***Failed 4.57 sec
Start 5: FP16Test
5/25 Test #5: FP16Test .........................***Exception: SegFault 0.05 sec
Start 6: Float16ConvertTest
6/25 Test #6: Float16ConvertTest ............... Passed 0.06 sec
Start 7: GConvTest
7/25 Test #7: GConvTest ........................Exit code 0xc0000374
***Exception: 4.72 sec
Start 8: I64Test
8/25 Test #8: I64Test .......................... Passed 1.20 sec
Start 9: I8DepthwiseTest
9/25 Test #9: I8DepthwiseTest .................. Passed 45.08 sec
Start 10: I8DirectconvTest
10/25 Test #10: I8DirectconvTest ................. Passed 5.64 sec
Start 11: I8SpmdmTest
11/25 Test #11: I8SpmdmTest ...................... Passed 14.93 sec
Start 12: Im2ColFusedRequantizeTest
12/25 Test #12: Im2ColFusedRequantizeTest ........ Passed 18.40 sec
Start 13: PackedRequantizeAcc16Test
13/25 Test #13: PackedRequantizeAcc16Test ........ Passed 1.61 sec
Start 14: PackedRequantizeTest
14/25 Test #14: PackedRequantizeTest ............. Passed 31.11 sec
Start 15: QuantUtilsTest
15/25 Test #15: QuantUtilsTest ................... Passed 0.16 sec
Start 16: RadixSortTest
16/25 Test #16: RadixSortTest .................... Passed 0.05 sec
Start 17: RequantizeOnlyTest
17/25 Test #17: RequantizeOnlyTest ............... Passed 0.04 sec
Start 18: RowWiseSparseAdagradFusedTest
18/25 Test #18: RowWiseSparseAdagradFusedTest .... Passed 27.23 sec
Start 19: SparseAdagradTest
19/25 Test #19: SparseAdagradTest ................ Passed 0.53 sec
Start 20: SparseDenseMMFP32Test
20/25 Test #20: SparseDenseMMFP32Test ............ Passed 3.37 sec
Start 21: SparseDenseMMInt8Test
21/25 Test #21: SparseDenseMMInt8Test ............ Passed 5.57 sec
Start 22: SparsePackUnpackTest
22/25 Test #22: SparsePackUnpackTest ............. Passed 0.08 sec
Start 23: TransposeTest
23/25 Test #23: TransposeTest .................... Passed 0.07 sec
Start 24: TransposedRequantizeTest
24/25 Test #24: TransposedRequantizeTest ......... Passed 0.06 sec
Start 25: UniConvTest
25/25 Test #25: UniConvTest ...................... Passed 26.91 sec
88% tests passed, 3 tests failed out of 25
Total Test time (real) = 193.31 sec
The following tests FAILED:
4 - EmbeddingSpMDMTest (Failed)
5 - FP16Test (SEGFAULT)
7 - GConvTest (Exit code 0xc0000374
)
Errors while running CTest
Output from these tests are in: C:/path/pytorch-help/FBGEMM/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
make: *** [Makefile:71: test] Error 8
I am open for feedback and suggestions.
In case you're curious in the previous commit, you can also build using the shared library of asmjit that you can get with your msys distribution, but this probably polutes the project, so I took it out.
Deploy Preview for pytorch-fbgemm-docs canceled.
| Name | Link |
|---|---|
| Latest commit | 77cb1f98585d77aebd6ba1d1d9a456b8054386cb |
| Latest deploy log | https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/654e9b2722be1c0008d384ea |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @Kreijstal!
Thank you for your pull request and welcome to our community.
Action Required
In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.
Process
In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.
Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.
If you have received this in error or have any questions, please contact us at [email protected]. Thanks!
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
The installation procedure is documented here https://www.msys2.org/
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
The installation procedure is documented here https://www.msys2.org/
The reason I asked this question is because the instructions at msys2 don't seem to apply well to the Windows CI runners.
The default gcc appears to be already installed in the runners, but are stuck at 8.0, while the more recent-versioned gcc packaged with mingw or msys2 (both installed through choco) can't replace the default one in %PATH%.
I've been trying to integrate MinGW builds into GitHub CI over at https://github.com/pytorch/FBGEMM/pull/2134, but compilation is failing at the linking stage (see here). Could you see if you can integrate the changes from that PR into this PR?
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
The installation procedure is documented here https://www.msys2.org/
The reason I asked this question is because the instructions at msys2 don't seem to apply well to the Windows CI runners. The default gcc appears to be already installed in the runners, but are stuck at 8.0, while the more recent-versioned gcc packaged with mingw or msys2 (both installed through choco) can't replace the default one in %PATH%.
I've been trying to integrate MinGW builds into GitHub CI over at #2134, but compilation is failing at the linking stage (see here). Could you see if you can integrate the changes from that PR into this PR?
Sure, I can work on that, do you mind if I use this helper? https://github.com/marketplace/actions/setup-msys2
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
The installation procedure is documented here https://www.msys2.org/
The reason I asked this question is because the instructions at msys2 don't seem to apply well to the Windows CI runners. The default gcc appears to be already installed in the runners, but are stuck at 8.0, while the more recent-versioned gcc packaged with mingw or msys2 (both installed through choco) can't replace the default one in %PATH%. I've been trying to integrate MinGW builds into GitHub CI over at #2134, but compilation is failing at the linking stage (see here). Could you see if you can integrate the changes from that PR into this PR?
Sure, I can work on that, do you mind if I use this helper?
https://github.com/marketplace/actions/setup-msys2
That should be fine, yes.
Could you let me know what version of msys2 gcc you are using, and how to install them on a Windows machine?
The installation procedure is documented here https://www.msys2.org/
The reason I asked this question is because the instructions at msys2 don't seem to apply well to the Windows CI runners. The default gcc appears to be already installed in the runners, but are stuck at 8.0, while the more recent-versioned gcc packaged with mingw or msys2 (both installed through choco) can't replace the default one in %PATH%. I've been trying to integrate MinGW builds into GitHub CI over at #2134, but compilation is failing at the linking stage (see here). Could you see if you can integrate the changes from that PR into this PR?
Sure, I can work on that, do you mind if I use this helper?
https://github.com/marketplace/actions/setup-msys2That should be fine, yes.
please forgive the delays, I have lots of uni stuff at the moment, I'll catch up when I can :)