oneapi-src
MKLDNN library size increase from v1.0 to v1.4
Hi,
We are analyzing the library size difference between v1.0 and v1.4 on windows,debian,.mac platforms. We found almost 2x increase in library size from v1.0 to v1.4. We have followed the same build steps mentioned in the documentation. Let me know if this is expected?
Is there a way to decrease the size of libs in v1.4? FYI we are interested only FP32 deployment on CPUs. It would be of great help if you guide us through this.
Below are the numbers for your reference
| v1.0 | v1.4 linux | 17961kb | 31291kb mac | 20359kb | 38916kb windows | 8927kb |16867kb
Thanks in advance, Praveen.
Hi @uu-praveeng . Thank you for your question. The size of the oneDNN library did indeed increase from v1.0 and v1.4, especially given that many new features were added to the library in the default CPU build, including new primitives (lnorm, binary, matmul, logsoftmax, resampling), new algorithms for existing primitives, various data type support, new optimized primitive implementations etc. In addition to the default CPU build new threading runtime support was added, such as TBB, which may also affect the size of the library.
That being said, we also attempt to minimize the size of the library as much as possible, especially for binary distribution, which is a separate challenge. It may not be straightforward to minimize the size of the library by turning off some library features (such as data type support) using CMake flags. This may require reorganization of the library, with enabling non-x86-64 platform support as a recent example.
Do you have any limitation with respect to the size of the library? Do you expect f32 support on CPU to be your main requirement for using the library, or are there other features that you require too?
Thank you,
Silviu
Hi Silviu,
Yes we do have a limitation with size increase.Since it is bundled with our product . It increases the product size. F32 is our primary requirement. It would be great only to include f32 kernels while build instead of including everything.
Let me know if this is possible during build.
Thanks, Praveen.
Get Outlook for Androidhttps://aka.ms/ghei36
From: Silviu Bogusevschi [email protected] Sent: Thursday, October 22, 2020 11:53:29 PM To: oneapi-src/oneDNN [email protected] Cc: Praveen Kumar Gajula [email protected]; Mention [email protected] Subject: Re: [oneapi-src/oneDNN] MKLDNN library size increase from v1.0 to v1.4 (#855)
Hi @uu-praveenghttps://github.com/uu-praveeng . Thank you for your question. The size of the oneDNN library did indeed increase from v1.0 and v1.4, especially given that many new features were added to the library in the default CPU build, including new primitives (lnorm, binary, matmul, logsoftmax, resampling), new algorithms for existing primitives, various data type support, new optimized primitive implementations etc. In addition to the default CPU build new threading runtime support was added, such as TBB, which may also affect the size of the library.
That being said, we also attempt to minimize the size of the library as much as possible, especially for binary distribution, which is a separate challenge. It may not be straightforward to minimize the size of the library by turning off some library features (such as data type support) using CMake flags since. This may require reorganization of the library, with enabling non-x86-64 platform support as a recent example.
Do you have any limitation with respect to the size of the library? Do you expect f32 support on CPU to be your main requirement for using the library, or are there other features that you require too?
Thank you,
Silviu
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/oneapi-src/oneDNN/issues/855#issuecomment-714675354, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGE77UYUM6A7X4IJKRBSBTSMB2CDANCNFSM4SZTWCGQ.
@uu-praveeng, do you use static or dynamic linkage with oneDNN?
Disclaimer: unfortunately, there is no standard way to disable library pieces. However, in case of the static linkage it could be a little easier to disable irrelevant parts.
We use dynamic linking. Could you take this as feature request to add disabling of portions that we can discard during build.
Get Outlook for Androidhttps://aka.ms/ghei36
From: Evarist [email protected] Sent: Friday, October 23, 2020 9:02:35 AM To: oneapi-src/oneDNN [email protected] Cc: Praveen Kumar Gajula [email protected]; Mention [email protected] Subject: Re: [oneapi-src/oneDNN] MKLDNN library size increase from v1.0 to v1.4 (#855)
@uu-praveenghttps://github.com/uu-praveeng, do you use static or dynamic linkage with oneDNN?
Disclaimer: unfortunately, there is no standard way to disable library pieces. However, in case of the static linkage it could be a little easier to disable irrelevant parts.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/oneapi-src/oneDNN/issues/855#issuecomment-714886849, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGE77RTOLFMKOIMA6HBFU3SMD2NHANCNFSM4SZTWCGQ.
Hi @uu-praveeng . Indeed we will look into this. Our schedule is a bit busy currently, so we will do our best to have an update as soon as possible :)
Just to confirm, what would be suitable upper limit for the size of oneDNN library that will satisfy your product requirements? Also, are you interested in a particular version of the library, or are you fine with using the latest release?
Thanks,
Silviu
Relative difference between 2 releases should not be more than 30 mb.If more than 30mb then flags are raised.
Get Outlook for Androidhttps://aka.ms/ghei36
From: Silviu Bogusevschi [email protected] Sent: Friday, October 23, 2020 11:49:42 PM To: oneapi-src/oneDNN [email protected] Cc: Praveen Kumar Gajula [email protected]; Mention [email protected] Subject: Re: [oneapi-src/oneDNN] MKLDNN library size increase from v1.0 to v1.4 (#855)
Hi @uu-praveenghttps://github.com/uu-praveeng . Indeed we will look into this. Our schedule is a bit busy currently, so we will do our best to have an update as soon as possible :)
Just to confirm, what would be suitable upper limit for the size of oneDNN library that will satisfy your product requirements?
Thanks,
Silviu
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/oneapi-src/oneDNN/issues/855#issuecomment-715500607, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGE77THIEBTQAVEW27KSGDSMHCL5ANCNFSM4SZTWCGQ.
Could you please clarify? The library build does not increase by more than 30 mb between releases for one OS. You can see the difference between v1.5 and v1.6.
I understand for 1.5 to 1.6 it might not be increasing but we are upgrading lib from 1.0 to 1.4 hence we have seen this issue
Get Outlook for Androidhttps://aka.ms/ghei36
From: Silviu Bogusevschi [email protected] Sent: Saturday, October 24, 2020 5:06:00 AM To: oneapi-src/oneDNN [email protected] Cc: Praveen Kumar Gajula [email protected]; Mention [email protected] Subject: Re: [oneapi-src/oneDNN] MKLDNN library size increase from v1.0 to v1.4 (#855)
Could you please clarify? The library build does not increase by more than 30 mb between releases for one OS. You can see the difference between v1.5https://github.com/oneapi-src/oneDNN/releases/tag/v1.5 and v1.6https://github.com/oneapi-src/oneDNN/releases/tag/v1.5.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/oneapi-src/oneDNN/issues/855#issuecomment-715632749, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGE77X6XONOTGHLPOJLONLSMIHOBANCNFSM4SZTWCGQ.
Hi @uu-praveeng . Implementation of a feature is prioritized for new releases instead of as a patch release for existing build. Would implementing this feature in a new release work for you? Is there a reason you wish to upgrade to 1.4 specifically instead of the latest version?
There is no specific reason. By the time we have taken up the upgradation task,latest version available was 1.4. Hence when we went with that.
Get Outlook for Androidhttps://aka.ms/ghei36
From: Silviu Bogusevschi [email protected] Sent: Saturday, October 24, 2020 9:07:34 PM To: oneapi-src/oneDNN [email protected] Cc: Praveen Kumar Gajula [email protected]; Mention [email protected] Subject: Re: [oneapi-src/oneDNN] MKLDNN library size increase from v1.0 to v1.4 (#855)
Hi @uu-praveenghttps://github.com/uu-praveeng . Implementation of a feature is prioritized for new releases instead of as a patch release for existing build. Would implementing this feature in a new release work for you? Is there a reason you with to upgrade to 1.4 specifically instead of the latest version?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/oneapi-src/oneDNN/issues/855#issuecomment-715941701, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGE77XEPRAJD4AG7NUH5N3SMLYD5ANCNFSM4SZTWCGQ.
Hi @uu-praveeng . We will look into this and will provide an update once such a feature is being worked on. As I mentioned, it will target a new release instead of an existing version.
Thank you,
Silviu
We analyzed top contributors to oneDNN binary size and there are several factors that have the biggest impact:
- GEMM autogenerated kernels
- Templated simple reorder implementations
- Growth in functionality and extensive use of templates
We cleaned up unnecesary templates in oneDNN v2.4 and introduced build time options that allow to disable functionality that is not needed (see #1057). This addresses item (3) in the list. Dealing with (2) is the next on the list, and (1) will take the most time due to extensive amount of changes required to replace autogeneraged GEMM.
We promoted changes addressing item (2) from the list, these are targeting oneDNN v2.5 release.
Build options that allow limit ISA-specific optimizations in the library landed into the main branch. The feature is targeting oneDNN v2.5.
I'm looking at the mentioned build options and it seems allow restricting a maximum ISA (eg. AVX2 ISA so that AVX512 is not built). Typically developers build the other way -- with a minimum ISA. That is, we would expect that a client would have at least AVX1 or AVX2. That way, if there is competing implementations (SSE4.1, AVX1, AVX2 and AVX512), and you set a minimum AVX2, then you would only build in the AVX2 implementation with dispatch for AVX512. This way, you save building SSE4.1 and AVX1 and still get full performance on all systems in the past ~8 years.
Right now, as a space-conscious developer, I can only save space by removing AMX as I have no machines currently using this. However, when my software is later used with AMX machines it will not get the advantage. I feel Intel is shooting itself in the foot here by encouraging software to not take advantage of the latest ISA's rather than encouraging users to move to newer ISA's.
Hi @xsacha, thank you for your feedback. Our data shows that biggest growth comes from AVX512+ ISA so if end user application is fine to include AVX512, our guess it's also fine to include some minor piece for extra precaution since the library uses a waterfall approach when picking an implementation to run.
There are devices that don't have any AVX sets and they have limited capacity of memory to run specific tasks. This feature is targeting primarily those scenarios when machines don't need the part of the library they could never take advantage of. Thank you.
