unified-memory-framework
unified-memory-framework copied to clipboard
A library for constructing allocators and memory pools. It also contains broadly useful abstractions and utilities for memory management. UMF allows users to manage multiple memory pools characterized...
Unified Memory Framework
Introduction
The Unified Memory Framework (UMF) is a library for constructing allocators and memory pools. It also contains broadly useful abstractions and utilities for memory management. UMF allows users to manage multiple memory pools characterized by different attributes, allowing certain allocation types to be isolated from others and allocated using different hardware resources as required.
⚠️ Work-In-Progress disclaimer:
Please note that this project is pre-production software, it should not be considered complete or fully functional. It has not been fully tested yet (including security testing). It is not recommended to be used in production as part of a larger system. Note that this warning is temporary - we plan to release a stable version within six months. This project is not eligible for Intel® Bug Bounty Program.
The API is not yet stable, may change without notice, and will not maintain backward compatibility.
Usage
For a quick introduction to UMF usage, please see examples documentation, which includes the code of the basic example and the more advanced one that allocates USM memory from the GPU device using the Level Zero API and UMF Level Zero memory provider.
Build
Requirements
Required packages:
- libhwloc-dev >= 2.3.0 (Linux) / hwloc >= 2.3.0 (Windows)
- C compiler
- CMake >= 3.14.0
For development and contributions:
- clang-format-15.0 (can be installed with
python -m pip install clang-format==15.0.7
) - cmake-format-0.6 (can be installed with
python -m pip install cmake-format==0.6.13
) - black (can be installed with
python -m pip install black==24.3.0
)
For building tests, multithreaded benchmarks and Disjoint Pool:
- C++ compiler with C++17 support
For Level Zero memory provider tests:
- Level Zero headers and libraries
- compatible GPU with installed driver
Linux
Executable and binaries will be in build/bin
$ mkdir build
$ cd build
$ cmake {path_to_source_dir}
$ make
Windows
Generating Visual Studio Project. EXE and binaries will be in build/bin/{build_config}
$ mkdir build
$ cd build
$ cmake {path_to_source_dir} -G "Visual Studio 15 2017 Win64"
Benchmark
UMF comes with a single-threaded micro benchmark based on ubench.
In order to build the benchmark, the UMF_BUILD_BENCHMARKS
CMake configuration flag has to be turned ON
.
UMF also provides multithreaded benchmarks that can be enabled by setting both
UMF_BUILD_BENCHMARKS
and UMF_BUILD_BENCHMARKS_MT
CMake
configuration flags to ON
. Multithreaded benchmarks require a C++ support.
Sanitizers
List of sanitizers available on Linux:
- AddressSanitizer
- UndefinedBehaviorSanitizer
- ThreadSanitizer
- Is mutually exclusive with other sanitizers.
- MemorySanitizer
- Requires linking against MSan-instrumented libraries to prevent false positive reports. More information here.
List of sanitizers available on Windows:
- AddressSanitizer
Listed sanitizers can be enabled with appropriate CMake options.
CMake standard options
List of options provided by CMake:
Name | Description | Values | Default |
---|---|---|---|
UMF_BUILD_SHARED_LIBRARY | Build UMF as shared library | ON/OFF | OFF |
UMF_BUILD_LEVEL_ZERO_PROVIDER | Build Level Zero memory provider | ON/OFF | ON |
UMF_BUILD_LIBUMF_POOL_DISJOINT | Build the libumf_pool_disjoint static library | ON/OFF | OFF |
UMF_BUILD_LIBUMF_POOL_JEMALLOC | Build the libumf_pool_jemalloc static library | ON/OFF | OFF |
UMF_BUILD_LIBUMF_POOL_SCALABLE | Build the libumf_pool_scalable static library | ON/OFF | OFF |
UMF_BUILD_TESTS | Build UMF tests | ON/OFF | ON |
UMF_BUILD_GPU_TESTS | Build UMF GPU tests | ON/OFF | OFF |
UMF_BUILD_BENCHMARKS | Build UMF benchmarks | ON/OFF | OFF |
UMF_BUILD_EXAMPLES | Build UMF examples | ON/OFF | ON |
UMF_BUILD_GPU_EXAMPLES | Build UMF GPU examples | ON/OFF | OFF |
UMF_ENABLE_POOL_TRACKING | Build UMF with pool tracking | ON/OFF | ON |
UMF_DEVELOPER_MODE | Treat warnings as errors and enables additional checks | ON/OFF | OFF |
UMF_FORMAT_CODE_STYLE | Add clang, cmake, and black -format-check and -format-apply targets to make | ON/OFF | OFF |
USE_ASAN | Enable AddressSanitizer checks | ON/OFF | OFF |
USE_UBSAN | Enable UndefinedBehaviorSanitizer checks | ON/OFF | OFF |
USE_TSAN | Enable ThreadSanitizer checks | ON/OFF | OFF |
USE_MSAN | Enable MemorySanitizer checks | ON/OFF | OFF |
USE_VALGRIND | Enable Valgrind instrumentation | ON/OFF | OFF |
Architecture: memory pools and providers
A UMF memory pool is a combination of a pool allocator and a memory provider. A memory provider is responsible for coarse-grained memory allocations and management of memory pages, while the pool allocator controls memory pooling and handles fine-grained memory allocations.
Pool allocator can leverage existing allocators (e.g. jemalloc or tbbmalloc) or be written from scratch.
UMF comes with predefined pool allocators (see include/pool) and providers (see include/provider). UMF can also work with user-defined pools and providers that implement a specific interface (see include/umf/memory_pool_ops.h and include/umf/memory_provider_ops.h).
More detailed documentation is available here: https://oneapi-src.github.io/unified-memory-framework/
Memory providers
OS memory provider
A memory provider that provides memory from an operating system. It supports two types of memory mappings
- private memory mapping (
UMF_MEM_MAP_PRIVATE
) - shared memory mapping (
UMF_MEM_MAP_SHARED
- supported on Linux only yet)
If the shared memory mapping is used then an anonymous file descriptor for memory mapping is created using:
-
memfd_secret()
syscall - (if it is implemented and) if theUMF_MEM_FD_FUNC
environment variable does not contain the "memfd_create" string or -
memfd_create()
syscall - otherwise (and if it is implemented).
Requirements
Required packages for tests (Linux-only yet):
- libnuma-dev
Level Zero memory provider (Linux-only yet)
A memory provider that provides memory from L0 device.
Requirements
- Linux or Windows OS
- The
UMF_BUILD_LEVEL_ZERO_PROVIDER
option turnedON
(by default)
Additionally, required for tests:
- Linux OS
- The
UMF_BUILD_GPU_TESTS
option turnedON
- System with Level Zero compatible GPU
- Required packages:
- liblevel-zero-dev
Memory pool managers
proxy_pool (part of libumf)
This memory pool is distributed as part of libumf. It forwards all requests to the underlying memory provider. Currently umfPoolRealloc, umfPoolCalloc and umfPoolMallocUsableSize functions are not supported by the proxy pool.
libumf_pool_disjoint
TODO: Add a description
Requirements
To enable this feature, the UMF_BUILD_LIBUMF_POOL_DISJOINT
option needs to be turned ON
.
libumf_pool_jemalloc
libumf_pool_jemalloc is a jemalloc-based memory pool manager built as a separate static library.
The UMF_BUILD_LIBUMF_POOL_JEMALLOC
option has to be turned ON
to build this library.
Requirements
- The
UMF_BUILD_LIBUMF_POOL_JEMALLOC
option turnedON
- Required packages:
- libjemalloc-dev (Linux) or jemalloc (Windows)
libumf_pool_scalable
libumf_pool_scalable is a oneTBB-based memory pool manager built as a separate static library.
The UMF_BUILD_LIBUMF_POOL_SCALABLE
option has to be turned ON
to build this library.
Requirements
- The
UMF_BUILD_LIBUMF_POOL_SCALABLE
option turnedON
- Required packages:
- libtbb-dev (libtbbmalloc.so.2) on Linux or tbb (tbbmalloc.dll) on Windows
Memspaces (Linux-only)
TODO: Add general information about memspaces.
Host all memspace
Memspace backed by all available NUMA nodes discovered on the platform. Can be retrieved using umfMemspaceHostAllGet.
Highest capacity memspace
Memspace backed by all available NUMA nodes discovered on the platform sorted by capacity. Can be retrieved using umfMemspaceHighestCapacityGet.
Proxy library
UMF provides the UMF proxy library (umf_proxy
) that makes it possible
to override the default allocator in other programs in both Linux and Windows.
Linux
In case of Linux it can be done without any code changes using the LD_PRELOAD
environment variable:
$ LD_PRELOAD=/usr/lib/libumf_proxy.so myprogram
The memory used by the proxy memory allocator is mmap'ed:
- with the
MAP_PRIVATE
flag by default or - with the
MAP_SHARED
flag only if theUMF_PROXY
environment variable contains thepage.disposition=shared
string.
Windows
In case of Windows it requires:
- explicitly linking your program dynamically with the
umf_proxy.dll
library - (C++ code only) including
proxy_lib_new_delete.h
in a single(!) source file in your project to override also thenew
/delete
operations.
Contributions
All contributions to the UMF project are most welcome! Before submitting an issue or a Pull Request, please read Contribution Guide.
Logging
To enable logging in UMF source files please follow the guide in the web documentation.