xla
xla copied to clipboard
[IFRT] Introduce Client::AllocateDevices() and DeviceAllocation
[IFRT] Introduce Client::AllocateDevices() and DeviceAllocation
xla::ifrt::Client::AllocateDevices() is a new API that processes a user
request for getting an ordered set of devices that satisfies constraints
specified in the request. It returns a new runtime object DeviceAllocation,
which represents the allocated devices as a collection, which will be used
where DeviceList is used today.
Main use cases of the new API will include:
-
The user can get a set of devices in a platform-specific manner (e.g., TPU user code typically uses a logical "mesh" that consists of devices ordered in a certain way to make collectives run efficiently).
-
The user can get a set of devices in relation to other set of devices (finding CPU devices colocated with accelerator devices).
-
The user can request allocating a new set of devices that do not overlap with in-use device to run computations in parallel.
This initial change implements the interface of the API and object type, and a
simple implementation BasicDeviceAllocation that wraps around a DeviceList
to help transition from DeviceList to DeviceAllocation in the future.