xla icon indicating copy to clipboard operation
xla copied to clipboard

[IFRT] Introduce Client::AllocateDevices() and DeviceAllocation

Open copybara-service[bot] opened this issue 1 year ago • 0 comments

[IFRT] Introduce Client::AllocateDevices() and DeviceAllocation

xla::ifrt::Client::AllocateDevices() is a new API that processes a user request for getting an ordered set of devices that satisfies constraints specified in the request. It returns a new runtime object DeviceAllocation, which represents the allocated devices as a collection, which will be used where DeviceList is used today.

Main use cases of the new API will include:

  • The user can get a set of devices in a platform-specific manner (e.g., TPU user code typically uses a logical "mesh" that consists of devices ordered in a certain way to make collectives run efficiently).

  • The user can get a set of devices in relation to other set of devices (finding CPU devices colocated with accelerator devices).

  • The user can request allocating a new set of devices that do not overlap with in-use device to run computations in parallel.

This initial change implements the interface of the API and object type, and a simple implementation BasicDeviceAllocation that wraps around a DeviceList to help transition from DeviceList to DeviceAllocation in the future.

copybara-service[bot] avatar Jul 25 '24 23:07 copybara-service[bot]