[Target] Add target_device_type attribute to override default device_type

[kparzysz-quic]

Implement Target::GetTargetDeviceType (C++) or get_target_device_type (python) to get the device type (kDL...) for a given target.

The attribute "target_device_type" can be used to override the default device type associated with the target kind.

[kparzysz-quic]

I tried to start a discussion on the forum, but it's not attracting a lot of attention...

[kparzysz-quic]

It started the way you described, but the Vulkan target defines a property "device_type", which is unrelated to the TVM's device type. I'd actually prefer to avoid the extra "target" word, but I'm not sure how do deal with the Vulkan name conflict. "Device type" is actually the term used by Vulkan docs, but maybe we can rename it in TVM...

[areusch]

for context for the Community Meeting tomorrow: hexagon is a backend which effectively compiles code in the same way as the LLVM target, but which at runtime needs a different Device API. At compile time, there's no reason that hexagon needs to use a different target.build. However, at present, we tie the runtime device_type to the Target kind.

There are a few different options at least for how we could resolve this:

1. allow this PR through, which lets the user configure it. However, a user may not necessarily understand which device type to use, and there may exist cases where the target device type shouldn't be included in tuning logs.
2. leave device_type for hexagon to be kDLCPU, and override the CPU Device API via patching the runtime::Registry.
3. allow targets to be flexible in determining their device_type, and make llvm -mcpu=hexagonvNN choose hexagon as the device type. this avoids the need to patch the registry.

[areusch]

We discussed this in the TVM Community Meeting this morning. Here are notes:

• All targets derive from a given target kind
• Targets derived from the llvm target kind share common components but it is special cased in TVM
  • LLVM targets are presumed to generate a shared library
• Target kind’s also share a device type relationship, and for llvm this is exclusivel kDLCPU
• Hexagon is an llvm target, however the HexagonDeviceAPI used for Hexagon is unique and cannot be supported by the CPUDeviceAPI
• Desire: Use the llvm target for the compiler and codegeneration, but utilize a unique runtime DeviceAPI
  • Q: (and possibly a unique DLDeviceType?)
  • Right now to achieve this one has to override the global registration for a kDLCPU DeviceAPI with the HexagonDeviceAPI
• This PR proposes that just like a target can derive it’s properties from a target kind, the target can override the default device type so that a target is not bound to a specific device (ie for llvm the user wouldn’t be required to utilize a kDLCPU device type for the backing runtime apis)
  • GetTargetDeviceType is the proposed api in the PR to offer the ability for different target to provide their own DeviceAPI.
• Are there other analogies
• Andrew:
  • Two things being coordinated
    • 1. A runtime module is produced (not contentious)
    • 2. The C++ DeviceAPI, this can change between targets
  • Work has been done in the C-API to support different type of hardware runtimes for accelerators
    • Currently makes global function calls based on names with a device name prefix
• Two places to extend the abstraction
• Current approach in PR has the advantage of changing existing code
  • If we were going to map multiple device types to multiple device api’s there would likely need to be more code changes.
• The DeviceAPI is some what a metadata structure that is provided
• Virtual device: goal to map between device type and particular subtarget
• Making a change to allow a target to have it’s own target device type would require making some changes about virtual device subtargetting
• KP: If this PR goes in, I would support it with Hexagon and then would try to address different the overuse of kDLCPU as host devices

[kparzysz-quic]

Hi @areusch ... So what should we do with this PR? I don't think there was any opposition to this, but we didn't really reach the final decision in the community meeting either...

[areusch]

@tvm-bot rerun

[areusch]

can you sync this one though and get it through CI again? usually we break main by submitting a test that passed CI weeks ago