[Misc] Investigate whether we need buffer for inference in #6557

[frozenbugs]

🔨Work Item

https://github.com/dmlc/dgl/pull/6557

IMPORTANT:

• This template is only for dev team to track project progress. For feature request or bug report, please use the corresponding issue templates.
• DO NOT create a new work item if the purpose is to fix an existing issue or feature request. We will directly use the issue in the project tracker.

Project tracker: https://github.com/orgs/dmlc/projects/2

Description

Depending work items or issues