[Eager]Support Lazy initialization for nn.Layer

[Aurelius84]

PR types

New features

PR changes

APIs

Describe

what's New?

• 动态图 Layer 实例化时，新增模型参数延迟初始化功能

• 新增 LazyInit 类接口，支持两种方式触发参数的Lazy初始化

  • 方式一：LazyInit(Model)(...)，即在实例化时，用 LazyInit 进行包裹
  • 方式二：with LazyInit() as lz 方式，在with block内的所有Parameter参数都会lazy初始化

• 给EagerParamBase 新增一个initialize() 接口，用于应用Lazy hook函数，以自定义初始化param的数据

import paddle
from paddle.nn import Linear
from paddle.fluid.framework import LazyInit
paddle.set_device('cpu')

weight_attr = paddle.ParamAttr(
    name="weight",
    initializer=paddle.nn.initializer.Constant(value=0.5))

bias_attr = paddle.ParamAttr(
    name="bias",
    initializer=paddle.nn.initializer.Constant(value=0.3))

# Usage 1:  LazyInit(Model)
model = LazyInit(Linear)(2, 4, weight_attr=weight_attr, bias_attr=bias_attr)

print(model.weight)       # <------ not initialized
program = model.startup_program

# initialize param
for param in model.parameters():
    param.initialize()     # <------ trigger lazy hook



# Usage 2: with context
with LazyInit() as lz:
     model = Linear(2, 4, weight_attr=weight_attr, bias_attr=bias_attr)
      
    print(model.weight)       # <------ not initialized
    print(lz.startup_program())


# initialize param
for param in model.parameters():
    param.initialize()     # <------ trigger lazy hook

[paddle-bot[bot]]

你的PR提交成功，感谢你对开源项目的贡献! 请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。 Your PR has been submitted. Thanks for your contribution! Please wait for the result of CI firstly. See Paddle CI Manual for details.

[Aurelius84]

现在实现OK，但是还是希望能够有更好的办法隐藏LazyInit调用。

如昨天讨论，暂时还没有比较好的方式能够实现概念的完全隐藏