add cache for VariableWrapper

[MingMingShangTian]

PR types

New features

PR changes

OPs

Describe

Add cache for VariableWrapper when type promotion occurs in real and complex computing of Paddle Quantum.

The test demo is as follows.

import paddle
import numpy as np
import time
np.random.seed(47)
class Optimization_ex1(paddle.nn.Layer):
    def __init__(self, param_attr=paddle.nn.initializer.Uniform(low=-5., high=5.), dtype='float32'):
        super(Optimization_ex1, self).__init__()
        self.shape = (10, 10)
        self.theta0 = self.create_parameter(shape=self.shape, attr=param_attr, dtype=dtype, is_bias=False)
        self.theta1 = self.create_parameter(shape=self.shape, attr=param_attr, dtype=dtype, is_bias=False)
        self.A = paddle.to_tensor(np.random.random(self.shape).astype(dtype), stop_gradient = False)
        self.B = paddle.to_tensor(np.random.random(self.shape).astype(dtype) + np.random.random(self.shape).astype(dtype) * 1j, stop_gradient = False)
        self.C = paddle.to_tensor(np.random.random(self.shape).astype(dtype) + np.random.random(self.shape).astype(dtype) * 1j, stop_gradient = False)
    def forward(self):
        jj = paddle.to_tensor(np.array([1j], dtype=np.complex64))
        self.theta = self.theta0 + self.theta1 * jj
        t1 = paddle.matmul(self.A, self.B)
        t2 = paddle.matmul(self.A, self.C)
        t3 = paddle.matmul(t1, t2)
        loss = paddle.sum(paddle.matmul(self.theta, t3))
        return loss.real()
if __name__ == "__main__":
    ITR = 1000
    LR = 0.1
    loss_list = []
    parameter_list = []
    myLayer = Optimization_ex1()
    optimizer = paddle.optimizer.Adam(learning_rate=LR, parameters=myLayer.parameters())
    T1 = time.time()
    for itr in range(ITR):
        loss = myLayer()
        loss.backward()
        optimizer.step()
        optimizer.clear_grad()
        #print(itr, loss.numpy())
        loss_list.append(loss.numpy()[0])
        parameter_list.append(myLayer.parameters()[0].numpy())
    T2 = time.time()
    print(T2 - T1)

In the test, after the first time self.A setting the complex variable cache, every iterations will hit the cache 4 times. It will reduce about 5% ~ 6% time cost when hitting the cache.

Test result:
1000 iterations， the develop branch and the cache branch time cost：

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-bot-old]

Sorry to inform you that e3d123e's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[chenwhql]

LGTM，测试的小demo也放到pr描述里吧，方便了解测试模型的规模