Transfer-Learning-Library icon indicating copy to clipboard operation
Transfer-Learning-Library copied to clipboard

是否需要加入训练多核对应的权重的过程?

Open Sibo-Tian opened this issue 3 years ago • 0 comments

我在阅读DAN时,注意到原文是一个minimax过程,在学习网络参数(\Theta)外,还需要学习多核权重(\beta);而后者似乎未在此代码文件中体现,5个核被分配了相同的权重;我写了一个二次规划求核权重的函数,供参考讨论 :-)

` def train_weights(train_source_iter: DataLoader, train_traget_iter: DataLoader, model: ImageClassifier, kernels: list): print("=>Train Beta(weights) of the kernels") num_kernels = len(kernels) #协方差矩阵(原文中的矩阵Q) covar_matrix = torch.zeros((num_kernels, num_kernels))

for i in range(num_kernels):
    for j in range(i, num_kernels):
        kernel1 = kernels[i]
        kernel2 = kernels[j]
        count = 0
        #线性时间估计Cov(u,u')
        for x_s, x_t in zip(train_source_iter, train_traget_iter):
            x_s = model(x_s[0])[1]
            x_t = model(x_t[0])[1]

            idx = 0
            res = []

            while idx < len(x_s) // 4:
                t1 = (kernel1(x_s[4 * idx : 4 * idx + 2]) + kernel1(x_t[4 * idx : 4 * idx + 2]) - kernel1(
                    torch.cat((x_s[4 * idx].unsqueeze(dim=0), x_t[4 * idx + 1].unsqueeze(dim=0)))) \
                     - kernel1(torch.cat((x_s[4 * idx + 1].unsqueeze(dim=0), x_t[4 * idx].unsqueeze(dim=0)))))[0][1]
                t2 = (kernel1(x_s[4 * idx+2 : 4 * idx + 4]) + kernel1(x_t[4 * idx+2 : 4 * idx + 4]) - kernel1(
                    torch.cat((x_s[4 * idx+2].unsqueeze(dim=0), x_t[4 * idx + 3].unsqueeze(dim=0)))) \
                      - kernel1(torch.cat((x_s[4 * idx + 3].unsqueeze(dim=0), x_t[4 * idx+2].unsqueeze(dim=0)))))[0][1]
                t3 = (kernel2(x_s[4 * idx: 4 * idx + 2]) + kernel2(x_t[4 * idx: 4 * idx + 2]) - kernel2(
                    torch.cat((x_s[4 * idx].unsqueeze(dim=0), x_t[4 * idx + 1].unsqueeze(dim=0)))) \
                      - kernel2(torch.cat((x_s[4 * idx + 1].unsqueeze(dim=0), x_t[4 * idx].unsqueeze(dim=0)))))[0][1]
                t4 = (kernel2(x_s[4 * idx + 2: 4 * idx + 4]) + kernel2(x_t[4 * idx + 2: 4 * idx + 4]) - kernel2(
                    torch.cat((x_s[4 * idx + 2].unsqueeze(dim=0), x_t[4 * idx + 3].unsqueeze(dim=0)))) \
                      - kernel2(torch.cat((x_s[4 * idx + 3].unsqueeze(dim=0), x_t[4 * idx + 2].unsqueeze(dim=0)))))[
                    0][1]

                res.append((t1 - t2) * (t3 - t4))
                idx += 1
            res = sum(res) / len(res)
            covar_matrix[i][j] = covar_matrix[i][j] * count / (count + 1) + res / (count + 1)
            count += 1
        covar_matrix[j][i] = covar_matrix[i][j]

epsilon = .001
reg = torch.eye(num_kernels) * epsilon
covar_matrix = covar_matrix + reg
#使用cvxopt包求解二次规划问题(from cvxopt import matrix, solvers)
P = covar_matrix.t().numpy().tolist()
q = [0. for i in range(num_kernels)]
G = (torch.eye(num_kernels) * -1).t().numpy().tolist()
h = [0. for i in range(num_kernels)]
A = [[1.] for i in range(num_kernels)]
P = matrix(P)
q = matrix(q)
G = matrix(G)
h = matrix(h)
A = matrix(A)
b = matrix([1.0])
result = solvers.qp(P, q, G, h, A, b)
print('The qp result')
print(result['x'])
return list(result['x'])

`

Sibo-Tian avatar Sep 22 '22 17:09 Sibo-Tian