Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

对于训练过程里的代码有个疑问150~161 #3

Open
sweetxing opened this issue Jul 5, 2020 · 6 comments
Open

对于训练过程里的代码有个疑问150~161 #3

sweetxing opened this issue Jul 5, 2020 · 6 comments

Comments

@sweetxing
Copy link

sweetxing commented Jul 5, 2020

optimizer_c中优化的是vnet的参数,但是损失l_g_meta中我没有看到vnet网络的参与,我有点想不清楚网络是如何迭代的

@xjtushujun
Copy link
Owner

请参考文章关于更新MW-Net参数的数学表达式。

@jingzhengli
Copy link

optimizer_c中优化的是vnet的参数,但是损失l_g_meta中我没有看到vnet网络的参与,我有点想不清楚网络是如何迭代的
损失l_g_meta是meta_model和vnet网络的共同参与计算的,所以是在一个计算图中的。这也是meta_model存在的意义,做一个桥梁作用。
226行:
l_g_meta.backward()
这一步更新的不仅是临时模型的参数,也包含了vnet

@justlovebarbecue
Copy link

@xjtushujun @jingzhengli 您好,我也是看到这个地方,有些不理解。meta_model 本身是不包含vnet的,vnet的梯度是怎么计算出来的呢?我发现meta_model.update_params这行代码应该很重要,我去掉这行,vnet的梯度就不能计算了。可是这行代码调用的函数 只和meta_model相关,是怎么能和vnet有联系的呢?如果方便 麻烦讲解一下,谢谢您!

@YAaNnGg
Copy link

YAaNnGg commented Apr 27, 2021

It's related to the update_params() method defined in MetaModule class in resnet.py.

@trafficpre
Copy link

关于网络参数更新的问题想请教一下,
optimizer_c.zero_grad()
l_g_meta.backward()
optimizer_c.step()
这几行应该是要更新vnet网络的参数,但是vnet.state_dict()观察发现网络参数并没有变化。还有后边优化model参数的时候也是一样的问题,麻烦解释一下,谢谢!

@Dawn-LX
Copy link

Dawn-LX commented Aug 11, 2022

same problem, can anyone explain this? I understand all the math in the paper, but I dont understand the code for gradient calculation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants