Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请教一个L1正则中的疑惑 #790

Open
CantBlues opened this issue Apr 30, 2023 · 1 comment
Open

请教一个L1正则中的疑惑 #790

CantBlues opened this issue Apr 30, 2023 · 1 comment

Comments

@CantBlues
Copy link

在A2.16.3(L1正则)Line 177 :
当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
实在没搞懂这句话的意思,不应该是“相当于加/减lambda的值以作为惩罚吗”。最后的效果是乘以w的绝对值??????
附关于L1反向传播的前后文:

def backward(self, delta_in, idx):
    dZ = delta_in
    m = self.x.shape[1]
    if self.regular == RegularMethod.L2:
        self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * self.weights.W) / m
    elif self.regular == RegularMethod.L1:
        self.weights.dW = (np.dot(dZ, self.x.T) + self.lambd * np.sign(self.weights.W)) / m
    else:
        self.weights.dW = np.dot(dZ, self.x.T) / m
    # end if
    self.weights.dB = np.sum(dZ, axis=1, keepdims=True) / m
    ......

符号函数的效果如下:

>>> a=np.array([1,-1,2,0])
>>> np.sign(a)
>>> array([ 1, -1,  1,  0])

当w为正数时,符号为正,值为1,相当于直接乘以w的值;当w为负数时,符号为负,值为-1,相当于乘以(-w)的值。最后的效果就是乘以w的绝对值。
望赐教

@xiaowuhu
Copy link
Collaborator

感谢提出问题,我这里写错了,如你所说,确实应该是"加减lambda的值"。
你能否提交一个PR修改一下?也算是对本站做了贡献. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants