anton/mlpractical
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Minor typos in momentum docstring

Browse Source
This commit is contained in:
Joachim 2016-10-29 15:57:00 +01:00 committed by GitHub
parent a6f775b1ff
commit bea3502210
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
mlp/learning_rules.py
View File

@ -84,12 +84,12 @@ class MomentumLearningRule(GradientDescentLearningRule):
For parameter p[i] and corresponding momentum m[i] the updates for a For parameter p[i] and corresponding momentum m[i] the updates for a
scalar loss function `L` are of the form scalar loss function `L` are of the form
m[i] := mom_coeff * m[i] - learning_rate * dL/dp[i] m[i] := mom_coeff * m[i-1] - learning_rate * dL/dp[i]
p[i] := p[i] + m[i] p[i] := p[i] + m[i]
with `learning_rate` a positive scaling parameter for the gradient updates with `learning_rate` a positive scaling parameter for the gradient updates
and `mom_coeff` a value in [0, 1] that determines how much 'friction' there and `mom_coeff` a value in [0, 1] that determines how much 'friction' there
is the system and so how quickly previous momentum contributions decay. is in the system and so how quickly previous momentum contributions decay.
""" """
def __init__(self, learning_rate=1e-3, mom_coeff=0.9): def __init__(self, learning_rate=1e-3, mom_coeff=0.9):