anton/mlpractical
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #43 from jfainberg/patch-1

Browse Source 
Minor typos in momentum docstring
This commit is contained in:
Matt Graham 2016-10-29 21:17:24 +01:00 committed by GitHub
commit d5ae8b2caf
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
mlp/learning_rules.py
View File

@ -84,12 +84,12 @@ class MomentumLearningRule(GradientDescentLearningRule):
For parameter p[i] and corresponding momentum m[i] the updates for a
scalar loss function `L` are of the form
m[i] := mom_coeff * m[i] - learning_rate * dL/dp[i]
m[i] := mom_coeff * m[i-1] - learning_rate * dL/dp[i]
p[i] := p[i] + m[i]
with `learning_rate` a positive scaling parameter for the gradient updates
and `mom_coeff` a value in [0, 1] that determines how much 'friction' there
is the system and so how quickly previous momentum contributions decay.
is in the system and so how quickly previous momentum contributions decay.
"""
def __init__(self, learning_rate=1e-3, mom_coeff=0.9):