Clarify Exercise 1
This commit is contained in:
kungfujam
parent
3cb5f81736
commit
40156f3778
@ -97,7 +97,7 @@
|
||||
"\\end{equation}\n",
|
||||
"$\n",
|
||||
"\n",
|
||||
"where both $\\mathbf{X}\\in\\mathbb{R}^{B\\times D}$ and $\\mathbf{Y}\\in\\mathbb{R}^{B\\times K}$ are matrices, and $\\mathbf{b}$ needs to be <a href=\"http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\">broadcasted</a> $B$ times (numpy will do this by default). However, we will not make an explicit distinction between a special case for $B=1$ and $B>1$ and simply use equation (3) instead, although $\\mathbf{x}$ and hence $\\mathbf{y}$ could be matrices. From an implementation point of view, it does not matter.\n",
|
||||
"where $\\mathbf{W} \\in \\mathbb{R}^{D\\times K}$ and both $\\mathbf{X}\\in\\mathbb{R}^{B\\times D}$ and $\\mathbf{Y}\\in\\mathbb{R}^{B\\times K}$ are matrices, and $\\mathbf{b}\\in\\mathbb{R}^{1\\times K}$ needs to be <a href=\"http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\">broadcasted</a> $B$ times (numpy will do this by default). However, we will not make an explicit distinction between a special case for $B=1$ and $B>1$ and simply use equation (3) instead, although $\\mathbf{x}$ and hence $\\mathbf{y}$ could be matrices. From an implementation point of view, it does not matter.\n",
|
||||
"\n",
|
||||
"The desired functionality for matrix multiplication in numpy is provided by <a href=\"http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html\">numpy.dot</a> function. If you haven't use it so far, get familiar with it as we will use it extensively."
|
||||
]
|
||||
@ -132,9 +132,16 @@
|
||||
"source": [
|
||||
"## Exercise 1 \n",
|
||||
"\n",
|
||||
"Using `numpy.dot`, implement **forward** propagation through the linear transform defined by equations (3) and (4) for $B=1$ and $B>1$. As data ($\\mathbf{x}$) use `MNISTDataProvider` introduced last week. For the case when $B=1$, write a function to compute the 1st output ($y_1$) using equations (1) and (2). Check if the output is the same as the corresponding one obtained with numpy. \n",
|
||||
"Using `numpy.dot`, implement **forward** propagation through the linear transform defined by equations (3) and (4) for $B=1$ and $B>1$ i.e. use parameters $\\mathbf{W}$ and $\\mathbf{b}$ with data $\\mathbf{X}$ to determine $\\mathbf{Y}$. Use `MNISTDataProvider` (introduced last week) to generate $\\mathbf{X}$. We are going to write a function for each equation:\n",
|
||||
"1. `y1_equation_1`: Return the value of the $1^{st}$ dimension of $\\mathbf{y}$ (the output of the first output node) given a single training data point $\\mathbf{x}$ using a sum\n",
|
||||
"1. `y1_equation_2`: Repeat above using vector multiplication (use `numpy.dot()`)\n",
|
||||
"1. `y_equation_3`: Return the value of $\\mathbf{y}$ (the whole output layer) given a single training data point $\\mathbf{x}$\n",
|
||||
"1. `Y_equation_4`: Return the value of $\\mathbf{Y}$ given $\\mathbf{X}$\n",
|
||||
"\n",
|
||||
"Tip: To generate random data you can use `random_generator.uniform(-0.1, 0.1, (D, 10))` from above."
|
||||
"We have initialised $\\mathbf{b}$ to zeros and randomly generated $\\mathbf{W}$ for you. The constants introduced above are:\n",
|
||||
"* The number of data points $B = 3$\n",
|
||||
"* The dimensionality of the input $D = 784$\n",
|
||||
"* The dimensionality of the output $K = 10$"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -148,9 +155,11 @@
|
||||
"from mlp.dataset import MNISTDataProvider\n",
|
||||
"\n",
|
||||
"mnist_dp = MNISTDataProvider(dset='valid', batch_size=3, max_num_batches=1, randomize=False)\n",
|
||||
"\n",
|
||||
"B = 3\n",
|
||||
"D = 784\n",
|
||||
"K = 10\n",
|
||||
"irange = 0.1\n",
|
||||
"W = random_generator.uniform(-irange, irange, (784,10)) \n",
|
||||
"W = random_generator.uniform(-irange, irange, (D, K)) \n",
|
||||
"b = numpy.zeros((10,))\n"
|
||||
]
|
||||
},
|
||||
@ -176,20 +185,21 @@
|
||||
" #use numpy.dot\n",
|
||||
" raise NotImplementedError()\n",
|
||||
"\n",
|
||||
"def y_equation_4(x, W, b):\n",
|
||||
"def Y_equation_4(x, W, b):\n",
|
||||
" #use numpy.dot\n",
|
||||
" raise NotImplementedError()\n",
|
||||
"\n",
|
||||
"for x, t in mnist_dp:\n",
|
||||
" y1e1 = y1_equation_1(x[0], W, b)\n",
|
||||
" y1e2 = y1_equation_2(x[0], W, b)\n",
|
||||
" ye3 = y_equation_3(x, W, b)\n",
|
||||
" ye4 = y_equation_4(x, W, b)\n",
|
||||
"for X, t in mnist_dp:\n",
|
||||
" n = 0\n",
|
||||
" y1e1 = y1_equation_1(x[n], W, b)\n",
|
||||
" y1e2 = y1_equation_2(x[n], W, b)\n",
|
||||
" ye3 = y_equation_3(x[n], W, b)\n",
|
||||
" Ye4 = Y_equation_4(x, W, b)\n",
|
||||
"\n",
|
||||
"print 'y1e1', y1e1\n",
|
||||
"print 'y1e1', y1e1\n",
|
||||
"print 'ye3', ye3\n",
|
||||
"print 'ye4', ye4\n",
|
||||
"print 'Ye4', ye4\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
@ -632,7 +642,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.9"
|
||||
"version": "2.7.10"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@ -55,19 +55,19 @@
|
||||
"***\n",
|
||||
"### Note on storing matrices in computer memory\n",
|
||||
"\n",
|
||||
"Consider you want to store the following array in memory: $\\left[ \\begin{array}{ccc}\n",
|
||||
"Suppose you want to store the following matrix in memory: $\\left[ \\begin{array}{ccc}\n",
|
||||
"1 & 2 & 3 \\\\\n",
|
||||
"4 & 5 & 6 \\\\\n",
|
||||
"7 & 8 & 9 \\end{array} \\right]$ \n",
|
||||
"\n",
|
||||
"In computer memory the above matrix would be organised as a vector in either (assume you allocate the memory at once for the whole matrix):\n",
|
||||
"If you allocate the memory at once for the whole matrix, then the above matrix would be organised as a vector in one of two possible forms:\n",
|
||||
"\n",
|
||||
"* Row-wise layout where the order would look like: $\\left [ \\begin{array}{ccccccccc}\n",
|
||||
"1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 \\end{array} \\right ]$\n",
|
||||
"* Column-wise layout where the order would look like: $\\left [ \\begin{array}{ccccccccc}\n",
|
||||
"1 & 4 & 7 & 2 & 5 & 8 & 3 & 6 & 9 \\end{array} \\right ]$\n",
|
||||
"\n",
|
||||
"Although `numpy` can easily handle both formats (possibly with some computational overhead), in our code we will stick with modern (and default) `c`-like approach and use row-wise format (contrary to Fortran that used column-wise approach). \n",
|
||||
"Although `numpy` can easily handle both formats (possibly with some computational overhead), in our code we will stick with the more modern (and default) `C`-like approach and use the row-wise format (in contrast to Fortran that used a column-wise approach). \n",
|
||||
"\n",
|
||||
"This means, that in this tutorial:\n",
|
||||
"* vectors are kept row-wise $\\mathbf{x} = (x_1, x_1, \\ldots, x_D) $ (rather than $\\mathbf{x} = (x_1, x_1, \\ldots, x_D)^T$)\n",
|
||||
@ -76,18 +76,18 @@
|
||||
"x_{21} & x_{22} & \\ldots & x_{2D} \\\\\n",
|
||||
"x_{31} & x_{32} & \\ldots & x_{3D} \\\\ \\end{array} \\right]$ and each row (i.e. $\\left[ \\begin{array}{cccc} x_{11} & x_{12} & \\ldots & x_{1D} \\end{array} \\right]$) represents a single data-point (like one MNIST image or one window of observations)\n",
|
||||
"\n",
|
||||
"In lecture slides you will find the equations following the conventional mathematical column-wise approach, but you can easily map them one way or the other using using matrix transpose.\n",
|
||||
"In lecture slides you will find the equations following the conventional mathematical approach, using column vectors, but you can easily map between column-major and row-major organisations using a matrix transpose.\n",
|
||||
"\n",
|
||||
"***\n",
|
||||
"\n",
|
||||
"## Linear and Affine Transforms\n",
|
||||
"\n",
|
||||
"The basis of all linear models is so called affine transform, that is a transform that implements some linear transformation and translation of input features. The transforms we are going to use are parameterised by:\n",
|
||||
"The basis of all linear models is the so called affine transform, which is a transform that implements a linear transformation and translation of the input features. The transforms we are going to use are parameterised by:\n",
|
||||
"\n",
|
||||
" * Weight matrix $\\mathbf{W} \\in \\mathbb{R}^{D\\times K}$: where element $w_{ik}$ is the weight from input $x_i$ to output $y_k$\n",
|
||||
" * Bias vector $\\mathbf{b}\\in R^{K}$ : where element $b_{k}$ is the bias for output $k$\n",
|
||||
" * A weight matrix $\\mathbf{W} \\in \\mathbb{R}^{D\\times K}$: where element $w_{ik}$ is the weight from input $x_i$ to output $y_k$\n",
|
||||
" * A bias vector $\\mathbf{b}\\in R^{K}$ : where element $b_{k}$ is the bias for output $k$\n",
|
||||
"\n",
|
||||
"Note, the bias is simply some additve term, and can be easily incorporated into an additional row in weight matrix and an additinal input in the inputs which is set to $1.0$ (as in the below picture taken from the lecture slides). However, here (and in the code) we will keep them separate.\n",
|
||||
"Note, the bias is simply some additive term, and can be easily incorporated into an additional row in weight matrix and an additional input in the inputs which is set to $1.0$ (as in the below picture taken from the lecture slides). However, here (and in the code) we will keep them separate.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
@ -136,7 +136,7 @@
|
||||
"\\end{equation}\n",
|
||||
"$\n",
|
||||
"\n",
|
||||
"where both $\\mathbf{X}\\in\\mathbb{R}^{B\\times D}$ and $\\mathbf{Y}\\in\\mathbb{R}^{B\\times K}$ are matrices, and $\\mathbf{b}$ needs to be <a href=\"http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\">broadcasted</a> $B$ times (numpy will do this by default). However, we will not make an explicit distinction between a special case for $B=1$ and $B>1$ and simply use equation (3) instead, although $\\mathbf{x}$ and hence $\\mathbf{y}$ could be matrices. From an implementation point of view, it does not matter.\n",
|
||||
"where $\\mathbf{W} \\in \\mathbb{R}^{D\\times K}$ and both $\\mathbf{X}\\in\\mathbb{R}^{B\\times D}$ and $\\mathbf{Y}\\in\\mathbb{R}^{B\\times K}$ are matrices, and $\\mathbf{b}\\in\\mathbb{R}^{1\\times K}$ needs to be <a href=\"http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\">broadcasted</a> $B$ times (numpy will do this by default). However, we will not make an explicit distinction between a special case for $B=1$ and $B>1$ and simply use equation (3) instead, although $\\mathbf{x}$ and hence $\\mathbf{y}$ could be matrices. From an implementation point of view, it does not matter.\n",
|
||||
"\n",
|
||||
"The desired functionality for matrix multiplication in numpy is provided by <a href=\"http://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html\">numpy.dot</a> function. If you haven't use it so far, get familiar with it as we will use it extensively."
|
||||
]
|
||||
@ -152,14 +152,13 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy\n",
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"#initialise the random generator to be used later\n",
|
||||
"seed=[2015, 10, 1]\n",
|
||||
@ -172,14 +171,21 @@
|
||||
"source": [
|
||||
"## Exercise 1 \n",
|
||||
"\n",
|
||||
"Using numpy.dot, implement **forward** propagation through the linear transform defined by equations (3) and (4) for $B=1$ and $B>1$. As data ($\\mathbf{x}$) use `MNISTDataProvider` from previous laboratories. For case when $B=1$ write a function to compute the 1st output ($y_1$) using equations (1) and (2). Check if the output is the same as the corresponding one obtained with numpy. \n",
|
||||
"Using `numpy.dot`, implement **forward** propagation through the linear transform defined by equations (3) and (4) for $B=1$ and $B>1$ i.e. use parameters $\\mathbf{W}$ and $\\mathbf{b}$ with data $\\mathbf{X}$ to determine $\\mathbf{Y}$. Use `MNISTDataProvider` (introduced last week) to generate $\\mathbf{X}$. We are going to write a function for each equation:\n",
|
||||
"1. `y1_equation_1`: Return the value of the $1^{st}$ dimension of $\\mathbf{y}$ (the output of the first output node) given a single training data point $\\mathbf{x}$ using a sum\n",
|
||||
"1. `y1_equation_2`: Repeat above using vector multiplication (use `numpy.dot()`)\n",
|
||||
"1. `y_equation_3`: Return the value of $\\mathbf{y}$ (the whole output layer) given a single training data point $\\mathbf{x}$\n",
|
||||
"1. `Y_equation_4`: Return the value of $\\mathbf{Y}$ given $\\mathbf{X}$\n",
|
||||
"\n",
|
||||
"Tip: To generate random data you can use `random_generator.uniform(-0.1, 0.1, (D, 10))` from the preceeding cell."
|
||||
"We have initialised $\\mathbf{b}$ to zeros and randomly generated $\\mathbf{W}$ for you. The constants introduced above are:\n",
|
||||
"* The number of data points $B = 3$\n",
|
||||
"* The dimensionality of the input $D = 10$\n",
|
||||
"* The dimensionality of the output $K = 10$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
@ -188,15 +194,17 @@
|
||||
"from mlp.dataset import MNISTDataProvider\n",
|
||||
"\n",
|
||||
"mnist_dp = MNISTDataProvider(dset='valid', batch_size=3, max_num_batches=1, randomize=False)\n",
|
||||
"\n",
|
||||
"B = 3\n",
|
||||
"D = 784\n",
|
||||
"K = 10\n",
|
||||
"irange = 0.1\n",
|
||||
"W = random_generator.uniform(-irange, irange, (784,10)) \n",
|
||||
"W = random_generator.uniform(-irange, irange, (D, K)) \n",
|
||||
"b = numpy.zeros((10,))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
@ -207,13 +215,9 @@
|
||||
"text": [
|
||||
"y1e1 0.55861474982\n",
|
||||
"y1e2 0.55861474982\n",
|
||||
"ye3 [[ 0.55861475 0.79450077 0.17439693 0.00265688 0.66272539 -0.09985686\n",
|
||||
" 0.56468591 0.58105588 -0.18613727 0.08151257]\n",
|
||||
" [-0.43965864 0.59573972 -0.22691119 0.26767124 -0.31343979 0.07224664\n",
|
||||
" -0.19616183 0.0851733 -0.24088286 -0.19305162]\n",
|
||||
" [-0.20176359 0.42394166 -1.03984446 0.15492101 0.15694745 -0.53741022\n",
|
||||
" 0.05887668 -0.21124527 -0.07870156 -0.00506471]]\n",
|
||||
"ye4 [[ 0.55861475 0.79450077 0.17439693 0.00265688 0.66272539 -0.09985686\n",
|
||||
"ye3 [ 0.55861475 0.79450077 0.17439693 0.00265688 0.66272539 -0.09985686\n",
|
||||
" 0.56468591 0.58105588 -0.18613727 0.08151257]\n",
|
||||
"Ye4 [[ 0.55861475 0.79450077 0.17439693 0.00265688 0.66272539 -0.09985686\n",
|
||||
" 0.56468591 0.58105588 -0.18613727 0.08151257]\n",
|
||||
" [-0.43965864 0.59573972 -0.22691119 0.26767124 -0.31343979 0.07224664\n",
|
||||
" -0.19616183 0.0851733 -0.24088286 -0.19305162]\n",
|
||||
@ -223,36 +227,37 @@
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"\n",
|
||||
"mnist_dp.reset()\n",
|
||||
"\n",
|
||||
"#implement following functions, then run the cell\n",
|
||||
"def y1_equation_1(x, W, b):\n",
|
||||
" y1=0\n",
|
||||
" for j in xrange(0, x.shape[0]):\n",
|
||||
" y1 += x[j]*W[j,0]\n",
|
||||
" return y1 + b[0]\n",
|
||||
" k = 0\n",
|
||||
" s = 0\n",
|
||||
" for j in xrange(len(x)):\n",
|
||||
" s += x[j] * W[j,k]\n",
|
||||
" return b[k] + s\n",
|
||||
" \n",
|
||||
"def y1_equation_2(x, W, b):\n",
|
||||
" return numpy.dot(x, W[:,0].T) + b[0]\n",
|
||||
" k = 0\n",
|
||||
" return numpy.dot(x, W[:,k]) + b[k]\n",
|
||||
"\n",
|
||||
"def y_equation_3(x, W, b):\n",
|
||||
" return numpy.dot(x,W) + b\n",
|
||||
" return numpy.dot(x, W) + b\n",
|
||||
"\n",
|
||||
"def y_equation_4(x, W, b):\n",
|
||||
" return numpy.dot(x,W) + b\n",
|
||||
" return numpy.dot(x, W) + b\n",
|
||||
"\n",
|
||||
"for x, t in mnist_dp:\n",
|
||||
" y1e1 = y1_equation_1(x[0], W, b)\n",
|
||||
" y1e2 = y1_equation_2(x[0], W, b)\n",
|
||||
" ye3 = y_equation_3(x, W, b)\n",
|
||||
" ye4 = y_equation_4(x, W, b)\n",
|
||||
"for X, t in mnist_dp:\n",
|
||||
" n = 0\n",
|
||||
" y1e1 = y1_equation_1(X[n], W, b)\n",
|
||||
" y1e2 = y1_equation_2(X[n], W, b)\n",
|
||||
" ye3 = y_equation_3(X[n], W, b)\n",
|
||||
" Ye4 = y_equation_4(X, W, b)\n",
|
||||
"\n",
|
||||
"print 'y1e1', y1e1\n",
|
||||
"print 'y1e2', y1e2\n",
|
||||
"print 'ye3', ye3\n",
|
||||
"print 'ye4', ye4\n",
|
||||
" "
|
||||
"print 'Ye4', Ye4"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -268,24 +273,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[[-0.00683757 -0.13638553 0.00203203 ..., 0.02690207 -0.07364245\n",
|
||||
" 0.04403087]\n",
|
||||
" [-0.00447621 -0.06409652 0.01211384 ..., 0.0402248 -0.04490571\n",
|
||||
" -0.05013801]\n",
|
||||
" [ 0.03981022 -0.13705957 0.05882239 ..., 0.04491902 -0.08644539\n",
|
||||
" -0.07106441]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"y = y_equation_3(x, W, b)\n",
|
||||
"z = numpy.dot(y, W.T)\n",
|
||||
@ -315,7 +307,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
@ -339,19 +331,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Well done!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"irange = 0.1 #+-range from which we draw the random numbers\n",
|
||||
"\n",
|
||||
@ -373,7 +357,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
@ -408,19 +392,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Well done!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"irange = 0.1 #+-range from which we draw the random numbers\n",
|
||||
"\n",
|
||||
@ -449,7 +425,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
@ -462,20 +438,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"my_dot timings:\n",
|
||||
"10 loops, best of 3: 726 ms per loop\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print 'my_dot timings:'\n",
|
||||
"%timeit -n10 my_dot_mat_mat(x, W)"
|
||||
@ -483,20 +450,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"numpy.dot timings:\n",
|
||||
"10 loops, best of 3: 1.17 ms per loop\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print 'numpy.dot timings:'\n",
|
||||
"%timeit -n10 numpy.dot(x, W)"
|
||||
@ -564,78 +522,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Observations: [[-0.12 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13]\n",
|
||||
" [-0.11 -0.1 0.09 -0.06 -0.09 -0. 0.28 -0.12 -0.12 -0.08]\n",
|
||||
" [-0.13 0.05 -0.13 -0.01 -0.11 -0.13 -0.13 -0.13 -0.13 -0.13]\n",
|
||||
" [ 0.2 0.12 0.25 0.16 0.03 -0. 0.15 0.08 -0.08 -0.11]\n",
|
||||
" [-0.13 -0.12 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13]\n",
|
||||
" [-0.1 0.51 1.52 0.14 -0.02 0.77 0.11 0.79 -0.02 0.08]\n",
|
||||
" [ 0.24 0.15 -0.01 0.08 -0.1 0.45 -0.12 -0.1 -0.13 0.48]\n",
|
||||
" [ 0.13 -0.06 -0.07 -0.11 -0.11 -0.11 -0.13 -0.11 -0.02 -0.12]\n",
|
||||
" [-0.06 0.28 -0.13 0.06 0.09 0.09 0.01 -0.07 0.14 -0.11]\n",
|
||||
" [-0.13 -0.13 -0.1 -0.06 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13]]\n",
|
||||
"To predict: [[-0.12]\n",
|
||||
" [-0.12]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.1 ]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.08]\n",
|
||||
" [ 0.24]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.02]\n",
|
||||
" [-0.13]]\n",
|
||||
"Observations: [[-0.09 -0.13 -0.13 -0.03 -0.05 -0.11 -0.13 -0.13 -0.13 -0.13]\n",
|
||||
" [-0.03 0.32 0.28 0.09 -0.04 0.19 0.31 -0.13 0.37 0.34]\n",
|
||||
" [ 0.12 0.13 0.06 -0.1 -0.1 0.94 0.24 0.12 0.28 -0.04]\n",
|
||||
" [ 0.26 0.17 -0.04 -0.13 -0.12 -0.09 -0.12 -0.13 -0.1 -0.13]\n",
|
||||
" [-0.1 -0.1 -0.01 -0.03 -0.07 0.05 -0.03 -0.12 -0.05 -0.13]\n",
|
||||
" [-0.13 -0.13 -0.13 -0.13 -0.13 -0.13 0.1 -0.13 -0.13 -0.13]\n",
|
||||
" [-0.01 -0.1 -0.13 -0.13 -0.12 -0.13 -0.13 -0.13 -0.13 -0.11]\n",
|
||||
" [-0.11 -0.06 -0.11 0.02 -0.03 -0.02 -0.05 -0.11 -0.13 -0.13]\n",
|
||||
" [-0.01 0.25 -0.08 0.04 -0.1 -0.12 0.06 -0.1 0.08 -0.06]\n",
|
||||
" [-0.09 -0.09 -0.09 -0.13 -0.11 -0.12 -0. -0.02 0.19 -0.11]]\n",
|
||||
"To predict: [[-0.13]\n",
|
||||
" [-0.11]\n",
|
||||
" [-0.09]\n",
|
||||
" [-0.08]\n",
|
||||
" [ 0.19]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.03]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.11]]\n",
|
||||
"Observations: [[-0.08 -0.11 -0.11 0.32 0.05 -0.11 -0.13 0.07 0.08 0.63]\n",
|
||||
" [-0.07 -0.1 -0.09 -0.08 0.26 -0.05 -0.1 -0. 0.36 -0.12]\n",
|
||||
" [-0.03 -0.1 0.19 -0.02 0.35 0.38 -0.1 0.44 -0.02 0.21]\n",
|
||||
" [-0.12 -0. -0.02 0.19 -0.11 -0.11 -0.13 -0.11 -0.02 -0.13]\n",
|
||||
" [ 0.09 0.1 -0.03 -0.05 0. -0.12 -0.12 -0.13 -0.13 -0.13]\n",
|
||||
" [ 0.21 0.05 -0.12 -0.05 -0.08 -0.1 -0.13 -0.13 -0.13 -0.13]\n",
|
||||
" [-0.04 -0.11 0.19 0.16 -0.01 -0.07 -0. -0.06 -0.03 0.16]\n",
|
||||
" [ 0.09 0.05 0.51 0.34 0.16 0.51 0.56 0.21 -0.06 -0. ]\n",
|
||||
" [-0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.13 -0.09 0.49]\n",
|
||||
" [-0.06 -0.11 -0.13 0.06 -0.01 -0.12 0.54 0.2 -0.1 -0.11]]\n",
|
||||
"To predict: [[ 0.1 ]\n",
|
||||
" [ 0.09]\n",
|
||||
" [ 0.16]\n",
|
||||
" [-0.13]\n",
|
||||
" [-0.13]\n",
|
||||
" [ 0.04]\n",
|
||||
" [-0.1 ]\n",
|
||||
" [ 0.05]\n",
|
||||
" [-0.1 ]\n",
|
||||
" [-0.11]]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from mlp.dataset import MetOfficeDataProvider\n",
|
||||
"\n",
|
||||
@ -658,7 +549,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
@ -691,7 +582,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
@ -749,74 +640,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"MSE training cost after 1-th epoch is 0.017213\n",
|
||||
"MSE training cost after 2-th epoch is 0.016103\n",
|
||||
"MSE training cost after 3-th epoch is 0.015705\n",
|
||||
"MSE training cost after 4-th epoch is 0.015437\n",
|
||||
"MSE training cost after 5-th epoch is 0.015255\n",
|
||||
"MSE training cost after 6-th epoch is 0.015128\n",
|
||||
"MSE training cost after 7-th epoch is 0.015041\n",
|
||||
"MSE training cost after 8-th epoch is 0.014981\n",
|
||||
"MSE training cost after 9-th epoch is 0.014936\n",
|
||||
"MSE training cost after 10-th epoch is 0.014903\n",
|
||||
"MSE training cost after 11-th epoch is 0.014879\n",
|
||||
"MSE training cost after 12-th epoch is 0.014862\n",
|
||||
"MSE training cost after 13-th epoch is 0.014849\n",
|
||||
"MSE training cost after 14-th epoch is 0.014839\n",
|
||||
"MSE training cost after 15-th epoch is 0.014830\n",
|
||||
"MSE training cost after 16-th epoch is 0.014825\n",
|
||||
"MSE training cost after 17-th epoch is 0.014820\n",
|
||||
"MSE training cost after 18-th epoch is 0.014813\n",
|
||||
"MSE training cost after 19-th epoch is 0.014813\n",
|
||||
"MSE training cost after 20-th epoch is 0.014810\n",
|
||||
"MSE training cost after 21-th epoch is 0.014808\n",
|
||||
"MSE training cost after 22-th epoch is 0.014805\n",
|
||||
"MSE training cost after 23-th epoch is 0.014806\n",
|
||||
"MSE training cost after 24-th epoch is 0.014804\n",
|
||||
"MSE training cost after 25-th epoch is 0.014796\n",
|
||||
"MSE training cost after 26-th epoch is 0.014798\n",
|
||||
"MSE training cost after 27-th epoch is 0.014801\n",
|
||||
"MSE training cost after 28-th epoch is 0.014802\n",
|
||||
"MSE training cost after 29-th epoch is 0.014801\n",
|
||||
"MSE training cost after 30-th epoch is 0.014799\n",
|
||||
"MSE training cost after 31-th epoch is 0.014799\n",
|
||||
"MSE training cost after 32-th epoch is 0.014793\n",
|
||||
"MSE training cost after 33-th epoch is 0.014800\n",
|
||||
"MSE training cost after 34-th epoch is 0.014796\n",
|
||||
"MSE training cost after 35-th epoch is 0.014799\n",
|
||||
"MSE training cost after 36-th epoch is 0.014800\n",
|
||||
"MSE training cost after 37-th epoch is 0.014798\n",
|
||||
"MSE training cost after 38-th epoch is 0.014799\n",
|
||||
"MSE training cost after 39-th epoch is 0.014799\n",
|
||||
"MSE training cost after 40-th epoch is 0.014794\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"(array([[ 0.01],\n",
|
||||
" [ 0.03],\n",
|
||||
" [ 0.03],\n",
|
||||
" [ 0.04],\n",
|
||||
" [ 0.06],\n",
|
||||
" [ 0.07],\n",
|
||||
" [ 0.26]]), array([-0.]))"
|
||||
]
|
||||
},
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"#some hyper-parameters\n",
|
||||
@ -882,7 +710,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.9"
|
||||
"version": "2.7.10"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
Loading…
Reference in New Issue
Block a user