308 lines
14 KiB
Plaintext
308 lines
14 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Below a skeleton class and associated test functions for the `fprop`, `bprop` and `grads_wrt_params` methods of the ConvolutionalLayer class are included.\n",
|
|
"\n",
|
|
"The test functions assume that in your implementation of `fprop` for the convolutional layer, outputs are calculated only for 'valid' overlaps of the kernel filters with the input - i.e. without any padding.\n",
|
|
"\n",
|
|
"It is also assumed that if convolutions with non-unit strides are implemented the default behaviour is to take unit-strides, with the test cases only correct for unit strides in both directions."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The three test functions are defined in the cell below. All the functions take as first argument the *class* corresponding to the convolutional layer implementation to be tested (**not** an instance of the class). It is assumed the class being tested has an `__init__` method with at least all of the arguments defined in the skeleton definition above. A boolean second argument to each function can be used to specify if the layer implements a cross-correlation or convolution based operation (see note in [seventh lecture slides](http://www.inf.ed.ac.uk/teaching/courses/mlp/2016/mlp07-cnn.pdf))."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"def test_conv_layer_fprop(layer_class, do_cross_correlation=False):\n",
|
|
" \"\"\"Tests `fprop` method of a convolutional layer.\n",
|
|
" \n",
|
|
" Checks the outputs of `fprop` method for a fixed input against known\n",
|
|
" reference values for the outputs and raises an AssertionError if\n",
|
|
" the outputted values are not consistent with the reference values. If\n",
|
|
" tests are all passed returns True.\n",
|
|
" \n",
|
|
" Args:\n",
|
|
" layer_class: Convolutional layer implementation following the \n",
|
|
" interface defined in the provided skeleton class.\n",
|
|
" do_cross_correlation: Whether the layer implements an operation\n",
|
|
" corresponding to cross-correlation (True) i.e kernels are\n",
|
|
" not flipped before sliding over inputs, or convolution\n",
|
|
" (False) with filters being flipped.\n",
|
|
"\n",
|
|
" Raises:\n",
|
|
" AssertionError: Raised if output of `layer.fprop` is inconsistent \n",
|
|
" with reference values either in shape or values.\n",
|
|
" \"\"\"\n",
|
|
" inputs = np.arange(96).reshape((2, 3, 4, 4))\n",
|
|
" kernels = np.arange(-12, 12).reshape((2, 3, 2, 2))\n",
|
|
" if do_cross_correlation:\n",
|
|
" kernels = kernels[:, :, ::-1, ::-1]\n",
|
|
" biases = np.arange(2)\n",
|
|
" true_output = np.array(\n",
|
|
" [[[[ -958., -1036., -1114.],\n",
|
|
" [-1270., -1348., -1426.],\n",
|
|
" [-1582., -1660., -1738.]],\n",
|
|
" [[ 1707., 1773., 1839.],\n",
|
|
" [ 1971., 2037., 2103.],\n",
|
|
" [ 2235., 2301., 2367.]]],\n",
|
|
" [[[-4702., -4780., -4858.],\n",
|
|
" [-5014., -5092., -5170.],\n",
|
|
" [-5326., -5404., -5482.]],\n",
|
|
" [[ 4875., 4941., 5007.],\n",
|
|
" [ 5139., 5205., 5271.],\n",
|
|
" [ 5403., 5469., 5535.]]]]\n",
|
|
" )\n",
|
|
" \n",
|
|
" layer = layer_class(\n",
|
|
" num_input_channels=kernels.shape[1], \n",
|
|
" num_output_channels=kernels.shape[0], \n",
|
|
" input_dim_1=inputs.shape[2], \n",
|
|
" input_dim_2=inputs.shape[3],\n",
|
|
" kernel_dim_1=kernels.shape[2],\n",
|
|
" kernel_dim_2=kernels.shape[3]\n",
|
|
" )\n",
|
|
" layer.params = [kernels, biases]\n",
|
|
" layer_output = layer.fprop(inputs)\n",
|
|
" \n",
|
|
" assert layer_output.shape == true_output.shape, (\n",
|
|
" 'Layer fprop gives incorrect shaped output. '\n",
|
|
" 'Correct shape is \\n\\n{0}\\n\\n but returned shape is \\n\\n{1}.'\n",
|
|
" .format(true_output.shape, layer_output.shape)\n",
|
|
" )\n",
|
|
" assert np.allclose(layer_output, true_output), (\n",
|
|
" 'Layer fprop does not give correct output. '\n",
|
|
" 'Correct output is \\n\\n{0}\\n\\n but returned output is \\n\\n{1}\\n\\n difference is \\n\\n{2}.'\n",
|
|
" .format(true_output, layer_output, true_output-layer_output)\n",
|
|
" )\n",
|
|
" return True\n",
|
|
"\n",
|
|
"def test_conv_layer_bprop(layer_class, do_cross_correlation=False):\n",
|
|
" \"\"\"Tests `bprop` method of a convolutional layer.\n",
|
|
" \n",
|
|
" Checks the outputs of `bprop` method for a fixed input against known\n",
|
|
" reference values for the gradients with respect to inputs and raises \n",
|
|
" an AssertionError if the returned values are not consistent with the\n",
|
|
" reference values. If tests are all passed returns True.\n",
|
|
" \n",
|
|
" Args:\n",
|
|
" layer_class: Convolutional layer implementation following the \n",
|
|
" interface defined in the provided skeleton class.\n",
|
|
" do_cross_correlation: Whether the layer implements an operation\n",
|
|
" corresponding to cross-correlation (True) i.e kernels are\n",
|
|
" not flipped before sliding over inputs, or convolution\n",
|
|
" (False) with filters being flipped.\n",
|
|
"\n",
|
|
" Raises:\n",
|
|
" AssertionError: Raised if output of `layer.bprop` is inconsistent \n",
|
|
" with reference values either in shape or values.\n",
|
|
" \"\"\"\n",
|
|
" inputs = np.arange(96).reshape((2, 3, 4, 4))\n",
|
|
" kernels = np.arange(-12, 12).reshape((2, 3, 2, 2))\n",
|
|
" if do_cross_correlation:\n",
|
|
" kernels = kernels[:, :, ::-1, ::-1]\n",
|
|
" biases = np.arange(2)\n",
|
|
" grads_wrt_outputs = np.arange(-20, 16).reshape((2, 2, 3, 3))\n",
|
|
" outputs = np.array(\n",
|
|
" [[[[ -958., -1036., -1114.],\n",
|
|
" [-1270., -1348., -1426.],\n",
|
|
" [-1582., -1660., -1738.]],\n",
|
|
" [[ 1707., 1773., 1839.],\n",
|
|
" [ 1971., 2037., 2103.],\n",
|
|
" [ 2235., 2301., 2367.]]],\n",
|
|
" [[[-4702., -4780., -4858.],\n",
|
|
" [-5014., -5092., -5170.],\n",
|
|
" [-5326., -5404., -5482.]],\n",
|
|
" [[ 4875., 4941., 5007.],\n",
|
|
" [ 5139., 5205., 5271.],\n",
|
|
" [ 5403., 5469., 5535.]]]]\n",
|
|
" )\n",
|
|
" true_grads_wrt_inputs = np.array(\n",
|
|
" [[[[ 147., 319., 305., 162.],\n",
|
|
" [ 338., 716., 680., 354.],\n",
|
|
" [ 290., 608., 572., 294.],\n",
|
|
" [ 149., 307., 285., 144.]],\n",
|
|
" [[ 23., 79., 81., 54.],\n",
|
|
" [ 114., 284., 280., 162.],\n",
|
|
" [ 114., 272., 268., 150.],\n",
|
|
" [ 73., 163., 157., 84.]],\n",
|
|
" [[-101., -161., -143., -54.],\n",
|
|
" [-110., -148., -120., -30.],\n",
|
|
" [ -62., -64., -36., 6.],\n",
|
|
" [ -3., 19., 29., 24.]]],\n",
|
|
" [[[ 39., 67., 53., 18.],\n",
|
|
" [ 50., 68., 32., -6.],\n",
|
|
" [ 2., -40., -76., -66.],\n",
|
|
" [ -31., -89., -111., -72.]],\n",
|
|
" [[ 59., 115., 117., 54.],\n",
|
|
" [ 114., 212., 208., 90.],\n",
|
|
" [ 114., 200., 196., 78.],\n",
|
|
" [ 37., 55., 49., 12.]],\n",
|
|
" [[ 79., 163., 181., 90.],\n",
|
|
" [ 178., 356., 384., 186.],\n",
|
|
" [ 226., 440., 468., 222.],\n",
|
|
" [ 105., 199., 209., 96.]]]])\n",
|
|
" layer = layer_class(\n",
|
|
" num_input_channels=kernels.shape[1], \n",
|
|
" num_output_channels=kernels.shape[0], \n",
|
|
" input_dim_1=inputs.shape[2], \n",
|
|
" input_dim_2=inputs.shape[3],\n",
|
|
" kernel_dim_1=kernels.shape[2],\n",
|
|
" kernel_dim_2=kernels.shape[3]\n",
|
|
" )\n",
|
|
" layer.params = [kernels, biases]\n",
|
|
" layer_grads_wrt_inputs = layer.bprop(inputs, outputs, grads_wrt_outputs)\n",
|
|
" assert layer_grads_wrt_inputs.shape == true_grads_wrt_inputs.shape, (\n",
|
|
" 'Layer bprop returns incorrect shaped array. '\n",
|
|
" 'Correct shape is \\n\\n{0}\\n\\n but returned shape is \\n\\n{1}.'\n",
|
|
" .format(true_grads_wrt_inputs.shape, layer_grads_wrt_inputs.shape)\n",
|
|
" )\n",
|
|
" assert np.allclose(layer_grads_wrt_inputs, true_grads_wrt_inputs), (\n",
|
|
" 'Layer bprop does not return correct values. '\n",
|
|
" 'Correct output is \\n\\n{0}\\n\\n but returned output is \\n\\n{1}\\n\\n difference is \\n\\n{2}'\n",
|
|
" .format(true_grads_wrt_inputs, layer_grads_wrt_inputs, layer_grads_wrt_inputs-true_grads_wrt_inputs)\n",
|
|
" )\n",
|
|
" return True\n",
|
|
"\n",
|
|
"def test_conv_layer_grad_wrt_params(\n",
|
|
" layer_class, do_cross_correlation=False):\n",
|
|
" \"\"\"Tests `grad_wrt_params` method of a convolutional layer.\n",
|
|
" \n",
|
|
" Checks the outputs of `grad_wrt_params` method for fixed inputs \n",
|
|
" against known reference values for the gradients with respect to \n",
|
|
" kernels and biases, and raises an AssertionError if the returned\n",
|
|
" values are not consistent with the reference values. If tests\n",
|
|
" are all passed returns True.\n",
|
|
" \n",
|
|
" Args:\n",
|
|
" layer_class: Convolutional layer implementation following the \n",
|
|
" interface defined in the provided skeleton class.\n",
|
|
" do_cross_correlation: Whether the layer implements an operation\n",
|
|
" corresponding to cross-correlation (True) i.e kernels are\n",
|
|
" not flipped before sliding over inputs, or convolution\n",
|
|
" (False) with filters being flipped.\n",
|
|
"\n",
|
|
" Raises:\n",
|
|
" AssertionError: Raised if output of `layer.bprop` is inconsistent \n",
|
|
" with reference values either in shape or values.\n",
|
|
" \"\"\"\n",
|
|
" inputs = np.arange(96).reshape((2, 3, 4, 4))\n",
|
|
" kernels = np.arange(-12, 12).reshape((2, 3, 2, 2))\n",
|
|
" biases = np.arange(2)\n",
|
|
" grads_wrt_outputs = np.arange(-20, 16).reshape((2, 2, 3, 3))\n",
|
|
" true_kernel_grads = np.array(\n",
|
|
" [[[[ -240., -114.],\n",
|
|
" [ 264., 390.]],\n",
|
|
" [[-2256., -2130.],\n",
|
|
" [-1752., -1626.]],\n",
|
|
" [[-4272., -4146.],\n",
|
|
" [-3768., -3642.]]],\n",
|
|
" [[[ 5268., 5232.],\n",
|
|
" [ 5124., 5088.]],\n",
|
|
" [[ 5844., 5808.],\n",
|
|
" [ 5700., 5664.]],\n",
|
|
" [[ 6420., 6384.],\n",
|
|
" [ 6276., 6240.]]]])\n",
|
|
" if do_cross_correlation:\n",
|
|
" kernels = kernels[:, :, ::-1, ::-1]\n",
|
|
" true_kernel_grads = true_kernel_grads[:, :, ::-1, ::-1]\n",
|
|
" true_bias_grads = np.array([-126., 36.])\n",
|
|
" layer = layer_class(\n",
|
|
" num_input_channels=kernels.shape[1], \n",
|
|
" num_output_channels=kernels.shape[0], \n",
|
|
" input_dim_1=inputs.shape[2], \n",
|
|
" input_dim_2=inputs.shape[3],\n",
|
|
" kernel_dim_1=kernels.shape[2],\n",
|
|
" kernel_dim_2=kernels.shape[3]\n",
|
|
" )\n",
|
|
" layer.params = [kernels, biases]\n",
|
|
" layer_kernel_grads, layer_bias_grads = (\n",
|
|
" layer.grads_wrt_params(inputs, grads_wrt_outputs))\n",
|
|
" assert layer_kernel_grads.shape == true_kernel_grads.shape, (\n",
|
|
" 'grads_wrt_params gives incorrect shaped kernel gradients output. '\n",
|
|
" 'Correct shape is \\n\\n{0}\\n\\n but returned shape is \\n\\n{1}.'\n",
|
|
" .format(true_kernel_grads.shape, layer_kernel_grads.shape)\n",
|
|
" )\n",
|
|
" assert np.allclose(layer_kernel_grads, true_kernel_grads), (\n",
|
|
" 'grads_wrt_params does not give correct kernel gradients output. '\n",
|
|
" 'Correct output is \\n\\n{0}\\n\\n but returned output is \\n\\n{1}.'\n",
|
|
" .format(true_kernel_grads, layer_kernel_grads)\n",
|
|
" )\n",
|
|
" assert layer_bias_grads.shape == true_bias_grads.shape, (\n",
|
|
" 'grads_wrt_params gives incorrect shaped bias gradients output. '\n",
|
|
" 'Correct shape is \\n\\n{0}\\n\\n but returned shape is \\n\\n{1}.'\n",
|
|
" .format(true_bias_grads.shape, layer_bias_grads.shape)\n",
|
|
" )\n",
|
|
" assert np.allclose(layer_bias_grads, true_bias_grads), (\n",
|
|
" 'grads_wrt_params does not give correct bias gradients output. '\n",
|
|
" 'Correct output is \\n\\n{0}\\n\\n but returned output is \\n\\n{1}.'\n",
|
|
" .format(true_bias_grads, layer_bias_grads)\n",
|
|
" )\n",
|
|
" return True"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"An example of using the test functions if given in the cell below. This assumes you implement a convolution (rather than cross-correlation) operation. If the implementation is correct "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from mlp.layers import ConvolutionalLayer\n",
|
|
"fprop_correct = test_conv_layer_fprop(ConvolutionalLayer, False)\n",
|
|
"bprop_correct = test_conv_layer_bprop(ConvolutionalLayer, False)\n",
|
|
"grads_wrt_param_correct = test_conv_layer_grad_wrt_params(ConvolutionalLayer, False)\n",
|
|
"if fprop_correct and grads_wrt_param_correct and bprop_correct:\n",
|
|
" print('All tests passed.')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"anaconda-cloud": {},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.2"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|