{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout and maxout\n",
    "In this lab we will explore the methods of [dropout](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf), a regularisation method which stochastically drops out activations from the model during training, and [maxout](http://www.jmlr.org/proceedings/papers/v28/goodfellow13.pdf), another non-linear transformation that can be used in multiple layer models. This is based on material covered in the [fifth lecture slides](http://www.inf.ed.ac.uk/teaching/courses/mlp/2016/mlp05-hid.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1: Implementing a dropout layer\n",
    "\n",
    "During training the forward propagation through a dropout layer produces outputs where a subset of the input dimensions are set to zero ('dropped out'). The dimensions to be dropped out are randomly sampled for each new batch, with each dimension having a probability $p$ of being included and the inclusion (or not) of each dimension independent of all the others. If the inputs to a dropout layer are $D$ dimensional vectors then we can represent the dropout operation by an elementwise multiplication by a $D$ dimensional *binary mask* vector $\\boldsymbol{m} = \\left[m_1 ~ m_2 ~\\dots~ m_D\\right]^{\\rm T}$ where $m_d \\sim \\text{Bernoulli}(p) ~~\\forall d \\in \\lbrace 1 \\dots D\\rbrace$. \n",
    "\n",
    "As a first step implement a `random_binary_mask` function in the cell below to generate a binary mask array of a specified shape, where each value in the outputted array is either a one with probablity `prob_1` or zero with probability `1 - prob_1` and all values are sampled independently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def random_binary_mask(prob_1, shape, rng):\n",
    "    \"\"\"Generates a random binary mask array of a given shape.\n",
    "    \n",
    "    Each value in the outputted array should be an indepedently sampled\n",
    "    binary value i.e in {0, 1} with the probability of each value\n",
    "    being 1 being equal to `prob_1`.\n",
    "    \n",
    "    Args:\n",
    "        prob_1: Scalar value in [0, 1] specifying probability each\n",
    "            entry in output array is equal to one.\n",
    "        shape: Shape of returned mask array.\n",
    "        rng (RandomState): Seeded random number generator object.\n",
    "    \n",
    "    Returns:\n",
    "        Random binary mask array of specified shape.\n",
    "    \"\"\"\n",
    "    return rng.uniform(size=shape) < prob_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test your `random_binary_mask` function using the cell below (if your implementation is incorrect you will get an `AssertionError` - look at what the assert statement is checking for a clue as to what is wrong)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "test_shapes = [(1, 1000), (10, 10, 10)]\n",
    "test_probs = [0.1, 0.5, 0.7]\n",
    "for i in range(10):\n",
    "    for shape in test_shapes:\n",
    "        for prob in test_probs:\n",
    "            output = random_binary_mask(prob, shape, np.random)\n",
    "            # Check generating correct shape output\n",
    "            assert output.shape == shape\n",
    "            # Check all outputs are binary values\n",
    "            assert np.all((output == 1.) | (output == 0.))\n",
    "            # Check proportion equal to one plausible\n",
    "            # This will be noisy so there is a chance this will error\n",
    "            # even for a correct implementation\n",
    "            assert np.abs(output.mean() - prob) < 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given a randomly sampled binary mask $\\boldsymbol{m}$, the outputs $\\lbrace \\boldsymbol{y}^{(b)} \\rbrace_{b=1}^B$ of the stochastic forward propagation through a dropout layer given a batch of inputs $\\lbrace \\boldsymbol{x}^{(b)} \\rbrace_{b=1}^B$ can be calculated by simply performing an elementwise multiplication of the inputs with the mask\n",
    "\n",
    "\\begin{equation}\n",
    "  y^{(b)}_d = m_k x^{(b)}_d \\qquad \\forall d \\in \\lbrace 1 \\dots D \\rbrace\n",
    "\\end{equation}\n",
    "\n",
    "The corresponding partial derivatives required for implementing back-propagation through a dropout layer are\n",
    "\n",
    "\\begin{equation}\n",
    "  \\frac{\\partial y^{(b)}_k}{\\partial x^{(b)}_d} = \n",
    "  \\begin{cases}\n",
    "      m_k & \\quad k = d \\\\\n",
    "      0   & \\quad k \\neq d\n",
    "  \\end{cases}\n",
    "  \\qquad \\forall k,\\,d \\in \\lbrace 1 \\dots D \\rbrace\n",
    "\\end{equation}\n",
    "\n",
    "As discussed in the lecture slides, when using a model trained with dropout at test time dimensions are no longer stochastically dropped out and instead all activations are deterministically fed forward through the model. So that the expected (mean) outputs of each layer are the same at test and training we scale the forward propagated inputs during testing by $p$ the probability of each dimension being included in the output. If we denote the deterministically forward-propagated batch of outputs of a dropout layer at test time as $\\lbrace \\boldsymbol{z}^{(b)} \\rbrace_{b=1}^B$ then we have\n",
    "\n",
    "\\begin{equation}\n",
    "  z^{(b)}_d =\n",
    "  \\mathbb{E}\\left[ y^{(b)}_d \\right] = \n",
    "  \\sum_{m_d \\in \\lbrace 0,1 \\rbrace} \\left( \\mathbb{P}\\left[\\mathrm{m}_d = m_d\\right] m_d x^{(b)}_d \\right) =\n",
    "  (p) (1) x^{(b)}_d  + (1-p) (0) x^{(b)}_d =\n",
    "  p x^{(b)}_d  \\qquad \\forall d \\in \\lbrace 1 \\dots D \\rbrace\n",
    "\\end{equation}\n",
    "\n",
    "To allow switching between this stochastic training time behaviour and deterministic test time behaviour, a new abstract `StochasticLayer` class has been defined in the `mlp.layers` module. This acts similarly to the layer objects we have already encountered other than adding an extra boolean argument `stochastic` to the `fprop` method interface. When `stochastic = True` (the default) a stochastic forward propagation should be caculated, for dropout this corresponding to $\\boldsymbol{x}^{(b)} \\to \\boldsymbol{y}^{(b)}$ above. When `stochastic = False` a deterministic forward-propagation corresponding to the expected output of the stochastic forward-propagation should be calculated, for dropout this corresponding to $\\boldsymbol{x}^{(b)} \\to \\boldsymbol{z}^{(b)}$ above.\n",
    "\n",
    "Using the skeleton `DropoutLayer` class definition below, implement the `fprop` and `bprop` methods. You may wish to store the binary mask used in the forward propagation as an attribute of the class for use in back-propagation - it is fine to assume that the `fprop` and `bprop` will always be called in sync."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlp.layers import StochasticLayer\n",
    "\n",
    "class DropoutLayer(StochasticLayer):\n",
    "    \"\"\"Layer which stochastically drops input dimensions in its output.\"\"\"\n",
    "    \n",
    "    def __init__(self, rng=None, incl_prob=0.5, share_across_batch=True):\n",
    "        \"\"\"Construct a new dropout layer.\n",
    "        \n",
    "        Args:\n",
    "            rng (RandomState): Seeded random number generator.\n",
    "            incl_prob: Scalar value in (0, 1] specifying the probability of\n",
    "                each input dimension being included in the output.\n",
    "            share_across_batch: Whether to use same dropout mask across\n",
    "                all inputs in a batch or use per input masks.\n",
    "        \"\"\"\n",
    "        super(DropoutLayer, self).__init__(rng)\n",
    "        assert incl_prob > 0. and incl_prob <= 1.\n",
    "        self.incl_prob = incl_prob\n",
    "        self.share_across_batch = share_across_batch\n",
    "        \n",
    "    def fprop(self, inputs, stochastic=True):\n",
    "        \"\"\"Forward propagates activations through the layer transformation.\n",
    "\n",
    "        Args:\n",
    "            inputs: Array of layer inputs of shape (batch_size, input_dim).\n",
    "            stochastic: Flag allowing different deterministic\n",
    "                forward-propagation mode in addition to default stochastic\n",
    "                forward-propagation e.g. for use at test time. If False\n",
    "                a deterministic forward-propagation transformation\n",
    "                corresponding to the expected output of the stochastic\n",
    "                forward-propagation is applied.\n",
    "\n",
    "        Returns:\n",
    "            outputs: Array of layer outputs of shape (batch_size, output_dim).\n",
    "        \"\"\"\n",
    "        if stochastic:\n",
    "            mask_shape = (1,) + inputs.shape[1:] if self.share_across_batch else inputs.shape\n",
    "            self._mask = (rng.uniform(size=mask_shape) < self.incl_prob)\n",
    "            return inputs * self._mask\n",
    "        else:\n",
    "            return inputs * self.incl_prob\n",
    "    \n",
    "    def bprop(self, inputs, outputs, grads_wrt_outputs):\n",
    "        \"\"\"Back propagates gradients through a layer.\n",
    "\n",
    "        Given gradients with respect to the outputs of the layer calculates the\n",
    "        gradients with respect to the layer inputs. This should correspond to\n",
    "        default stochastic forward-propagation.\n",
    "\n",
    "        Args:\n",
    "            inputs: Array of layer inputs of shape (batch_size, input_dim).\n",
    "            outputs: Array of layer outputs calculated in forward pass of\n",
    "                shape (batch_size, output_dim).\n",
    "            grads_wrt_outputs: Array of gradients with respect to the layer\n",
    "                outputs of shape (batch_size, output_dim).\n",
    "\n",
    "        Returns:\n",
    "            Array of gradients with respect to the layer inputs of shape\n",
    "            (batch_size, input_dim).\n",
    "        \"\"\"\n",
    "        return grads_wrt_outputs * self._mask \n",
    "\n",
    "    def __repr__(self):\n",
    "        return 'DropoutLayer(incl_prob={0:.1f})'.format(self.incl_prob)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test your implementation by running the cell below (if your implementation is incorrect you will get an `AssertionError` - look at what the assert statement is checking for a clue as to what is wrong)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "seed = 31102016 \n",
    "rng = np.random.RandomState(seed)\n",
    "test_incl_probs = [0.1, 0.5, 0.7]\n",
    "input_shape = (5, 10)\n",
    "for incl_prob in test_incl_probs:\n",
    "    layer = DropoutLayer(rng, incl_prob)\n",
    "    inputs = rng.normal(size=input_shape)\n",
    "    grads_wrt_outputs = rng.normal(size=input_shape)\n",
    "    for t in range(100):\n",
    "        outputs = layer.fprop(inputs, stochastic=True)\n",
    "        # Check outputted array correct shape\n",
    "        assert outputs.shape == inputs.shape\n",
    "        # Check all outputs are either equal to inputs or zero\n",
    "        assert np.all((outputs == inputs) | (outputs == 0))\n",
    "        grads_wrt_inputs = layer.bprop(inputs, outputs, grads_wrt_outputs)\n",
    "        # Check back-propagated gradients only non-zero for non-zero outputs\n",
    "        assert np.all((outputs != 0) == (grads_wrt_inputs != 0))\n",
    "        assert np.all(grads_wrt_outputs[outputs != 0] == grads_wrt_inputs[outputs != 0])\n",
    "    det_outputs = layer.fprop(inputs, stochastic=False)\n",
    "    # Check deterministic fprop outputs are correct shape\n",
    "    assert det_outputs.shape == inputs.shape\n",
    "    # Check deterministic fprop outputs scaled correctly\n",
    "    assert np.allclose(det_outputs, incl_prob * inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optional extension\n",
    "\n",
    "Above we assumed the same dropout mask was applied to each input in a batch, as specified in the lecture slides. In practice sometimes a different mask is sampled for each input. As an extension you could try implementing this per-input form of dropout either by defining a new layer or adding an extra argument to the constructor of the above layer which allows you to switch between the two forms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2: Training with dropout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Experiment with training models with dropout layers to classify MNIST digits. Code has been provided below as a starting point for setting up the model objects though feel free to use any additional adaptive learning rules or learning rule schedulers you wrote during the coursework instead. You may also wish to change the model architecture to use a larger model with more parameters in which the regularisation provided by dropout is likely to have a more pronounced effect. You will probably also find that models with dropout generally need to be trained over more epochs than those without (can you suggest why this might be?).\n",
    "\n",
    "You should training with a few different `incl_prob` settings for the dropout layers and try to establish how the values chosen affect the training performance. You may wish to experiment with using a different dropout probability at the input than for the intermediate layers (why?).\n",
    "\n",
    "You may wish to start reading through and implementing exercise 3 while waiting for training runs to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import logging\n",
    "from mlp.data_providers import MNISTDataProvider\n",
    "from mlp.models import MultipleLayerModel\n",
    "from mlp.layers import ReluLayer, AffineLayer\n",
    "from mlp.errors import CrossEntropySoftmaxError\n",
    "from mlp.initialisers import GlorotUniformInit, ConstantInit\n",
    "from mlp.learning_rules import MomentumLearningRule\n",
    "from mlp.optimisers import Optimiser\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Seed a random number generator\n",
    "seed = 31102016 \n",
    "rng = np.random.RandomState(seed)\n",
    "\n",
    "# Set up a logger object to print info about the training run to stdout\n",
    "logger = logging.getLogger()\n",
    "logger.setLevel(logging.INFO)\n",
    "logger.handlers = [logging.StreamHandler()]\n",
    "\n",
    "# Create data provider objects for the MNIST data set\n",
    "train_data = MNISTDataProvider('train', batch_size=50, rng=rng)\n",
    "valid_data = MNISTDataProvider('valid', batch_size=50, rng=rng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1570e8d049a44c52b2608f768bf7431f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0ba46a7ab1344643b0918cdc4be9cca2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3a07d519729341d3830025aae432be59",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1c257f5e31344710b8b70d079779f5f1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dce6a1b493d1458ba621933e0b142790",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "16cb7154e175437abe3442279be3495d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 5: 5.0s to complete\n",
      "    error(train)=4.50e-01, acc(train)=8.63e-01, error(valid)=3.95e-01, acc(valid)=8.75e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fef614f668e047adb56e9b46072b5313",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6dbffb51864e49b482d2d3cb19614d32",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "95eb02808ab64475a6f7df48e18ced07",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aad36029708246879ae1115fbbe28467",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d1c268026f0d4553837075abfd269951",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 10: 5.1s to complete\n",
      "    error(train)=3.72e-01, acc(train)=8.86e-01, error(valid)=3.35e-01, acc(valid)=9.00e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d27023b34df34004aa0c88b8859c7286",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2ba31ddcc89243358ecb5af656ac5da1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "acb54a7df3ca4d178d95bc6e80e21af2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "adf7558500464c81b76b9a873dcdfd3a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e834c74b669d4f9c9bc7a24cb02ef55d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 15: 6.3s to complete\n",
      "    error(train)=3.33e-01, acc(train)=8.99e-01, error(valid)=3.23e-01, acc(valid)=9.04e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8df1ec88360243dcbc5ae9fa2c936c95",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a3ab83357cf14be587cd3513ebe2d1f1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7ca9850b800d4175b32fb521aefd0efe",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "30d0300afce0437a88d50726255b7de4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "209cdfede4f64daca753d20ea94fbb74",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 20: 7.8s to complete\n",
      "    error(train)=3.09e-01, acc(train)=9.06e-01, error(valid)=2.85e-01, acc(valid)=9.15e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1daa05b3785c486daf4f1c5392ad6aa9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c453707b73094c18921585d433db7c0d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ea96921abe4241c894c630d7bb3411a0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "475ab1b604ce4303a3e8ee6a9eb853bb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "278bb04b0b1d4022a1b9dc3e7c4cf94f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 25: 4.4s to complete\n",
      "    error(train)=2.98e-01, acc(train)=9.10e-01, error(valid)=2.87e-01, acc(valid)=9.13e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3d550f1782be4ef2b3c292ada8270585",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d9d958a3603841e4986535355f051346",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b6d9aabde2f7431999bee9512044e122",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ba3f1f442317416a992f5089047a630c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "75550efdb5074490a4638a365e777421",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 30: 4.7s to complete\n",
      "    error(train)=2.92e-01, acc(train)=9.10e-01, error(valid)=2.71e-01, acc(valid)=9.20e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "17d66dc35b014d4b9ee81ac16f295099",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2256aef45a814d83ad8109385a0857a2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64641729794b42d4948e0102b8390375",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d8e1fb104f094c1dbf0a4144d2ec39f2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2f466aa9dd104da7a1b222b707fcfc95",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 35: 7.4s to complete\n",
      "    error(train)=2.83e-01, acc(train)=9.14e-01, error(valid)=2.87e-01, acc(valid)=9.15e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0884d3d86ec94e658497215e65566d80",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e820d07bc4b144bb8d3675a12992be70",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00bbd01802ca4896985099582e4dae63",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd33d19947ba446081f8cfc96f9aab55",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e44bc5dd043f4ee19ca28044fb440410",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 40: 4.9s to complete\n",
      "    error(train)=2.74e-01, acc(train)=9.17e-01, error(valid)=2.56e-01, acc(valid)=9.24e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "43358e22a2f2416795e4808e118ef138",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e2c9235d716442a3974242d3b69077f2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6e1607a023d24a50b375b13c92fccd35",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a3dee9b0f99247aa93ad6cdc0b16cf4c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0bf336981b6b4761af29fb783d5037d9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 45: 9.7s to complete\n",
      "    error(train)=2.73e-01, acc(train)=9.18e-01, error(valid)=2.64e-01, acc(valid)=9.22e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eee9189d58274a1289aac7e09ee0b4bc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5429d270e0da42e9becefe000112eaa4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8885af6289624c99b6337d13e41fbd63",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ede51727693c4c23863475afd864631e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a73a7b5dc4114240896160fd5e987447",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 50: 9.0s to complete\n",
      "    error(train)=2.72e-01, acc(train)=9.18e-01, error(valid)=2.62e-01, acc(valid)=9.26e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "18d11f77d5e44ecc9d395c63c06c9317",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ef9359b9222646ef89544e40d4da8076",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "05836a0d5e5a44b7846c09f57a4c56ca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "35014e8b5c284f80bc6b210ee40910fa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d9c0357cffbf4edd8573ecda737cbe28",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 55: 10.8s to complete\n",
      "    error(train)=2.62e-01, acc(train)=9.21e-01, error(valid)=2.50e-01, acc(valid)=9.26e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "095e101ea8414d2f8e21f03a3fcdbfff",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ed9da8a4db5643e289c1b16cf26d170d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fccf93993b5242df9b976c11fb3846ae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b3985082d8f24531a5dd7b0dccd074ae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ac075e7579d84afbac8bf5108ba49aca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 60: 7.0s to complete\n",
      "    error(train)=2.61e-01, acc(train)=9.21e-01, error(valid)=2.48e-01, acc(valid)=9.27e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4ab6620b5100414c9af5fc6465c99bc7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5d84a2a69eab41af86bea91820aa9d09",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b444569827ea47f787d6399871ff8acc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5d5ba3c9163647ed98f41d6b74f7c5a8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a285e64ceaaa4b8d8a7abb6424f15d90",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 65: 8.5s to complete\n",
      "    error(train)=2.58e-01, acc(train)=9.22e-01, error(valid)=2.62e-01, acc(valid)=9.24e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6eaa64732a9d4cfe8cba94b7ad519182",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d8ca76d20c145a3947ca7f9d7f68bbb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ad2a274f76a448099a41fc9b743aa68c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a119af1dc2924ea3a844ece31f4de437",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "668f3f9c541c4baea191481db86260cd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 70: 6.5s to complete\n",
      "    error(train)=2.53e-01, acc(train)=9.24e-01, error(valid)=2.39e-01, acc(valid)=9.29e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "15b5beb781ad4dc2bbb6a8a0a740378b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4d9680a210ca4139aead1d91635eb483",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "47a5a602989c4e40bd8e0c9eb2c726c3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "70f7e81581364cbfa33bc05983857b17",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bfe4a1bc800145d4862158d2ff6bd148",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 75: 4.8s to complete\n",
      "    error(train)=2.55e-01, acc(train)=9.23e-01, error(valid)=2.52e-01, acc(valid)=9.28e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1adf3ded095347a7a32dd514a4a8d44e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "46216a3cdb774ccdab27bdfaad7491b3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd851493e6a241e5b0ad402785d9e3bf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a3c918d4fe0b44039421c8f50106490e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e3ac9dc85ac64c1db29a15cb0aa53e9d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 80: 5.5s to complete\n",
      "    error(train)=2.49e-01, acc(train)=9.25e-01, error(valid)=2.52e-01, acc(valid)=9.27e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "abfad44544a843858505c3f1388c38e0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "32f89d02068d487c9c2fafb1462fb4eb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4485b100a7724386ad2bf6ea14241cef",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8a599deab8f14eeaaabc45e97e263e54",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b414cae29a10405ead343f58858d6d8a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 85: 5.4s to complete\n",
      "    error(train)=2.51e-01, acc(train)=9.26e-01, error(valid)=2.53e-01, acc(valid)=9.29e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "70042b7a5cd64378a8b51b11f3f1f385",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "10b961c8c6254297ba70df418afe67bb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2c36eb7fb8844be09ba83b979a3afd0c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "50c5549d709e47a7a6b6709c7e7cb74d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7ce364d66cf742108a468ec40c4e3913",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 90: 5.0s to complete\n",
      "    error(train)=2.45e-01, acc(train)=9.26e-01, error(valid)=2.48e-01, acc(valid)=9.27e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5cdcaa8fc3354cc1be2227d1d0b2ec85",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6ceb8c00fc9844678cecb81604620ee8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c2f182aeeaa546e7a544425843bbba96",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1419f1fea31c48a5a765a087e9ca47d0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f8b87965d23476a9d1dcba586b7209f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 95: 5.2s to complete\n",
      "    error(train)=2.47e-01, acc(train)=9.25e-01, error(valid)=2.40e-01, acc(valid)=9.29e-01\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f100ae52a704bec8a696fbae64f6114",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dc4c1d2a13d04ca78bbabf3ac96986d8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3a9592147abd412fa7a06e23b700d0cd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2b26af7df3a64327adeb2c285fc57d4d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "50e49d26598747c9bd4f3b511370ba09",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=1000), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Epoch 100: 5.3s to complete\n",
      "    error(train)=2.50e-01, acc(train)=9.25e-01, error(valid)=2.30e-01, acc(valid)=9.31e-01\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f214a6de710>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe0AAAEKCAYAAAA/9Q3BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8leX9//HXJzshISEQwiaMMMKGAG5BwL2xzlqpX7W4\nql9tta1b22/Vn7VatVr3rHtRR1VUFJEVQGWKbMIMCQnZ8/r9cR8gQiDrJCfhvJ+PRx7nnPvc45Pj\nkXeu+77u6zLnHCIiItLyhQS6ABEREakbhbaIiEgrodAWERFpJRTaIiIirYRCW0REpJVQaIuIiLQS\nCm0REZFWQqEtIiLSSii0RUREWomwQBewrw4dOriUlJRAlyEiItJsFixYsMM5l1Tbei0utFNSUsjI\nyAh0GSIiIs3GzNbXZT2dHhcREWklFNoiIiKthEJbRESklWhx17RFRKT5lJeXk5mZSUlJSaBLCQpR\nUVF069aN8PDwBm2v0BYRCWKZmZnExcWRkpKCmQW6nEOac47s7GwyMzPp1atXg/ZRp9PjZnaimf1o\nZqvM7A8HWW+ymTkzS/e9TjGzYjP7zvfzRIOqFBGRJlFSUkL79u0V2M3AzGjfvn2jzmrU2tI2s1Dg\nMWASkAnMN7Npzrll+6wXB1wHzN1nF6udc8MbXKGIiDQpBXbzaexnXZeW9hhglXNujXOuDHgNOKOG\n9e4B7gNazIWRwtIK/jljFRnrcgJdioiISKPVJbS7Ahurvc70LdvDzEYC3Z1zH9awfS8zW2RmX5nZ\n0TUdwMyuMLMMM8vIysqqa+21Cgs1/vnlat5emOm3fYqISOvz0EMP8eKLLwLw/PPPs3nz5nrv44kn\nntizjwNZvHgxU6ZMaUiJddLojmhmFgI8CEyp4e0tQA/nXLaZjQLeM7NBzrld1Vdyzj0JPAmQnp7u\nGlvTbpFhoRzbP4npy7fzlypHSIhOAYmItEaVlZWEhoYe8PWBVFRUAPDss8+ycOFCwAvtwYMH06VL\nl1qPU93UqVNrPd6QIUPIzMxkw4YN9OjRo9b166suLe1NQPdqr7v5lu0WBwwGZpjZOuAwYJqZpTvn\nSp1z2QDOuQXAaqCfPwqvq0kDk8nKL+WHTXnNeVgREamHl19+mTFjxjB8+HB+85vfUFlZSWxsLDfe\neCPDhg1j9uzZpKSkcPPNNzNy5EjefPNNvvvuOw477DCGDh3KWWedxc6dOwEYN24c119/Penp6Tz8\n8MN88cUXjBw5krCwMN566y0yMjK46KKLGD58OMXFxfvt96mnnmL06NEMGzaMyZMnU1RUBMCdd97J\nAw88sOcYN998M2PGjKFfv37MnDlzz+9y2mmn8dprrzXJ51SXlvZ8INXMeuGF9fnAhbvfdM7lAR12\nvzazGcDvnHMZZpYE5DjnKs2sN5AKrPFj/bUa1z+J0BBj+rJtDO+e0JyHFhFpVe76z1KWbd5V+4r1\nkNalLXecNuig6yxfvpzXX3+dWbNmER4ezlVXXcUrr7xCYWEhY8eO5W9/+9ueddu3b7+nxTx06FAe\neeQRjj32WG6//XbuuusuHnroIQDKysr2zGNxxx13MGrUKADOOeccHn30UR544AHS09Nr3G92djaX\nX345ALfeeivPPPMM11577X51V1RUMG/ePD766CPuuusupk+fDkB6ejr33nsvN910U4M+s4OpNbSd\ncxVmdg3wCRAKPOucW2pmdwMZzrlpB9n8GOBuMysHqoCpzrlm7RWWEBPB6JR2fLZsG787oX9zHlpE\nROrg888/Z8GCBYwePRqA4uJiOnbsSGhoKJMnT/7Zuueddx4AeXl55ObmcuyxxwJwySWX8Itf/GK/\n9QC2bNnCwIEDD1pD9fWXLFnCrbfeSm5uLgUFBZxwwgk1bnP22WcDMGrUKNatW7dneceOHRt0zbwu\n6nRN2zn3EfDRPstuP8C646o9fxt4uxH1+cXEgcn8+cPlbMguokf7mECXIyLSItXWIm4qzjkuueQS\n/vrXv/5s+QMPPLDf9eU2bdrUaZ/V14uOjq713ujq60+ZMoX33nuPYcOG8fzzzzNjxowat4mMjAQg\nNDR0z7Vz8O59j46OrlOd9RUUY49PSksGYPrybQGuRERE9jVhwgTeeusttm/fDkBOTg7r1x98psr4\n+HjatWu351rySy+9tKfVva+BAweyatWqPa/j4uLIz88/4L7z8/Pp3Lkz5eXlvPLKK/X9dVi5ciWD\nBw+u93Z1ERTDmPZs34Z+ybFMX76NS49q2NBxIiLSNNLS0vjzn//M8ccfT1VVFeHh4Tz22GO1bvfC\nCy8wdepUioqK6N27N88991yN65100klcfPHFe15PmTKFqVOnEh0dzezZs/db/5577mHs2LEkJSUx\nduzYgwZ8Tb788ktOOeWUem1TV+ac3+6w8ov09HS3u/OAP93/3xX86+s1LLx1EvExDRuoXUTkULN8\n+fJar/ceCs466yzuv/9+UlNTm/Q4paWlHHvssXzzzTeEhdXcLq7pMzezBc659Bo3qCYoTo8DTExL\nprLKMWPl9kCXIiIizezee+9ly5YtTX6cDRs2cO+99x4wsBsraEJ7eLcEOsRG8NkyXdcWEQk2/fv3\n55hjjmny46SmpjJu3Lgm23/QhHZIiDFhQDJf/ZhFWUVVoMsRERGpt6AJbfB6keeXVjBvrSYQERGR\n1ieoQvvIvh2ICg/RrV8iItIqBVVoR0eEclTfJD5bto2W1mteRESkNkEV2gCT0jqyKbeY5Vvqd9+d\niIi0XtWn5qyvKVOm8NZbbwFw2WWXsWzZsv3Wef7557nmmmsAePTRR3n22WcbXuxBBF1oHzcgGTON\njiYi0ppUVlYe9PWBVFRUUFFRwbPPPsuFF15Y+wa1ePrpp0lLSzvoOpdeeimPPPJIo49Vk6AL7aS4\nSEZ0T1Boi4i0IM01NeeKFSsYM2bMnuOuW7eOIUOGAHD33XczevRoBg8ezBVXXFHjZdRx48btmT3s\nueeeo1+/fowZM4ZZs2btWScmJoaUlBTmzZvn988pKIYx3dfEtGTu/++PbM0roVN8VKDLERFpGT7+\nA2xd7N99dhoCJ9170FWac2rOAQMGUFZWxtq1a+nVqxevv/76nhm+rrnmGm6/3ZsL6+KLL+aDDz7g\ntNNOq7HmLVu2cMcdd7BgwQLi4+MZP348I0aM2PN+eno6M2fO/NkfCP4QdC1tgEkDvQlEPl+h1raI\nSKBVn5pz+PDhfP7556xZs6beU3N+/fXX+60HXsAmJSXteX3uuefy+uuvA/wstL/88kvGjh3LkCFD\n+OKLL1i6dOkBa547dy7jxo0jKSmJiIiInx0Pmm56zqBsafftGEvP9jF8tmwbF43tGehyRERahlpa\nxE2luafmPO+88/jFL37B2WefjZmRmppKSUkJV111FRkZGXTv3p0777yz1uk8D6appucMypa2mTFx\nYDLfrsqmsLSi9g1ERKTJNPfUnH369CE0NJR77rlnTwt5d0B36NCBgoKCPb3FD2Ts2LF89dVXZGdn\nU15ezptvvvmz95tqes6gbGmDNzraM9+sZeZPWZw4uHOgyxERCVrNPTUneK3t3//+96xduxaAhIQE\nLr/8cgYPHkynTp0YPXr0QY/duXNn7rzzTg4//HASEhIYPnz4z96fNWsWd955Z62/Q30FzdSc+6qo\nrGLUn6czcWAyfzt3WJMfT0SkJdLUnP63aNEiHnzwQV566aUa39fUnA0QFhrCcQM68sWKbVRWtaw/\nXERExL+aa2pOgB07dnDPPfc0yb6DNrQBJg5MZmdROQvW7wx0KSIi0oSaa2pOgEmTJpGSktIk+w7q\n0D6mXwfCQ00DrYhIUGtpl0kPZY39rIM6tOOiwjm8TwemL1Noi0hwioqKIjs7W8HdDJxzZGdnExXV\n8EG9grb3+G6TBnbktveXsjqrgD5JsYEuR0SkWXXr1o3MzEyysrICXUpQiIqKolu3bg3ePuhDe8LA\nZG57fynTl22jz7EKbREJLuHh4fTq1SvQZUgdBfXpcYAuCdEM6tKWz3SKXEREWrigD23wepEv2LCT\n7ILSQJciIiJyQAptvNHRnIMvVmwPdCkiIiIHpNAGBnVpS+f4KN36JSIiLZpCm70TiHy9cgcl5ZWB\nLkdERKRGCm2fiWnJFJdX8u3qHYEuRUREpEYKbZ/DeicSGxnGZ8t0XVtERFomhbZPZFgox/ZL4vPl\n26jSBCIiItICKbSrmZjWke35pSzelBfoUkRERPaj0K5mfP+OhIZoAhEREWmZFNrVJMREkN6znUZH\nExGRFkmhvY9Jacms2JrPxpyiQJciIiLyM3UKbTM70cx+NLNVZvaHg6w32cycmaVXW/ZH33Y/mtkJ\n/ii6KU1KSwbQKXIREWlxag1tMwsFHgNOAtKAC8wsrYb14oDrgLnVlqUB5wODgBOBf/r212L1bN+G\n1I6xCm0REWlx6tLSHgOscs6tcc6VAa8BZ9Sw3j3AfUBJtWVnAK8550qdc2uBVb79tWgT05KZuyaH\nvOLyQJciIiKyR11CuyuwsdrrTN+yPcxsJNDdOfdhfbf1bX+FmWWYWUZLmIh94sBkKqocM37UQCsi\nItJyNLojmpmFAA8CNzZ0H865J51z6c659KSkpMaW1GjDuyfQITaC6csV2iIi0nKE1WGdTUD3aq+7\n+ZbtFgcMBmaYGUAnYJqZnV6HbVuk0BDjuAEd+XjJVsoqqogIUyd7EREJvLqk0Xwg1cx6mVkEXsey\nabvfdM7lOec6OOdSnHMpwBzgdOdchm+9880s0sx6AanAPL//FgfjHFTVf+auSWmdyC+pYP66nCYo\nSkREpP5qDW3nXAVwDfAJsBx4wzm31Mzu9rWmD7btUuANYBnwX+Bq51zzzX1ZuAOeOAoWvVzvTY/q\n24HIsBANtCIiIi2GOdeyJsdIT093GRkZ/tmZc/DPwyE8Gq74st6bX/bCfFZszWfmTePxnfoXERHx\nOzNb4JxLr229Q/tirRmMmgKbF8KW7+u9+cSByWTuLGbF1nz/1yYiIlJPh3ZoAww7D8KiYMHz9d70\nuIEdMYPpOkUuIiItwKEf2tHtYNBZ8MObUFpQr007xkUxvHuCRkcTEZEW4dAPbfBOkZflw9J36r3p\nxIHJfJ+Zx7ZdJbWvLCIi0oSCI7S7j4WkAZDxXL033T2ByOcaaEVERAIsOEK7ER3SUjvG0iMxhs+W\nbW2a2kREROooOEIbYOjuDmkv1GszM2PiwGRmrc6msLSiiYoTERGpXfCEdkwipJ0JP7wBZYX12nRS\nWjJlFVXM/GlHExUnIiJSu+AJbdjbIW3J2/XaLD2lHfHR4epFLiIiARVcod3jMK9DWj3v2Q4PDWF8\n/yS+WLGdyqqWNYKciIgEj+AK7d0d0jYtgC0/1GvTiWnJ5BSWsXDDzqapTUREpBbBFdrgdUgLjax3\na/vYfkmEh5pGRxMRkYAJvtCOSfSNkFa/DmlxUeEc1rs9n+m6toiIBEjwhTZU65BWvxHSJqUlsyar\nkNVZ9RsOVURExB+CM7R7HAYd+sOC+o2QNmHg7tHR1NoWEZHmF5yhbQbpv653h7SuCdGkdW7LZ7qu\nLSIiARCcoQ17O6QtrN8IaRPTklmwfifZBaVNVJiIiEjNgje0YxJhUP1HSDs+LZkqB1/+mNWExYmI\niOwveEMbvA5ppbvq1SFtUJe2dGobxadLNYGIiIg0r+AO7R6H+zqkPV/nTcyM04Z1ZvrybSzZlNd0\ntYmIiOwjuEN7zwhpGbB1cZ03u2Z8Ku1iIrjt/SVUaVhTERFpJsEd2gDDzq/3CGnxMeH86eSBLNqQ\nyxsZG5uuNhERkWoU2g3skHb2yK6MSUnkvv+uYGdhWRMWKCIi4lFow94OaUvfrfMmZsY9Zw5mV0kF\n93+youlqExER8VFow94OaRn1GyGtf6c4Lj0yhVfnbdTsXyIi0uQU2tDgDmkA103sR3LbSG57bwkV\nlVVNU5+IiAgK7b32dEir3whpsZFh3H7qIJZu3sXLc9Y3UXEiIiIK7b1iEiHtDPjhdSgrqtemJw/p\nxNGpHfjbpyvZnl/SRAWKiEiwU2hXl/5rX4e0+k3ZaWbcdfogSiuq+L8PlzdRcSIiEuwU2tX1OBw6\n9KvXPdu79U6K5TfH9ua97zYze3W2/2sTEZGgp9CubneHtMz5sHVJvTe/alxfurWL5vb3l1BWoU5p\nIiLiXwrtfQ27oN4jpO0WHRHKXacP4qftBTw7a63/axMRkaCm0N5XIzqkAUwYmMzEgck8PP0nNucW\nN0GBIiISrBTaNWnACGnV3XFaGg7H3f9Z5t+6REQkqCm0a9LzCF+HtPqNkLZb98QYrj0ulf8u3cqM\nH7f7uTgREQlWCu2aNLJDGsBlR/eid4c23DFtKSXllf6tT0REglKdQtvMTjSzH81slZn9oYb3p5rZ\nYjP7zsy+MbM03/IUMyv2Lf/OzJ7w9y/QZIZdAKERsLB+I6TtFhkWyt1nDGZ9dhFPfLXaz8WJiEgw\nqjW0zSwUeAw4CUgDLtgdytX82zk3xDk3HLgfeLDae6udc8N9P1P9VXiT290h7fuGdUgDOCq1A6cO\n7cw/Z6xmfXbdp/0UERGpSV1a2mOAVc65Nc65MuA14IzqKzjndlV72QZw/isxgEb9GkrzGtwhDeC2\nU9OICA3hjmlLce7Q+FhERCQw6hLaXYGN1V5n+pb9jJldbWar8Vrav632Vi8zW2RmX5nZ0TUdwMyu\nMLMMM8vIysqqR/lNrOcR0D61Qfds75bcNorrJ6Yy48csPlm6zX+1iYhI0PFbRzTn3GPOuT7AzcCt\nvsVbgB7OuRHADcC/zaxtDds+6ZxLd86lJyUl+aukxtvTIW0ebFva4N1MOSKFAZ3iuPs/Sykqq/Bf\nfSIiElTqEtqbgO7VXnfzLTuQ14AzAZxzpc65bN/zBcBqoF/DSg2Q4Rd6HdIa0doOCw3hnjMHszmv\nhH98vsp/tYmISFCpS2jPB1LNrJeZRQDnA9Oqr2BmqdVengL85Fue5OvIhpn1BlKBNf4ovNn4oUMa\nwOiURM4Z1Y2nZ67hp235fixQRESCRa2h7ZyrAK4BPgGWA28455aa2d1mdrpvtWvMbKmZfYd3GvwS\n3/JjgB98y98Cpjrncvz+WzS1UVO8DmnL3mvUbv540gDaRIZx2/tL1ClNRETqzVpaeKSnp7uMjIxA\nl/FzzsGjoyG6HVz2WaN29fKc9dz63hIePn84Zwzfrz+fiIgEITNb4JxLr209jYhWF37qkAZwwZge\nDO0Wzz0fLGdXSbl/6hMRkaCg0K6r3SOkLWjYCGm7hYYYfz5zMNmFpTz46Uo/FSciIsFAoV1XbdrD\nwNPhh9ca1SENYGi3BC4a24MXZ69jyaY8/9QnIiKHPIV2faT/Gkoa3yEN4PfHD6BdTAS3vb+EqqqW\n1a9ARERaJoV2ffQ8Etr3bdQ927vFx4Tzx5MHsmhDLm8u2Fj7BiIiEvQU2vWxu0PaxrmwbVmjdzd5\nZFdGp7Tj3o9XsLOwrPH1iYjIIU2hXV/DfCOkzfuXdytYI5gZ95w5mF0lFdz/yQo/FSgiIocqhXZ9\ntWkPQ8/1TpE/fwpkNu6e8gGd2vLrI1J4dd5GFm7Y6Z8aRUTkkKTQbohTH4KTH4AdK+HpCfDGryB7\ndYN3d/2kfiS3jeS295ZQUVnlx0JFRORQotBuiNBwGHM5/HYRHPsH+Gk6PDYGPrwRCrbXe3exkWHc\ncdoglm7exQ1vfK/gFhGRGim0GyMyDsb/0QvvkZdAxnPwjxEw414oLajXrk4e0pmbTuzPtO83c+Ob\nCm4REdmfQtsf4pLh1Afh6nnQdwLM+Cv8YzjMewoq6z5U6VXj+vL7E/rz/neb+d2b31Op+7dFRKQa\nhbY/degL574Il30OHfrBR7+Dx8bC0vfq3NP86vFecL+n4BYRkX0otJtCt3SY8iFc8Lp3e9ibl8DT\nE2HdrDptfvX4vtw4qR/vLtrE7xXcIiLio9BuKmbQ/0S4chac/ijs2gzPnwz/Pg+2L69182snpHLD\npH68s2gTN731g4JbREQIC3QBh7yQUBh5MQyeDHOfgG8egsePgOEXwrg/QfyB59T+7YRUqpzjoek/\nYQb3Tx5KSIg1Y/EiItKSKLSbS0QMHH2DNwzqzL/BvCdh8Vtw2JVw5PUQnVDjZtdP7Idz8PDnP2HA\nfQpuEZGgpdPjzS0mEU74C1yTAWlneC3vfwyH2Y9BRWmNm/zvpH78dkIqby7I5A/v/KBZwUREgpRC\nO1Da9YSzn4TffA1dRsAnf4JHR0NuzTN+/e/EVK49ri9vZGTyp3cXK7hFRIKQQjvQOg+Fi9/1fgp3\neKOq1XB7mJlxw6R+XDO+L6/N38gt7ym4RUSCjUK7pehzHIz/E/z0CSx7v8ZVzIwbj+/HVeP68Oq8\njdzy3hIFt4hIEFFHtJZk7FRY/AZ8fDP0GQ9R8futYmb8/oT+OODxGasJMbjnjMHqnCYiEgTU0m5J\nQsPgtIehcDt8fvcBVzMzbjqhP1OP7cMrczdw+7QluEbO7S0iIi2fWtotTZcRMOY33j3dQ8+D7mNq\nXM3MuPnE/jjn+NfXazCMu88YhJla3CIihyq1tFui426Btl3gP9cddMIRM+MPJw3gimN689Kc9dwx\nbala3CIihzCFdksUGQcnPwDbl8HsRw+6qpnxx5MGcNlRvXhx9nru+s8yBbeIyCFKp8dbqgEnw4BT\nYcZ9kHYmJPY64Kpmxi2nDMQBz3yzFoA7TkvTqXIRkUOMWtot2cn/D0LCDnjvdnVmxq2nDOTSI3vx\n/LfruPsDtbhFRA41Cu2WrG0XmHAbrP4clrxd6+pmxm2nDuTXR6bw3Kx13PPBcgW3iMghRKfHW7rR\nl8EPr8N//+ANwBKTeNDVzYzbT03DOXh21lpCDG45ZaBOlYuIHALU0m7pQkLh1IegKAem31mnTcyM\nO05L45LDe/L0N2u54qUFLN+yq2nrFBGRJqfQbg06D4XDr4KFL8D6b+u0iZlx5+mD+P0J/ZmzOpuT\nHp7Jb17KYOnmvCYuVkREmoq1tGue6enpLiMjI9BltDxlhfDYYRAeDVNnQlhknTfNKyrnmVlreW7W\nWvJLKpiUlsx1E1IZ3HX/YVJFRKT5mdkC51x6beuppd1aRLSBU/4GO36EWf+o16bxMeHcMKkf39x8\nHNdPTGXummxOfeQbLnthPosz1fIWEWkt1NJubd6cAis+giu/hQ59G7SLXSXlPD9rHc98s5a84nKO\nG9CR6yakMqx7gn9rFRGROqlrS1uh3drkb4VHx3jXuS/5DzSiV3h+STkvzl7PUzPXkFtUzrj+SVw3\nIZURPdr5sWAREamNX0+Pm9mJZvajma0ysz/U8P5UM1tsZt+Z2TdmllbtvT/6tvvRzE6o368h+4nr\nBBPvgHUz4fvXGrerqHCuHt+Xb24+jptO7M/3G3M565/fcvEzc1mwPsdPBYuIiL/U2tI2s1BgJTAJ\nyATmAxc455ZVW6etc26X7/npwFXOuRN94f0qMAboAkwH+jnnKg90PLW066CqCp47EXb8BNdkQJv2\nftltYWkFL81Zz1NfryG7sIyj+nbguompjE45+L3hIiLSOP5saY8BVjnn1jjnyoDXgDOqr7A7sH3a\nALv/EjgDeM05V+qcWwus8u1PGiMkxLt3u3QXfHab33bbJjKMqcf2YebN4/nTyQNYsXUXv3hiNhc+\nNYe5a7L9dhwREWmYuoR2V2BjtdeZvmU/Y2ZXm9lq4H7gt/Xc9gozyzCzjKysrLrWHtyS0+CI38J3\nr8Dar/2665iIMK44pg8zbzqOW08ZyE/bCzjvyTmc/+RsZq9WeIuIBIrfbvlyzj3mnOsD3AzcWs9t\nn3TOpTvn0pOSkvxV0qHv2JugXQr853ooL/H77qMjQrns6N7MvGk8t5+axpqsQi54ag7n/ms205dt\no6C0wu/HFBGRA6vL2OObgO7VXnfzLTuQ14DHG7it1Ed4NJz6d3jpLPjmQRj/pyY5TFR4KJce1YsL\nx/bgtXkbePyr1Vz2YgahIcagLm0ZnZLImF6JjE5JJLFNRJPUICIideuIFobXEW0CXuDOBy50zi2t\ntk6qc+4n3/PTgDucc+lmNgj4N3s7on0OpKojmp+9fTksfReunAVJ/Zv8cKUVlcxbm8P8tTnMXZvD\ndxtzKa2oAiC1YyxjeiXu+ekcH93k9YiItHZ+vU/bzE4GHgJCgWedc38xs7uBDOfcNDN7GJgIlAM7\ngWt2h7qZ3QJcClQA1zvnPj7YsRTaDVCQBY+mQ8c0mPKh11GtGZVWVLI4M49563KYtzaHjHU795w6\n754YzeiURMb6WuK9OrTRjGMiIvvQ4CrBZuGLMO1aOP0RGPmrgJZSWeVYvmUX89Z6IT5/XQ7ZhWUA\ndIiN9AV4O8b0ak//TnGEhijERSS4KbSDjXPw/Cmwbal373Zsy+nQ55xjdVYh830t8Xlrc9iUWwxA\nXFTYnmviY3olMrxbAiEKcREJMgrtYJS1Eh4/AgadBZOfCnQ1B5W5s8gX4juZtzab1VmFAHRNiObM\nEV04e2Q3+iTFBrhKEZHmodAOVl/+H3x1H1z8LvQ5LtDV1NmOglJmrdrBu4s28fXKLKocDO+ewOSR\nXTl1aBfaqVe6iBzCFNrBqrwEnjgSqirgytkQERPoiupt+64S3v9uM28vzGTF1nzCQ43jBnRk8shu\njOvfkYgwzSgrIocWhXYwW/s1vHAaHHWDN7lIK7Zs8y7eWZjJe99tZkdBKe1iwjl9mHf6fGi3+Np7\noldVAtbsPepFROpDoR3s3rsKfngdfvM1JA+q+3bOQfFObwrQ/C0HeNwKMYne+OfdRjXd71BNRWUV\nM3/awdsLM/l02TbKKqro2zGWs0d25czhXemSUMP94Kumw/vXQNdRcO5LCm4RabEU2sGuKMe7dzux\nD1z6iTfvdml+LWHse6ws3X9/UQkQ19mbGjSuE6yd6a1/7E1w9I0QGt5sv1pecTkfL97COws3MW9d\nDmZwRJ/2nD2iGycO7kSbkHL47A6Y9y+I7QQFW2HCHXD0Dc1Wo4hIfSi0Bb57Fd6bCm27ea3n8sL9\n14mI2xvMS6nlAAAcbUlEQVTEe0J538dO3pCp1RXnwsc3ea35rqPgrCehQ9/m+b2q2ZBdxLuLNvHO\nokzWZxcxKmIDj0U9Tqey9VSNvZKQiXfA+1d7I8b9ahr0OrrZaxQRqY1CW7xT3Z/eCrs2HyCQkyEy\nrnHHWPIOfPC/UFEKJ/wZ0v/Ha9U3M1dZwaaP7qfTwr+R7eK4sWwqq+NGc+aIrpwzOJ4+754GJXkw\ndab3+4uItCAKbWk+uzZ7rdnVX0DfSXDGo80bjLkb4N2psH4WpJ1ByYkPMn1dGe8s3MRXK7OorHKc\n0imXh3bdAF1HEj7lPxBal7lyRESah0JbmpdzMP9pr2UfHgOnPQRpZzT9MX94Az76nff85P8Hw87/\nWUs/K7+Uad9v5p2FmfTb+iF/j3icjxIuIGTinYwfkERkWGjT1igiUgcKbQmMrJXw7hWweREMuwBO\nug+i4v1/nOKd8MENsPQd6HE4nPUvaNfzoJus2LqL4revYUTW+1xa9jsWRo3ltKFdOHtkV4Z3T9BE\nJiISMAptCZzKcvjqfpj5gNcJ7qzHIeUo/+1/zVfw3pVQsM2bQ/zI6yGkji3m8hLcM5OoyFnPX7o9\nwasrjdKKKnontWHyyG6cOaIrXWu6fUxEpAkptCXwNs73Wt05a+GIa+G4WyEssuH7Ky+BL+6B2Y9C\nh35w9pPQZUT995OzBv41Dtr3Jv/CD/h4+U7eWpjJvLU5ABzeuz1nj+zKSUM6Exupa98i0vQU2tIy\nlBZ417kXPAcdB3lB22lw/fezbSm8fTlsXwqjL4dJdzduiNblH8DrF3n7OuUBADbm+G4fW5jJuuwi\nosNDOXFwJ84e2ZUj+nTQFKIi0mQU2tKyrPzEG52sJBeOuw0Ov7pup7SrqmDOP+Hzu7wBXs54DPod\n75+aPrnFa7VPfgaGnLNnsXOOhRtyeXthJh98v5ldJRUkt43kzBFdmTyyG/2SG3mbnIjIPhTa0vIU\n7oD/XAcrPoCeR3nXuhN6HHj9vE3e4DBrv4YBp8JpD0ObDv6rp7Icnj8Vti6GK76EpP77rVJSXskX\nK7bz9oJMZvhuHxvSNZ7j05JJioskISac+OgI4qPDfc/DiYkIVac2EakXhba0TM7Bd/+Gj2/2bs06\n6f79btMCYMnb3qAtlRVw0r0w4uKmGbRl12Z44mjvj4HLv4CINgdcdUdBKdN8s48t3bzrgOuFhxrx\n0eF7fhJiIkiIDqdttWDf/RgfHVHteTjhoRofXSQYKbSlZdu5zhsQZcNsGHi6N/lIm/beqGUf/d4b\nHrXbaO8aeGLvpq1l9Zfw0lkw9Fzv1rE6/HFQUFpBXnE5uUVl5BWXk1dU7r0u9j0WlbOruJzc4rI9\nr/OKyskvrTjofsNCjMiwECLDQ73HsBAiw0KJDK/2PCzE97raOgdZv3+nOAZ2buuvT0tEmkBdQ1td\nYyUw2qXAlA/h23/AF3+BjXPhqP+F2Y95rd9xf/JNRNIMX9E+42HcH2HG/3n3fKf/utZNYiPDiI0M\nq/ftYRWVVewq8QL/Z6HvC/6SikpKy6soraiitKLSeyzf+7y4vJLc4rIa1ympqORAf4MP757ARWN7\ncOrQLkRHaEAZkdZKLW0JvC0/wDtXQNZyr1V99lPQrdY/OP2rqgpeOQfWfQP/8yl0Gd68x/cD5xwV\nVc4X4ntD/qsfs3hl7npWZxXSNiqMyaO6cdHYHvTtqA51Ii2FTo9L61JeAis/9sYuj4wNTA2F2fCv\noyEkzJuHPDohMHU0Aeccc9fm8MrcDfx3yRbKKx1jeyVy0WE9OWFQsoZzFQkwhbZIQ2ycB8+dBKkn\nwPmvBGTGsqa2o6CUNzMy+fe89WzMKaZ9mwh+kd6dC8f0oEf7Rtz7LiINptAWaajZ/4RP/giT7oEj\nfxvoappMVZVj5qod/HvueqYv305llePo1A5cNLYnEwd2JEw92UWajUJbpKGcgzd+BSs+9DrL9Tw8\n0BU1ua15Jbw+fyOvzd/AlrwSkttGct7oHpw/ujtdNBa7SJNTaIs0RkkePDkOyovhNzMhNqlpj5ez\nFlZNh9AIby7y2I4Q2wnaJDXr3N8VlVV86eu49tXKLAw4bkAyFx3Wg2NSkzSUq0gTUWiLNNbWxfD0\nROg+Fi5+t+4zidXVrs2w9F1vIJlNCw6wknkDv8T6gjyuE8Qmez9xyXufxyb7vQPfxpwiXpu/gdfn\nb2RHQRnd2kVzwZgenJvenaS4Rkz8IiL7UWiL+MPCl2DaNXDMTXDcLY3fX+EOWPa+F9TrvwUcdB4G\ngyd7g8yEhHlTjuZv9R53/+Rvg4KtULDde11VwyAtEbE1B3rPI6HH2AaXXFZRxWfLtvHK3PV8uzqb\nsBDjmH5JJLeNom10GPHR4bSN8kZ88577lvmWR4Tp2rhIbRTaIv7y3tXw3Stw0VuQOrH+2xfnetfH\nl7wNa2aAq4QO/b1JSgadDR361m9/VVVQvNML8fzdQe57rB72+dugLN/bpvd4b2rURt7/vjqrgFfn\nbmDGyqw9o76VVVYddJvo8NCfhfveQK8W7r73kttG0iUhmqTYSEIaeyo+60f49DbvDoDJzwTuVkKR\nOlBoi/hLWZF3mjx/C0ydCfHd6rBNIaz8Lyx+G1Z9BpVlkNDTa1EPOQc6pjXP7WQlu2Dhi/DNg1CU\nDf1OhPF/8lr3/jpEeSW7fKO67SopZ1dxxZ7neUW+x+J9lhd7gZ9fWlHjKG5hIUZy2yi6JETRJSGa\nzvHRdEmIonN8NJ3jvWXtYsJrnpilZBd8dR/MfQLCY7z/Ft3S4aI3ISreb7+3iD8ptEX8accqr2Na\nxwEw5SMIi9h/nYpSrzPZkrfhx4+hvAjiOnut6cGToevIwN33XVoA8/4Fs/7hTY868DRvqNjktMDU\n41NV5cgvrdgT+lvzStiSV8zmvBK25Poe84rZmldCeeXP/62KCg+hS3w0nX1h3qVtBEcUfsbIlQ8T\nXpJN+bBfEnH8nd4od2//D3QaAr98B2ISA/PLihyEQlvE35a+B29eAmOv9GYeA28WsrUzYMk7sPwD\nKM2DmPaQdoYX1D2OgJAWdE23JM+7D33OP6E036tx3B/rf4q+mVVVOXYUlrI5t1qY5xazJa+EzXnF\nJOQs5rdlTzEiZBULq/pyR/kUFrvexEWF0ScplgsTlnDO6luoat+PsCnT/DvFq4gfKLRFmsLHf4C5\nj8PEOyF3Iyx7zzvtHNnWa70OPht6HQuh4YGu9OCKcuDbR7xTyBUlMOwCOPYmbyKX1qQgCz6/Cxa9\njIvtyM4jbmF151PYnFfKFl+wL9+Sz3eZuRxWtYgnwx9ka2gnXun/CANTUxmdkki3dtGa/1wCTqEt\n0hQqyuD5kyFzPoRFQ/+TvNZq34kQHhXo6uqvIAtmPQTzn/Z6pI/4JRzz+7pdtw+kynKv5i//CuWF\ncNiVXg//qJqnIC2tqGRxZh6Ziz7hpB+uZ4trx/klf2Ir7enUNor0lHaM6ZVIes9E+neKa9b70Ssq\nq9ieX8q2XSV0axej2+mClEJbpKkUZsPGOV6L+lDpkbxrC8z8Gyx43rvuPmqKNzVqXKdAV7a/NV/B\nxzd7s8L1mQAn3gtJ/eq+/YY5uJfPoTyyHR+MeIIvt8Uwf20OW3eVABAXFcaonu0YnZLI6JREhnaL\nJyq8YffoO+fYUVDmXafP9a7Pb8krYbPv1P6W3GK25ZdSWbX33+G+HWM5rHcih/fuwNjeiXSIVYgH\nA4W2iNRf7kb4+v/Bope9U/yjL/PmOW8J14BzN8Ant8Dyad5p/BP+6p3paMip7cwF8PJZEBEHl0zD\nJfYmc2cxGetzmLd2JxnrcvhpewEAEaEhDO0Wz+heiYxOaceononER4fjnGNXcQWb84p/Hsq5Jb5l\nJWzJLdnvlriIsBCvB7yvE93ux45xUazOKmDOmmzmr82hsKwSgNSOsRzWuz2H9W4fmBCvLPc6VZYX\nez3xy4u92xbbp0KEJpjxF7+GtpmdCDwMhAJPO+fu3ef9G4DLgAogC7jUObfe914lsNi36gbn3OkH\nO5ZCW6QFyFkDX90PP7zuXQY4bCocfk1gel6XF8Osh+GbvwMGx9wIh1/b+MsRW76HF8/0ho69ZBok\n9f/Z2zmFZWSsyyFj/U7mrc1hyaY8KqocZtA1IZqdhWV7gnW30BCjU9soOsdH0Tkhmi7x1Z974dy+\nTUTN19ALtnuD77hKKioqWZ2Vz5KNOSzdtJMft+RRVl5BCI7u7SIZ1DmWQZ1j6d+xDfFRoeCqoKrS\ne3RVXqg65z2vLPM+w32Dd8+yomrvVXteXuxdeqhpIB8AC4WOA727IrqM9B47prX8/hwtlN9C28xC\ngZXAJCATmA9c4JxbVm2d8cBc51yRmV0JjHPOned7r8A5V+dziAptkRYkayXM+CssfcfrbHf41d71\n4+a439k5WP4fr3Wdt8G7de74e/x7vX3bMnjxDC/cfvU+dBp8wFWLyypZtHEnGet28tP2AjrERtAl\nPtq7j9zXYk6Ki6z/9fDCbPjqXsh49sAB6U9hURAeDeFtfI/R3v3sETHeY3g0LiyaUoukiEgKqyIo\nqAxnV2U4eRXh7CwPo6yikmHhG+lTvpKYrO+xkty9++40xBfio7wgT+zTsu6gaKH8GdqHA3c6507w\nvf4jgHPurwdYfwTwqHPuSN9rhbZIa7dtKXz5f7DiA4hK8E6bJ/X3DZnayfuJbOu/+9C3r4D/3uyN\nINdxEJx0H/Q62j/73teOn+CF06Gi2BtjvsuIpjnOvspLvHvnv/6bN3LdyF95I9dZiPcTErr3efWf\nkFAqHKzZUcLizfks3pzPki0FFJc7qjC6t49lSLd2DO2eyNAeibSLjaY8NJqdFWHklIaSU1hBdmEZ\nOYVlvsdS73mBtyynsIydRWVUHSAa4iLDCA8LIaewDIAObSI4tXsJE+I3MYTVxO9cjG353muxA0TG\nQ5dhPw/ytl1b5Vz1zjl2FpWzJa+Ybu1iiI/231kFf4b2OcCJzrnLfK8vBsY65645wPqPAludc3/2\nva4AvsM7dX6vc+69Gra5ArgCoEePHqPWr19fW90iEgibF3nh/dOn+78XFu0b87xTzY9xnb3nMYkH\n/ge7JA9m3OeFWUQbGH8rpF/a9DOd5az1grskD375FnQf03THcs4bgOfzu7zr9KnHw6S7vVPNDVRe\nWcXiTXnMWZPNnDU5ZKzLoch36j4uKoz8kppb8GaQEB1OYpsI2reJJLFNBImxEbRvE0G7mAjax0Z4\ny3zvt2sTTmRYKM451mcXMWdNNrPXZDN7dTbb80sB6BgXyRG94jm+Yx6jI9bRIW8ptnmh94dfVbl3\n4DYdq51W9wV5gAe9cc6RW1TOZt9gPpvzStjq66OwJW9vJ8LSCq+PwlO/SmdSWrLfjh+Q0DazXwLX\nAMc650p9y7o65zaZWW/gC2CCc271gY6nlrZIK1CyyzfO+da9k5nkb9079vnux9Jd+28bEl5tUpNq\nwR4aBnMe967rjroEjrsd2rRvvt8pdyO8cBoUZsGFb0DKkf4/xoY53un+TRmQPBiO/zP0Ge/3w+wO\n8dmrs9m+q4TENpEktgn3Pe4N44TocMJCG3/q2jnH2h2FzFmTw+w12cxZk02WL8Q7tY3isN6JHJkS\ny1Fx2+hUsMwL8U0LYcdKwJdBCT29z2LQWdDzKL/+oeacI6+4nM25JWzd5XUa3OobmMcbhc8L5ZLy\nn3ca3N1HodPuvgnxe4fSHZXSjo5x/rvNs9lPj5vZROARvMDefoB9PQ984Jx760DHU2iLHELKivYJ\n9m3eGO7Vgz1/KxTneOt3Hwsn3Q9dhgem3l1b4MXTvQC/4FX/BWrOGvjsDq/ne2wnmHCbN6CNv6d7\nbSGcc6zOKtzTEp+7JpsdBd7p9C7xUV5v+D7tOaJrBN1KVsLmhd7YB6u/hLICiOkAaad7/Rh6HrHf\n51RSXsnOojJ2FpaTW1xGblE5uUXl7CwqI6+4nJ2FZeQWl5Nb5J3235JXQnH5/p0Gk+Mi6ZwQTaf4\nKLrER9EpPtr36I1v3yG2AX0UGsifoR2G1xFtArAJryPahc65pdXWGQG8hdci/6na8nZAkXOu1Mw6\nALOBM6p3YtuXQlskCFWUeTOXxXYM/LXOgu1er/LsVXDey9Dv+IbvqygHvn4A5j3p9ao+8jo44lrv\n1H8Qcc6xanvBnhCfsyZnzzXxrgnRHN7Hu6UtPryCmHVfkJz5MT12fE1EVQl5oe2YE3kUn4Ycybel\nfckprthziromUeEhtIuJID46nHYx3hmF3T34q7eWG9RpsAn5+5avk4GH8G75etY59xczuxvIcM5N\nM7PpwBBgi2+TDc65083sCOBfQBUQAjzknHvmYMdSaItIwBXlwEtner3Lf/GcN0RtfVSUeSO2fXWf\nd518xEXe9fm2nZum3lamqsrx0+4QX53N3LXZ7Cwq/9k6bUNLOTlqCSfbbMZWLCCSUvLCOrCy/XFs\n7HIipZ1GkRATSUJMBO3ahJMQHUFCTHiDB8IJNA2uIiLSGMW58PJkr/Pd5Ke84Wpr45x3CvyzO2Dn\nWq83+PF/PuitZOKF+KqsAsoqqkiI8VrIMRGhe+9nLy3wprpd+i789BlUlno90NPO9K6Bd0sP/Bma\nRlJoi4g0Vmk+vHKuN2ztGY/B8AsPvG7mAvj0FtgwG5IGemHdd0KrD5MWp2TX3gBfNd0bPCa+Owzy\nBXiXAE6B2wgKbRERfygrhFcvgLVfw6l/h/Rf//z9neu927eWvA1tkmD8LTDi4qa/TU28Sw8rPvIC\nfPUX3i1lCT298B50FnQe1moCXKEtIuIv5cXw+sWw6jOvd/vY33iBMfNvMOcJLxgOvwaOuh4i4wJd\nbXAq3gkrPvQCfM0Mb3S5xN7eZY2Rv4KEHoGu8KAU2iIi/lRRCm/+Gn780Ltd66dPvbnUh10Ax90G\n8V0DXaHsVpTjDYG79B3vDAl4g9ik/493yaIF3mqn0BYR8bfKcnjncq81l3K0d906UPeUS93kbvSm\nnF34IhRu91rco6bAiF9BbFKgq9tDoS0i0hSqKr3xypP6t5rrpYJ3G96KD7yJWdbN9EbmSzvda333\nPCLg/y0V2iIiIjXJWumF9/f/9vomJA3wxrgfdn7zzGBXA4W2iIjIwZQVede95z/jDaUaHgNDzvFa\n38182UOhLSIiUlebF3nhvfgtb5rWrqO88B50ljfXeBNTaIuIiNRXcS788LoX4Dt+9E6XD7/IO33e\nIbXJDqvQFhERaSjnYP0sL7yX/8cbuKXXMV7re8Ap3gQwflTX0NaQPSIiIvsyg5SjvJ+C7d4tYwte\ngDcv8eaDP/Of0Hdis5fV+NnPRUREDmWxHeGY38F138GFb0Dn4dCuV0BKUUtbRESkLkJCod8J3k+g\nSgjYkUVERKReFNoiIiKthEJbRESklVBoi4iItBIKbRERkVZCoS0iItJKKLRFRERaCYW2iIhIK9Hi\nxh43syxgfaDrOMR0AHYEuohDjD7TpqHP1f/0mTYNf3+uPZ1zSbWt1OJCW/zPzDLqMhC91J0+06ah\nz9X/9Jk2jUB9rjo9LiIi0kootEVERFoJhXZweDLQBRyC9Jk2DX2u/qfPtGkE5HPVNW0REZFWQi1t\nERGRVkKhfQgxs+5m9qWZLTOzpWZ2nW95opl9ZmY/+R7bBbrW1sbMQs1skZl94Hvdy8zmmtkqM3vd\nzCICXWNrY2YJZvaWma0ws+Vmdri+q41jZv/r+39/iZm9amZR+q7Wn5k9a2bbzWxJtWU1fjfN8w/f\n5/uDmY1sytoU2oeWCuBG51wacBhwtZmlAX8APnfOpQKf+15L/VwHLK/2+j7g7865vsBO4H8CUlXr\n9jDwX+fcAGAY3uer72oDmVlX4LdAunNuMBAKnI++qw3xPHDiPssO9N08CUj1/VwBPN6UhSm0DyHO\nuS3OuYW+5/l4/wh2Bc4AXvCt9gJwZmAqbJ3MrBtwCvC077UBxwFv+VbRZ1pPZhYPHAM8A+CcK3PO\n5aLvamOFAdFmFgbEAFvQd7XenHNfAzn7LD7Qd/MM4EXnmQMkmFnnpqpNoX2IMrMUYAQwF0h2zm3x\nvbUVSA5QWa3VQ8BNQJXvdXsg1zlX4XudiffHkdRdLyALeM532eFpM2uDvqsN5pzbBDwAbMAL6zxg\nAfqu+suBvptdgY3V1mvSz1ihfQgys1jgbeB659yu6u8573YB3TJQR2Z2KrDdObcg0LUcYsKAkcDj\nzrkRQCH7nArXd7V+fNdYz8D7g6gL0Ib9T/GKHwTyu6nQPsSYWTheYL/inHvHt3jb7tM1vsftgaqv\nFToSON3M1gGv4Z1qfBjvFFiYb51uwKbAlNdqZQKZzrm5vtdv4YW4vqsNNxFY65zLcs6VA+/gfX/1\nXfWPA303NwHdq63XpJ+xQvsQ4rvW+gyw3Dn3YLW3pgGX+J5fArzf3LW1Vs65PzrnujnnUvA69Xzh\nnLsI+BI4x7eaPtN6cs5tBTaaWX/fognAMvRdbYwNwGFmFuP7t2D3Z6rvqn8c6Ls5DfiVrxf5YUBe\ntdPofqfBVQ4hZnYUMBNYzN7rr3/Cu679BtADbwa1c51z+3aykFqY2Tjgd865U82sN17LOxFYBPzS\nOVcayPpaGzMbjte5LwJYA/waryGh72oDmdldwHl4d5IsAi7Du76q72o9mNmrwDi8mby2AXcA71HD\nd9P3B9KjeJciioBfO+cymqw2hbaIiEjroNPjIiIirYRCW0REpJVQaIuIiLQSCm0REZFWQqEtIiLS\nSii0RQLIzCrN7LtqP36bIMPMUqrPUtTczGzc7lnRRMQ/wmpfRUSaULFzbnigi2iJzCzUOVcZ6DpE\nWhK1tEVaIDNbZ2b3m9liM5tnZn19y1PM7AvfvL2fm1kP3/JkM3vXzL73/Rzh21WomT3lm2P5UzOL\nruFYz/vmA/7WzNaY2Tm+5T9rKZvZo2Y2pVp9f/WdHcgws5Fm9omZrTazqdV239bMPjSzH83sCTML\n8W1/vJnNNrOFZvamb7z83fu9z8wWAr/w/ycr0roptEUCK3qf0+PnVXsvzzk3BG+0pYd8yx4BXnDO\nDQVeAf7hW/4P4Cvn3DC8MbyX+panAo855wYBucDkA9TRGTgKOBW4t461b/CdJZiJN//wOXjzuN9V\nbZ0xwLVAGtAHONvMOgC3AhOdcyOBDOCGattkO+dGOudeq2MdIkFDp8dFAutgp8dfrfb4d9/zw4Gz\nfc9fAu73PT8O+BWA75Rynm/Wp7XOue986ywAUg5wrPecc1XAMjOr63SY03yPi4FY3xzu+WZWamYJ\nvvfmOefWwJ6hIY8CSvBCfJY3AiQRwOxq+329jscXCToKbZGWyx3geX1UH2O6Etjv9HgN65nvsYKf\nn42LOsA2VftsX8Xef1v2rdv59v+Zc+6CA9RSeIDlIkFPp8dFWq7zqj3ubol+izfbGMBFeKemAT4H\nrgSvA5eZxfvh+OuBNDOL9LWcJzRgH2PMrJfvWvZ5wDfAHODIatfp25hZPz/UK3LIU0tbJLCizey7\naq//65zbfdtXOzP7Aa8Vu7tVei3wnJn9HsjCmxkL4DrgSTP7H7wW9ZVAo6YHdM5tNLM3gCXAWrwZ\nouprPt41+b54U0S+65yr8nVoe9XMIn3r3QqsbEy9IsFAs3yJtEBmtg5Id87tCHQtItJy6PS4iIhI\nK6GWtoiISCuhlraIiEgrodAWERFpJRTaIiIirYRCW0REpJVQaIuIiLQSCm0REZFW4v8DIK9cRBw5\nvXoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f214d28e080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe0AAAEKCAYAAAA/9Q3BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4lNXd//H3N/sespCwhEDYd0EioLhQEAVUsOKGdcFa\nqNa1tYvVtlrr82h/tYt93IrWWq0b0rpU2TdxZZU1hBAQSIDsZIOsM+f3x5lAiEAmySSTSb6v65pr\nZu65lzPDkM/c55z7HDHGoJRSSqn2z8/bBVBKKaWUezS0lVJKKR+hoa2UUkr5CA1tpZRSykdoaCul\nlFI+QkNbKaWU8hEa2koppZSP0NBWSimlfISGtlJKKeUjArxdgIbi4+NNnz59vF0MpZRSqs1s2rSp\nwBjTtbH12l1o9+nTh40bN3q7GEoppVSbEZED7qyn1eNKKaWUj9DQVkoppXyEhrZSSinlI9pdm/bp\n1NTUkJ2dTWVlpbeL0iGEhISQlJREYGCgt4uilFKqCXwitLOzs4mMjKRPnz6IiLeL49OMMRQWFpKd\nnU1KSoq3i6OUUqoJfKJ6vLKykri4OA1sDxAR4uLitNZCKaV8kE+ENqCB7UH6WSqllG9yK7RFZKqI\n7BaRTBF56DSv9xaRlSKyTUTWiEhSveWbRWSLiOwUkTs9/QaUUkqpVmcMlB6G9I9h1RNQ9I1XitFo\naIuIP/AcMA0YCswWkaENVnsaeM0YMxJ4HHjStfwIcL4xZhQwDnhIRHp4qvC+xhjDpEmTKC0tpbi4\nmOeff75Z+5k+fTrFxcVnXeenP/0pq1atatb+lVKq0yvPh4xlsOb38OYN8MdB8Kch8PZN8OmfIGe7\nV4rlTke0sUCmMWYfgIi8DcwE0uqtMxT4ievxauB9AGNMdb11gvGh6vjWsGjRIs455xyioqLYv38/\nzz//PD/60Y++tV5tbS0BAWf+p1m0aFGjx7r33nuZO3cukyZNalGZlVKqwzteBEe2wKHNcPhrOLwF\nSrNdLwp0HQT9JkGP0faWOByCwrxSVHdCuyeQVe95Nvasub6twDXAM8B3gUgRiTPGFIpIL+BjoD/w\nM2PM4ZYX2zuuvvpqsrKyqKys5P7772fevHksWbKEhx9+GIfDQXx8PCtXrqS8vJx7772XjRs3IiI8\n+uijzJo1izfeeIN58+YB8NBDD7F3715GjRrFlClTuOKKK/j1r39NTEwM6enpZGRknPZ4cHKo1/Ly\ncqZNm8aFF17IF198Qc+ePfnggw8IDQ2ld+/eFBYWkpOTQ7du3bz5sSmlVPtRWQJHtrrC2XU7uv/k\n67H9IHn8yYDuPhKCI71W3IY8dcnXT4FnRWQOsBY4BDgAjDFZwEhXtfj7IrLQGJNbf2MRmQfMA0hO\nTj7rgX77352kHS71ULGtoT2iePSqYY2u98orrxAbG0tFRQXnnXceM2fOZO7cuaxdu5aUlBSKiooA\n+N3vfkd0dDTbt9vqk6NHjwLw+eef87e//Q2Ap556ih07drBlyxYA1qxZw+bNm9mxY8eJS7EaHm/W\nrFnExcWdUqY9e/bw1ltv8dJLL3H99dfz73//m5tvvhmAc889l88//5xZs2Z54FNSSikfU1tV7+zZ\ndSvcc/L1Lsk2mMfMcQX0KAjt4rXiusOd0D4E9Kr3PMm17ATX2fM1ACISAcwyxhQ3XEdEdgAXAQsb\nvDYfmA+Qmppqmvge2sxf//pX3nvvPQCysrKYP38+F1988YmQjY2NBWDFihW8/fbbJ7aLiYkBoKio\niMjIM/9iGzt27CnXTjc83p49e74V2ikpKYwaNQqAMWPGsH///hOvJSQkcPiwz1ZsKNX+GAObX4ON\nr9gq017j7FlZ1yHg16lb/9oPpxMOfAbb34W0D+yZNUBkDxvMI284eRYdHnf2fbVD7oT2BmCAiKRg\nw/pG4Kb6K4hIPFBkjHECvwRecS1PAgqNMRUiEgNcCPy5JQV254y4NaxZs4YVK1bw5ZdfEhYWxsSJ\nExk1ahTp6elu7yMgIACn04nfGf5zh4eHn/V4p7u2Ojg4+MRjf39/KioqTjyvrKwkNDTU7fIppc6i\nsgT++wDs/A8kDIW9q2HbO/a14GjodR70Gm9DvOcYr7V5tipjID/dtvl2PwcShkB7uITUGMjZBtsW\nwI7/QNlhCAyHIVfCkBmQlAqRHaOZsNHQNsbUisg9wFLAH3jFGLNTRB4HNhpjPgQmAk+KiMFWj9/t\n2nwI8EfXcgGeNsZ4p8tdC5WUlBATE0NYWBjp6el89dVXVFZWsnbtWr755psT1eOxsbFMmTKF5557\njr/85S+ArR6PiYlh0KBB7Nu3j/79+xMZGUlZWVmTjtdUGRkZXHfddc1+z0opl+xNsPB2KMmGyb+B\nCQ+A+MHRb+DgOsj6yt5nPmHX9wuAbiNtgNedjftqaFSVwb41sGc5ZK6s10ELiOgG/b5jO2n1nQgR\nCW1btqJ9sH2hPasuyLCfe/8pcNnvYND0DvnDya02bWPMImBRg2W/qfd4IQ2qvF3LlwMjW1jGdmHq\n1Km8+OKLDBkyhEGDBjF+/Hi6du3K/Pnzueaaa3A6nSQkJLB8+XJ+9atfcffddzN8+HD8/f159NFH\nueaaa7jiiitYs2YN/fv3Jy4ujgkTJjB8+HCmTZvGFVdc0ejxmqKmpobMzExSU1M9+TEo1bk4nfDl\n/8HKxyGyO9y+GJLr9cON7Wtvo2bb5xVHIWsDHPwSstbZavSvXJd2dul9aoi31yp1YyAvzRXSK+x7\ncdZCUCT0vQQu+ZmtSTj8NexdBRlLYOtbdtvEEa4Q/w4knw+BrVDTV55nz6a3vwuHNtplvSfA+Ltg\n6NUQFuv5Y7YjYkz7akJOTU01GzduPGXZrl27GDJkiJdK5DlHjhzh1ltvZfny5a1+rPfee4/Nmzfz\nu9/97rSvd5TPVKlWU54P7/0Q9q60Vawz/gqhMU3bR221rbY9+NXJs/Fjefa1kGhIGmt/BPTycpV6\nZak9m86sO5t2dVtKGAYDLrVnr73GQUDQt7d1Omxv7H2rbZPBwa/AWQMBITa4+02yIZ44vPlV6ZWl\nkP6RDep9a8A47Q+EEdfaW3RSc995uyEim4wxjZ5l+cSEIR1F9+7dmTt3LqWlpURFRbXqsWpra3nw\nwQdb9RhKdVh7V9vAriiGK/4Eqd9vXuAEBNn21KRU4B57FtuwSn2V60e8X4DtzRzdy96feOx6HtkD\n/D30J9sYyN1pQ3rPClsWZy0ER7nOpn8B/S+F6J6N78vPH3qea28XPQhV5XDgC1eIr4Llv4blQHiC\nrUKvq0qP6n72/dZW2TP9bQvs2Xxtpf0cLvwxjLjOtqd3Qnqm3UnpZ6rajfJ8yNkKBXvs2VzPc71X\nFkcNrP5f+OzPED8QrvsHJLZy59e6KvWsdVC0F4qzoCQLynNPXU/8IarHqWF+SsAnQUDw6Y8BtiNd\n/bbpMteVJYnDbUAPcJ1N+3t4yt6SQ/a4e1fZ++MFdnnCUOjrag/vfYGtZXA64cDnsH3ByZ7fYfEw\n/Bob1EnntY+Ob61Az7SVUu2L0wnF++HINjsEZM42+7g859T1eo2H8XfC4Ks8d2bpjqMH4N8/gOz1\ncO6tMPUpCApvfLuWCo2BgZfZW301lbbjW8lBG+TFB22YF2fB/s9s6BrnqdtEdKsX5q5Aryy1Z6xZ\n61xn09HQb6Kt8u4/2f4QaE3RPWH09+zN6YTc7bYmY+8q2PAyfPUc+AfZpoKiffZ9BUXA4CttUPe9\nxPM/JHyYhrZSyvNqq+2lQfXDOXcHVLkGRhJ/6DrYVpN2H2l7WsemwK7/wrq/wbtzICoJxv4Azr2t\n9TsXpX0AH9xrQ3DW3207qbcFhkB8f3s7HUeNncCixBXoxVknA/7w1/azdNbYdbuNgAvus2fTSed5\nLwT9/OylYt3PoXLcvRQUHaVi76f4f/MJUTlfURM+kJrzHiYh9WpCwtrPKGTtiVaPd1L6mSqPqSy1\n7aN14ZyzzQa2wzX1QGCYrYKtC+duI2zVaGDI6ffndEDGUlj3AnyzFgJC4ZwbYNxdkDDYs2WvqYCl\nD9te3j3OhWtfsT8eOgKn01ax+/m36aVYVbUO8suqyC2tIq+0kryyKnJLK+3zskrySqvILauk+HjN\nGffhJ5AcG8bAxEgGJkYyIDGCgYmR9O0aTnCAf5u9l7ak1eNKdXZOh62GLNrr+X1XlULODhvQRftO\nLg+Lt+Hc7y4b0N3PsZdE+TXhD62fPwyebm+5O2Hdi7D1bdj0qm0DHX+Xrdpt6eVSeen22uu8NHsW\nOunXp+8d7av8/Brv7NVArcNJtcNJVY2Tqlon1bVOqmodVNU6XTf7uLLaQX55/TC2AZ1bWsnR04Rx\ngJ+QEBlM16gQeseFMTYlloTIYBKjQkiIsvddI4M5eqya3bllZOSWsye3jIzcMlam5+Fw2pNLfz+h\nd1wYAxMiGZgYwQBXqKfEhxMU0A4vn2sFGtptyBjD5MmTef/995vVezwiIoLy8nIOHz7Mfffdx8KF\n37o0nokTJ/L000+TmprKpZdeyrvvvntiGFXVSRwvgq//ZdsLiw+03nG69LYBfc5N9uy5+0h7LbMn\nOwolDoMZ/weTH4NN/7Dv6c3r7aQO434Io25q+mQOdUORLv6FbbP+3r/tZU0dzJ7cMv677QjpR0pP\nCdzqegFc97juvi4c3eXvCuOEyGB6xYaR2ieGhMgQEqOCSYgKITHShnJsWBB+fo1/L+IjghmQeOq/\nZ1Wtg28Kjp0S5Bm5ZSxLy6GuuAF+Qp/4cAa6zsjtLYLeceEE+p8a5rUOJ5W1TiprHPVuzlPva+sv\nc/1QabDunAl9GNYjukmflydoaLeh+lNztkSPHj1OG9gN3XLLLTz//PM88sgjLTqe8hFHtsL6l+y1\nrLWVdsCJSx+DlEs83+PWPwiCIzy7z7MJj4OLfwoT7rftz+tehMU/h1VPwOhbYOxc96q16w9FmnIJ\nXDPfd0cqO439Bcf4aNthPtp2hPScMkRgQEIEoUEBBAf4EREcQHC4H0EBfgQH+BMcUPfYPj/52I8g\n1+vBgX4E+fsRHHhy/ZAAf7pGBhMX7l4Yt0RwgD+Du0UxuNupfzcraxzszS9nT265K8jL2Xm4lMU7\ncqhr9Q30FxIiQ6iqF8K1TfxhUl9IoB8hgf6EBPhz5Tmt3IHvDDS0m8DTU3P26tWLu++2I74+9thj\nREREcOeddzJz5kyOHj1KTU0NTzzxBDNnzjylHPv37+fKK69kx44dVFRUcPvtt7N161YGDx58ytjj\nM2bM4KKLLtLQ7shqq2HXh7B+vu0dHBgG59wI582FbsO9XTrP8w88OaBG9ibb7r3+b3bUsUHTbdV5\nnwtP/yMleyMs/P6pQ5E2pdq+nTpUXMHHrqDelm0nx0jtHcNjVw1l+ojuJESdoe+AjwsJ9GdYj+hv\nne1WVNswrwvyvLLKE0F7InTrhW/wiWX+hATUe9xgneAAP6QdXG7me6G9+CHbI9WTuo2AaU81upon\np+a84YYbeOCBB06E9oIFC1i6dCkhISG89957REVFUVBQwPjx45kxY8YZvywvvPACYWFh7Nq1i23b\ntnHuuSevcY2JiaGqqorCwsJvzQ6mfFzpYdvGu/EfdoSt2L5w+ZO2uridTy3oMUljIOllmPI4bPi7\nrT7f/bHt9DbuTnu5UGBI40OR+qC8skoWbTvCf7cdYdMB+/dlZFI0j0wfwvSR3enZpfNOFBQa5M/w\nntEM79n2VddtwfdC24s8OTXn6NGjycvL4/Dhw+Tn5xMTE0OvXr2oqanh4YcfZu3atfj5+XHo0CFy\nc3Pp1u30VXhr167lvvvuA2DkyJGMHHnqUO9103NqaHtQ3fXGUUlt23HJGDvS1Pr59nIe44QBl8HY\neXaAivY4jnVbiOoBk39tq8+3vwtfvQgf3gMrHoUxt7vGyG7BUKTtRNGxapbsyOG/Ww/z1TeFGAOD\nu0Xys8sHccWI7vSJb4NrypXX+V5ou3FG3BpaY2rO6667joULF5KTk8MNN9wAwBtvvEF+fj6bNm0i\nMDCQPn36nHZKTnfp9Jwe5nTCu7fa0PQLsKNmJQyFxKH2DC9hqB2ZypPVaNXH7FCO61+CvJ0Q0gXO\n/xGk3tFxLk/yhMBQOyjK6Ftg/6c2vD/9o21/b8lQpC4Op2HHoRI+yyxg15FS4iNsr+fu0SF0i7b3\niVEhhAR6rsq9tLKGZTtz+e/Ww3yWWYDDaegbH869kwZw1cju3+q0pTo+3wttL/H01Jxgq8jnzp1L\nQUEBn3zyyYnjJCQkEBgYyOrVqzlw4Oy9fy+++GLefPNNJk2axI4dO9i2bduJ14wx5OTk0KdPn9b5\nUDqjNf9rA3vcXbbqNTfNtiXvqNcxMDjajoucONT2fk4YZh+HNLG6rnCv7S399RtQVWKbcWb8Hwy/\ntkNOOegxIpBysb0VZ9kfV0289Ans/58Dhcf5LLOAzzML+GJvISUV9nKmpJhQSipqKKus/dZ2MWGB\ndIsOPRnmUSEkukLdLgslIvjMf3qPVdWyYlcuH207wie786l2OOnZJZS5F/XlqnO6M7R7VLtoW1Xe\noaHtJk9PzQkwbNgwysrK6NmzJ9272z8q3/ve97jqqqsYMWIEqampDB589sEk7rrrLm6//XaGDBnC\nkCFDGDNmzInXNm3axPjx4wkI0H9mj9i+ENb+wTXE5ZOnnrVVFEPeLnsmnJtmr/3dvtAO2lEnKsmG\neOLQk0EeN+DUKnanw44NvX6+rdL1C7DTDY6dB73Gdthxl1tNl15NWr2wvIrP9xby+Z4CPsss4FCx\n7djZIzqEy4clMqF/PBf0i6drpB3ju7yqlpySSnJKKjlSUkFuaSVHTjyvZEtWMUXHqr91nIjggBNn\n591cZ+ux4UFs2H+Ulem5VNY4SYwK5ubxvbnqnO6M6tVFg1oBOiJam2rLqTkB7r//fmbMmMHkyZO/\n9VpH+UzbzKFN8I/pdtSsWz9wry3bGNtTOS/NDhJSd1+QYceABvALtFXsiUNt2+zO9+211ZHdbXvs\nmNs61CVJ7U1FtYP1+4v4PLOAz/YUkHbEDrMaGRLABf3iuLB/PBP6x5MSH97s0KyscZBXWsWRkgpy\n6oV6TkklR0orySmpIK+sCmMgLjyIaSO6cdXIHpzXJ7bVL6dS7YeOiNYOteXUnADDhw8/bWCrJio9\nDG/dZIeCvOF19zufibgmbegFAy8/uby2Ggr3uM7Id9ogP/AllGbba6un/NZOlqCTJHicw2nYfqiE\nzzML+HRPPpsPFFPtcBLk78eY3jH87PJBTOgfz/AeUQT4e6ZjX0igP8lxYSTHnblJo8bhpLC8mviI\nII8dV3VMGtpt7Prrr2+zY82dO7fNjtVh1VTA2zdBdTncsgzC41u+z4AgVzX5MOC6k8trqzvWMJrt\ngDGGbwqO2TPpzAK+3FtIqasdemj3KOZM6MOE/vGM7RNLaJD3rtkO9PejW3THvJ5aeZbPhLYxRtt0\nPKS9NYm0W8bAB/fA4S1w45utP6+yBvYJxhiOVzsor6qlrLKW8qpayitrKa+qOfV5dd1ye19WVe+5\n63G1w05f2bNLKNNHdHe1S8cRF3GWuaeVaqd8IrRDQkJODBCiwd0yxhgKCwsJCdFf9Y369I+2V/jk\nR+3kFapF3Jn9qbSy5kQYu/PbMjjAj8iQACKCA4hw3ffoEkpEsL/reSBJMaFc2D+e3nFh+vdD+Ty3\nQltEpgLPAP7Ay8aYpxq83ht4BegKFAE3G2OyRWQU8AIQBTiA/zHGvNPUQiYlJZGdnU1+fn5TN1Wn\nERISQlJSkreL0b7t+i+s+h2MuB4u/LG3S9OuVdc6T8z4lOcK4ObM/pQcF0aX0EAiQgKIPBHCga57\nf/s4OOBESIcHB3SamZ2UqtNo73ER8QcygClANrABmG2MSau3zrvAR8aYf4rIJOB2Y8wtIjIQMMaY\nPSLSA9gEDDHGFJ/peKfrPa5Um8rZDn+/3M7dPGfRmed97mSMMezJK2d5Wi4b9heRU1JJflkVhae5\npMnfT+gaEXxytqeo4BbN/qRUR+fJ3uNjgUxjzD7Xjt8GZgJp9dYZCvzE9Xg18D6AMSajbgVjzGER\nycOejZ8xtJXyqvJ8eGu2HQjlxjc7fWDXOpxs2H+UFbtyWZ6Wy8Gi44AdPjMpJoxze8eQ6ArjujmR\nE6PsNcf+GsZKeZw7od0TyKr3PBtoONr+VuAabBX6d4FIEYkzxhTWrSAiY4EgYG+LSqw6JqcTstdD\nzzHeu9SptgreuRmOFcD3F3fa66PLq2pZm5HP8rRcVqXnUVJRQ5C/Hxf0j+OHl/Tl0iGJJHbQmaOU\nau881RHtp8CzIjIHWAscwrZhAyAi3YHXgduMMc6GG4vIPGAeQHJysoeKpHxGbTV88CM72UPiCJj5\nLPQY1bZlMAY++glkfQXX/gN6jG7b43tZTknlibPpL/cWUu1w0iUskMlDEpgyJJGLB3Yl/CxDbyql\n2oY7/wsPAfXHAkxyLTvBGHMYe6aNiEQAs+rarUUkCvgYeMQY89XpDmCMmQ/MB9um3cT3oHxZZSks\nuAX2rYExc2D3EnhpEky4Dy55qO2qp796Hrb8Cy75BQy/pm2O6UXGGNJzylielsuKXbkn5mHuHRfG\nref3ZsrQRMb0jtGBPpRqZ9wJ7Q3AABFJwYb1jcBN9VcQkXigyHUW/UtsT3JEJAh4D3jNGLMQpeor\ny4U3rrUjgs18HkZ/Dy49Cst+BZ/92fbgnvEs9D6/dcuxZ7k95pAZ9odCB1XjcLLhmyKWuYI6+2gF\nIjCqVxd+PnUQU4Yk0j8hQi+LUqodazS0jTG1InIPsBR7ydcrxpidIvI4sNEY8yEwEXhSRAy2evxu\n1+bXAxcDca6qc4A5xpgtnn0byucUZMK/roFj+XDTOzBgil0eGgMzn4Phs+C/98M/psHYufZa6eAI\nz5cjfzcs/L4dOOW7L7bpnNTGGA4VV7Alq5ivDxaz83AJTmOvPbY3f4ID/Aiqex7oT5B/3WM/+ziw\n/joN1g/wJyjAj11HSlmxK5fV6XmUVtYSHODHhf3juec7/Zk0JIGESG2fVspX+MSEIaqDyd4Ib7qG\nc73pXUgac/r1qsrttdLr/gbRveCqv0B/D46lfrzIVsVXH4O5q5o8I1RTHauqZVt2CV9nHWXLwWK+\nziomv6wKsEE9tEcUQf5+VNU6qa51UlXroNrhpKrGecoyZzP+y8aGBzF5cAKXDk3kogHxhAVp+7RS\n7YlOGKLap4xl8O5tEN4VbnkP4vqded3gCJj2exj2XTuc6L+ugVE3w+VP2DPylnDU2HKUHoI5H3s8\nsJ1OQ2Z+uSucj/L1wWIycstOBG7f+HAu6h/P6OQujOoVw+DukQS60X5sjKHWaVwBfjLIGz6uqq0L\newc9u4QyOjlGL8FSqgPQ0FZt5+t/wYf3Qbfh8L2FdtYsdySPhzs/g09+D58/A5nL4Yo/wZArm1+W\nJQ/BN2vh6hftPNUtVFhedaKae0tWMVuziimrshNTRIUEMCo5hsuHdWNUchdGJXUhJrx544yLCIH+\nQqC/H+E6dLZSnY6Gtmp9xsCnT8OqJ6Dvd+z0lsGRTdtHYAhc+igMnQkf3gPvfA+GXg3T/+B++NfZ\n8LK9XXAfjJrdtG05Ob3j1wePngjqukFH/P2Ewd0imTm6B6N6xTA6uQspceE66pdSyiO0TVu1LqcD\nFv/chuSI620ns5bOZuWosWfcn/wegsJh6u9h5PV2/urG7PsEXv8u9L8UZr8Ffu5Px3ikpIJ3NmTx\nzoYsjpRUAtAtKsRVxd2F0ckxjOgZ7dUpHpVSvsndNm0NbdV6airhPz+wl25dcB9c+lvP9s7O323b\nurPXw4DL4Mo/Q/RZJkIp3Gs7nkV2gzuWQ0hUo4dwOA1rM/J5Y91BVqXn4jRw0YB4rh2TxNiUWLpH\nh3ru/SilOi3tiKa8q+IovHUTHPwSpj4F4+/y/DG6DoLvL4H182Hl4/DceLjscTh3zrd/HFSW2DHF\nxQ9mv91oYOeWVrJgQxZvb8jiUHEF8RFB/PCSfsw+L5nkuDDPvxellHKDhrbyvJJs+Ne1ULQXrn2l\ndUcY8/O3PwgGTrXXdX/0Y9jxH7jqmZM9050OWHiHLc8t70Nsyml35XQa1u7J5811B1mZnofDabiw\nfzyPXDGES4ck6jSQSimv09BWnpW3C16/BqrL4eZ/Q8rFbXPc2BS49QPY/Jod3eyFCTDpERj/I1j+\nG9vj/Mo/Q8pF3y5yWSXvbszmrfUHyT5aQVx4ED+4KIXZ5yXTJz68bcqvlFJu0NDuyIyB1f8Ljiob\nnsnn245breXAF/DWjRAQCrcvgm4jWu9YpyMCY26zo6t9/KAN702vQmEmjJ0Hqd8/sarTafgss4A3\n1x1kxa5cap2G8/vG8Yupg7lsWCLBAdqZTCnV/mhHtI5sy1vw/p22Hdc4wS8QklJtgKdcYh8HeOhi\n37QP4d8/gC7JcMt/7L03GQM7/m17rncfBTctAP8A8suqeHdTFm+vz+Jg0XFiwgK5dkwSs8cm07dr\nKwyTqpRSbtDe451dWS48N9Z21rr535C13g4m8s1aOLLFhnhAqB24pC7Eu58D/s2ofFn/Eiz6GSSd\nZ8cRD4v1/PtprtpqnPjx5f5i3lx3kGVpOdQ4DONSYrlpXDKXD+tGSKCeVSulvEt7j3d2i38GNRV2\nlqzgSDtmd9243RXFtiq7LsRX/tYuD46C3hNcIX4xJAw9+yVaxtixwT/9IwyaDrP+DkHtp2d1eVUt\nb6/P5l9fHWB/4XG6hAVy6/l9mD02mf4JelatlPI9GtodUdqHkPYBTP4NdB347ddDu8Dg6fYGUJ4P\n+z89GeIZi+3ysDjoc9HJM/G4ficHMHHUwH8fsHNQn3ubHVa0OWfpraCgvIpXP9/Pa1/up7SyltTe\nMdx/6QCmDe+uZ9VKKZ/WPv7KKs85XmQ7YXUbYQc0cUdEV3tZVt2lWSXZ8E1diH8Cae/b5ZE9Tp6F\np70Pe5bBxF/CJb9wbzSyVnaw8DgvfbqPBRuzqHY4uXxoN+6c2I9Rvbp4u2hKKeURGtodzdJH4Hgh\n3LwQ/APy6lUHAAAgAElEQVSbt4/oJDsm96jZtgq8aN/Js/DMFbDtbdu57apnYMwcjxa/OdIOl/Li\nJ3v5aNth/P2Ea0YnMe+SvvTTjmVKqQ5GQ7sj2bMCtr4JFz1oO5V5goitFo/rB6m32xDPS7M90U9X\n9d5GjDGs+6aIF9bs5ZOMfMKD/PnBRX35/oQUukWHeK1cSinVmjS0O4qqMvjoAYgfCBf/vPWOIwKJ\nw1pv/41wOg3Ld+Xywpq9bMkqJi48iJ9dPoibx/UmOqyZNQtKKeUjNLQ7ihW/tW3Rdyyz01h2MNW1\nTt7fcoi/fbKXvfnH6BUbyu+uHs51Y5K0c5lSqtPQ0O4IDnwBG16CcXdBr7HeLo1H2cu2DvLyp9+Q\nU1rJkO5R/HX2aKYP70aAv44FrpTqXDS0fV1NhZ2esksyTP61t0vjMYXlVbz6xX5e+/IAJRU1jO8b\ny1OzRnDJwK5IO+iprpRS3uBWaIvIVOAZwB942RjzVIPXewOvAF2BIuBmY0y267UlwHjgM2PMlR4s\nuwJY8+TJ2atac1zxNpJVdJyXP93HOxuzqKxxcvmwRO68pB+jk2O8XTSllPK6RkNbRPyB54ApQDaw\nQUQ+NMak1VvtaeA1Y8w/RWQS8CRwi+u1PwBhwA89WnIFhzbDF/8Ho2+Bft/xdmkA21GsstZBRbWD\nihoHlTUOKmucVNQ0XFb33L5WVeMg6+hxlu7MxU/g6lE9+eElfemfEOntt6SUUu2GO2faY4FMY8w+\nABF5G5gJ1A/tocBPXI9XA+/XvWCMWSkiEz1SWnVSbbWtFo9IhMueaLPDllfV8vKn+/gkI5+K6gah\nXOOgutbZrP2GBPoRERzI7Rf04Y6LUugeHerhkiullO9zJ7R7Aln1nmcD4xqssxW4BluF/l0gUkTi\njDGFHiml+rbP/wJ5O+HGt+ywpK2sxuHkrfUH+evKPRSUV3Nenxh6xYYRGuhvb0H+BAf6nfI8JNDe\nTi7zIzjAvla3LCTQn+AAP/z8tJ1aKaUa46mOaD8FnhWROcBa4BDgcHdjEZkHzANITvbylI6+IG8X\nfPL/YPisk+OHtxJjDIu25/CHpensLzzOuJRYXr5tiA4NqpRSXuBOaB8CetV7nuRadoIx5jD2TBsR\niQBmGWOK3S2EMWY+MB/s1JzubtcpOR22WjwkCqb9v1Y91Ff7CnlycTpbs4oZmBjBK3NS+c6gBO29\nrZRSXuJOaG8ABohICjasbwRuqr+CiMQDRcYYJ/BLbE9y1Rq+egEObbTTYIbHt8ohdueU8fsl6axK\nz6N7dAj/79qRzDo3CX+twlZKKa9qNLSNMbUicg+wFHvJ1yvGmJ0i8jiw0RjzITAReFJEDLZ6/O66\n7UXkU2AwECEi2cAdxpilnn8rnUDhXlj1BAycZqvGPexISQV/WpbBvzdnEx4cwC+mDub2CX10xDGl\nlGonxJj2VRudmppqNm7c6O1itD9OJ7w2A45shbvXQVQPj+26pKKGF9bs5R+ff4MxcOv5vbn7O/2J\nCQ/y2DGUUkqdmYhsMsakNraejojmKzb/E/Z/aqfD9FBgV9U6eP3LAzy7OpOSihquHtWTn0wZSK/Y\nMI/sXymllGdpaPuCkkOw7NeQcjGce1uLd+d0Gj7Yeoinl2ZwqLiCiwbE89C0wQzrEe2BwiqllGot\nGtrtnTHw0Y/BOOCqv9qpMVtgbUY+Ty1OJ+1IKcN6RPH7WSO5cEDrdGhTSinlWRra7d32d2HPUrj8\nSYhNafZudhwq4anF6XyWWUBSTCjP3DiKq0b20EFNlFLKh2hot2fl+bD4F5A0FsY1b+j2rKLjPL1s\nNx9sOUyXsEB+feVQbh6fTHCA9ghXSilfo6Hdni3+GVSXw4z/A7+mh+wXmQXMfW0jtU7Djyb244eX\n9CM6NLAVCqqUUqotaGi3V7s+gp3vwXd+BQmDm7z5ou1HeODtLaTEh/P3OakkxWiPcKWU8nUa2u1R\nRTF8/CAkjoALH2jy5q9/dYDffLCDMckx/P2284gO07NrpZTqCDS026Nlj8CxfLjpHfB3P3CNMTyz\ncg9/WbGHyYMTePamcwkN0rZrpZTqKDS025u9q+Drf8GFP4Yeo9zezOE0PPbhTl7/6gCzzk3iqVkj\nCPT3a8WCKqWUamsa2u1JVTl8eD/EDYBLHnJ/s1oHP1mwlY+3HeGHF/floWmDdSYupZTqgDS025OV\nj0NJFnx/CQSGuLVJeVUtd76+ic8yC3h4+mDmXdyvlQuplFLKWzS024OqMvjiWVg/H8bOg+Txbm1W\nWF7F7a9uYOfhUp6+7hyuHZPUygVVSinlTRra3uSogU2vwie/tx3Phl4Nk3/j1qZZRce57ZX1HC6p\nYP4tY5g8JLF1y6qUUsrrNLS9wRhI+8BWhxfthd4TYPbbkNTorGwApOeUctsr66modvDGD8Yxpnds\nKxdYKaVUe6Ch3db2fw7LfwOHNkLXITD7HRh4udsTgWzcX8T3X91AaJA/7955AYO6RbZygZVSSrUX\nGtptJW8XrHgMMpZAZA+Y8SyMuqlJw5Ou3JXLj97YTM8uobx2x1gd5UwppToZDe3WVnII1vwvbHkT\ngiJg8qMw7k4Ialrgvrsxi4f+s51hPaL4x5zziIsIbqUCK6WUaq80tFtLZQl89mf46gUwThh3F1z8\nUwhrevvz3z7Zy5OL07mwfzwv3jKGiGD9Z1NKqc5I//p7Wm0VbHgZ1v4BKo7CiOth0q8gpneTd+V0\nGp5aks78tfu4cmR3/nj9OTqlplJKdWJujXMpIlNFZLeIZIrIt4bqEpHeIrJSRLaJyBoRSar32m0i\nssd1u82ThW9XnE7YtgCeTYWlD0P3UfDDtTDrpWYFdo3Dyc8WbmP+2n3cdn5v/nrjaA1spZTq5Bo9\n0xYRf+A5YAqQDWwQkQ+NMWn1VnsaeM0Y808RmQQ8CdwiIrHAo0AqYIBNrm2PevqNeNXeVbD8UcjZ\nBt1GwM3/gf6Tm727imoHd7+5mVXpefxkykDundRfhyVVSinlVvX4WCDTGLMPQETeBmYC9UN7KPAT\n1+PVwPuux5cDy40xRa5tlwNTgbdaXvR24MhWG9b7VkN0Mnx3Poy4DvyaP1FHyfEavv/PDWw+eJQn\nrh7OzeObfpaulFKqY3IntHsCWfWeZwPjGqyzFbgGeAb4LhApInFn2LZns0vbXpTl2Gutt70DoTFw\n2f/A2LkQ0LIe3Tklldz6yjr2Fxzn+ZvOZdqI7h4qsFJKqY7AUx3Rfgo8KyJzgLXAIcDh7sYiMg+Y\nB5CcnOyhIrWij34MmSthwgN2Cs3QLi3eZU5JJbNe+IKSihpe/f55XNAv3gMFVUop1ZG4U497COhV\n73mSa9kJxpjDxphrjDGjgUdcy4rd2da17nxjTKoxJrVr165NfAttrPq4bcNOvR2m/NYjgQ3w5OJd\nFJRX8dbc8RrYSimlTsud0N4ADBCRFBEJAm4EPqy/gojEi0jdvn4JvOJ6vBS4TERiRCQGuMy1zHft\nWwO1lTBwqsd2+fXBo3yw5TBzL+rLiKRoj+1XKaVUx9JoaBtjaoF7sGG7C1hgjNkpIo+LyAzXahOB\n3SKSASQC/+Patgj4HTb4NwCP13VK81kZiyE4yk7y4QHGGJ74eBfxEcHcOVHnwlZKKXVmbrVpG2MW\nAYsaLPtNvccLgYVn2PYVTp55+zanEzKWQr9JEBDkkV0u3pHDpgNHefKaETrSmVJKqbNq/rVJndGR\nr6E8FwZN88juqmodPLl4F4O7RXJ9aq/GN1BKKdWpaWg3xe4lIH4w4DKP7O6fX+wnq6iCR64Ygr+f\nDp6ilFLq7DS0myJjMfQa16xJPxoqLK/i/1Zm8p1BXbloQDvvMa+UUqpd0NB2V0k25Gz3WK/xZ1bu\n4XiNg4enD/HI/pRSSnV8Gtruylhi7z3Qnp2ZV8Yb6w5y09hkBiRGtnh/SimlOgcNbXftXgIxKRA/\nsMW7+t9F6YQF+vPApQM8UDCllFKdhYa2O6qPwTdr7Vl2C2fb+nRPPqvS87hnUn/iIlo2VrlSSqnO\nRUPbHXtXg6Oqxe3ZDqfhfz7eRa/YUG67oI9nyqaUUqrT0NB2R8ZiCI6G3he0aDfvbswiPaeMh6YO\nISTQ30OFU0op1VloaDfG6YSMZdB/MvgHNns35VW1PL0sgzG9Y5g+opsHC6iUUqqz0NBuzOHNcCyv\nxb3GX1yzl4LyKn51xRCkhe3iSimlOicN7cbsXgziD/0vbfYuDhdX8NKn+5hxTg9GJ8d4sHBKKaU6\nEw3txmQsgeTxLRoF7Q9Ld2OAn08d5LlyKaWU6nQ0tM+m+CDk7mhRr/GtWcW89/UhfnBhCkkxYR4s\nnFJKqc5GQ/tsMpba+2a2Z9u5stOIjwjiLp0rWymlVAtpaJ9NxhKI7QfxzRu5bMmOHDbsP8pPpgwi\nMqT5Pc+VUkop0NA+s6ryk6OgNWfzWgdPLk5nUGIk16cmebhwSimlOiMN7TPZtxoc1c1uz37tiwMc\nLDrOI1cMIcBfP2allFItp2lyJruXQEi07TneREXHqvnrqj1MHNSViwfqXNlKKaU8Q0P7dJxO2LMU\n+k9p1ihoz6zI4Hi1g0d0rmyllFIe5FZoi8hUEdktIpki8tBpXk8WkdUi8rWIbBOR6a7lQSLyDxHZ\nLiJbRWSih8vfOg5tgmP5zWrPzswr51/rDjJ7bC+dK1sppZRHNRraIuIPPAdMA4YCs0VkaIPVfgUs\nMMaMBm4EnnctnwtgjBkBTAH+KCLt/+w+o24UtMlN3vTJRbtcc2W3fN5tpZRSqj53AnQskGmM2WeM\nqQbeBmY2WMcAUa7H0cBh1+OhwCoAY0weUAyktrTQrW73Ekg+H0KbNuToZ3sKWJmex92T+hOvc2Ur\npZTyMHdCuyeQVe95tmtZfY8BN4tINrAIuNe1fCswQ0QCRCQFGAP0alGJW1vxQcjbCYOa1mvc4bQD\nqfTsEsocnStbKaVUK/BUVfVs4FVjTBIwHXjdVQ3+CjbkNwJ/Ab4AHA03FpF5IrJRRDbm5+d7qEjN\ntHuJvR/YtPbshZtcc2VPG6xzZSullGoV7oT2IU49O05yLavvDmABgDHmSyAEiDfG1BpjfmyMGWWM\nmQl0ATIaHsAYM98Yk2qMSe3a1cuXSGUshrj+EN/f7U2OuebKHp3chStHdm/FwimllOrM3AntDcAA\nEUkRkSBsR7MPG6xzEJgMICJDsKGdLyJhIhLuWj4FqDXGpHms9J5WVQb7P2vygCp/+2Qv+WVV/PrK\noTpXtlJKqVYT0NgKxphaEbkHWAr4A68YY3aKyOPARmPMh8CDwEsi8mNsp7Q5xhgjIgnAUhFxYs/O\nb2m1d+IJe1fZUdCacKnX4eIK5n+6j6vO6cG5Ole2UkqpVtRoaAMYYxZhO5jVX/abeo/TgAmn2W4/\n4DuTSO9eAiFdoJf7o6A9vXQ3TgM/v9x33qZSSinf1P6vmW4rTocdBW3AFPB367cM27KL+c/Xh7jj\nwhR6xepc2UoppVqXhnad7I1wvNDt9mxjDE98tIu48CB+pHNlK6WUagMa2nUyFoNfAPS/1K3Vl+7M\nYf3+In5y2UCdK1sppVSb0NCuc2IUtC6Nrlo3V/bAxAhuSG3fY8UopZTqODS0AY7uh/xdbvca/9dX\nBzlQeJxHrhiqc2UrpZRqM5o4UG8UNPfasxdsyGJM7xgu0bmylVJKtSENbbDt2fEDIa7xDmUHCo+x\nO7eMacO7tUHBlFJKqZM0tCtLYf/nbp9lL0/LBeCyoRraSiml2paG9t5V4Kxxuz17WVoug7tFkhyn\n12UrpZRqWxraGUvsvNlJYxtdtehYNRv3FzFlaGIbFEwppZQ6VecObacD9iyDAZe5NQrayl25OI1W\njSullPKOzh3a2RuaNAra8rRcukeHMLxnVCsXTCmllPq2zh3au+tGQZvc6KoV1Q7W7slnytBEnX5T\nKaWUV3Tu0M5YAr0nQEh0o6t+lllAZY1T27OVUkp5TecN7aJvID/d/V7jO3OIDAlgXEpcKxdMKaWU\nOr3OG9oZ7o+C5nAaVqbn8Z1BCQQFdN6PTCmllHd13gTavRi6DobYlEZX3XTgKEXHqrVqXCmllFd1\nztCuLIEDTRkFLYdAf2HiIB1rXCmllPd0ztDOXAnOWrdC2xjDsrRczu8Xr/NmK6WU8qrOGdoZSyA0\nFno1PgranrxyDhQe5zKtGldKKeVlboW2iEwVkd0ikikiD53m9WQRWS0iX4vINhGZ7loeKCL/FJHt\nIrJLRH7p6TfQZI7ak6Og+fk3unrdBCHanq2UUsrbGg1tEfEHngOmAUOB2SIytMFqvwIWGGNGAzcC\nz7uWXwcEG2NGAGOAH4pIH88UvZmy10PFURjkXnv2sp05nNOrC4lRIa1cMKWUUurs3DnTHgtkGmP2\nGWOqgbeBmQ3WMUDd2J7RwOF6y8NFJAAIBaqB0haXuiV2Lwa/QOjX+ChouaWVbM0u0apxpZRS7YI7\nod0TyKr3PNu1rL7HgJtFJBtYBNzrWr4QOAYcAQ4CTxtjilpS4BbLWAJ9JkBI4+OHa9W4Ukqp9sRT\nHdFmA68aY5KA6cDrIuKHPUt3AD2AFOBBEenbcGMRmSciG0VkY35+voeKdBqFe6EgAwa6P3d2n7gw\nBiREtF6ZlFJKKTe5E9qHgF71nie5ltV3B7AAwBjzJRACxAM3AUuMMTXGmDzgcyC14QGMMfONManG\nmNSuXVvxWui6UdDcaM8uq6zhy70FOkGIUkqpdsOd0N4ADBCRFBEJwnY0+7DBOgeByQAiMgQb2vmu\n5ZNcy8OB8UC6Z4reDLsXQ9chENOn0VXX7M6nxmG4bJjOna2UUqp9aDS0jTG1wD3AUmAXtpf4ThF5\nXERmuFZ7EJgrIluBt4A5xhiD7XUeISI7seH/D2PMttZ4I42qKIaDX7rda3x5Wi5x4UGcmxzTygVT\nSiml3BPgzkrGmEXYDmb1l/2m3uM0YMJptivHXvblfZkrXKOgNd6eXV3rZHV6HlOHd8PfT6vGlVJK\ntQ+dZ0S0jKUQFgdJ32pS/5Z13xRSVlWrVeNKKaXalc4R2idGQbvc7VHQQgL9uLB/fBsUTimllHJP\n5wjtrHVQWexWe7YxhuVpuVw8oCuhQY0HvFJKKdVWOkdoZywG/yDoN6nRVXccKuVISaUOqKKUUqrd\n6RyhvXsJ9LkQgiMbXXVZWg5+ApOHaGgrpZRqXzp+aBfuhcI9bo+Ctjwtl9Q+scSGB7VywZRSSqmm\n6fihvXuxvXejPftg4XHSc8p0ghCllFLtUscP7YwlkDAMuiQ3uuqytBxAJwhRSinVPnXs0K44Cge+\naNIoaIMSI+kdF97KBVNKKaWazq0R0XxWcDTMXWkHVWlE0bFqNuwv4u7v9G+DgimllFJN17FD288P\neox2a9VV6Xk4jVaNK6WUar86dvV4EyzbmUO3qBBG9Iz2dlGUUkqp09LQBiprHHy6R+fOVkop1b5p\naAOf7SmgosahVeNKKaXaNQ1t7KVekcEBjO/beIc1pZRSyls6fWg7nIaVu/KYODiBoIBO/3EopZRq\nxzp9Sm0+eJTCY9U6CppSSql2r9OH9vK0XAL9hYmDunq7KEoppdRZderQNsawbGcO5/eLJzIk0NvF\nUUoppc6qU4d2Zl45+wuPa69xpZRSPsGt0BaRqSKyW0QyReSh07yeLCKrReRrEdkmItNdy78nIlvq\n3ZwiMsrTb6K5lqXlAjBF585WSinlAxoNbRHxB54DpgFDgdkiMrTBar8CFhhjRgM3As8DGGPeMMaM\nMsaMAm4BvjHGbPHkG2iJZWm5nJMUTbfoEG8XRSmllGqUO2faY4FMY8w+Y0w18DYws8E6BohyPY4G\nDp9mP7Nd27YLuaWVbM0q1qpxpZRSPsOdCUN6Aln1nmcD4xqs8xiwTETuBcKBS0+znxv4dth7zXJX\n1fhlw7p5uSRKKaWUezzVEW028KoxJgmYDrwuIif2LSLjgOPGmB2n21hE5onIRhHZmJ+f76Eind3y\ntFx6x4UxICGiTY6nlFJKtZQ7oX0I6FXveZJrWX13AAsAjDFfAiFAfL3XbwTeOtMBjDHzjTGpxpjU\nrl1b/3rpssoavthbwGU6QYhSSikf4k5obwAGiEiKiARhA/jDBuscBCYDiMgQbGjnu577AdfTjtqz\nP8nIp8ZhmDJUq8aVUkr5jkZD2xhTC9wDLAV2YXuJ7xSRx0Vkhmu1B4G5IrIVe0Y9xxhjXK9dDGQZ\nY/Z5vvjNszwtl9jwIMb0jvF2UZRSSim3udMRDWPMImBRg2W/qfc4DZhwhm3XAOObX0TPqnE4WZWe\nx9Rh3fD306pxpZRSvqPTjYi2bl8RZZW1eqmXUkopn9PpQntZWg4hgX5cNEAnCFFKKeVbOlVoG2NY\nkZbLRQO6Ehrk7+3iKKWUUk3SqUJ75+FSDpdUatW4Ukopn9SpQnvZzhz8BCYPTvB2UZRSSqkm61yh\nnZZLau9Y4iKCvV0UpZRSqsk6TWhnFR0nPaeMy4Zp1bhSSinf1GlC+8Tc2dqerZRSykd1ntDemcOg\nxEh6x4V7uyhKKaVUs3SK0D56rJoN+4v0LFsppZRP6xShvSo9D6fRqnGllFK+rVOE9rK0HLpFhTCi\nZ7S3i6KUUko1W4cP7coaB2szCrh0aAJ+OkGIUkopH9bhQ/uzPQVU1Di4TOfOVkop5eM6fGgvT8sl\nMjiA8X3jvF0UpZRSqkU6dGg7nIYVu3KZODiBoIAO/VaVUkp1AgHeLkBrKq+qZeKgBKYO16pxpZRS\nvq9Dh3Z0aCB/vP4cbxdDKaWU8gitM1ZKKaV8hIa2Ukop5SPcCm0RmSoiu0UkU0QeOs3rySKyWkS+\nFpFtIjK93msjReRLEdkpIttFJMSTb0AppZTqLBpt0xYRf+A5YAqQDWwQkQ+NMWn1VvsVsMAY84KI\nDAUWAX1EJAD4F3CLMWariMQBNR5/F0oppVQn4M6Z9lgg0xizzxhTDbwNzGywjgGiXI+jgcOux5cB\n24wxWwGMMYXGGEfLi62UUkp1Pu6Edk8gq97zbNey+h4DbhaRbOxZ9r2u5QMBIyJLRWSziPy8heVV\nSimlOi1PdUSbDbxqjEkCpgOvi4gftvr9QuB7rvvvisjkhhuLyDwR2SgiG/Pz8z1UJKWUUqpjcSe0\nDwG96j1Pci2r7w5gAYAx5ksgBIjHnpWvNcYUGGOOY8/Cz214AGPMfGNMqjEmtWvXrk1/F0oppVQn\n4M7gKhuAASKSgg3rG4GbGqxzEJgMvCoiQ7ChnQ8sBX4uImFANXAJ8OezHWzTpk0FInKgSe9CNSYe\nKPB2IToY/Uxbh36unqefaevw9Ofa252VGg1tY0ytiNyDDWB/4BVjzE4ReRzYaIz5EHgQeElEfozt\nlDbHGGOAoyLyJ2zwG2CRMebjRo6np9oeJiIbjTGp3i5HR6KfaevQz9Xz9DNtHd76XMVmq+rI9D+t\n5+ln2jr0c/U8/Uxbh7c+Vx0RTSmllPIRGtqdw3xvF6AD0s+0dejn6nn6mbYOr3yuWj2ulFJK+Qg9\n01ZKKaV8hIZ2ByIivVwTt6S5Jmi537U8VkSWi8ge132Mt8vqa0TE3zUhzkeu5ykiss41ic47IhLk\n7TL6GhHpIiILRSRdRHaJyPn6XW0ZEfmx6//+DhF5S0RC9LvadCLyiojkiciOestO+90U66+uz3eb\niHxrLBJP0tDuWGqBB40xQ4HxwN2uCVweAlYaYwYAK13PVdPcD+yq9/z3wJ+NMf2Bo9gBhlTTPAMs\nMcYMBs7Bfr76XW0mEekJ3AekGmOGYy/RvRH9rjbHq8DUBsvO9N2cBgxw3eYBL7RmwTS0OxBjzBFj\nzGbX4zLsH8Ge2Ale/ula7Z/A1d4poW8SkSTgCuBl13MBJgELXavoZ9pEIhINXAz8HcAYU22MKUa/\nqy0VAIS6ZlgMA46g39UmM8asBYoaLD7Td3Mm8JqxvgK6iEj31iqbhnYHJSJ9gNHAOiDRGHPE9VIO\nkOilYvmqvwA/B5yu53FAsTGm1vX8dJPoqLNLwY6a+A9Xs8PLIhKOflebzRhzCHgaO0LlEaAE2IR+\nVz3lTN9NdybV8hgN7Q5IRCKAfwMPGGNK67/mGqlOLxlwk4hcCeQZYzZ5uywdTAB2HoIXjDGjgWM0\nqArX72rTuNpYZ2J/EPUAwvl2Fa/yAG9+NzW0OxgRCcQG9hvGmP+4FufWVde47vO8VT4fNAGYISL7\nsXPJT8K2xXZxVUHC6SfRUWeXDWQbY9a5ni/Ehrh+V5vvUuAbY0y+MaYG+A/2+6vfVc8403fTnUm1\nPEZDuwNxtbX+HdhljPlTvZc+BG5zPb4N+KCty+arjDG/NMYkGWP6YDv1rDLGfA9YDVzrWk0/0yYy\nxuQAWSIyyLVoMpCGfldb4iAwXkTCXH8L6j5T/a56xpm+mx8Ct7p6kY8HSupVo3ucDq7SgYjIhcCn\nwHZOtr8+jG3XXgAkAweA640xDTtZqEaIyETgp8aYK0WkL/bMOxb4GrjZGFPlzfL5GhEZhe3cFwTs\nA27Hnkjod7WZROS3wA3YK0m+Bn6AbV/V72oTiMhbwETsTF65wKPA+5zmu+n6gfQstiniOHC7MWZj\nq5VNQ1sppZTyDVo9rpRSSvkIDW2llFLKR2hoK6WUUj5CQ1sppZTyERraSimllI/Q0FbKi0TEISJb\n6t08NkGGiPSpP0tRWxORiXWzoimlPCOg8VWUUq2owhgzytuFaI9ExN8Y4/B2OZRqT/RMW6l2SET2\ni8j/E5HtIrJeRPq7lvcRkVWueXtXikiya3miiLwnIltdtwtcu/IXkZdccywvE5HQ0xzrVdd8wF+I\nyD4Ruda1/JQzZRF5VkTm1Cvfk67agY0icq6ILBWRvSJyZ73dR4nIxyKyW0ReFBE/1/aXiciXIrJZ\nRN51jZdft9/fi8hm4DrPf7JK+TYNbaW8K/T/t3fvoFEFURjH/6cxioIKNnaKjyKiQpCAmEotraKw\niFDkz48AAAIpSURBVGBjlSIIgl0aO7FRIjY2EkSiNoqVEFKIT3xA8FVYGB+lCgYRFHQ/i5nV65or\nGxPYve73a2Z29j6m2uGce3dOU3q8VvhuRtJm0m5Lp/LYaWBM0hbgAjCax0eBG5K2kvbwfpbHNwBn\nJG0CPgJ7S+axGhgA9gDHW5z7m5wluEmqP7yPVMf9WOGYfmAY6AXWAYMRsQoYAXZL6gMeAkcK53yQ\n1CfpYovzMOsaTo+btdff0uPjhfZk7m8HBnP/PHAi93cCBwFySnkmV32aljSVj3kErCm511VJdeB5\nRLRaDvNabp8Ay3IN908R8TUiVuTv7kt6CT+3hhwAvpAW8dtpB0gWAXcL173U4v3Nuo4XbbPOpZL+\nXBT3mP4O/JEen+W4yO03fs/GLS45p950fp1fvy3N81a+/oSk/SVz+Vwybtb1nB4361y1QtuIRO+Q\nqo0BHCClpgEmgSFIL3BFxPIFuP9roDcienLkvOsfrtEfEWvzs+wacAu4B+woPKdfGhEbF2C+Zv89\nR9pm7bUkIqYKn69Lavzta2VEPCZFsY2odBg4FxFHgXekylgAh4GzEXGIFFEPAfMqDyjpbURcBp4C\n06QKUXP1gPRMfj2pROQVSfX8Qtt4RPTk40aAF/OZr1k3cJUvsw4UEa+AbZLet3suZtY5nB43MzOr\nCEfaZmZmFeFI28zMrCK8aJuZmVWEF20zM7OK8KJtZmZWEV60zczMKsKLtpmZWUX8AGmS8JQ4I/fr\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f214a722f98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Probability of input being included in output in dropout layer\n",
    "incl_prob = 0.5\n",
    "\n",
    "input_dim, output_dim, hidden_dim = 784, 10, 125\n",
    "\n",
    "# Use Glorot initialisation scheme for weights and zero biases\n",
    "weights_init = GlorotUniformInit(rng=rng, gain=2.**0.5)\n",
    "biases_init = ConstantInit(0.)\n",
    "\n",
    "# Create three affine layer model with rectified linear non-linearities\n",
    "# and dropout layers before every affine layer\n",
    "model = MultipleLayerModel([\n",
    "    DropoutLayer(rng, incl_prob),\n",
    "    AffineLayer(input_dim, hidden_dim, weights_init, biases_init), \n",
    "    ReluLayer(),\n",
    "    DropoutLayer(rng, incl_prob),\n",
    "    AffineLayer(hidden_dim, hidden_dim, weights_init, biases_init), \n",
    "    ReluLayer(),\n",
    "    DropoutLayer(rng, incl_prob),\n",
    "    AffineLayer(hidden_dim, output_dim, weights_init, biases_init)\n",
    "])\n",
    "\n",
    "# Multiclass classification therefore use cross-entropy + softmax error\n",
    "error = CrossEntropySoftmaxError()\n",
    "\n",
    "# Use a momentum learning rule - you could use an adaptive learning rule\n",
    "# implemented for the coursework here instead\n",
    "learning_rule = MomentumLearningRule(0.02, 0.9)\n",
    "\n",
    "# Monitor classification accuracy during training\n",
    "data_monitors={'acc': lambda y, t: (y.argmax(-1) == t.argmax(-1)).mean()}\n",
    "\n",
    "optimiser = Optimiser(\n",
    "    model, error, learning_rule, train_data, valid_data, data_monitors, notebook=True)\n",
    "\n",
    "num_epochs = 100\n",
    "stats_interval = 5\n",
    "\n",
    "stats, keys, run_time = optimiser.train(num_epochs=num_epochs, stats_interval=stats_interval)\n",
    "\n",
    "# Plot the change in the validation and training set error over training.\n",
    "fig_1 = plt.figure(figsize=(8, 4))\n",
    "ax_1 = fig_1.add_subplot(111)\n",
    "for k in ['error(train)', 'error(valid)']:\n",
    "    ax_1.plot(np.arange(1, stats.shape[0]) * stats_interval, \n",
    "              stats[1:, keys[k]], label=k)\n",
    "ax_1.legend(loc=0)\n",
    "ax_1.set_xlabel('Epoch number')\n",
    "\n",
    "# Plot the change in the validation and training set accuracy over training.\n",
    "fig_2 = plt.figure(figsize=(8, 4))\n",
    "ax_2 = fig_2.add_subplot(111)\n",
    "for k in ['acc(train)', 'acc(valid)']:\n",
    "    ax_2.plot(np.arange(1, stats.shape[0]) * stats_interval, \n",
    "              stats[1:, keys[k]], label=k)\n",
    "ax_2.legend(loc=0)\n",
    "ax_2.set_xlabel('Epoch number')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3: Implementing maxout\n",
    "\n",
    "[Maxout](http://www.jmlr.org/proceedings/papers/v28/goodfellow13.pdf) can be considered a generalisation of the rectified linear transformation implemented in the previous lab. \n",
    "\n",
    "For a rectified linear (`Relu`) layer the forward propagation corresponds to\n",
    "\n",
    "\\begin{equation}\n",
    "    y^{(b)}_k = \n",
    "    \\max\\left\\lbrace 0,\\,x^{(b)}_k \\right\\rbrace\n",
    "\\end{equation}\n",
    "\n",
    "i.e. each output corresponds to an pairwise maximum of a constant (0) and the input.\n",
    "\n",
    "Instead of taking the maximum of the input and a constant, we could instead consider taking the maximum over sets of inputs of a fixed size $s$.\n",
    "\n",
    "\\begin{equation}\n",
    "    y^{(b)}_k = \n",
    "    \\max\\left\\lbrace x^{(b)}_{(k-1)s + 1},\\, x^{(b)}_{(k-1)s + 2},\\, \\dots ,\\, x^{(b)}_{ks} \\right\\rbrace\n",
    "\\end{equation}\n",
    "\n",
    "If these inputs $x^{(b)}_d$ are themselves the outputs of an affine layer, then this corresponds to taking the maximum of a series of affine functions of the previous layer outputs. Like a rectified linear layer this leads to piecewise linear input-output relationships (which have well-behaved gradients which do not suffer from the saturation problems of logistic sigmoid / hyperbolic tangent transformations) but unlike the rectified linear case we do not end force a portion of the outputs to be zero. \n",
    "\n",
    "Experimentally this form of transformation has been found to give good performance, with the name *maxout* chosen because the *out*put is the *max*imum of a set of inputs. Maxout is also commonly used with dropout layers however note they are not directly related - maxout defines a deterministic non-linear transformation which can help improve the representational capacity and trainability of models; dropout defines a stochastic transformation which is mainly aimed at regularising a model to reduce overfitting.\n",
    "\n",
    "Using layers which take the maximum of fixed sized sets of inputs is also a common technique in models with convolutional layers which we will cover later in the course, with here the layer commonly being termed a *max-pooling* layer (with there being natural generalisation to other choices of reduction functions over pools such as the mean). We will adopt this terminology here for a layer implementing the transformation described above and we will be able to reuse our code implementing this maximum operation when experimenting with convolutional models.\n",
    "\n",
    "The partial derivatives of this max-pooling transformation are sparse (lots of values are zero), with only the partial derivative of the output of a pool with respect to the maximum input in the pool non-zero. This can be expressed as\n",
    "\n",
    "\\begin{equation}\n",
    "    \\frac{\\partial y^{(b)}_k}{\\partial x^{(b)}_d} = \n",
    "    \\begin{cases} \n",
    "      1 & \\quad (k-1)s + 1 \\leq d  \\leq ks \\quad\\textrm{and} &x^{(b)}_d = \\max\\left\\lbrace x^{(b)}_{(k-1)s + 1},\\, x^{(b)}_{(k-1)s + 2},\\, \\dots ,\\, x^{(b)}_{ks} \\right\\rbrace \\\\\n",
    "      0 & \\quad \\textrm{otherwise}\n",
    "    \\end{cases}.\n",
    "\\end{equation}\n",
    "\n",
    "Using these definitions implement the `fprop` and `bprop` methods of the skeleton `MaxPoolingLayer` class below.\n",
    "\n",
    "Some hints\n",
    "\n",
    "  * One way of organising the inputs into non-overlapping pools is using the `numpy.reshape` function.\n",
    "  * The `numpy.max` function has an `axis` argument which allows you specify the axis (dimension) of the input array to take the maximum over.\n",
    "  * It may help to construct a binary mask corresponding to the definitions of the partial derivatives above to allow you to implement the `bprop` method. \n",
    "  * As with the `DropoutLayer` it is fine to temporarily store values calculated in the `fprop` method as attributes of the object (e.g. `self.val = val`) to use in the `bprop` method (although you don't necessarily need to do this)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlp.layers import Layer\n",
    "\n",
    "class MaxPoolingLayer(Layer):\n",
    "    \n",
    "    def __init__(self, pool_size=2):\n",
    "        \"\"\"Construct a new max-pooling layer.\n",
    "        \n",
    "        Args:\n",
    "            pool_size: Positive integer specifying size of pools over\n",
    "               which to take maximum value. The outputs of the layer\n",
    "               feeding in to this layer must have a dimension which\n",
    "               is a multiple of this pool size such that the outputs\n",
    "               can be split in to pools with no dimensions left over.\n",
    "        \"\"\"\n",
    "        self.pool_size = pool_size\n",
    "    \n",
    "    def fprop(self, inputs):\n",
    "        \"\"\"Forward propagates activations through the layer transformation.\n",
    "        \n",
    "        This corresponds to taking the maximum over non-overlapping pools of\n",
    "        inputs of a fixed size `pool_size`.\n",
    "\n",
    "        Args:\n",
    "            inputs: Array of layer inputs of shape (batch_size, input_dim).\n",
    "\n",
    "        Returns:\n",
    "            outputs: Array of layer outputs of shape (batch_size, output_dim).\n",
    "        \"\"\"\n",
    "        assert inputs.shape[-1] % self.pool_size == 0, (\n",
    "            'Last dimension of inputs must be multiple of pool size')\n",
    "        pooled_inputs = inputs.reshape(\n",
    "            inputs.shape[:-1] + \n",
    "            (inputs.shape[-1] // self.pool_size, self.pool_size))\n",
    "        pool_maxes = pooled_inputs.max(-1)\n",
    "        self._mask = pooled_inputs == pool_maxes[..., None]\n",
    "        return pool_maxes\n",
    "\n",
    "    def bprop(self, inputs, outputs, grads_wrt_outputs):\n",
    "        \"\"\"Back propagates gradients through a layer.\n",
    "\n",
    "        Given gradients with respect to the outputs of the layer calculates the\n",
    "        gradients with respect to the layer inputs.\n",
    "\n",
    "        Args:\n",
    "            inputs: Array of layer inputs of shape (batch_size, input_dim).\n",
    "            outputs: Array of layer outputs calculated in forward pass of\n",
    "                shape (batch_size, output_dim).\n",
    "            grads_wrt_outputs: Array of gradients with respect to the layer\n",
    "                outputs of shape (batch_size, output_dim).\n",
    "\n",
    "        Returns:\n",
    "            Array of gradients with respect to the layer inputs of shape\n",
    "            (batch_size, input_dim).\n",
    "        \"\"\"\n",
    "        return (self._mask * grads_wrt_outputs[..., None]).reshape(inputs.shape)\n",
    "\n",
    "    def __repr__(self):\n",
    "        return 'MaxPoolingLayer(pool_size={0})'.format(self.pool_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test your implementation by running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_inputs = np.array([[-3, -4, 5, 8], [0, -2, 3, -8], [1, 5, 3, 2]])\n",
    "test_outputs_1 = np.array([[8], [3], [5]])\n",
    "test_grads_wrt_outputs_1 = np.array([[10], [5], [-3]])\n",
    "test_grads_wrt_inputs_1 = np.array([[0, 0, 0, 10], [0, 0, 5, 0], [0, -3, 0, 0]])\n",
    "test_outputs_2 = np.array([[-3, 8], [0, 3], [5, 3]])\n",
    "test_grads_wrt_outputs_2 = np.array([[3, -1], [2, 5], [5, 3]])\n",
    "test_grads_wrt_inputs_2 = np.array([[3, 0, 0, -1], [2, 0, 5, 0], [0, 5, 3, 0]])\n",
    "layer_1 = MaxPoolingLayer(4)\n",
    "layer_2 = MaxPoolingLayer(2)\n",
    "# Check fprop with pool_size = 4\n",
    "assert np.allclose(layer_1.fprop(test_inputs), test_outputs_1)\n",
    "# Check bprop with pool_size = 4\n",
    "assert np.allclose(\n",
    "    layer_1.bprop(test_inputs, test_outputs_1, test_grads_wrt_outputs_1),\n",
    "    test_grads_wrt_inputs_1\n",
    ")\n",
    "# Check fprop with pool_size = 2\n",
    "assert np.allclose(layer_2.fprop(test_inputs), test_outputs_2)\n",
    "# Check bprop with pool_size = 2\n",
    "assert np.allclose(\n",
    "    layer_2.bprop(test_inputs, test_outputs_2, test_grads_wrt_outputs_2),\n",
    "    test_grads_wrt_inputs_2\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4: Training with maxout\n",
    "\n",
    "Use your `MaxPoolingLayer` implementation in a multiple layer models to experiment with how well maxout networks are able to classify MNIST digits. As with the dropout training exercise, code has been provided below as a starting point for setting up the model objects, but again feel free to substitute any components.\n",
    "\n",
    "If you have time you may wish to experiment with training a model using a combination of maxout and dropout or another regularisation method covered in the last lab notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import logging\n",
    "from mlp.data_providers import MNISTDataProvider\n",
    "from mlp.models import MultipleLayerModel\n",
    "from mlp.layers import AffineLayer\n",
    "from mlp.errors import CrossEntropySoftmaxError\n",
    "from mlp.initialisers import GlorotUniformInit, ConstantInit\n",
    "from mlp.learning_rules import MomentumLearningRule\n",
    "from mlp.optimisers import Optimiser\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Seed a random number generator\n",
    "seed = 31102016 \n",
    "rng = np.random.RandomState(seed)\n",
    "\n",
    "# Set up a logger object to print info about the training run to stdout\n",
    "logger = logging.getLogger()\n",
    "logger.setLevel(logging.INFO)\n",
    "logger.handlers = [logging.StreamHandler()]\n",
    "\n",
    "# Create data provider objects for the MNIST data set\n",
    "train_data = MNISTDataProvider('train', batch_size=50, rng=rng)\n",
    "valid_data = MNISTDataProvider('valid', batch_size=50, rng=rng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "train() got an unexpected keyword argument 'notebook'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-f63f26d97836>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     33\u001b[0m \u001b[0mstats_interval\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     34\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 35\u001b[0;31m \u001b[0mstats\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkeys\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_time\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moptimiser\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_epochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnum_epochs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstats_interval\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstats_interval\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnotebook\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     36\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     37\u001b[0m \u001b[0;31m# Plot the change in the validation and training set error over training.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: train() got an unexpected keyword argument 'notebook'"
     ]
    }
   ],
   "source": [
    "# Size of pools to take maximum over\n",
    "pool_size = 2\n",
    "\n",
    "input_dim, output_dim, hidden_dim = 784, 10, 100\n",
    "\n",
    "# Use Glorot initialisation scheme for weights and zero biases\n",
    "weights_init = GlorotUniformInit(rng=rng)\n",
    "biases_init = ConstantInit(0.)\n",
    "\n",
    "# Create three affine layer model interleaved with max-pooling layers\n",
    "model = MultipleLayerModel([\n",
    "    AffineLayer(input_dim, hidden_dim * pool_size, weights_init, biases_init), \n",
    "    MaxPoolingLayer(pool_size),\n",
    "    AffineLayer(hidden_dim, hidden_dim * pool_size, weights_init, biases_init), \n",
    "    MaxPoolingLayer(pool_size),\n",
    "    AffineLayer(hidden_dim, output_dim, weights_init, biases_init)\n",
    "])\n",
    "\n",
    "# Multiclass classification therefore use cross-entropy + softmax error\n",
    "error = CrossEntropySoftmaxError()\n",
    "\n",
    "# Use a momentum learning rule - you could use an adaptive learning rule\n",
    "# implemented for the coursework here instead\n",
    "learning_rule = MomentumLearningRule(0.02, 0.9)\n",
    "\n",
    "# Monitor classification accuracy during training\n",
    "data_monitors={'acc': lambda y, t: (y.argmax(-1) == t.argmax(-1)).mean()}\n",
    "\n",
    "optimiser = Optimiser(\n",
    "    model, error, learning_rule, train_data, valid_data, data_monitors, notebook=True)\n",
    "\n",
    "num_epochs = 100\n",
    "stats_interval = 5\n",
    "\n",
    "stats, keys, run_time = optimiser.train(num_epochs=num_epochs, stats_interval=stats_interval)\n",
    "\n",
    "# Plot the change in the validation and training set error over training.\n",
    "fig_1 = plt.figure(figsize=(8, 4))\n",
    "ax_1 = fig_1.add_subplot(111)\n",
    "for k in ['error(train)', 'error(valid)']:\n",
    "    ax_1.plot(np.arange(1, stats.shape[0]) * stats_interval, \n",
    "              stats[1:, keys[k]], label=k)\n",
    "ax_1.legend(loc=0)\n",
    "ax_1.set_xlabel('Epoch number')\n",
    "\n",
    "# Plot the change in the validation and training set accuracy over training.\n",
    "fig_2 = plt.figure(figsize=(8, 4))\n",
    "ax_2 = fig_2.add_subplot(111)\n",
    "for k in ['acc(train)', 'acc(valid)']:\n",
    "    ax_2.plot(np.arange(1, stats.shape[0]) * stats_interval, \n",
    "              stats[1:, keys[k]], label=k)\n",
    "ax_2.legend(loc=0)\n",
    "ax_2.set_xlabel('Epoch number')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}