nvidia-AI.summerschool17-no.../teaching-ncso/27-simple-convnnet.ipynb

595 lines
621 KiB
Text
Raw Permalink Normal View History

2017-09-22 17:38:09 +00:00
{
"metadata": {
"name": "50-simple-convnnet"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Title: Introduction to Convolutional Networks\n",
"Author: Thomas Breuel\n",
"Institution: UniKL"
]
},
{
"cell_type": "code",
"collapsed": true,
"input": [
"\n",
"from pylab import *\n",
"from scipy.ndimage import filters\n",
"def F(a): return array(a,'f')"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 2
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"A Nonlinear Image Processing Problem"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's start by constructing a simple non-linear image processing filter. Can we learn this filter?\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"image = 1.0*(mean(imread(\"page.png\"),2))\n",
"roi = (slice(500,1000),slice(500,1000))\n",
"image = image[roi]\n",
"image -= mean(image)\n",
"out = 1.0*(8*maximum(0,filters.gaussian_filter(image,2.0,order=(1,0))))\n",
"gray()\n",
"subplot(121); imshow(image)\n",
"subplot(122); imshow(out)\n",
"amax(out)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 5,
"text": [
"0.91207802"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAC2CAYAAAAr6G0hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvWecldXV/v89vZfpvTCFgZEZYOgoIFUHEVGKhZaoEUtM\nQmKJxid5kmh+mhiNvaAxkMRuwG5AQQGll6HD0Ibp9czp/Zz/Cz57e4aimGjI//nMegNzyn3usvfa\na13rutZWxOPxOL3Wa73Wa732/2tTnu8T6LVe67Ve67V/33qdea/1Wq/12v8B63XmvdZrvdZr/wes\n15n3Wq/1Wq/9H7BeZ95rvdZrvfZ/wHqdea/1Wq/12v8B+06c+UcffUS/fv0oLS3loYce+i5+otd6\n7bxY79jutf9WU3zbPPNoNEpZWRkff/wxOTk5DBs2jFdeeYX+/ft/mz/Ta732H7fesd1r/832rUfm\nmzdvpqSkhMLCQjQaDddccw1vv/32t/0zvdZr/3HrHdu99t9s37ozb2xsJC8vT/6dm5tLY2Pjt/0z\nvdZr/3HrHdu99t9s6m/7gAqF4lv5TK/12r9j30WXit6x3Wv/DXa2sf2tO/OcnBzq6+vl3/X19eTm\n5p72uaNHj6JWqzGZTPh8PtxuN0lJSWi1WkKhkHw9Go3S3t5OLBYjHo9jNpsxGo1EIhGUSiXRaBSd\nTodKpSIQCGA0GonFYqjVauLxOJFIhHA4jMFgwO128+STT3LTTTeh1Wrx+XzodDqUSiWbNm3ilVde\n4aGHHsJkMhGPx9FoNCgUCsLhMJFIhB/84AfodDqee+45urq6+PnPf07//v356U9/Kn9bo9EA4PP5\n6OrqIjk5GZ1OxyOPPMJNN92ERqPB7/djMplQqVT4fD5sNhtKpRKFQkE0GiUQCGAwGFAoFDidThob\nG3n++ef54Q9/SFlZGR0dHcTjcRQKhTzPzs5ObDYbRqORYDCIVqvF5XJhs9l45JFHuP3224nFYrzw\nwgtMnToVo9GI2WxGoVCg1WrlfY/FYuj1ejweD/F4HL/fj1Kp5JZbbmHgwIEsXryYWCxGLBZDpVJh\nMpnwer1otVo50MR9u//++1m8eDF6vZ5AIEA4HKa9vZ2MjAzUajWRSAS1Wo3BYCAejxMOh+Uz0+v1\nhEIh1OqTQzTx8+Ie/eUvf2HPnj04nU6efvppDAYDsViM0tLSb3tYf6OxnWg6nQ6z2Uw0GqW7uxsA\npVJJVlYWEyZMoG/fvmg0Gurq6lizZg0HDx78yoVIqTyZTMdisW/hipBj6Ovs2/7dM5nVamXRokXk\n5uaydOlStm/fLt8zGAwEg8HTfv+bnL9arUapVBIIBP6l81MoFCiVSnQ6HfF4nEAg8G8FDSqVSvq1\nb8O+dWc+dOhQamtrOX78ONnZ2bz22mu88sorp33OYrEQDAblxDWbzWg0Gjnp3W43Wq0Wr9eL1WpF\nr9fj9XpRKBRoNBra29vJy8uju7sbr9eLxWIhHA7j9/vRaDQolUo8Hg9er5d4PM6RI0fkzVepVIRC\nIXbt2oXFYqGqqop+/foRCoWw2WzE43EOHz5MRkaGdHgej4fvfe97PPDAA8RiMSKRCLFYDLfbLR1i\nNBolGo2iUCiIxWK0tbURj8cxGo3ymlJSUvB4PMRiMXQ6HZFIhO7ubsLhMGazWS4cwqmp1WrsdjvJ\nyckolUo6OzvRaDT4fD66u7vl+ykpKZhMJsLhMAqFglAoJAeLOKfu7m5effVVtFot1113Hfv37+fR\nRx/l6aeflgNVpVIBoNfriUQiBINBVCoVSUlJ5OTkEA6H5cKp1+vx+XxycVWr1fL3xAKhVCppaWkh\nKSmJSCRCWloaOp0On89HIBDAZrPh9/uJRCJoNBqam5tZunQpt99+OyaTSY4XhUKBz+eT5/bpp59y\n8OBBbr75Zp566ikikYh08t+VnevYTrRwOIzD4ejhhGKxGI2Njbz22mvodDo0Gg1er5dgMPi15/BN\nnalKpWL+/PmUl5fzt7/9jf379xMOh+X78Xgcm82G3W7nxIkTcryGQiEikcg3/l2VSkU0Gj3n8xsx\nYgSTJk3inXfeYffu3Tz88MPyvBKPeTbHea6OMBaLEQqFAKRTD4VC5ObmsmjRIpqbm/nb3/6Gy+X6\nymuLxWIEg8FvdI1ns687xjXXXENZWRl//vOfewQRZ7Nv3Zmr1WqefPJJLrnkEqLRKDfccMMZq/1u\ntxuVSoVWqyUQCEiHE41GiUQiuN1u9Hq9HFjCeXR3dxOLxTCbzXg8Hrlai9f8fj8ejwej0YharSYc\nDqPRaLDZbJjNZgC8Xi8Gg4GjR49SVlZGJBIhLy+Pxx9/nGAwiEKhID09ndbWVtRqtVxgYrEYycnJ\nuFwuTCYTmZmZdHd34/P5pCMVE9Lr9WK329FqtWg0GqLRKKmpqSgUCpKTk+XntFotbrebtrY2cnNz\ncTgcpKSk4HA4ADCZTKjVam6++WaZiahUKvx+P7FYTN4/lUpFOBzG4/HQ2dmJ3W4HkAuHOJ7BYMBm\ns+Hz+SgvL2f8+PH84Ac/4JlnniEQCGCxWDAajUSjURn9A/z2t7+VC6OYCOLz4joSMyK/3w9AXV1d\nj6jf7XYTCoWIx+PyGDqdjkAggN/v59ixY+j1ehoaGujTp490/ImLjNfr5fXXXyc7Oxun08kvf/lL\nQqGQjL6+KzvXsZ1oX+UEQ6GQdDDfFTwTjUZ59dVXMRgMcgycak6nE6fTKbMfsRgnOvOvMpVKRXJy\nMl6vVy7m52qbNm1i06ZN8u8zOedzPZ5Wq5UL1YUXXkh+fj4vv/zyGY8nfqehoYHf//73RCIROWbP\nZud6P74t+/DDD9m2bds5OXL4Dpw5QHV1NdXV1V/5GYfDwRNPPEFWVhYtLS1YrVY6OzsZOXIkBw4c\nYM+ePTz55JPyposV32Qy8cknn7BmzRrUajUlJSUATJ8+Hbvdzu7du1myZAlWq5Uf/ehHGI1G3nrr\nLXbv3s2iRYuoqalh48aNtLa28tRTTzFx4kSysrLweDy8+eabLFiwAL1ez7PPPktbWxtVVVVcfvnl\ncvEIh8P4fD7MZrN0mhqNhrVr17Jjxw7i8ThFRUXs27cPh8PBnDlzqKioYOjQofJ7cHLQPvPMM4wd\nO5bm5mY6OzuJxWI8//zz3HHHHXz00Uc0NTVx1113sW7dOlauXMmCBQtIS0vjueeeIxAIkJOTQ01N\nDYsXL6asrAyv18umTZtYs2YNNpuNBQsWAFBSUkJtbS0FBQVoNBpUKhUqlQqDwcDVV1/Nvn37WL9+\nPWPGjJFwiUajIRaLEQgEqKurY9WqVaSlpXHFFVfQ0NDAG2+8gdfrxe1289BDDxGJRKQDiEajhEIh\nDhw4wJo1axg6dChKpZKGhgZuvPFGsrKyiMfjPPHEE6SkpHDBBRcwZswYNmzYwJIlSygpKeG9997j\nBz/4gYzY1Wo1oVAIj8fDCy+8wKFDh3A4HFx99dUEAgHWr1/PsWPHvnbc/bt2LmP7XzGFQiEzum/b\nAoHA12YsBoNBOsJzyRBONYfD8R93dqdaKBSS9/HAgQPk5uYyceJEPvnkE6AnJJO4QLjd7vNyvl9l\nIshzOp3n/J3zpgDVaDTccMMNvPTSS0yYMIExY8awatUq+vbty+zZsxk/fnwPuEHAB2q1mtGjR5Od\nnc0nn3zCoEGDKC8vZ/bs2bS0tPCLX/yCK664gsrKSu677z7C4bCEFF577TWKi4upqalh8ODBGAwG\nrr32Wmw2G1lZWbz//vvY7XaOHDnC6tWruf/++1m6dCkOhwOXy0VmZqbExQU8pFarcblcDBkyhI0b\nN/Lyyy9TXl6OWq1m0KBBBINBurq6qK6uxmg0SsjG5/OxatUqnn76aaLRKH//+99JTk5m3bp1OBwO\nhgwZwscff0xnZycDBw5k79696PV6UlNTpROfNWsWkyZN4qWXXkKtVrNt2zZeeOEFfvjDH/L555/z\n0UcfEY/HefTRR4lGozidTgKB
}
],
"prompt_number": 5
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Learning a Linear Filter"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that if this were a linear filter, we might think we can derive it from the Fourier transform:\n",
"\n",
"convolution (linear filter):\n",
"\n",
"$$y = f * x$$\n",
"\n",
"fundamental theorem:\n",
"\n",
"$${\\cal F}[y] = {\\cal F}[f] \\cdot {\\cal F}[x]$$\n",
"\n",
"solving the equation:\n",
"\n",
"$${\\cal F}[f] = \\frac{{\\cal F}[y]}{{\\cal F}[x]}$$\n",
"\n",
"transforming back:\n",
"\n",
"$$ f = {\\cal F}^{-1} \\left[ \\frac{ {\\cal F}[y] } { {\\cal F}[x] } \\right] $$\n",
"\n",
"However, this may or may not be the filter we want.\n"
]
},
{
"cell_type": "code",
"collapsed": true,
"input": [
"\n",
"def sigmoid(x): return 1/(1+exp(-x))"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 14
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Single Layer Convolutional Neural Network"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The simplest convolutional neural network has no hidden layer; it is the equivalent\n",
"of a sigmoid regression. The formula is:\n",
"\n",
"$$ Y = \\sigma(F*X+\\theta) $$\n",
"\n",
"Here\n",
"\n",
"- $F$ is the filter\n",
"- $X$ is the input image\n",
"- $Y$ is the cleaned up output\n",
"- $\\theta$ is a threshold"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we can view the convolution basically as a large number of independent\n",
"training problems. \n",
"\n",
"Consider the image $I_\\hat{p}$, which is the image\n",
"$I$ shifted by $-p$. If we keep the filter fixed and shift the image,\n",
"then the output $C$ at pixel $p$ is given by:\n",
"\n",
"$$Y(p) = F \\cdot X_{\\hat{p}} + \\theta$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Generally, the filter $F$ has a small footprint, meaning that\n",
"it is zero outside a small region around the origin.\n",
"\n",
"In essence, this problem is just like training with lots of separate\n",
"training instances, except that we're trying to use a convolution\n",
"operation to implement this.\n",
"We need to keep track of the coordinates in the right way.\n",
"\n",
"$$\n",
"\\begin{eqnarray}\n",
"\\frac{\\partial}{\\partial F_{ij}} \\sum_p (~T(p)-Y(p)~)^2 &=& \\sum_p 2(T(p)-Y(p)) ~~ \\sigma'(F\\cdot I_{\\hat{p}}) ~~ I_{\\hat{p}}\\\\\\\n",
"&=&\\sum_p 2(T(p)-Y(p)) ~~ Y(p)(1-Y(p)) ~~ I_{\\hat{p},i,j}\n",
"\\end{eqnarray}\n",
"$$\n",
"\n",
"Here, we define $\\delta(p)$ as before:\n",
"\n",
"$$ \\delta(p) = 2(T(p)-Y(p)) ~~ Y(p)(1-Y(p)) $$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's generate a random mask to start with.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": true,
"input": [
"r=5\n",
"filter = F(0.01*randn(2*r+1,2*r+1))"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's look at the output this filter generates.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"pred = sigmoid(filters.convolve(image,filter))\n",
"subplot(121); imshow(pred)\n",
"subplot(122); imshow(pred,vmin=0,vmax=1)\n",
"amin(pred),amax(pred)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 8,
"text": [
"(0.48624954, 0.52211583)"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAC2CAYAAAAr6G0hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsfdmPHNd1/le97/ve0zPdPfvGbShSJCVKDiXZgmPBsOEk\nDgIYyWOQP8B5seO8xHkP8ho4QBD7xbEMw5At2dZibZRkiaPhDIez7z2973t3/R4m56imVN0zlClT\nv2AOQEyzuupudfs75373nHMFURRFnMmZnMmZnMn/16J61A04kzM5kzM5kz9ezsD8TM7kTM7k/4Cc\ngfmZnMmZnMn/ATkD8zM5kzM5k/8DcgbmZ3ImZ3Im/wfkDMzP5EzO5Ez+D8jnAuYvvfQSJiYmMDo6\nin/913/9PKo4kzN5JHI2t8/kiyrCw/Yz73Q6GB8fxyuvvIJwOIzHHnsM//3f/43JycmHWc2ZnMmf\nXM7m9pl8keWhW+a3b9/GyMgIotEotFot/uqv/govvvjiw67mTM7kTy5nc/tMvsjy0MF8b28PkUiE\n/z8wMIC9vb2HXc2ZnMmfXM7m9pl8kUXzsAsUBOGh3HMmZ/LHyOeRpeJsbp/JF0F6ze2HDubhcBg7\nOzv8/52dHQwMDHzqvlu3bqHb7QIA/zUYDBAEAfV6Hc1mE7FYDHq9HqFQCMvLy+h2u8jn8wiFQtjd\n3YXdbke1WkWn04FWq4Uoimg2mxBFETqdDs1mE4IgwGw2o9PpoNPpYGdnB5FIBA6HA1tbWxgaGoLL\n5UKhUMDW1hamp6eRzWaRyWRgs9nQ7XbRaDTgdrvRarVQr9fx9NNPI5lM4je/+Q2+/OUvQ6PR4Pbt\n2+h2u1Cr1dBqtVCpVKhWq7BYLGg0GlheXkYwGEQ4HEYymYTRaIRGo0G9XodarQYAmEwmHBwcwGw2\nQ6vVotvtQqvVwmazYX9/HyMjIygUCrBYLCgUCgiFQlhfX4cgCOh0OlCpVBBFEfV6HXq9Ht1uFyqV\nCvfv38e5c+dw7tw5/OIXv8BXvvIVNBoN1Go1vPLKK/ibv/kbLC0tIZFIQK/XQxRFiKIIlUqFbreL\noaEhrK2tIRwOQ61WI5PJoFKpQK/Xo9PpAPhkgjUaDajVapjNZiwuLmJmZgalUgk2mw21Wg0qlQo6\nnQ6NRgOiKEKtVkMURbTbbZjNZlgsFtRqNXi9Xuzu7qJWq/FYCIIAtVqNcDiM3d1duN1ujI6O4s6d\nO/D5fDg4OECtVsOvf/3rhz2tH2huf+973/vUNQJ5Gif6P40xfScIAv+VXpM+q1Tuq6++iqeffvpT\nZdBnep/0uZfSoXto7giCgG63+6l2S+XVV1/FU0899am2y6VXGdK2Sful1F9p+dRnajf9Bkg6nQ40\nGg33p1+7pPUr9UHeNvl49xP5PdJ3Ih8zab9pbkjv/+d//uee9Tx0muXy5ctYWVnB5uYmms0mfvKT\nn+CFF1741H1arRZqtRoGgwFqtRoajYYBRq/Xw2AwIJVK4eDgAL/61a/Q7XZhMBgQiUSQzWZhMpkg\nCAJUKhWX0el0oNPpuDyTyQSDwQCNRgO1Ws0vWq/Xw+fzIZlMYnh4GNFoFGq1GsFgEPV6HZVKBTqd\nDrVaDSaTCRaLBUNDQ3C73XA6nbDZbBgaGoLJZIJarYZarUar1QJwpJg0Gg3a7TZMJhNarRa/DLPZ\njHK5zD8QaotG84lONZlM0Gg06HQ6EAQBgUAArVYLVqsVqVQKuVwOq6urUKlUSKVS8Hg80Ov13GeV\nSgWtVnuszyqVCnq9HuFwGPV6HX6/H91uF7lcDrdu3cKvfvUrjI2NwWKxQBAEaLVanlwmk4nLsNvt\naLfbDMg0xhqNhvtEbSCQbzab8Pl8EAQBdrsdWq0WZrMZRqOR20VtDYfDSKVSiEQiMBqNAACLxQLg\nOPhFIhHU63VcvHgRDocDDocDlUoFjUYDer3+YU9pltPObQJR6Q+4F4jSONFz0r/Sz/IfvLx8ed1S\nMKfrvSw6eXn9gKyXnHZFotQ3advkf09bj5ISJEzoNVa9ynuQ1dVpV2tKiq7X+yblIv3/aZTGQwdz\njUaDf/u3f8OXv/xlTE1N4S//8i8Vd/uffPJJmM1m+P1+XLt2DRcuXIDRaMS3vvUtRCIRiKKIJ554\nAjabDXa7HfV6HblcDu12GwaDAT6fD3Nzc7hw4QK0Wi0mJiZw6dIlVCoVXLt2DTqdDrOzszh37hwr\nh6997WsAgKeffhrT09MYGRlBt9tlwN3Y2MCf//mf4+LFixBFEdevX0en00EsFkO73cbh4SEqlQps\nNhs0Gg0ymQzUajVmZmbgcrlgMpkwMjKCmzdvIhKJQKfT4caNGwiFQsc0v16vRzQaRSaTwVe+8hXY\nbDYEg0E88cQTKBaL+Pa3vw2j0QidTodYLAadToeNjQ1861vfwle/+lVYLBb4/X7cunUL6XQaIyMj\niEQisNlsEEURIyMjaLfbuHLlCmKxGCsDo9EIQRDg8XgwODgIi8WCTqcDi8WCzc1NBAIBiKKITqeD\ndrvNVjNJMBjEM888AwCIRqOIRqNoNpuYmZlhJaLVaqHT6eD1emG1WmE2mzEyMoKnnnoK1WoVjz32\nGEKhEAwGA1wuFy5evAij0YgbN26wZe73+yGKImKxGK92NBoNtFot/H4/arUaHA4HEokEzGYz9Ho9\nZmdnEQ6HEQ6HH/aUfuC5Lf0hkiKkVYVUKfUSek76Y6f5Q6s4OUBJy1UCL3pWXr8U/JRWAVKgl/ZF\nWpfS/XJAos/Ufvm9ZHjQtdP2Uw76SopObpX3WuEo1amkQOXtob5Jx0M6xtJx6KUwpW2TA3+vdyqX\nz8XP/Pnnn8fy8jJWV1fxj//4j4r3aLVaPPHEE3jnnXcQj8cRCoWQy+Xg8/lQr9cRCoWQTCbRbrd5\nma1SqVAoFBAIBJDL5XD79m0Eg0GcP38er7/+Om7evIlyuQy73Y6xsTFsbm7C6XRiamoKpVIJer0e\nk5OTODw8hE6nQyKRQCAQQDabxcTEBNLpNMrlMiqVCjKZDKanp5HP5zE4OAi1Wg2n04larYZut4ti\nsQgAbOHG43Go1WocHh7C6XRiZWUFgiBgd3cXpVIJbreb+6LT6eByuWCxWJBMJnH58mVUq1WEQiEA\nQKvVwuTkJGq1GgRBQDQaRbvdRjKZRKlUQqfTweHhIbrdLux2Ow4ODvD444+jUqlAo9Hg8uXLTFWd\nP38eGo0GU1NTAI4mS6vVQi6XQ71eRzqdxs2bN5HJZODz+WAwGNBut3l1IIoitFotTCYTkskk9Ho9\nFhYW0Gw24fV6USqVUKlUjk28TqeDQqGAkZERJJNJXg3k83l0u12cP38eu7u7CIfDGBsbQzqdhlar\nxdjYGPb396HT6eB2u/lZaZvS6TScTicqlQoikQiGh4dxcHCA7e1tPP7441haWvo8pvQDzW0aBylV\nIf+R9qIc6DsCIGk58nukQBKLxRQBSE6vKJVBcpI1TPdLaSFRFBGNRj91v3Q+9KIZpJ+ldcvbIe27\ntC+xWOxT30uflys/aR/k7aVnqG/9VjJUt7wP8v4q9fOksuWrKjnl2U8eWQTowMAAYrEYbty4AQDw\n+XyIxWLwer0wmUwIh8NIp9MoFAq8VNJoNPD7/QiFQvD5fGz16XQ6phquXLkCvV6PwcFB5HI5ttZU\nKhUcDgfsdju8Xi+q1Sqi0ShMJhNmZmZgt9tht9thMBig1Wpx7tw5GI1GmM1mWK1W+Hw+flE0eUwm\nE5rNJsLhMJrNJr70pS+hVCrh/PnzmJychN1uR6lUQr1eh8/ng9FoRKfTgdlshs/nw+HhIWKxGJLJ\nJEZHR1GtVmEymRCPx6HX62EymRAMBjE4OAiNRgO3241wOAyfz4fBwUHEYjEYDAa2itvtNi5dugSr\n1YqpqSmoVCqYzWYYDAYYDAYAR7RFtVqFRqNBPp+H3+/H+vo6arUadDodBEFgC56op8HBQej1eqYx\nnnzySQQCATgcDjz++OMIBAJs
}
],
"prompt_number": 8
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As before, we compute a delta between the output and the desired output.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"delta0 = (out-pred)\n",
"subplot(121); imshow(delta0)\n",
"delta = (out-pred)*pred*(1-pred)\n",
"subplot(122); imshow(delta)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 10,
"text": [
"<matplotlib.image.AxesImage at 0x374ffd0>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAC2CAYAAAAr6G0hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvWl0nMWV///tp/dudbdaUmtfrc3yIlkyGC8Yr3g3iYMB\nG8w6QCYkECYwDGQSZjgzZ0LeTGYyy4vMOTMnc5gwZpIQIEwIgYADMQ5gYywvsmzZsrbW3rt6f57/\nC/3udT2PuiXZmIj/HN1zdCT18jxPVX2q6tate2/pFEVRMC/zMi/zMi//vxZprh9gXuZlXuZlXj67\nzA/m8zIv8zIv/wdkfjCfl3mZl3n5PyDzg/m8zMu8zMv/AZkfzOdlXuZlXv4PyPxgPi/zMi/z8n9A\nPpfB/I033sDChQtRX1+P73//+5/HLeZlXuZE5tmely+q6K61n3k6nUZjYyPeeustlJWV4frrr8eL\nL76Ipqama3mbeZmXP7rMsz0vX2S55pr5hx9+iLq6OlRXV8NoNGLfvn145ZVXrvVt5mVe/ugyz/a8\nfJHlmg/m/f39qKio4P/Ly8vR399/rW8zL/PyR5d5tufliyyGa31BnU53TT4zL/PyWeTzyFIxz/a8\nfBEkG9vXfDAvKytDb28v/9/b24vy8vIpnzObzaq/c3JykE6n4ff7AQCSJKG4uBgbNmxAbW0tzGYz\nenp68O677+Ls2bMAJgtFnUf8W5Ikfo1+gMmOlkqlYDAY+PP0vk6n4//ps/S3JEmQZZmfV5Zl6HQ6\n6PV6pNNpKIoCvV7P902n06prZLuv9j70v1ge8XOyLCM3NxcPPvggSktL8cILL+D48eP8GZvNhlgs\npioTPQ+VQa/Xq8qi1+uRSqUgSRL0ej3/0HXEMlC5tc8tDmA6nQ6SJMFsNiOVSiGRSEzbVtq/Seh1\nqjNZllXPIUkSv05/UxvG4/EpvF0LmS3bJpOJy3SlbHd0dExhgoTqFgDXB0k6nYZer1d9Vss2td9s\n2KbXFUVhNqi+tZJOpzOynen5s5VLlmW4XC489NBDs2Jbe19iO51Oc93PxDaQuZ9lek56XZIkmEwm\npFIpJJNJvsaViMi2LMuqOhX7g5ZtANyfMtbvtd4ATaVSaGxsxNtvv43S0lKsWLFiyiaRTqdTDebT\ngWIymWCxWKDX6xGJRDJ2VHHAIck0WNKgSvejxpckCffccw8WLVqEF154AWfPnp3SUA6HA263Gz09\nPUin07Db7Uin00gkEnwt7b1FEcEzGAw8CWQqi3gdRVGwatUqbNiwAb/85S9x8uRJVWOTiNfU1geV\nOZ1Ow2g0IplMQpIk/pwkSVwPJEajEZIkIR6Po7y8HA899BD6+/vx3//93wgGg1mfm8oIAPF4XPW/\ntj203yd4qVzixCHWidhx9+3bh4aGBvz4xz9GX18fFEXhNrnWciVs0/2JCyqT2D4mkwlWq5XZjsVi\nV63Zi4N5JrabmprwX//1X+jo6GC2SWbLdrY6Fe9N952NKIqClStXYuPGjXj11Vdx6tQpHrzEiYgU\nEO39xft+nmwDUA2+pDCIfX42Zc3EtlaI7VQqhX379qGxsRE//vGPWYmYju1rPpgDwK9+9Ss8/vjj\nSKfT+JM/+RM888wz6pv+v4oRG06rPYiNIDaM2CkyadTi97Sao6jdAVBBYrPZYLPZMDIyAkmSsl7H\nYDAgHo/DYrHAbDYjFApxA9DzZytPQUEBJiYmIMsyJiYmpoAmNqQ4CFAdpFIpFdz0ea1WQfcUf1Mn\ns9vtiEajAIA1a9agsrISBw8enKLNi3VAWlM8HsfExMSUe4rlJVEUhTsbXZvaT/ysOIBrn19sO1HE\nAdHpdKKwsBDnzp3jtkomk5/LYA5cHdtUnyLPVG6S6djO1Ce09ZhMJmE0Gpk/Gng+C9vBYBCyLMNg\nMMz4XAUFBYhEIlAUhdkWFSeaFLRs0/dTqRQAtWJHnGtXg3RvWZa5zLIsM9uKouDGG29ktjMpB9Qf\nZmJb205UD5lWBZ+VbbGfKIoCh8OBoqIinDt3TtXOf9TBfCYhcIBJTS6ZTHIFi42tHSSokOJSihpK\nr9cz0PQ+VR69T8AQFEajkQdI6gA0s9P1SOPV6XSw2WxIpVKsQdF74oALqLVwKoN4TXpuUYuhDmYw\nGHhpSK9RPdHzi+YcUTMR/xaXcOKkRfUjSRLy8/OxadMmDA0N4d133+WOqzV3iIOntgNlgphEhJmu\nTasH8fm0r4t1InZgei66r16vRyKRUJVPr9cjHo9/boP5TCKyLTIntmumlSQNIlfDdqaBU+RWy7b4\neZHtdDqdkW3RvDNbtsXJScu2OJiKdUOfpftoB3bx/vRsVCeZ2N64cSOGh4dVbFN5MplZtNfTTiai\niFq/drU9099iObVCdWcwGJBIJFTmRJpss7Gt/+u//uu/nobNz0Wee+456HQ6XH/99YhEIohGozCZ\nTNy5zWYzbr/9dlgsFgwODgK4PKjYbDZs3rwZeXl5GB4exq5du5CXl4f+/n7IsoySkhLU19cjHo8j\nHo9DlmVUVVUhEomguroad955J/Ly8gAAeXl5WLJkCRRF4c/Ksgy3242ioiI4nU7WTgCo4JMkiRse\nAJqbm+HxeDAyMgKj0YimpiasWbMGiqIgEAhwRyWhDp9Op1FfX4+qqioEg0Gk02mk02kUFBRwx6qo\nqEAsFkMymYTFYoEkSbBYLFi5ciUkSYLf71dpvmVlZQiFQtzoBJFer4fJZOKlWjQahdfrhcvlQn9/\nf0ZziNFoRGFhIS8txU5gMplQVFSEiYkJAJe1eUVRkJubi+uuuw4GgwEejwcrV67E0NAQ2yv1ej2c\nTidaW1sxMDDA1y0rK8MNN9yA1atXIxQKIRAI8PMAk2a3xYsXw+FwwOFwwGAwYGJiAiaTCTabDYlE\nAul0GnOANQA12xMTE1PYtlgsuO2222CxWOD1evl7IttutxsjIyPTsh2LxVRs19TUYN++fcjPzweg\nZps+m4ltcYKkdhHZVhQFLS0tV8y2TqfLynZ+fj7i8fiMbOt0Or6+lm0ScS8oG9tkftOKlm1xZaRl\nm5QHRVHgdruzsk3P4nK50NraqupXZWVlWLFixRS2SUwmExYtWnTVbM+ZZq7X61FZWYna2locOnQI\nsixj+fLlOHbsGEwmEzZs2IBPPvkEo6OjKk1GkiTYbDZYLBZYLBYAwMTEBMbGxpBIJGCz2RCPx7Fo\n0SKMjo5ieHgYer0eBQUFKC8vx/XXX49kMol3330XfX19DBUAXm7l5uYiHA4jEAjAZDIhmUzC5XIh\nmUwimUyydmE2m5FIJCDLMvLy8nDrrbfil7/8JQYHB+FwOLB+/XqcP38eXV1dDDuVw2azsUZz0003\noaysDL/61a/4vtRBZFmGyWRiG6YkSSgsLEQ6neYl8fj4OJLJJEwmE3/vlltuQTQaxVtvvcWdizQN\n0RZqNptRUlKC4eFhhMNh1QoDmNzAI+0kGo0iNzcXdXV1OHbs2JSVRCZTk8lkgtvthizLiMfjGB8f\n5wnb6XSyNkj17nA4eMCfmJiA2WzmpWUsFoPZbIYkSYjFYjAYDDCZTCgtLcXFixcRi8VYW59LzTwT\n221tbcz2xo0bmW1R9Ho9rFYrrFYrsx2JRDA+Ps5sx2IxLF68+IrYpsk1FovB7XZnZDuRSCCVSjHb\n9B5NAHv37lWxvW7dOnR1daGrq0u1ugUAu91+VWzrdDoUFRUxE1arFWNjY0ilUqoJUWSbJh9SImhP\njQb30tJSDA0NIRwOT9kgpmtKkqRi++jRo6wJa9kWnRuMRiPy8vL4+bOxTfVObMuyjGg0yuMHABXb\n0WgURqMRJpMJxcXFuHTpEqLRKGvr2di+5t4ss5XS0lLE43EYjUbWUp1OJw8mp0+fRjQa5QFIrMxg\nMIhQKASdTsdLR1HTBYAzZ87wUkan02F8fBzDw8M4f/48bDYbX1P7vWg0ing8ztekiotGo7BYLKrN\nI1oaApMTSnt7OxYuXIjh4WE4
}
],
"prompt_number": 10
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that each weight in our random filter contributes to all output pixels.\n",
"If we want to \"backpropagate\" the deltas, we need to sum up all the deltas from all the filter outputs.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"dw = array([[sum(delta*roll(roll(image,i,0),j,1)) for j in range(-r,r+1)] for i in range(-r,r+1)])\n",
"dw /= prod(image.shape)\n",
"imshow(dw)\n",
"amin(dw),amax(dw),dw.dtype"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 90,
"text": [
"(-0.0034652944, 0.0011683544, dtype('float32'))"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAFpCAYAAAC4UhIKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnVmsXVX9x38X6cNfCTjEtsptrSINbRlaLVEZnLBqSCQO\nmOCQEkB9MEYxxiFqfDARSmpUHJ6MQ00MmPiASLAqMSpQq5LiECVClAsVBAesikWRev4PZNfVxRp+\n4xrOXd+kOXv/prX2Pmd/9u+sc+7pwmw2m8HQ0NDQUPM6qvYEhoaGhoZwGsAeGhoa6kQD2ENDQ0Od\naAB7aGhoqBMNYA8NDQ11oqMtii4sLFiUHRoaGpp7pb64ZwJsAIAHHniAlXfFFVfA+9///mSM5IbA\nycXmcOMuu+wy+OAHP0iqlfJLci1ru76PfvSj8JGPfIRdKyfLpoH7TdjUMWNrasblYlJ+bO7HPvYx\n+NCHPpTMk9is8lP2XM7OnTvhve99bzQ3pZUrVyb9Y0lkaGhoqBMNYA8NDQ11ouaAfeaZZ9aeQnGd\nffbZtadQXC960YtqT6G4luMxa762e/ls7IwzzjCrvWDxp+kLCwvsNWxs/ZK51mvYnJje17Axmrc1\nbI2ava1hY+yl1qtLrWFLtHLlymSN5jrsoaGhoaGwugN2L2+LetU4v3mNc1RHpX+nrsXnuTtgDw0N\nDS1XdQfs8WuwthrnN69xjuqodMfb4vPcHbCHhoaGlqsGsIeGhoY6ERvYu3fvhpNOOglOPPFEuOKK\nKzTnNDQ0NPQYtbhEUVosYB86dAje8Y53wO7du+HXv/41XHXVVXDbbbdpz21oaGhoyBEL2D/5yU/g\n2c9+Nqxbtw5WrFgBF1xwAXzjG9/QntvQ0NDQYbX4NbvSYv1a3z333ANr1qw5vL+4uAg//vGPj4hx\nl0nOPPNMOOuss5hTHBoaGgqrxjKJ5pg333wz7NmzBx3PAjbmTpf7iVSJZrMZ+247nWxKPnY8zTjp\nMfbejbT6Z+l+fa2Ld2FhAVVLO64ntX48nPmdeeaZR/x+0s6dO5PxrCWR448/Hvbv3394f//+/bC4\nuMgpxdZsNhM9gdL8VF3NuNJqAfQtwzo0nvtPWqtkXE/nmaJWry0NsYC9detWuOOOO2BpaQkefvhh\n+NrXvgbnnXee9txQsgJvaJyW1Np8WlALEJGCWxvaErVwPrU0L9cLa0nk6KOPhs9+9rPwile8Ag4d\nOgSXXHIJbNiw4YgYyQnivFA4Sx3UvNJLI70tbWj8Ul9Px5uSfxy1gDGPSyOa0voFv1Ln2OznVf/y\nl7+o1SqVV+NnVHNxLfks61LjtPJqSOOnTzVrcX9GVevnR2vZStip6v7nVbkngpPX4vrz6I7y6gnW\nWGkek+Va9nJS7e4aoANgA/DXqTl52BxsjGY3RcnrEfTLBRwl/6MLalzvorzuWwAwVWb/a7qFJOvU\nFjm9rTHPoyRLKDUvTM2v51keS+3zlFKL74itx+yiw/ZVqtvWmos0RnveLV6Alp85xPK0vo7HVcll\nD+3xLOr1qNLXUpfABpAtk1jElqxVSzU+yNRWbKxa8K4B2hJjzBvMW7l+uwX2JO46tWas1pNpfQOa\nJ1l/C6gkwLWg3Qr8LVTqdd769WS2hi09cOt1au3vX48173IquXwSyrW4qLXWqi3WnFtex+ZK43hq\nfGjZbIdt+Q0PP0dLtb4R0rNa6WCp9Sw68No3c83xpT8b0ZJamk+zwJ7Uyp+dj7dk+irRKZe6IWhB\nXGO9v6XPDHqSZdetpeaBDUDvnK067ZIx86J56kApCkFca/49rFXXGJ/yF44WY5VQF8CexAE3tX6p\nGE5+rT+X7VW1oRVSrx8epr5dM1ROXQF7kmW3XapDXu4wtVbLICkFZM5XLVs+b65KrZHX+hpwTF0C\nG8C229aA9nJaGunxg8PasgbykJ1qXrfdAnuSVbddu4ueF5jXUC8gK/HhYOvnopXXeSvzyKnZ72ED\n4F9snO9ga3z/WuN71VZzb12UY5iH440p9x1nqT8V08L3qy3nYPVZksVYWDXdYddeq9aos5y/DlgD\ntPMI99Jf0+t9fTulFq8TipoGNkD7a9UaGt/yiGs5dOLjM4B+VPuabB7Yk1qGditdtuXY83bBh74X\nXfMYpV209JcLa6mFpsly6WRqOP2YmC2n7n4P22Jdu8RatOZ6+LysY+dk1V27sRRoTxeU1bprifXs\neVVrx439Ix7qvLsCNoD+jzZhY5cLJEuo5U4Wm4v5oS83rgWgUIC+nOEP0MbzFVI3SyK+LL6eZ730\nwXkRtPrCsVbva9f+kgv2z9NrLI1Iz1+L59/iuin5p+8xNfu1Ps0OWrMrt/RLu/ga7wJqf6OgR7Bb\nL22k8pdz5zwP39hqtsOmdNDa3XbLHyK2NIeUpMDRViuwnmT5i37zLs3XfgtdM0XNAhvA5pshJf7S\nURv4Lb+AYhpfVZOp1Z9InedznlIrYG8a2AB1OmiNGtwxeoQzReOPaf4nSyj3+mftvb/+reffPLAn\nYcFNjcN8h1Li5+RKvhIk/TpRLftQe2qlq8yJ+11uyfHVOg/dAHsSBrSUOEysBNzc3OUMbaubc2uS\nvEtr6XMW7uuNA1ounCVqCeLNfktkEuYPGShx2NhQTO7bJtJc31fLVtNOjaHEtaRawG3xxjWJMzfr\n7lq7M5eq+T+c4UAZ+0cNqdhUTC6fC+4YUP34mE0C7dCcUvYeoZ27abYiKcyp8GgVaFYdeavHi1FX\nSyKc5RCNWIvlkNRSCPcFQHkxSV9kmssjmkskPUgyT+1llFrdIgfGuRpac7GwaanpJRHtZQ5KbMmu\nOjZezkbpvrlj5PKxdVP2nM8iplVZwlyiee6uMTEtwBqg8Q4b2yVbxGp3zhhfyzaqnftuJad56rR9\naXfPOf88ddeaIOaML6lFUdPAdlUL3iV9IXvMFsrVtklBzIW2BFyUmJakccwWubH81rprzrjcY6gF\na4BGl0Sob2ktlk4oSyKxpQGuL/S23reFcjF5FFvpWKofIP98tv4BI0YWnXcr3bVGDQ6crWFtdS6b\n7LDdrteyo+Z2ehYdt5ZN2pFLX5BanTbGr1WjtjjAxfqlsu6uc6/VUh261vyxeVw1CWxfFkDG1k5B\nLOfj1PNtoTiOLVZfOoYF4DFjUGu0KsncuMddq0ss0V1r3ORjMdzrU1NdANtVqe475ovlpepRc3I1\nrLseaq41tDExPULbcs5Wx6sJKYvuOjdG7RipmlzDnkT5rWtMvD8nzvrnVENjHTtlL7mGLZmHdOzc\nHCiK1cb6W5IVzEvduCzGqdVdc7ttCzX9l44UwFrEp24GXHBjYSWBb0t/zk7Jx/gAZB80tgLt1AVe\n+t2E9ju2GksFIVmBt2a33dWSiOVyiBsf81FyNJZWuBdDiWUU6QvSElity3Kpg3pea51LardssXyi\nBfSS3XazSyLWyyGpHOvuORdvufRRo/vmdLWSTpmSG3oNxV5XsecCk495d+DnUqUFit466ZA0AM+J\nsT7+ZjtsSTdN7ahrdM+5Ojkbt4uu1X1TzxfWz/HFaqdsrj32usHYrN5ZaL0rsQCOdTetcewanXSJ\nm1WzwPYlgTEV+DFfSbsE5CXzpLaUHSMJxFqSFnApuRKIt9xxU89li510TE1/6BhT6GTVXBLRtGss\nh5TOk9pS9ti8KLktfMgokQXMawGndLetcW5aAnqza9iuqOvZnBytr+PF7Jrfquglj2JL2THqGdpW\n7zCotUu9o6opDcDXXBrpYkmEuhzi51guicRqUWpwlifmcckkZc/5cmoRHgAy4HJzWzkXLXbXvmqM\nmVIXwHblgxh7gig5VLhKAT3ZMWNRY2J5uRjNPIptspcGeg1pdsdSleikaz8/Ld4QqDIDdgis1H/c\nsbRyKJ21Vhees5WM0cyj2HI+LrRrA4MiKcxrH6v1+C3CtsQ5b7rD1gC4Ro4U0NTYnK1naFM1r9CW\nzEMC65I33FRcC4DVnE9o36JJ
}
],
"prompt_number": 90
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's put this all together into a gradient descent learning algorithm. \n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"theta = 0.0\n",
"for iter in range(1000):\n",
" pred = sigmoid(filters.convolve(image,filter)+theta)\n",
" err = sum((pred-out)**2)\n",
" delta = (pred-out)*pred*(1-pred)\n",
" delta /= prod(image.shape)\n",
" dw = array([[sum(delta*roll(roll(image,i,0),j,1)) for j in range(-r,r+1)] for i in range(-r,r+1)])\n",
" if iter%50==0:\n",
" print iter,err,\":\",(amin(pred),amax(pred)),sum(abs(delta)),\n",
" print (amin(dw),amax(dw)),(amin(filter),amax(filter)),theta\n",
" filter -= dw\n",
" theta -= sum(delta)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"0 51495.0 : (0.48624954, 0.52211583) 0.10982 (-0.0013520373, 0.0032668943) (-0.029455993, 0.026091032) 0.0\n",
"50"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 4397.94 : (0.072140738, 0.36709112) 0.0162744 (-0.00070879137, 0.0011495821) (-0.10338775, 0.063890524) -1.71587491222\n",
"100"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 2627.21 : (0.031862784, 0.45811501) 0.0103208 (-0.00061922497, 0.00093237666) (-0.15040764, 0.088849939) -2.03730937187\n",
"150"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 1910.8 : (0.017386325, 0.56336188) 0.00819295 (-0.00056234887, 0.00077779568) (-0.18951969, 0.11478845) -2.19306036993\n",
"200"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 1473.71 : (0.010572175, 0.65190011) 0.00688511 (-0.00050368829, 0.0006372091) (-0.2218695, 0.13822004) -2.29702537518\n",
"250"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 1181.12 : (0.006968664, 0.71730405) 0.00591087 (-0.00044778202, 0.00052048423) (-0.24828075, 0.16170618) -2.37981959002\n",
"300"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 977.117 : (0.0049024085, 0.76385123) 0.00515023 (-0.00039510831, 0.00043040476) (-0.26993901, 0.18278031) -2.45108571986\n",
"350"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 829.998 : (0.0036344689, 0.79730797) 0.0045502 (-0.00034930339, 0.0003636154) (-0.28797349, 0.20138238) -2.51411203877\n",
"400"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 720.586 : (0.002809946, 0.82201749) 0.00407363 (-0.00031081311, 0.00031881803) (-0.30326092, 0.21787542) -2.57035250252\n",
"450"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 637.084 : (0.0022468881, 0.84082115) 0.0036917 (-0.00027875858, 0.00028317625) (-0.31643328, 0.23260616) -2.62074463977\n",
"500"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 571.935 : (0.0018463387, 0.8555268) 0.00338272 (-0.00025198792, 0.00025411052) (-0.32794094, 0.24586843) -2.66604685766\n",
"550"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 520.144 : (0.001551502, 0.86730045) 0.00313121 (-0.00022944857, 0.00022991268) (-0.33810905, 0.25789988) -2.70691331301\n",
"600"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 478.324 : (0.0013281747, 0.87691414) 0.00292533 (-0.00021028086, 0.00020941216) (-0.34791788, 0.26889029) -2.74391074048\n",
"650"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 444.095 : (0.0011548706, 0.88489538) 0.00275525 (-0.00019381418, 0.00019179737) (-0.35690808, 0.27899081) -2.77752619621\n",
"700"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 415.747 : (0.0010175834, 0.8916145) 0.00261426 (-0.00017952557, 0.0001764989) (-0.36504495, 0.28832325) -2.80817601702\n",
"750"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 392.03 : (0.00090688199, 0.89733911) 0.00249657 (-0.00016702792, 0.0001630753) (-0.37244996, 0.29698664) -2.83621535654\n",
"800"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 372.01 : (0.00081623549, 0.90226686) 0.0023978 (-0.00015600196, 0.00015120726) (-0.37922135, 0.30506241) -2.86194675113\n",
"850"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 354.975 : (0.00074100756, 0.90654665) 0.00231418 (-0.00014620741, 0.00014063941) (-0.38543916, 0.31261787) -2.88562904819\n",
"900"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 340.375 : (0.00067783217, 0.91029274) 0.00224326 (-0.00013745073, 0.00013117767) (-0.39117, 0.31970966) -2.9074842775\n",
"950"
]
},
{
"output_type": "stream",
"stream": "stdout",
"text": [
" 327.774 : (0.00062422006, 0.91359425) 0.0021827 (-0.00012957772, 0.00012265667) (-0.39646918, 0.32638583) -2.9277037502\n"
]
}
],
"prompt_number": 11
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After this training, let's look at the output.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"figsize(12,6)\n",
"subplot(121); imshow(out,interpolation='nearest')\n",
"subplot(122); imshow(pred,interpolation='nearest')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 12,
"text": [
"<matplotlib.image.AxesImage at 0x40dc350>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAsEAAAFOCAYAAABnv8DkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsfUmPnNd19lPzPFdXzxPZHEVSNEVKtOxIERxHQRwPcbxI\nkOyC7PMHssgym6yyySIIkAQwYiAeYMCZZEvyp4mSbFGkRFJks9nseaqha57rW3Sey1O371tdpEhR\nEusAje6ueoc7nvOc8do6nU4HAxrQgAY0oAENaEADGtATRPbH3YABDWhAAxrQgAY0oAEN6LOmAQge\n0IAGNKABDWhAAxrQE0cDEDygAQ1oQAMa0IAGNKAnjgYgeEADGtCABjSgAQ1oQE8cDUDwgAY0oAEN\naEADGtCAnjgagOABDWhAAxrQgAY0oAE9cfRIQPB//dd/4fjx4zhy5Aj+7u/+7lG8YkADGtCABvSQ\naMCzBzSgAT2JZHvYdYJbrRaOHTuGV155BePj47hw4QJ++MMf4sSJEw/zNQMa0IAGNKCHQAOePaAB\nDehJpYduCX733XcxNzeHmZkZuFwu/Omf/il+9rOfPezXDGhAAxrQgB4CDXj2gAY0oCeVnA/7gaur\nq5icnFT/T0xM4NKlS13X2Gy2h/3aAQ1oQAP6TOnLctjmgGcPaEADehLIxLMfOgh+EGZps9ngcrng\ndrvh8Xjgdrvh8/nQarXQaDRQqVRQr9fR6XRUJ+x2O4LBIEKhEL7yla9gdHQU6XQaP/nJT1Aul/e9\nY3p6GjMzM/B6vbh27Rqy2SyKxaKxPeFwGF6vF06nE8ViEfV6Hc1mE3a7HU6nE+12G+12G61WC61W\n6777+zDIZrPBbrd3vd9ut8Nmsz20NrndbjgcDvh8PjgcDnQ6HVQqFTQaDdTr9Z7tstvtaLfbcDqd\nqNVqXd87nU64XC7YbDZ0Oh00m000m0202201xsCem7bT6cBut8PtdsNut6PT6ajPudY6nU7XO3r1\nx+PxYHx8HKOjo5iamsL169exvLyM9fX1A++32+2qD6R2u63a6HK50Gq14HA4+mqPHBOuf44J1xbH\npRc5HA4AeGjz3msdORwONcfAXv+bzWbXNU6ns2tu9O8fhOTzDiKHwwGn0wm32632Kvcv+1apVL40\nIPbTUr8822azqbXmdDrhdDq7xpRrotPpqHHX7/d4PPB6vZidncXIyAjy+TyuXr2KTCaDZrOp5sTl\nciEWiyEUCsHtdmNnZwe1Wg3VanXfvJGn2Gw22Gw29W5eJ/snZYhp/uX3ANBsNuFyuYzX8338ju3n\nu/m93W6Hw+FQY2dFVnJFPkdeR+IccE54LXkJ29NqtbrGRd7LftTrdbhcLmNf+X7TOJnWkD4+kni/\n5HX8n/c5nU44HA74/X6Mj49jeHgYqVQKt27dQiaTwfb2NhqNRtezD2qT3j722e12q3s5Bxw72X/u\nAX1OTP0ytcnUNr6HvN60duRY9jPOUj7q93U6HTXP+v0kq/0j+yg/t/q71xjIa9kGfVz4nZT7+j4j\ncX7Il/R9ZCWHHjoIHh8fx/Lysvp/eXkZExMTPe/pdDpoNBpqI7PxXIAOh6OrUwQd9XodtVoNhUIB\nExMTSKVSeOGFF/DRRx+hUCigUqnA5XJhYmICp06dQiAQwMbGhgK1VtRoNBQIkCDEtFgIzD5rgUqG\n1263lYJwEFi6XyKDIZjtR1hKZsuxMX3PuZaCC7jHgIB7oI5MXwoB4N666Hfs6/U6RkZGcOrUKRw5\ncgRHjx5FNBpFLBZDpVJBLpfrq39yPUjGKTctlYB+iEyp3W4bGZqJMUh62IpYr3ZzHjh/drsdlUql\na78+DNCrUz9zrCsnklny537A9JNC/fJsfc9Kga+DYP7Wr2k2m6jX66hWq3C73UgkEpiYmFD8nLzX\n7/cjlUrB6/WiUCh0rS8THQQ6epEOik08y7QHKYdMfeUP+QAB8EHrT/IUq2tMIMsEvg7qswTJck71\nfso2W8kAq/sOkhlW40aZQAPI1NSUStz0+Xy4efMmdnd3uxQnSf3KKh0oyjZIZYPfS3kjx1vvq2nM\n9DEyvVMfg4N4vwn8mt6ht5H7tdeaNwFq07v0dWN6jj7W97OOpIyVCq6+3rmO+Ztyitd8ZiD4/Pnz\nuHXrFhYXFzE2NoZ///d/xw9/+MMD7+t0OqhWq6hWq3A6ncoiLJkMGSip0WigXC6rQRkZGcHFixfx\ne7/3e2oAAKBcLmN5eRk7OztYXFxEJpNBo9GwbEulUkGlUtn3OS2WcmA5oZ+1RbjRaPTsw8MgKiel\nUgnVahUAFFjr514rS3m73UatVrO0llrdo7/3QUB/MBjE4cOHcfjwYUxOTiKfz6Ner+P27dvI5/PG\nZ9LKq1tl9Y3PtSkZ2P0IZNMmlRYI+beuKH5WJIUD++ZyuRAIBFCr1Yz75rMiaf2iF2lAB1O/PFtX\n0HTAAMAICuVaabfbaDQaWF5ehsPhwNDQEJ599lmEQiHY7Xa1x4rFIra2tpDJZLCysoJqtaoMIJL4\nP3mGBOfyfyuhy3abhKrJkib7I/e5XHu9yAqsSu+ZlSWtH6DB+/sBWbL98lqd/1pddxBYN1Ev67D+\nTmn8oLI0PT2NnZ0d5PN5zM/P7xtPqYzpipoO6k3jR9J5nL4uTNZ46SUzAT65B0zP7If0e/pZb/J6\n3cIsgaVO/Vp0rdpgejbXue6B4HP08Za/Ke9M46e3537W5EMHwU6nE//wD/+Al19+Ga1WC3/5l395\n31nGdEFJIGES+ARTmUwGd+/eRalUgsvlwsjICDweD5xOJ5rNJtLpNBYXF7G9va1cKA9KejvuF+h8\nVvQwLdSfBeD+rKhQKGBlZQXtdluti+XlZUsA7HK5lPWrXwvnw1SIpGCT7lTdKv5ZkGRYrVZLrf3H\n5Q3pRZ+1cvBFpn55thRaJmuYDFMyAS5Sq9VCuVzGxsYGqtUq/H6/CoFzu92o1+uo1+vY2dlBOp1G\nqVTqCYAloLO6xkSmPpj6DJgthf0AYanAWr1P7iMJTmQbdSuefJ7Jmmn6fdA4yLZIsrJO93pPL1Cn\nz5W0SuqyFdgzLqyursLr9aJWq2F+fh6bm5vKU2kCuFbzpb+X434//MK0rvlbB3ZSIZF/m8aK/F0f\nlwcxqMjn6u0GoEIO9c/lff1Ya3t9bxp7qaTpc2alWPFzXSkxWd/1fWKl+Eh66CXS+iGbbS82jG58\nnex2O3w+H5xOJzqdjnKf9SKHwwGv14tIJIJgMAiXy6WYSbFYxPb2Nsrl8udKUPeieDyOoaEhzM3N\nwel0olQqYXFxEfl8HrlczjIml0Q3UiKRQKfTQTqdNlpeP68g/tNQKpVCMpnExMQEcrkcMpkM7ty5\no8BpNBpVXgZaua1Avt/vh91uR7VafSRu/gclPe6WiiK/m56extGjRxEMBlEsFpFOp3Hr1i3kcrkH\nYqameECGo3B8DnoGY9AOWruPiy5evIjh4WF4PB7cvXsXm5ubWFxc3HedzsSfBLLZbAiFQvvAmQkE\n8nOT21IKJYZ0+Xw+BAKBrjjuYrGIYrGIRqOxLySN1At89WttNP0tSQc1DNfSw2tcLheCwSB8Ph8i\nkYiSW+xDo9Houk8HtxwPCYKbzWZXzCv3G4CuUB8Zq9prbKz6bkW9rJkmoNPPs0x/cw0wZ6NWqymZ\nL0MiaJDgmqjVal3KkXSBs21yDZr6ZTVOMo6aa1wqKVyTeuw2vzdZg60sq7wnFAohFovB6XSiXC6j\nWq2iUCgo76u+j0wA0GrMTZ/LPurfyXHk+6z2sZXiqcdV83pazGWscy+FlM/Qx8DtdmNoaAiRSARe\nrxfpdFp58XVZbbPZUCqVjOv0oVuC+yWrWB5gb3JKpdJ9Pa/VaqFUKhnv+yICPZ/Ph2Qyieeffx7D\nw8PY2dnBL37xiwPjmUntdhvVahW5XA6hUAjRaBTlchn1el250p1OJyKRCGy2vUQhq0TBh0FSUHLT\nm5JcHgbFYjFMTk7iwoULKBaL
}
],
"prompt_number": 12
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"figsize(12,6)\n",
"subplot(121); imshow(out[100:200,100:200],interpolation='nearest')\n",
"subplot(122); imshow(pred[100:200,100:200],interpolation='nearest')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 13,
"text": [
"<matplotlib.image.AxesImage at 0x5963f50>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAArsAAAFNCAYAAAAAQKcbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnduvJeV55t/dB2ho6PNh0ye6gW6aBjvAwETyjCZOUGM5\nCoSJLWuSyEGW41xYkZwoUpS/wDTKxcSZmasoilqJFMdz47EijydBjqPx+JBg4mCbhoamN/SZQx+g\nTzR077kgq/itYj2131q1ah9qPT8J6eu116r66jtVUc/zve/E9PT0dBhjjDHGGNNBFs11BYwxxhhj\njGkLP+waY4wxxpjO4oddY4wxxhjTWfywa4wxxhhjOosfdo0xxhhjTGfxw64xxhhjjOksjR52v/Wt\nb8Xu3btj586d8eSTT46qTsYYY1rAa7YxZhyZGDbO7tWrV+POO++Mp556KjZv3hwPPvhg/PVf/3Xc\nddddo66jMcaYhnjNNsaMK0uG/eE//dM/xR133BHbt2+PiIj/8l/+S/yv//W/+hbOiYmJxhU0xpi5\noks5d7xmG2O6jlqzh37YPXbsWGzdurX495YtW+KHP/zhh763ffv22LFjRzz//PPFZydOnBj2tK3B\nRZ6L/y233FKUb7zxxqL83nvvFeXXX3+9KB86dCguXboUN9xwQ5w5c2YkdWMdbr755qJ85cqVonz2\n7Nm+31y4cKEoL1nyQTevX7++KPduehERO3bsKMqbNm0qyqtXrx54HB7/+PHj8c///M/x4IMPxk9+\n8pPi8x/96EdF+dKlS4MurTbLly8fWJ+I/j5h/drm+uuvL8pLly4typx0165dG/hbjrvFixcPLC9a\n9IHbqDyRec0cD++8806q7oPqccMNNxTlZcuWFeXrr78+3n777bj55pv76qeuk5/z+Oy36667riiz\n7fid8jXz2s6dO1eUOQ/Nh8mu2bt374433nijrw/Onz9flN99992ifPXq1aJc938Mst9Xc4Tzjuvi\nihUrijLXC84Vjpu33nqrWLO5TvH7qq6sD9tLrQmcy+U1Qc0XHovXtnHjxqLMe8TKlSsH1vXNN98s\nykePHo1XX301tm3bFidPniw+5z0r0xasJ6+NbaHKZXgONcbUeGM9VJ0WL14cV65cieuuu67vc5YJ\nj6P6f5j/OVTH4ueZuaGus1y+ePFi3HjjjX2fq/bKkDlv+ZjsN/Yt13Lev/h9de8cdO7Lly/Leg/9\nsJttoDNnzsS1a9fi8uXLsWzZsr4b6XyCD7JsdC6KqiHZGcuXL4+rV6/G8uXLR/awyxuNeiBi/SP6\nH1hY5sMuy2vWrCnKGzZsKMpcRLnQcsCuWrUqpqamYs+ePX0L1ltvvVWUn3322YHXVpdhHmIzi1Zd\nlixZEteuXYtFixbJB7/M4qXmkXqALE989XBZdVPlNfTgDbX8gNtj6dKlsXjx4rjuuutSDyA33XRT\nUeb/NPFmzAcBzi/emHkzLv9tVHNsHMiu2efOnYvLly/HkiVLYtmyZbFs2bK+ea2OqcaiIjsXOZY5\nXjL/o8SxybpyjF+5ciXefffdWLZsmXyYyvzPqiqzPurBt+o3nEfqZcUdd9xRlPkQzPnIeXPw4MF4\n77334sEHH4znnnuu+Pzw4cNF+fTp00WZc5NtRKoeuHqwPyL6r5P3Kba9elDimFT9QyYmJuK9996r\nfOAmbak6TR6i1YNm1QP+xMRELF68WPYDy5n5pc7FfiqvF3yorXp504P9XNW3165di2vXrqX6auiH\n3c2bN8eRI0eKfx85ciS2bNnyoe+tXLkytm3bFsePHx/2VMYYYxqSXbM3bdoUx48f73tQMsaY+cai\nRYuK/yL6H5I/9N1hT/LAAw/Eiy++GFNTU3HlypX4m7/5m3j00Uc/9D0lqXSZcbxJ7Ny5c66rMOuM\no7+Rb8TMwiK7Zkf02wLGAb7FGhcG/Y9O18m+1e0S4zi2BzF0zy9ZsiT++3//7/GJT3wirl69Gp//\n/OcH7updtGhRvP32232vuCm5X7x4cdgqjBS+yucrd0rxnCh80CnLKUuXLo1r1671HVNJPxloY6AE\nwIfq8oNXWS7qwWvjcSkHUzZjX1X5NG+//fa4du2a9H6uWrWqKJf9xW1Ar96tt95alGnR4IOb6nPW\nlRYKynpN/GUZj2uV7MXfKxlUyVdcBDOS8KJFi/radRCsA4+jbDWsG/uA7cv+iJid8dNFsmv21atX\n48Ybb+wbv+xLfs7+VuM941Esk/GCcixzHiivKeHnixYtKsYkj69k2cz1qN9WeUUzlgieg8flNXM9\n4j2Cc/zmm2+OPXv2FOUe6t7M9s34d7PWMbY368F7kLI38P7FMtcOrinvvvtuMY6VJ3RU1oXsi5CM\ndzYzLqr8soMednnNyrOs7EPsM2WxKz/vqDGjPO9N1hVFo//N+eQnPxmf/OQnmxzCGGPMLOE12xgz\njjiDmjHGGGOM6SytG1h60QwoLaidgE2k/mFQsgGjMahX7oSv7/lbtVOxLqwD25HnLdeN56aEoMK7\n8LgsUx7ijnqids7PZviviP7IEfQQ9+S6iP4Qa4wuQcnu6NGjRfnQoUNF+ZVXXinKvE7Vdk3IhnRR\n31M7wFVZzUmiZCqWlRWBUU2U75fff/vtt4tyOWwdZTmeW7UT6121gcG8T6/tuZYRNT4yu6yHCTdW\ntZu/h9qlr8aEirSgoozUlU/VmFNWh4j+OavsF+pew/ny2muvFWXaAXjMN954oyhzbqq2U17XTDQB\ntdu/XL/JycmizEgTXNf5e0aLOHbsWFHmJkxeJ9tIRXLIWBqaWA+q/jaqMqmyFvTg+OSYYplrsJqD\nVdEYMtYXdT9S95q69hO/2TXGGGOMMZ3FD7vGGGOMMaaztG5jGCSJ8ZW2klvryqo8Zvl1fUamUvKQ\n2nmspI+2ZVJ1/PKOSvVaX+1gpeWAcg8tCpScCH9LCY0yU92MXtyBy/MylB0jK0REX3Yo2hh27dpV\nlLdt21aUuVOZdWU/M0oFY0VzXIzKukDU+Kraaasyk6lIC2oXrpK+lNxFiWtUmfI4N2k3iejvd/Yh\nf8O60qKixnyX0gI3pddemUghKtMfyUi9VWS+VzeTW+aYmfuOahdSZV0gypLDscz5xbHM9Ys2LK6j\nrB+PoyxDmfsmyWRJLM9lWhe4Zu/evbsoM6Mn26WcbKYHo7WwXdTzgrI01LUxZDKLlf9d91iqHspm\no9byutep5rl6Tquy26moC4R9Unf+K/xm1xhjjDHGdBY/7BpjjDHGmM7Suo2hJxVS9lTyiMrLzN3b\nSrYl5VzKPDclZ35OqUhZBZQc1TQxQB2U3aKqrgoVfFsF9Fe76JVkzOOoduHn5aDnPShbM+vP5s2b\n++rBv7G8du3aokyphfWj3EXrBsuUx7iDeTapSiqRCb6vrDhqJ62aF5nA/U1QCQ0i+vuTFhdeDyXe\nzHw2H9Brr4xET9Q4U5KukmfLqN3Yag2uK9GqOinpVp0rQ5WlQQXSJ2qdpiWHa1MmioKa+5nEA5mo\nC5yja9as6fv9+vXrB/6NyS0I1x1eJ9dvfs77UTnBRI8mSQsy1gNl+yj/Rn1eNyFRJmpO3flC1LWp\nSD8R/XYzlfBI1TtzbY7GYIwxxhhjxho/7BpjjDHGmM7Suo1hpp3qfP2srA581c3X4ZTVlcRergMl\nDiXjjwpeG3fFUqJRO8hZN7XDvakMyzamDEYpizIQbQZVOy97UMqgLYF9yHbn9aid//x+2cbBa3j9\n9dcHfs5zMNICA5EzkcTU1FRRZjSGNqT7YWA7ZfKUq7Jqb/X5bFI1ztkPrB9tDJzz/NwMZtDYrhtR\nQVkAuCYoOTOif2yqSD4qAk9GllVSbGYHesbOpuqQXbMz31NjX+32zySqUd/P2NBUndkW5ecBWsOY\ntIf2OdaD8/fEiRNFmREoVCIJFWUpM14y1z9MdBHVxpl6KHlffZ4p1+3PrAWCcz0zZ5R9Tt3vbGMw\nxhhjjDFjjR92jTHGGGNMZ2nd
}
],
"prompt_number": 13
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"imshow(filter,interpolation='nearest')"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 93,
"text": [
"<matplotlib.image.AxesImage at 0x12377150>"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAFpCAYAAAC4UhIKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE4FJREFUeJzt3W1MlXX8x/EPAklE4E3cpAemSwgO3kBqbk7X0qjlpvOG\nmnfJ1OpBualzcz7ugeF6kJj2xC1Hy2mtdUPN2HLW0hyZw8olqXMcPd62RVAEihyv/wP/fyb97SAX\n55wfX3m/HnGO51zfL8veXh4510nyPM8TAGDQG+Z6AQDAvSHYAGAEwQYAIwg2ABhBsAHAiJR4HDQp\nKSkehwWA+160H9yLS7Alqba21tfzPv30Uy1atGhAszMzMwf0fL+ysrJ8Pa+2tlZVVVUDmv3II48M\n6Pl++f3D+d1339Vrr702oNnp6ekDer5fN27c8PW8Xbt26fXXXx/Q7NbW1gE936+2tjZfz/vggw+0\ncuXKAc1uaWkZ0PP9unXrlq/nffLJJ1q8eLGv57700ktRf52XRADACIINAEYMumAXFxe7XiHhpkyZ\n4nqFhJs+fbrrFRJuKH7PkydPdr1CwpWUlMTt2IMu2PH8ZgersrIy1ysk3FCM15NPPul6hYQj2LE1\n6IINALg7gg0ARhBsADCCYAOAEQQbAIwg2ABghO9g19fXq7i4WIWFhdq2bVssdwIA3IWvYEciEa1b\nt0719fU6deqU9u3bp6ampljvBgC4g69gHzt2TBMmTNC4ceOUmpqqpUuX6vPPP4/1bgCAO/i6Wt+l\nS5eUn5/fczsQCOiHH37o9ZhPP/205+vi4uIh+Q5GAIimqampX69O+Ar2vVxSc6CXSAWA+11JSUmv\nk9k7T3TvxtdLImPHjlU4HO65HQ6HFQgE/BwKAHCPfAV72rRpOnv2rEKhkLq6uvThhx9qwYIFsd4N\nAHAHXy+JpKSkaOfOnXruuecUiUS0du1aXqMGgDjz/RFhzz//vJ5//vlY7gIAiIJ3OgKAEQQbAIwg\n2ABgBMEGACMINgAYQbABwAiCDQBGEGwAMIJgA4ARBBsAjPD91vQ+D5wSt0MPWsOHD3e9QsLl5eW5\nXiHhuru7h9zstrY2J3MlKS0tzcncjo4OJ3Oj4QwbAIwg2ABgBMEGACMINgAYQbABwAiCDQBGEGwA\nMIJgA4ARBBsAjCDYAGAEwQYAIwg2ABhBsAHACIINAEYQbAAwgmADgBEEGwCMINgAYATBBgAjCDYA\nGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABgBMEGACMINgAYQbABwIiUeB14+PDh8Tp0n7q7u53M\nvXXrlpO5kpSTk+Nkbnp6upO5kpSamupkrud5TuZKUmdnp5O5Y8aMcTJXks6dO+dk7rBhg+98dvBt\nBAC4K4INAEYQbAAwgmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAIgg0ARhBsADDCV7DD4bCefvpp\nlZaWauLEidqxY0es9wIA/Iuviz+lpqbq7bffVllZmdrb2zV16lRVVFSopKQk1vsBAP6XrzPsvLw8\nlZWVSZIyMjJUUlKiy5cvx3QxAEBvA768aigU0okTJzRjxoxe93/00Uc9X5eWlqq0tHSgowDgvnLq\n1Ck1NTXd8+MHFOz29nZVVlaqpqZGGRkZvX7txRdfHMihAeC+FwwGFQwGe25/8sknUR/v+6dEbt68\nqSVLlmjlypVauHCh38MAAO6Rr2B7nqe1a9cqGAxqw4YNsd4JAHAXvoL9/fff64MPPtA333yj8vJy\nlZeXq76+Pta7AQDu4Os17FmzZjn9/EIAGIp4pyMAGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABg\nBMEGACMINgAYQbABwIgBXw/7v3R3d8fr0H1KS0tzNtuV1tZWJ3MzMzOdzJVuf/KRC52dnU7mSu5+\nb58/f97JXElKT093MncwXn6DM2wAMIJgA4ARBBsAjCDYAGAEwQYAIwg2ABhBsAHACIINAEYQbAAw\ngmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABgBMEGACMINgAY\nQbABwAiCDQBGEGwAMIJgA4ARBBsAjCDYAGAEwQYAI1LideC2trZ4HbpPN2/eHFJzJam7u9vJXJff\nc0pK3H77RpWUlORkriT99ddfTub+/fffTuZKUnt7u5O5nZ2dTuZGwxk2ABhBsAHACIINAEYQbAAw\ngmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAIgg0ARvgOdiQSUXl5uebPnx/LfQAA/8F3sGtqahQM\nBp1eCAcAhhJfwb548aIOHDigl19+WZ7nxXonAMBd+Lo+5caNG/XWW29FvdTjF1980fN1UVGRHn/8\ncT+jAOC+9dtvv+n06dP3/Ph+B/vLL79UTk6OysvL9e233/7n43htGwCiKy4uVnFxcc/turq6qI/v\n90siR48eVV1dncaPH69ly5bp0KFDWrVqVf83BQD0S7+DvXXrVoXDYTU3N2v//v2aM2eO3n///Xjs\nBgC4w4B/DpufEgGAxBjQh+I99dRTeuqpp2K1CwAgCt7pCABGEGwAMIJgA4ARBBsAjCDYAGAEwQYA\nIwg2ABhBsAHACIINAEYQbAAwYkBvTY+mtbU1XofuU1tbm5O5169fdzJXktLS0pzMfeCBB5zMlaTh\nw4c7mTsUr59z48YNZ7Nd/X8ViUSczI2GM2wAMIJgA4ARBBsAjCDYAGAEwQYAIwg2ABhBsAHACIIN\nAEYQbAAwgmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABgBMEG\nACMINgAYQbABwAiCDQBGEGwAMIJgA4ARBBsAjCDYAGAEwQYAI1LideCWlpZ4HbpP//zzj5O5f//9\nt5O5ktTW1uZkrsvvOTk52cnc69evO5krSaNHj3YyNzs728lcScrKynIyNy0tzcncaDjDBgAjCDYA\nGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABgBMEGACMINgAYQbABwAjfwW5tbVVlZaVKSkoUDAbV\n0NAQy70AAP/i++JP69ev17x58/Txxx+ru7vb2QWXAGCo8BXstrY2HT58WLW1tbcPkpLi7IpaADBU\n+Ap2c3OzsrOztXr1av3888+aOnWqampqlJ6e3vOYI0eO9HxdUFCggoKCgW8LAPeRUCik8+fP3/Pj\nfQW7u7tbjY2N2rlzp6ZPn64NGzaourpab7zxRs9jZs2a5efQADBkjBs3TuPGjeu5/d1330V9vK9/\ndAwEAgoEApo+fbokqbKyUo2NjX4OBQC4R76CnZeXp/z8fJ05c0aSdPDgQZWWlsZ0MQBAb75/SuSd\nd97RihUr1NXVpccee0x79uyJ5V4AgH/xHewpU6boxx9/jOUuAIAoeKcjABhBsAHACIINAEYQbAAw\ngmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAI329N70tnZ2e8Dt0nV59+8/vvvzuZK0mXLl1yMvfC\nhQtO5kru/juPGjXKyVxJys7OdjL3+vXrTuZKUldXl5O5mZmZTuZGwxk2ABhBsAHACIINAEYQbAAw\ngmADgBEEGwCMINgAYATBBgAjCDYAGEGwAcAIgg0ARhBsADCCYAOAEQQbAIwg2ABgBMEGACMINgAY\nQbABwAiCDQBGEGwAMIJgA4ARBBsAjCDYAGAEwQYAIwg2ABhBsAHACIINAEYQbAAwgmADgBEp8Tpw\nenp6vA7dp1u3bjmZO2yYuz//urq6nMyNRCJO5krSzZs3nczt6OhwMleSHnjgASdzhw8f7mSuJCUn\nJw+pudFwhg0ARhBsADCCYAOAEQQbAIwg2ABgBMEGACMINgAYQbABwAiCDQBGEGwAMMJ3sN98802V\nlpZq0qRJWr58uW7cuBHLvQAA/+Ir2KFQSLt371ZjY6NOnjypSCSi/fv3x3o3AMAdfF38KTMzU6mp\nqero6FBycrI6Ojo0duzYWO8G
}
],
"prompt_number": 93
},
{
"cell_type": "code",
"collapsed": true,
"input": [],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 93
}
],
"metadata": {}
}
]
}