mirror of
https://github.com/saymrwulf/nvidia-AI.summerschool17-notebooks.git
synced 2026-11-21 09:54:43 +00:00
554 lines
443 KiB
Text
554 lines
443 KiB
Text
|
|
{
|
||
|
|
"metadata": {
|
||
|
|
"name": "107-bayesian-decision-theory"
|
||
|
|
},
|
||
|
|
"nbformat": 3,
|
||
|
|
"nbformat_minor": 0,
|
||
|
|
"worksheets": [
|
||
|
|
{
|
||
|
|
"cells": [
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"Title: Review of Bayesian Decision Theory\n",
|
||
|
|
"Author: Thomas Breuel\n",
|
||
|
|
"Institution: UniKL"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"Bayesian Decision Making\n",
|
||
|
|
"======================="
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Joint Density)\n",
|
||
|
|
"\n",
|
||
|
|
"In decision problems, we have items with:\n",
|
||
|
|
"\n",
|
||
|
|
"- some measurable property/properties $x\\in R^n$\n",
|
||
|
|
"- some class $\\omega\\in \\{0,...,C\\}$\n",
|
||
|
|
"\n",
|
||
|
|
"These are described by a _joint density_ $P(\\omega,x)$."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Conditional Probabilities)\n",
|
||
|
|
"\n",
|
||
|
|
"The _class conditional density_ is $p(x|\\omega) = P(\\omega,x) / P(\\omega)$.\n",
|
||
|
|
"\n",
|
||
|
|
"The _posterior distribution_ is $P(\\omega|x) = P(\\omega,x) / p(x)$."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Bayes Rule)\n",
|
||
|
|
"\n",
|
||
|
|
"These two combine into _Bayes Rule_:\n",
|
||
|
|
"\n",
|
||
|
|
"$$p(x|\\omega) P(\\omega) = P(\\omega|x) p(x)$$\n",
|
||
|
|
"\n",
|
||
|
|
"or\n",
|
||
|
|
"\n",
|
||
|
|
"$$P(\\omega|x) = \\frac{p(x|\\omega) P(\\omega)}{p(x)}$$"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Optimal Decision Rule)\n",
|
||
|
|
"\n",
|
||
|
|
"The _optimal decision rule_ under a zero-one loss function is \n",
|
||
|
|
"\n",
|
||
|
|
"$D(x) = \\arg\\max_\\omega P(\\omega|x)$"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Justification for Optimal Decision Rule)\n",
|
||
|
|
"\n",
|
||
|
|
"The reason for this is that the probability of error for each $x$ is\n",
|
||
|
|
"\n",
|
||
|
|
"$$P(\\hbox{error}|x) = 1 - \\max_\\omega P(\\omega|x)$$ \n",
|
||
|
|
"\n",
|
||
|
|
"and the total error is given by:\n",
|
||
|
|
"\n",
|
||
|
|
"$$P(\\hbox{error}) = \\int P(\\hbox{error}|x) p(x) dx $$"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"\n",
|
||
|
|
"\n",
|
||
|
|
"from scipy import stats\n",
|
||
|
|
"from scipy.stats import norm"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"prompt_number": 1
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(probability density functions)\n",
|
||
|
|
"\n",
|
||
|
|
"We previously computed with distributions by sampling from them. Let's compute now directly with the probability density functions, like $p(x)$.\n",
|
||
|
|
"\n",
|
||
|
|
"Remember:\n",
|
||
|
|
"\n",
|
||
|
|
"$$ p(x) \\geq 0 $$\n",
|
||
|
|
"\n",
|
||
|
|
"$$ \\int p(x) dx = 1 $$"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# normal density\n",
|
||
|
|
"x = linspace(-8.0,8.0,1000)\n",
|
||
|
|
"plot(norm.pdf(x))"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 3,
|
||
|
|
"text": [
|
||
|
|
"[<matplotlib.lines.Line2D at 0x3130090>]"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD9CAYAAACx+XApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10U2WeB/BvIFFGEIFaCiSVQlOaFNpSSXlRqxmldGAk\nQ4tnpspBD9PFLh6G47w5O7uzK3VmkK7LnGHs2bPVRUfEKZ0d1KJiZDqYAYG2gy3yUl5SppVQKDJI\nBQrYNt7945JI30LaJnnuzf1+zumxae69+eVa+s3zcp+rkyRJAhERadYQ0QUQEZFYDAIiIo1jEBAR\naRyDgIhI4xgEREQaxyAgItK4mwaB0+mExWJBUlISiouL+9zub3/7G/R6PbZs2dLvfYmISBxdoOsI\nvF4vkpOTUVlZCaPRiMzMTJSVlcFqtfbYLjs7G7fddhuWLVuGxYsXB70vERGJFbBFUFNTA7PZjISE\nBBgMBuTn56OioqLHdi+++CIeeeQRxMbG9ntfIiISK2AQNDc3Iz4+3v/YZDKhubm5xzYVFRVYsWIF\nAECn0wW9LxERiacP9KTvj3ogTz/9NNauXQudTgdJkuDraQpm3/5sR0REXYVqhaCALQKj0QiPx+N/\n7PF4YDKZumzz8ccfIz8/H5MmTcKWLVvw1FNPYevWrUHt6+MLEK1/Pfvss8JrUMqXqHOxbp2E1FQJ\nLS3y42PHJMTHS3jrLe2dCyV+8Vx8/RVKAVsENpsNbrcbTU1NmDBhAsrLy1FWVtZlm7///e/+75ct\nW4aFCxfC4XCgs7PzpvsSKcnhw8DzzwP79gFxcfLPpkwB/vhHwOEAZs8Gxo0TWyNROARsEej1epSU\nlCAnJwcpKSn43ve+B6vVitLSUpSWlgY8cF/7EinVqlXAs88CEyd2/fns2cDSpUBRkZi6iMIt4PTR\niBRwfWyBAJfLBbvdLroMRYj0udi9G3j8ceDYMUDfSzv5/HkgORn429+ASZMiVhYA/l7ciOfia6H8\n28kgIALwne8A3/oWcH3yW69++lP5vy+8EJmaiAJhEBCF0KlTQHo64PEAt93W93ZNTYDNBnz6KTB8\neMTKI+pVKP92cq0h0rxNm4BHHgkcAgCQkABkZgLvvBORsogihkFAmiZJwMaN8vhAMB59FPjDH8Jb\nE1GksWuINO3jj4HvfhdoaACCubbx4kUgPh5obATGjAl/fUR9YdcQUYi8/bbcLRTsBe4jRwLZ2cBb\nb4W3LqJIYhCQpm3dCixc2L99cnM5TkDRhV1DpFmffgrMmAGcPQsMHRr8fv/4B5CYCHz2GXDrreGr\njygQdg0RhcC77wLf/nb/QgAA7rwTSEkBdu0KT11EkcYgIM1yOoEFCwa274IFwHvvhbYeIlEYBKRJ\nnZ3yJ/pvfnNg+8+fLwcJUTRgEJAm1dUBJhMwduzA9s/IAFpa5PEFIrVjEJAm7dgx8NYAII8rZGUB\nf/1r6GoiEoVBQJr04YfAgw8O7hh2O+ByhaIaIrEYBKQ5HR3Anj3AAw8M7jgMAooWDALSnH375OsA\nBrtERHo6cOYMxwlI/RgEpDl79gD33Tf44wwdKt+9rKpq8MciEolBQJpTVSX/AQ+F2bOBvXtDcywi\nURgEpDmhDII5c9giIPW7aRA4nU5YLBYkJSWhuLi4x/MVFRVIT09HRkYGZsyYgR07dvifS0hIQFpa\nGjIyMjBz5szQVk40AKdOAdeuAZMnh+Z4M2fKS1l3dobmeEQiBFx0zuv1Ijk5GZWVlTAajcjMzERZ\nWRmsVqt/m7a2Ngy/ft++gwcPIjc3Fw0NDQCASZMm4eOPP8aYAKNyXHSOIulPfwJeey20q4empMg3\nq5k+PXTHJLqZiC06V1NTA7PZjISEBBgMBuTn56OioqLLNsNvuHnr5cuXceedd3Z5nn/kSUmqquTu\nnFDiOAGpXcAgaG5uRnx8vP+xyWRCc3Nzj+3efvttWK1WzJ8/H7/73e/8P9fpdJg7dy5sNhtefvnl\nEJZNNDChHB/w4cwhUjt9oCd1Qd62adGiRVi0aBF27dqFpUuX4tixYwCA3bt3Y/z48Th37hyys7Nh\nsViQlZXVY//Vq1f7v7fb7bDb7cG/A6IgdXbKawzZbKE97uzZwG9+E9pjEnXncrngCtMVjAGDwGg0\nwuPx+B97PB6YTKY+t8/KykJnZyfOnz+PmJgYjB8/HgAQGxuL3Nxc1NTU3DQIiMLlyBF5obmRI0N7\nXKsVOHkSuHwZGDEitMcm8un+IbmoqChkxw7YNWSz2eB2u9HU1IT29naUl5fD4XB02ebEiRP+cYDa\n2loAQExMDK5cuYJLly4BkAeUt2/fjtTU1JAVTtRfdXXA3XeH/rgGAzBtGvDJJ6E/NlEkBGwR6PV6\nlJSUICcnB16vFwUFBbBarSgtLQUAFBYWYsuWLdi4cSMMBgNGjBiBzZs3AwBaWlqQl5cHAOjs7MSS\nJUswb968ML8dor7V1oYnCAD5uLW1wL33huf4ROHEexaTZtx/P/Dss8BDD4X+2C+9JM8cevXV0B+b\nqDe8ZzFRP331FbB/v3xDmXC4+26564lIjRgEpAkNDfJqo4NdcbQv06YBx4/LVy0TqQ2DgDQhXAPF\nPsOGAUlJwKFD4XsNonBhEJAmhHOg2IfdQ6RWDALShEgFwfUZ1ESqwiCgqCdJ8if1cA0U+2RkMAhI\nnRgEFPXOnAGGDAGuX+geNmlpwOHD8gwlIjVhEFDUO3hQntUTbiNHAjExQGNj+F+LKJQYBBT1Dh0C\nIrW6SWqqHDxEasIgoKgXqRYBwCAgdWIQUNQ7eJAtAqJAGAQU1bxeefnpqVMj83oMAlIjBgFFtRMn\ngLg44PbbI/N6yclAUxOXmiB1YRBQVIvkQDEA3HILkJgot0KI1IJBQFEtkgPFPuweIrVhEFBUi3SL\nAGAQkPowCCiqsUVAdHMMAopaV68Cn34qD+BGEoOA1IZBQFHr6FF54PaWWyL7unfdBVy8CHz+eWRf\nl2igbhoETqcTFosFSUlJKC4u7vF8RUUF0tPTkZGRgRkzZmDHjh1B70sUTpG8kOxGQ4bI1y0cPhz5\n1yYaiIA3r/d6vUhOTkZlZSWMRiMyMzNRVlYGq9Xq36atrQ3Dhw8HABw8eBC5ubloaGgIal+AN6+n\n8HnmGWDUKOBf/zXyr/397wOzZwNPPhn51yZtiNjN62tqamA2m5GQkACDwYD8/HxUVFR02cYXAgBw\n+fJl3HnnnUHvSxROhw8DKSliXttqBerrxbw2UX/pAz3Z3NyM+Ph4/2OTyYTq6uoe27399tv4+c9/\njjNnzmD79u392hcAVq9e7f/ebrfDbrf35z0Q9eroUfkPsggpKUBlpZjXpujkcrngcrnCcuyAQaDT\n6YI6yKJFi7Bo0SLs2rULS5cuxdGjR/tVxI1BQBQKV68Czc3A5MliXt9q5dXFFFrdPyQXFRWF7NgB\nu4aMRiM8Ho//scfjgclk6nP7rKwsdHZ24vPPP4fJZOrXvkSh5HbLIWAwiHn9iROB8+eBS5fEvD5R\nfwQMApvNBrfbjaamJrS3t6O8vBwOh6PLNidOnPAPWNRev2FrTExMUPsShcvRo4DFIu71hw4FpkyR\n6yBSuoBdQ3q9HiUlJcjJyYHX60VBQQGsVitKS0sBAIWFhdiyZQs2btwIg8GAESNGYPPmzQH3JYqE\nI0fEjQ/4+AaMMzPF1kF0MwGnj0akAE4fpTB49FFgwQJg6VJxNfzqV8Dly8DateJqoOgVsemjRGol\numsI4IAxqQeDgKLOV18Bx4+LD4KUFF5LQOrAIKCoc/IkMHp05O5K1hezGfB4eLcyUj4GAUUdkReS\n3chgkKewHj8uuhKiwBgEFHWOHBHfLeTDcQJSAwYBRR0lDBT7cM0hUgMGAUUdJVxD4JOSwhYBKR+D\ngKKO0loEDAJSOl5QRlHl/Hl5gLa1FQhyzcSwunIFiImR1xzSB7yOn6h/eEEZUR98rQElhAAA3HYb\nMH48cOKE6EqI+sYgoKiipG4h
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 3
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# Class Conditional Densities\n",
|
||
|
|
"x = linspace(-8.0,8.0,1000)\n",
|
||
|
|
"p_x_given_1 = norm.pdf(x,loc=-1.0)\n",
|
||
|
|
"p_x_given_2 = norm.pdf(x,loc=1.0,scale=2.0)\n",
|
||
|
|
"plot(p_x_given_1); plot(p_x_given_2)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 3,
|
||
|
|
"text": [
|
||
|
|
"[<matplotlib.lines.Line2D at 0x26b4110>]"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD9CAYAAACx+XApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYVGX7B/DvwIwbrqziDAayDcqqgJKSaBhpSYhZmLll\nSpb5Wpq9Vr9eqDQxfTO1cknLLURfFbQQEXEyRUABd1FU0BFRETUR1IHh+f1xggRlGGRmziz357q4\nYGbOc87NuWDueXYBY4yBEEKIyTLjOwBCCCH8okRACCEmjhIBIYSYOEoEhBBi4igREEKIiaNEQAgh\nJq7JRJCSkgKpVApXV1fExcU1etzhw4chFAqxdevWZpclhBDCH4GqeQRKpRLu7u5IS0uDWCxGQEAA\n4uPj4eHh8dhxQ4YMQbt27TBx4kSMHDlS7bKEEEL4pbJGkJ2dDRcXFzg6OkIkEiEqKgpJSUmPHbd0\n6VK8+uqrsLGxaXZZQggh/FKZCIqLi+Hg4FD3WCKRoLi4+LFjkpKSMHXqVACAQCBQuywhhBD+CVW9\nWPumrsqMGTMwf/58CAQCMMZQ29KkTtnmHEcIIaQ+Ta0QpLJGIBaLIZfL6x7L5XJIJJJ6x+Tk5CAq\nKgpOTk7YunUr3n33XezYsUOtsrVqE4ipf/3nP//hPQZ9+VLnXqxfz9C9O8PFi9zjK1cYevZk+P57\n/uOnvwu6F9r+0iSVNQJ/f38UFBSgqKgI3bp1Q0JCAuLj4+sdc/HixbqfJ06ciOHDhyM8PBzV1dVN\nliXkaRUXAzNmAPv2AU5O3HNiMbB9O/Dss8DgwYBUym+MhBgKlTUCoVCIZcuWISwsDD179sTrr78O\nDw8PrFixAitWrFB54sbKEqIJs2cD77wDeHnVf97NDfj3v4E5c/iJixBDpHL4qE4C+LtvgQAymQwh\nISF8h6EXVN2Lc+eA/v2BixeBDh0ef/3+fcDVFUhMBPz9tRunLtDfxT/oXvxDk++dlAiIwZkyBbC3\nB2JjGz/m22+BI0eAjRt1FxchukSJgJise/cABwfgzBmga9fGj7t9G+jRA8jPB+zsdBcfIbqiyfdO\nWmuIGJStW4EBA1QnAQDo0gWIjATWr9dNXIQYMkoExKD8/DMwcaJ6x44eDSQkaDceQowBNQ0Rg1Fc\nDHh7AyUlQKtWTR9fXc0NKc3IAJydtR8fIbpETUPEJO3YAQwbpl4SAAChEBg5Eti8WbtxEWLoKBEQ\ng5GUBISHN6/MqFHAtm3aiYcQY0FNQ8Qg3L0LSCRc89CT5g40pqoKsLHh5h7Y2movPkJ0jZqGiMlJ\nTeWWjmhOEgAAkYhbbmL3bu3ERYgxoERADEJaGhAW9nRlX3wR2LVLs/EQYkwoERCDkJ7OfbJ/GkOH\ncjUKpVKzMRFiLCgREL0nlwO3bj2+wJy6HBwAa2vgxAnNxkWIsaBEQPTevn3AoEGAWQv+Wp97Dti/\nX3MxEWJMKBEQvZeeDjz/fMvOQYmAkMbR8FGi1xgDnnmG6yx2c3v688jlQJ8+wPXrAO2OSowBDR8l\nJqOoiJsL4OrasvM4OAAWFsDZsxoJixCjQomA6LXMTCAoSDOf4ql5iJAno0RA9NqhQ1wi0ARKBIQ8\nGSUCotc0mQiefZarYRBC6msyEaSkpEAqlcLV1RVxcXGPvZ6UlAQfHx/4+fmhT58+SE9Pr3vN0dER\n3t7e8PPzQ2BgoGYjJ0bv/n3g1Cmuk1cT3N2Bmze5L0LIP1SOGlIqlXB3d0daWhrEYjECAgIQHx8P\nDw+PumMqKipgYWEBADhx4gRGjBiB8+fPAwCcnJyQk5MDS0vLxgOgUUOkEQcOAB98ABw+rLlzhoYC\nM2dys40JMWQ6GzWUnZ0NFxcXODo6QiQSISoqCklJSfWOqU0CAHDv3j1YW1vXe53e5MnT0mSzUK3A\nQCA7W7PnJMTQqUwExcXFcHBwqHsskUhQXFz82HGJiYnw8PDA0KFDsWTJkrrnBQIBQkND4e/vj1Wr\nVmkwbGIKMjOBfv00e87AQCArS7PnJMTQCVW9KFBzzF5ERAQiIiLw559/YuzYsTj792DtgwcPwt7e\nHqWlpRgyZAikUimCg4MfKx8TE1P3c0hICEJCQtT/DYjRyskB5s/X7DkDA4G33+YmqtHEMmJIZDIZ\nZDKZVs6tMhGIxWLI5fK6x3K5HBKJpNHjg4ODUV1djbKyMlhZWcHe3h4AYGNjgxEjRiA7O7vJREAI\nAJSVcQvNaXqv4W7dgLZtgcJCoEcPzZ6bEG1q+CE5NjZWY+dW2TTk7++PgoICFBUVQaFQICEhAeEN\n9gq8cOFCXT9Abm4uAMDKygqVlZUoLy8HwHUop6amwutpl48kJic3F/Dza9lCc42h5iFC6lNZIxAK\nhVi2bBnCwsKgVCoxadIkeHh4YMWKFQCA6OhobN26FevWrYNIJEL79u2xadMmAMC1a9cQGRkJAKiu\nrsaYMWPwwgsvaPnXIcYiN1dzw0Yb6t0byMsDRo/WzvkJMTS06BzRS6+/DgwfDrz5pubPnZwMfPst\nsGeP5s9NiK7QonPE6OXmcp/ctcHXFzh6lOswJoRQIiB66K+/gJISbiawNtjbcyOGrl7VzvkJMTSU\nCIjeycsDvL0Bc3PtnF8g4Dqi8/K0c35CDA0lAqJ3tNlRXKu2eYgQQomA6CFt9g/UohoBIf+gRED0\nzrFjgI+Pdq9BNQJC/kHDR4leUSiATp24WcVt22rvOkold53iYu47IYaGho8So3X2LLdZvTaTAMB1\nRHt5cbUPQkwdJQKiV06c4N6gdcHHh5qHCAEoERA9c+IE4Ompm2t5eXE7oBFi6igREL1y8qTuagSe\nntz1CDF1lAiIXtFl01BtIqCxCsTUUSIgeuPuXaC0VHf7BFhZARYWwCNbbhBikigREL1x8iTQs6f2\nlpZ4EmoeIoQSAdEjumwWqkWJgBBKBESP6HLEUC1KBIRQIiB6RJcjhmpRIiCElpggeoIxwNqaG9ff\ntavurnvvHmBrC5SX67ZvgpCWoiUmiNEpKeE2qrez0+1127fnEs+FC7q9LiH6pMlEkJKSAqlUCldX\nV8TFxT32elJSEnx8fODn54c+ffogPT1d7bKE1KrtKBYIdH9tT0/u+oSYKpVNQ0qlEu7u7khLS4NY\nLEZAQADi4+Ph4eFRd0xFRQUsLCwAACdOnMCIESNw/vx5tcoC1DREOP/9L1BYCCxdqvtrf/IJ0Lo1\n8J//6P7ahDwtnTUNZWdnw8XFBY6OjhCJRIiKikJSUlK9Y2qTAADcu3cP1tbWapclpFZ+PtDgM4LO\nUI2AmDqhqheLi4vh4OBQ91gikSArK+ux4xITEzFnzhyUlJQgNTW1WWUBICYmpu7nkJAQhISENOd3\nIEbgzBlg9Gh+rt2rFzB3Lj/XJkRdMpkMMplMK+dWmQgEajbYRkREICIiAn/++SfGjh2L/Pz8ZgXx\naCIgpik/H5BK+bm2mxtw8SJQXQ0IVf5HEMKfhh+SY2NjNXZulU1DYrEY8kcWYpHL5ZBIJI0eHxwc\njOrqaty6dQsSiaRZZYnpunkTqKrS7bDRR7VtC3TrRiOHiOlSmQj8/f1RUFCAoqIiKBQKJCQkIDw8\nvN4xFy5cqOuwyM3NBQBYWVmpVZYQ4J/aAB8jhmp5eHDNU4SYIpUVYaFQiGXLliEsLAxKpRKTJk2C\nh4cHVqxYAQCIjo7G1q1bsW7dOohEIrRv3x6bNm1SWZaQhs6c4a+juFZtIoiI4DcOQvhAM4sJ72bO\n5Gb3fvwxfzGsWQPIZMC6dfzFQEhz0MxiYlT47CiuJZVS0xAxXZQICO/0pWkoP592KyOmiRIB4dX9\n+8DVq7rblawxXbpwu5VducJvHITwgRIB4dW5c4Czs36M36eRQ8RUUSIgvNKH/oFalAiIqaJEQHhF\niYAQ/lEiILzSh47iWpQIiKmiREB4RTUCQvhHE8oIb5RKoEMH4MYNbqcwvjEGdO7MLUBnZcV3NISo\nRhPKiFG4fJl7w9WHJABwax1J
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 3
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# Priors and Sample Distribution\n",
|
||
|
|
"p_1 = 0.8\n",
|
||
|
|
"p_2 = 0.2\n",
|
||
|
|
"p_x = p_x_given_1 * p_1 + p_x_given_2 * p_2\n",
|
||
|
|
"plot(p_x); plot(p_x_given_1*p_1); plot(p_x_given_2*p_2)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 4,
|
||
|
|
"text": [
|
||
|
|
"[<matplotlib.lines.Line2D at 0x2ecb310>]"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD9CAYAAACx+XApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPe5+PHPAAMCKi4ssikKCCiLKEjUaIgb0Rjiktto\njMlNjbFJ06S5bW/apv1V2zSJNzfNTWPb2GY1yY02za2YVIkxStxBA4o6LqggiIJEQRYXYPj+/jiC\ngjqCznBmmOed17xkzvrMCZxnznc1KKUUQgghnJaL3gEIIYTQlyQCIYRwcpIIhBDCyUkiEEIIJyeJ\nQAghnJwkAiGEcHI3TQSZmZlER0cTGRnJkiVLrlmfkZFBQkICiYmJjBgxgg0bNrSsCwsLIz4+nsTE\nREaOHGndyIUQQliFwVI/ArPZTFRUFOvXryc4OJjk5GQ++eQTYmJiWrapq6vD29sbgL179zJjxgyO\nHDkCwMCBA/n222/p06ePjT+GEEKIW2XxiSAnJ4eIiAjCwsIwGo3Mnj2bjIyMVts0JwGA2tpafH19\nW62X/mpCCGHfLCaC0tJSQkNDW96HhIRQWlp6zXarVq0iJiaGKVOm8Mc//rFlucFgYOLEiSQlJfG3\nv/3NimELIYSwFjdLKw0GQ7sOMn36dKZPn87mzZuZN28ehw4dAmDr1q0EBgZSUVHBpEmTiI6OZuzY\nsbd0DiGEEK1Zq8TF4hNBcHAwJSUlLe9LSkoICQm54fZjx46lsbGRM2fOABAYGAiAn58fM2bMICcn\n57r7KaXkpRS/+c1vdI/BXl5yLeRayLWw/LImi4kgKSmJgoICioqKqK+vZ+XKlaSnp7fa5ujRoy1B\n5ebmAtC3b1/Onz9PTU0NoFUor1u3jri4OKsGL4QQ4vZZLBpyc3Nj6dKlpKWlYTabmT9/PjExMSxb\ntgyAhQsX8tlnn7F8+XKMRiPdu3dnxYoVAJSVlTFz5kwAGhsbmTt3LpMnT7bxxxHOaN8+eOklKCqC\nCRPg5z+Hq9owCCFuwmLz0U4JwGCw+mOOo8rKyiI1NVXvMOxCe6/F11/DnDnwy1/CsGHw9tuwfz9k\nZYGPj83D7BTye3GFXIsrrHnvlEQgHNbx45CcDJ9+CnfdpS1TCp55Bo4cgTVrQNoiiK7KmvdOGWJC\nOKwnnoD/+I8rSQC0G/8f/gCnT8P77+sWmhAORZ4IhENatw6eflorBjIar12fmwvTpmlPBl5enR+f\nELYmTwTC6f3mN/D7318/CQAMHw6jRsFf/tK5cQnhiOSJQDicHTvgoYegoABcXW+83e7dcN99UFgI\nbhbbxwnheOSJQDi1N9+EH/3IchIArRXRgAGwenXnxCWEo5InAuFQqqshNBSOHYO+fW++/SefwHvv\naXUKQnQl8kQgnNY//wmpqe1LAgDTp0NODpSV2TQsIRyaJALhUD7+GB5+uP3be3pCejr8/e+2i0kI\nRyeJQDiMigrt2/20aR3bb84crYhICHF9kgiEw/jXv2DiRO1bfkdMnKi1MLpqIF0hxFUkEQiH8fnn\nWnPQjjIaYcoU+OIL68ckRFcgiUA4hEuXYP167YZ+K6ZNk0QgxI1IIhAO4ZtvYOhQ8Pe/dl3NpRq+\nPfktVRerbrh/Whps3gx1dTYMUggHJYlAOISvvoJ77rl2+Vu73iLsjTC+v/r7DHxjIK9te+26bat7\n9YKkJNiwoROCFcLBSCIQDmHjRhg/vvWyv377V/5723+zff529vxgD3kL83hv93u8vuP16x4jLU1L\nKEKI1qRnsbB7lZXQvz+cOQPu7tqyveV7mbB8Alu/v5XIvpEt25acK2Hk2yNZ9eAqUkJSWh1n5054\n7DFtRjMhHJ30LBZO5ZtvYPToK0lAKcWT/3qS397921ZJACDUJ5TXJr/Gwi8WYm4yt1qXmAgnTkB5\neWdFLoRjkEQg7N6GDa2LhdYUrKH6UjVPjHjiutvPiZ2Dt7s3n5o+bbXczU2bxGbjRltGK4TjkUQg\n7N7V9QNKKV7c/CK/GvcrXAzX//U1GAz8etyv+d2m39GkmlqtmzBBm+dYCHGFJAJh1777DoqLtWId\ngF0nd1FeW86smFkW90sLT8PV4EpWUVar5ePHS8shIdqSRCDs2vbtkJJyZWKZt/Pe5vHhj+PqYnky\nAoPBwILhC/hb7t9aLR86FGpqZLgJIa4miUDYte3btSknAerq6/h0/6f8+7B/b9e+c+PnsrZgLWfO\nn2lZZjBoFc/bttkgWCEc1E0TQWZmJtHR0URGRrJkyZJr1mdkZJCQkEBiYiIjRoxgw1XP3TfbV4ib\nuToR/N+B/2N06GiCegS1a98+nn24d/C9rNi3otVySQRCtKEsaGxsVOHh4aqwsFDV19erhIQEZTKZ\nWm1TW1vb8nN+fr4KDw9v976X+zBYCkE4sYYGpbp3V6qyUnuf/km6+nDPhx06xqoDq1Tq+6mtlm3e\nrFRSkrWiFEIf1rx3WnwiyMnJISIigrCwMIxGI7NnzyYjI6PVNt7e3i0/19bW4uvr2+59hbAkP1/r\nSNarF9TW17KxcCPTBndsMoLJ4ZPJO5VHRV1Fy7IRI8BkknGHhGjmZmllaWkpoaGhLe9DQkLIzs6+\nZrtVq1bxi1/8glOnTrHu8uSw7d0XYNGiRS0/p6amkpqa2pHPILqoq4uF1hSsYUz/MfTq1qtDx/A0\nepIWkcaqg6tYMGKBtswT4uJg1y6tX4EQjiArK4usrCybHNtiIjAYDO06yPTp05k+fTqbN29m3rx5\nHDx4sENBXJ0IhGi2fTvcfbf28z8P/pOZ0TNv6TizYmbx/u73WxIBXKknkEQgHEXbL8mLFy+22rEt\nFg0FBwdTclU7u5KSEkJCQm64/dixY2lsbOTs2bOEhIR0aF8h2tq5E0aOBHOTmXVH1zE1cuotHWfS\noElsLt7MxcaLLcukwliIKywmgqSkJAoKCigqKqK+vp6VK1eSnp7eapujR4+2DHyUm5sLQN++fdu1\nrxA3Ul2tjQsUEwO5p3IJ7B5IcM/gWzpWb8/exAfEs/n45pZlzYlAxjsU4iZFQ25ubixdupS0tDTM\nZjPz588nJiaGZcuWAbBw4UI+++wzli9fjtFopHv37qxYscLivkK0R14exMdrHcnWHV3H5PDJt3W8\ntPA0vjz6JZPCJwEQFATe3tpcxoMHWyNiIRyXDEMt7NIf/gDHjsHSpXDX+3fxizt/wT0R15mZpp2y\nT2Qzf/V89j11ZQzqWbPggQdgzhxrRCxE55JhqEWX9+23WjPPmks15J7KZdyAcbd1vKSgJE7WnORU\nzakry5K0lkNCODtJBMIu7dql3aizirJIDkrGy+h1W8dzdXFl7ICxbDq+qWWZJAIhNJIIhN25uqJ4\nU/Emxg8cf/Od2mFc/3FsKr6SCEaM0Ooimpos7CSEE5BEIOzO1RXFW4u3MiZ0jFWOe1fYXa2eCPr0\nAT8/OHzYKocXwmFJIhB2p7l+4ELDBfaU72Fk8EirHHdYv2EUnyvmu/PftSyT4iEhJBEIO9ScCHae\n3Emsfyze7t4336kd3FzcGBUyii3FW1qWSSIQQhKBsEO7d8OwYbCleAt39r/Tqse+a8BdUmEsRBuS\nCIRduXhR6z8wZMjlRBBq3UQwdsDYVk8Ew4driaex0aqnEcKhSCIQduXAAQgPB6N7E9tPbGdMf+tU\nFDcbHjic/RX7W8Yd8vGB4GDo4DiJQnQpkgiEXcnP11oM7T+9Hz8vP/y9/a16fC+jF1F9o9hdtrtl\nWVKSNsCdEM5KEoGwK82JYGvJVqs/DTRLCUkh+8SVuTESE7XiISGclSQCYVfy8yEhQWsxNDLIOs1G\n20oJTiG79EoiGDYM9uyxyamEcAiSCIRdaX4i2Fm6k+TgZJuco20iSEjQnghk7EPhrCQRCLtRXq61\n3vHxreNo5VHiA+Jtcp4o3yjOnD/TMo+xn582JPXx4zY5nRB2TxKBsBvNTwO7y/MY6jcUd1d3m5zH\nxeBCUlASOaU5LcuGDZN6AuG8
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 4
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# Conditional Distributions\n",
|
||
|
|
"p_1_given_x = p_x_given_1 * p_1 / p_x\n",
|
||
|
|
"p_2_given_x = p_x_given_2 * p_2 / p_x\n",
|
||
|
|
"plot(p_1_given_x); plot(p_2_given_x)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 5,
|
||
|
|
"text": [
|
||
|
|
"[<matplotlib.lines.Line2D at 0x2edd6d0>]"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD9CAYAAABdoNd6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVdXeP/DPgYMCzigyHRSZQeBA4jzhFGpGjjmUmpWZ\nZVZPz73d6nfv1Z7nUaluN8tb4q2s1EhvmkMJWurBAVFABhFUVNDD6AgqiMBh/f7YQaIMB9h7r304\n3/frxSvh7LPW1x1+WKy99toqxhgDIYSQds2CdwGEEEKkR2FPCCFmgMKeEELMAIU9IYSYAQp7Qggx\nAxT2hBBiBpoN++effx4ODg4IDAxs9Jjly5fDy8sLWq0WKSkpohZICCGk7ZoN+0WLFiE2NrbR1/fu\n3YsLFy4gOzsbGzZswNKlS0UtkBBCSNs1G/YjR45Ejx49Gn199+7dWLhwIQBg8ODBKCkpQXFxsXgV\nEkIIabM2z9nn5+fD1dW17nONRoO8vLy2NksIIUREajEaeXjHBZVK9cgxDX2NEEJI88TY1abNI3sX\nFxfo9fq6z/Py8uDi4tLgsYwx7h83ym/g6OWj+OT4J5j2wzTYRdphQNQAfHTsI1wruyZLDX//+9+5\nnwelfNC5kP5cpBelY3nMcrh+7Arvz7zx0p6XsDFlIzKKM3Cv6h73vzd9XzT9IZY2j+wjIiKwbt06\nzJkzBwkJCejevTscHBzEqE0SdjZ2GN5nOIb3GY7Xh7wOQ40Bh3IPYXP6Znh/5o1ngp7BOyPegXMX\nZ96lEtIm8fp4/O3Q35B5LROLByxGzDMx8Lf3p9+yzVSzYT937lzExcXh+vXrcHV1xcqVK1FVVQUA\nWLJkCSZPnoy9e/fC09MTnTp1wsaNGyUvWkyWFpYY7z4e493Ho/BOIT5O+BhBXwThnRHv4I0hb8DS\nwpJ3iYS0yI3yG3gt5jUcuXIE74e9j2eCnkEHyw68yyKcqZiYvyc01ZFKJeqvJFI6f+M8Fu9ZDCsL\nK2yZvgUOncX9TUWn0yEsLEzUNk0VnYs/iHEudLk6PLvjWcwJmIOVYSvRqUMncYqTGX1f/EGs7KSw\nb0R1TTVW6FZgU/om7H92P3x6+fAuiZAmbUrbhP/+9b+xedpmTPCYwLscIhIKe5l8nfI13jv4HvY/\nux+BDo3fRUwIT+uT1mPN0TWIeSYGfvZ+vMshIhIrO0VZetmePR/yPGzUNnji+ycQ/0I8NF01vEsi\npJ5tZ7bhfw7/D44sOgL3Hu68yyEKRSN7I3147ENsSt+EhBcTYGtly7scQgAAyQXJmLhlIg4sOIAg\nhyDe5RAJ0DSOzBhjWLBzAWytbBE1JYp3OYSgtKIUj214DKvHrcbT/Z/mXQ6RiFjZSVscG0mlUuFf\nk/+F3y79hh1ZO3iXQwhe3fsqwj3CKeiJUSjsW6Brx67YPG0zlu1dhtKKUt7lEDO2/+J+HNMfw4cT\nPuRdCjERNI3TCi/ufhFdOnbBP8P/ybsUxbtzB9Drgbw8oKQEKC8XPqqqAGtrwMYGsLUFHB0BFxfA\n2Rno2JF31cp2r+oeAr8IxNqJa/GE9xO8yyESozl7jq6VXUP/z/vj4MKDCOgdwLscxSgsBA4eBJKS\ngLQ0ID0duHcPcHUFNBrAzk4IdltbwNISqKgQPsrKgKIiID8fKC4Wjg0MBIKCgJEjgWHDhPcQQeTR\nSCTkJ+Cn2T/xLoXIgMKes4+Pf4xj+mPY/vR23qVww5gQ7D/8AMTGCmEfFgYMGSIEtVYrjNhbshVL\ndTWQnQ2cPg2kpgKHDwv/fewxYNo0YNYs4YeBuSqtKIXXZ16Iey6O1tObCQp7zsqryuHxqQf2PbvP\n7Ja83bwJREUBX30lBPncuUBEBBASIozYxXb3LhAXB/z4I7Brl/BDZNky4KmnALWZ3Snyd93fkVuS\ni2+nfsu7FCITCnsF+Pj4x4jXx+PHp3/kXYos8vKAVauA6GghaF95BRg4sGUj97a6fx/YuRP47DPh\nWsBbbwFLlpjHPH9pRSncP3VH4uJEunnKjNDSSwV4OfRlHL58GOdvnOddiqRKS4G//EUYUXfpAmRl\nAd98AwwaJG/QA0Koz54NHD0KbN8O7N8P+PgAmzYJ00rt2dcpX+Nxj8cp6EmrUNi3ga2VLV587EV8\nnvg571Iks2cPEBAAXL0qXHCNjBTm4ZUgNBT4+Wch6P/5T2DCBODCBd5VScNQY8CnJz/Fm0Pe5F0K\nMVEU9m30cujL2JS+CXcr7/IuRVR37wILFgBvvimE6ddfC0sjlWjkSODkSWDSJOHi8L//3f5G+bvO\n7YJTZycMchnEuxRioijs26hPtz4IcwvD5vTNvEsRzdmzwODBwsXP9HRhhY3SqdXC/P3Ro8DatcDC\nhcKSzvbi88TPsXzwct5lEBNGYS+CV0JfQVRy+9gv57ffgFGjgDfeEFbbmNr6dl9f4MQJYWQfFiZM\nP5m6yyWXkVqUiqm+U3mXQkwYhb0IxvQbg+vl15FxNYN3KW2ybRswb56wxHHxYvkvvoqlUyfgu++A\nyZOFG7IuXuRdUdtsSt+E2QGzYa225l0KMWEU9iKwUFngmcBnTHoqJzpaGM3/+qswsjd1KhWwcqUw\ntTN2LJCby7ui1mGM4ZvUb7BQu5B3KcTEUdiLZH7QfGw5vQU1rIZ3KS3288/Chdj9+4Xlle3J0qXA\nn/4EjBsn3CdgauL18ehg2QEDnQfyLoWYOAp7kfTv3R/2tvbQ5ep4l9IiCQnA888Du3cLSyzbo2XL\nhGmpKVOEVUamZFvmNswNmAuVqc6pEcWgsBfR7P6zsT3LdPbKKSgAZs4UllUOaucr+t5+W9hfZ/58\noMZEfvmqYTXYkbUDM/xn8C6FtAMU9iKa5jcNO8/uNImpnPv3hY3FXn1VGPG2dyoV8MUXwPXrwPvv\n867GOIn5iejcoTP87f15l0LaAQp7EXn39Ea3jt2QmJ/Iu5RmvfeesHf8X/7CuxL5dOworDhavx44\ncoR3Nc3bnrUdM/xoVE/EQWEvsml+07Dz3E7eZTTpwAFhW+IvvzTd5ZWt5eQk/L2ffRa4dYt3NY1j\njFHYE1FR2Itsqs9U/JSl3IdKlJQAixYJ8/Q9e/Kuho8pU4QtmV9/nXcljTt7/SyqDFUIdgzmXQpp\nJyjsRRbqHIo7lXeQfSObdykNeu89YQ+Zxx/nXQlfq1cLe+QfPMi7kobFXIjBJK9JtAqHiIbCXmQq\nlQqPezyOfRf38S7lEYmJwrbAa9bwroS/zp2BdeuAl18WHo2oNDEXYjDJcxLvMkg7QmEvgXCPcMWF\nvcEg3GAUGQn06MG7GmV48knhWbcffMC7kvruVt5FQl4CxvUbx7sU0o5Q2EtggvsExOXGodJQybuU\nOps3C6tRFizgXYmyfPSRsEtmcTHvSv5wKOcQBjoPRJeOXXiXQtoRCnsJ9LTtCd9evjh25RjvUgAI\na+r//ndhVE9TwPX16yf8AFTS2nuawiFSoLCXSLincqZyoqKErRBGjOBdiTK99x6wdSuQrZBr6gdz\nDmK8+3jeZZB2hsJeIo+7P47fLv3GuwzcvSs8JHzVKt6VKFevXsJGcCtX8q4EKLxTiKtlV6F1bGc7\n0hHuKOwlMshlEM5eP4vSilKudfz738KWxUFBXMtQvGXLgNhYICeHbx26XB1G9R0FCxX90yTiou8o\niXRUd8RAl4GI18dzq6GyEvj4Y2ETMNK0bt2Al14CPvyQbx26yzqEuYXxLYK0SxT2EhrVdxTiLsdx\n6z86GvDxAQYM4FaCSXnjDWEbiaIifjXocnUY4zaGXwGk3Wo27GNjY+Hr6wsvLy9ERkY+8vr169cx\nceJEBAcHIyAgAN98840UdZqk0X1Hcwv7mhphlPrnP3Pp3iT17i08lvHTT/n0X3CnANfLryPQIZBP\nAaRdazLsDQYDli1bhtjYWGRmZiI6OhpZWVn1jlm3bh1CQkKQmpoKnU6Ht956C9XV1ZIWbSqGaIbg\ndPFplFWWyd73gQOApSUwYYLs
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 5
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(Density vs Conditional Distributions)\n",
|
||
|
|
"\n",
|
||
|
|
"A _density_ is a non-negative function $f:R^n\\rightarrow R$ such that $\\int f(x) dx = 1$.\n",
|
||
|
|
"\n",
|
||
|
|
"A _conditional distribution_ is a function $f:R^n\\rightarrow R^c$ such that $\\sum_i f_i(x) = 1$ for all $x$.\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# Error at each Point\n",
|
||
|
|
"p_error_given_x = 1-maximum(p_1_given_x,p_2_given_x)\n",
|
||
|
|
"plot(p_error_given_x)\n",
|
||
|
|
"print sum(p_error_given_x*p_x)/sum(p_x)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "stream",
|
||
|
|
"stream": "stdout",
|
||
|
|
"text": [
|
||
|
|
"0.117715838421\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD9CAYAAABdoNd6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VOWdB/DvQCIQbkogQGaCwVzIABIuAbybFtgAqwHB\nrbFdrDRq1hZdurXr467dQrVI1nW3VLZtfBYvgA34WGtol4yAmMLKJUBQkCAESGDIjUC4hYshw7t/\nvJ3cCJOZyTnnPWfm+3mePOQyOefHSfKdd37nPe+xCSEEiIgopHVTXQAREemPYU9EFAYY9kREYYBh\nT0QUBhj2RERhgGFPRBQGOg17l8uFlJQUJCUlITc394avFxUVoX///hg3bhzGjRuHV199VZdCiYgo\neBG+vujxeLBgwQJs2rQJdrsdEydORGZmJpxOZ5vHPfjgg1i3bp2uhRIRUfB8juyLi4uRmJiI+Ph4\nREZGIisrCwUFBTc8jtdlERGZm8+wr6ysRFxcXPPHDocDlZWVbR5js9mwbds2pKamYubMmSgtLdWn\nUiIiCprPNo7NZut0A+PHj4fb7UZUVBQKCwsxe/ZsHD58OKhtERHRjbTonvgc2dvtdrjd7uaP3W43\nHA5Hm8f07dsXUVFRAIAZM2bg2rVrqK+v73B7Qgi+CYGf//znPr8OCCxfrr5OMxwLq74BArNn81gE\n+8Zj0fKmFZ9hn5aWhrKyMlRUVKCxsRFr165FZmZmm8fU1tY2F1RcXAwhBAYMGKBZgeHm2jX5b0WF\n0jJIA1VVqisgauGzjRMREYHly5cjIyMDHo8H2dnZcDqdyMvLAwDk5OTgww8/xG9/+1tEREQgKioK\na9asMaTwUFVXJ/89dUptHRQ8DQdjRJqxCS1fJ/jakc2m6UsSKysqKkJ6enqHX9u7Fxg/Hpg2Ddiw\nwdi6VPB1LKyqvh6IjgaSk4FDh/z/vlA8FsHisWihVXYy7E2msBCYPx+IiQH27VNdDQXjwAFg6lTg\n8mXg/HnV1ZDVaZWdXC7BZGprgdRUoLpadSUUrJoaICUFuHoVuHJFdTVEEsPeZGprgdGjgXPnWk7W\nkrXU1ACxscCQIfLnSWQGDHuTqa0Fhg4FBg1iUFhVdbUM+sGDZfATmQHD3mRqa2VIDBnCoLCqmhr5\n8+PInsyEYW8ytbUyJIYOZd/eqqqr5c+PT9hkJgx7k+HI3vq8I3u2cchMGPYmU1MjQ4Ije+vy9uzZ\nxiEzYdibSFOTnIUzcCBH9lZWU8M2DpkPw95E6uqAAQOA7t05sreqb74BLl2SP0e2cchMGPYm4u3X\nAzLsGRTW423D2Wwc2ZO5MOxNxDsTB5D/cmRvPd6Ts0BLz56rhJAZMOxNpPXI3jsqZFBYi/fkLAD0\n6SP/bWhQVw+RF8PeRFqHfVQU0KOHPGFL1uE9OevFVg6ZBcPeRFqHPcCTtFbUemQPcPolmQfD3kS8\nJ/e8OCq0ntY9e4Azcsg8GPYmwpG99bGNQ2bFsDeR1rNxAAaFFbGNQ2bFsDcRjuytr/3Inm0cMguG\nvUl4PPLepQMHtnyOI3trEYLnXci8GPYmcfo0cNttQEREy+c4sreW+nqgd2+gZ8+Wz7GNQ2bBsDeJ\n9i0cgKNCq2nfwgHYxiHzYNibRPuX/wBH9lbT/uQsIH+mXDKBzIBhbxLtZ+IAcuXEhga5kiKZX0cj\n+1695BuvhCbVGPYm0VEbp1s3tgGspKORPcCfIZkDw94kOgp7gH17K6msBOz2Gz/PnyGZAcPeJG4W\n9uzbW0dVFRAbe+PnOSOHzIBhbxIMe+urquLInsyLYW8SvsK+qsr4eihwlZUdj+zZsyczYNibREdT\nLwE5UqysNL4eCowQ8hUY2zhkVgx7E/B4gDNngJiYG79mt3NkbwX19fKGM7163fg1tnHIDBj2JnDm\nDHDrrW2XSvDiyN4abtbCAdjGIXNg2JvAzfr1AMPeKm42EwdgG4fMgWFvAu3vbtRadDRw6RJw5Yqx\nNVFgfI3sY2KAujrZriNShWFvAr5G9jabDBGO7s3tZtMuASAyEujfX7briFRh2JuAr7AH2MqxAl9t\nHICtHFKv07B3uVxISUlBUlIScnNzb/q4Xbt2ISIiAh999JGmBYYDf8KeM3LMzVcbB+CMHFLPZ9h7\nPB4sWLAALpcLpaWlyM/Px8GDBzt83Isvvojp06dDcC3XgPnq2QMc2VuBrzYOwBk5pJ7PsC8uLkZi\nYiLi4+MRGRmJrKwsFBQU3PC4N998E48++igGDRqkW6GhjG0c62Mbh8yug5ndLSorKxEXF9f8scPh\nwM6dO294TEFBATZv3oxdu3bBZrPddHuLFi1qfj89PR3p6enBVR1i/An7doedTKSpSc628fUzZBuH\n/FVUVISioiLNt+sz7H0Ft9fChQuxdOlS2Gw2CCF8tnFahz21uNlSCV4c2Ztbba28UXxk5M0fM3gw\n8OWXxtVE1tV+ILx48WJNtusz7O12O9xud/PHbrcbDoejzWP27NmDrKwsAMDp06dRWFiIyMhIZGZm\nalJgqPMuleCrA8apl+bWWQsH4Mie1PMZ9mlpaSgrK0NFRQViY2Oxdu1a5Ofnt3nMsWPHmt+fP38+\nHn74YQZ9ALxLJfgaFcbGykW2hJDz7slcOpuJA7BnT+r5DPuIiAgsX74cGRkZ8Hg8yM7OhtPpRF5e\nHgAgJyfHkCJDWWf9ekAurtWnD3D6tO9XAKRGZzNxAM7GIfVswqC5kt6ePrW1cSOQmwts2uT7cWPG\nACtXAmPHGlMX+e/ll4FbbgH+7d9u/hiPB+jZE7h82ferOKL2tMpOXkGrmD8je4Anac3MnzZO9+7y\nJG5dnTE1EbXHsFeMYW99/pygBdjKIbUY9op1Nu3Si2FvXidPAu0mqXWIM3JIJYa9YrW1vpdK8GLY\nm5fbDbS69vCmOCOHVGLYK+ZvGyc2louhmdH588D163L6bGfYxiGVGPaKsWdvbd5RvT/XP7CNQyox\n7BVjz97a/G3hAGzjkFoMe4X8WSrBa+BA4OJF4OpV/esi/wUS9mzjkEoMe4X8WSrBq1s3YOhQju7N\nJtCRPcOeVGHYK+RvC8dr2DAZLmQebOOQVTDsFfL35KwXw958Agn7224DLl1iK47UYNgr5O8ce69h\nw4ATJ/SrhwJ38qT/YW+zySd3ju5JBYa9QsGM7Bn25iFEYCN7gK0cUodhr1CgPfu4OIa9mdTXAz16\nyOWn/cUZOaQKw14hjuytLdBRPcAZOaQOw16hYHv2vC2AObjd/i2A1hrbOKQKw16hQEf2/fvL+fbn\nzulXE/mPI3uyEoa9QoH27AG2cswkmLBnz55UYdgrEshSCa1xrr15BDuyZxuHVGDYKxLIUgmtcWRv\nHmzjkJUw7BUJpoUDcPqlmbCNQ1bCsFck0JOzXhzZm8P163JRukBn4/TtK7+3oUGfuohuhmGvCMPe\n2k6dAvr1A3r1Cuz7bDb27UkNhr0igc6x92LYm8Px40B8fHDfy1YOqcCwVyTYnr3dLr+3qUn7msh/\nFRXBhz1P0pIKDHtFgm3jREYCMTFAdbX2NZH/uhr2bOOQ0Rj2igQb9gBbOWbQlbBnG4dUYNgrUlMT\nXM8e4PRLM2Abh6yGYa9IV0f2x49rWw8Fhm0cshqGvQLepRJiYoL7/uHDgfJybWsi/wkhn2xvvz24\n72cbh1Rg2Ctw6hQwYAAQERHc9zPs1Tp1CujdO7CblrTGNg6pwLBXoLoaGDo0+O+/4w6GvUpdaeEA\nLfeh5X0JyEgMewW6Gva33y7XZfF4tKuJ/NfVsI+KAm65BTh/XquKiDrHsFegq2HfsycQHS3XZiHj\ndTXsAbZyyHgMewW6GvYA+/YqaRX2nJFDRuo07F0uF1JSUpCUlITc3Nwbvl5QUIDU1FSMGzcOEyZM\nwObNm3UpNJRoEfbs26ujRdhzRg4Zzed8EI/HgwULFmDTpk2w2+2YOHEiMjMz4XQ6mx8zdepUzJo1\nCwCwf/9+PPLIIzhy5Ii+VVtc
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 7
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "heading",
|
||
|
|
"level": 1,
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"2D Example"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"Some preliminaries...\n",
|
||
|
|
"\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"xs,ys = meshgrid(linspace(-4.0,4.0,200),linspace(-4.0,4.0,200))\n",
|
||
|
|
"xys = c_[xs.ravel(),ys.ravel()]\n",
|
||
|
|
"\n",
|
||
|
|
"def mvpdf(x,mu=zeros(2),sigma=eye(2)):\n",
|
||
|
|
" return ((2*pi)**len(mu)*det(sigma))**-.5 * exp(-0.5*dot(x-mu,dot(inv(sigma),x-mu)))"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"prompt_number": 19
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# class conditional densities\n",
|
||
|
|
"p_x_given_1 = array([mvpdf(x,mu=array([1,1.0])) for x in xys]).reshape(xs.shape)\n",
|
||
|
|
"p_x_given_2 = array([mvpdf(x,sigma=diag([2.0,1])) for x in xys]).reshape(xs.shape)\n",
|
||
|
|
"subplot(121); imshow(p_x_given_1,cmap=cm.gray)\n",
|
||
|
|
"subplot(122); imshow(p_x_given_2,cmap=cm.gray)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 20,
|
||
|
|
"text": [
|
||
|
|
"<matplotlib.image.AxesImage at 0x5e6b190>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAC2CAYAAADEKgYfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztXV+sXUX1Xuf+AX1RUOml6W1yoS3UammaIOGFxASKT1Z4\nQarRhj/GYOILRNQ3NFHaB55QEmMwqS8CL1AepGl8EA0vjX94sTE0sWgpbSMUEgLY23t7fg/81uE7\n31lrzcw++5x7zu18yc6ePTN79uzZa7759prZ53S63W5XKioqKiqmFjNrXYGKioqKiuFQibyioqJi\nylGJvKKiomLKUYm8oqKiYspRibyioqJiylGJvKKiomLK0TqRHzlyRLZv3y7btm2TgwcPtl18RcWa\nodp2xaSi0+Y68tXVVbnxxhvlD3/4g2zatEm+9KUvye9+9zv5/Oc/39YlKirWBNW2KyYZrSryY8eO\nydatW2VpaUnm5+fl3nvvlcOHD7d5iYqKNUG17YpJRqtEfvr0adm8eXPveHFxUU6fPt3mJSoq1gTV\ntismGXNtFtbpdFrJU1ExLNr+5Ylq2xWTAM+uW1XkmzZtklOnTvWOT506JYuLi21eoqJiTVBtu2Ki\n0W0RFy9e7F5//fXdkydPdi9cuNDdtWtX9/jx4315RKRudRv51jaqbddtEjYPrbpW5ubm5Be/+IV8\n5StfkdXVVXnggQfqrH7FukC17YpJRqvLD7MuWP2IFWPAmM1aRKptV4wenl3XLzsrKioqphyVyCsq\nKiqmHJXIKyoqKqYclcgrKioqphyVyCsqKiqmHJXIKyoqKqYclcgrKioqphytfhBUUVFRgRhmbf1a\nfAswrahEXlFR0Qij/gAqp/xK9h+hEnlFRcUApuUr1Ur2H6ESeUVFhYhMD3mXAu9rvZJ6JfKKissY\n65W8PaxXUq9EXlFxmaAt0p5E8m9CytZ9TCu5VyKvqFjHaIN0J5G4GVzHpoQ8rYq9EnlFxTrEMOQ7\nLHGPgvhLSbUNYtcypoHQK5FXVKwTNCHQ3HPGSe4WcY7i/FyCngaVXom8omLKUUqybf+RdNsKfNgl\nhblqvAlBT6pKr0ReUTGFGIX6bpvghzknRZQlCjuHsEsJetJUeiXyioopQltKeRhSHxXhe+fnEGW3\n23WvieenBoAmvvVJUOn1Pzsr1iXW2392joPAvbS1IPVc8i5Na3JObn2a5C2F+0ZRibxiPWK9EHlb\nk5ElJD0qQm8LbRF4E7LPSS/NV4JK5BWXFdYDkQ9L4iXE2wahj6Nvl5JvDoG3reyb5MtFJfKKywrT\nTOTDuCtyCTkV15ToR9m/c1V1RNRNy/DiovjSPDnwyqmTnRUVE4Q2XSRWfAlZlx5H9UjlaZPoeKI0\nqhOm86Qn18mb1MyZ7LTKaxNVkVesS0ybIm9ThecQrEfobeRL1bcEwyjuNvKV5IniS/OUnluJvGJd\nYlqIvCmBD+MaKQk3KTOKK0WKVHPIOofAmwwApXFN8uSeU4l8HaJpG0/Chw1tYRqIvC0SH4a0LcJO\npeeqdC8uFyUkHhG4R8xNiT7n2ItrkicnfyXyKceo23NayX3SibyJL3xYAi+N47QSUs+9B0QbbpRu\nt5tN6qVxqfrk3kNpek7eOtk5JVirAdC77rQS/FqjbQIfRn3n7jU8rDqP4kXy/M5tEblOcuK+FNZE\nqTdxGt1fzmRpCo0V+dLSknzqU5+S2dlZmZ+fl2PHjsn58+fl61//uvz73/+WpaUlee655+Sqq64y\nK12RxrS01SSS+jB1GpVtj5vEc/YpsvbirL0XF90fI1K0uQSt+yguyhPtc8Ope/HQNL0xkV933XXy\n17/+VT7zmc/04h599FH53Oc+J48++qgcPHhQ3nnnHTlw4ED/BaeEnNYSbbfRuFT1JBH6MHUZhW03\nSWvqOikhcI4rPSfaR/eHcSUulaYEjmml50T7KOzdWxTfNG0oIv/LX/4in/3sZ3tx27dvl5dfflkW\nFhbk7Nmz8uUvf1n++c9/9l+wErmJJu0yzrZsa1JmXBiWyNu07VISzyHwXB93DnljOOfcpq6ZXJQq\nb93nhnNVe0TwUV04bB17cam01on8+uuvl09/+tMyOzsr3/3ud+U73/mOXH311fLOO+/0LviZz3ym\nd9y7YCXyPpS0x6S0XRuTM6PGMNdt07bbIvFSBZ4bzj2emZkxy+C4VF1zkKN6IwK+dOlSX1yKyEsH\ng2jvxVnHXlyT/I0nO1955RXZuHGj/Pe//5U9e/bI9u3b+9L54Vb0Y1QE3kabp0gw9Vps5V1rhV6C\ncdh2isRTZDgseXskzcfeOTkKP3VfjJS7IldBdzqdAZK+dOlSL56PMV8O8Bq4t9Kwrnh9bI82+kZj\nIt+4caOIiFxzzTVy9913y7Fjx3qvnddee62cOXNGNmzYMHQF1yNySCCXKEYxWHKZkaHlEnVbBjsO\ntGHbJUq8xJWi4ZQaTxG4ReQpAo/KtOrg3UuEUgL3FDZvem1V63gNa9P6cn2YsLmuTPBcjtVfvL5R\nIoJmkjkMfPDBB/Lee++JiMj7778vR48elZ07d8revXvl0KFDIiJy6NAhueuuu5oUvy7BRl+aJ+pU\nUfowW1R22/c3KWjDttsgcY8gZ2ZmXDKdmZnppeM+Z5udne3tU9vc3Fxv07j5+XmZm5vr7TmfxuOx\nblE+LQfLj+oRbXifOZvVjtjW0XPxnmGuHeSm9fI08ZGfPHlS7r77bhERWVlZkW9+85vy4x//WM6f\nPy/33HOP/Oc//5Glpbr8ENH0QXlpqXYsbeeUGZROvuSY1SgVetOyh7XtpiSeCucoX09tc9jaOE3z\ns1rHfUqhe/WOYKlwPLYUuKpsa2/5yy9duuQqccxvnZvjO4/85lHYOuY0L71+2TliNCVcT/U2uUZT\ntEHea0Xoa+HGiYi8KYnnkjcfMzmL2ETOKpPz8TneYKDXx4FDwT74COjD5nglYD22SNg7RnKPCJ3J\n28pjXR/jsM5tknlE5PXLzhGhCYGn4krIPacOCneU7wz67tg3iOdrPPv/mlxnPaENEs9V4dGWQ9Be\nnpywiBS7GBAppRuR7KVLl2RmZqYvfmZmZiDc6XR6cWp3WA5eU+/HU+T4rNgfjv3E85VbYes4B1WR\njwCl5Bp19Jxzmqr+1GtcKq70OHXNnPRcTIoiLyXxEgWOSpfDuQQ+Oztrkjmqc88/nKP6EVom3jM/\np2gyMkXkGEa3ioY53+rqaqjgrTQtj98cInXu7aOwd+yKoUrk7WIYEs/t9DllDYtSAys59uKi+FJM\nApGPk8RzFDgStYj0SNybGEWy12t6pO4NKt69WbAIzyPPFHkjSWMan7u6ujpQZsr9YhE71ndUZB4R\neXWttIRREHiq40fnNoXlJoleEa3zrfx4HJVh5Z1G5D6XHBLPXRZoKfGIzL3jVB4rnKpb1AYIJkFv\ns4gXw+pKUZfL6urqQBy6WnSv9bTeADAOXS4a5vvgPmTtLXto4mapRN4CSkg8R6XlKrec8qP65RhS\nCpbBWmGsR86AUFqPSUKKsDxiKyVxS5V7y+ZEPlbgqMSZkDHNC6cGCKvu0X2L+IqVV5F4qhnjVldX\nB8JK5lpHJW891jQFkjpeV8/FNA8emePzjkid80WoRD4kUkrLi8sl6yZEn6obp7ORoOFYRoQGynHW\neU0Je5rJHJHzDC0S5zhPfXuuDovUkcwxjxc/NzfXF++p96hOCmtFCwIVrx5bLpSUy0QV8srKSp8K\nV3tida4krm4WtFscSHSP5I7E7t1TStxEfScXlciHQC6JD0Pg3BlyiD6qmwfPN2cZH/rq2BBTxJ/r\nesH7mCYyz3n2uSTubaUkzsqaiZjjcvLkulmY0FNErnvLP+25U9Qfzm4TJmuMV/LWjd0qItLLi2F0\nrVhh776GJfNUP6hE3hBtkXjOviRvqn4eeFIFjS2HYD0/IF+jibFOG5nnIkXopSQe+bwtxY1fO2qa\n9QWkF2ZFLyJ9yh3vid0sGmYb
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 20
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# Priors and Sample Distribution\n",
|
||
|
|
"p_1 = 0.4\n",
|
||
|
|
"p_2 = 0.6\n",
|
||
|
|
"p_x = p_x_given_1 * p_1 + p_x_given_2 * p_2\n",
|
||
|
|
"imshow(p_x,cmap=cm.gray)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 26,
|
||
|
|
"text": [
|
||
|
|
"<matplotlib.image.AxesImage at 0x693e190>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAQIAAAD9CAYAAABX/HkoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfV+MXVX1/7qdmbevCigdmpkmA22hVkvTBAkvJCZQfLLC\nC1KMNgLG4CNE1DcwUdoHnlASYzCpLwIvUB6kaXwQCS/jPxJjY2jioGVoG6GQEEhsZ+b8Hvyty5o1\n69/eZ597z53Zn+TmnrP32vvsvc9en/XZ+5y5M2iapoGKiootjW3jbkBFRcX4UYmgoqKiEkFFRUUl\ngoqKCqhEUFFRAZUIKioqoAMiOHXqFOzduxf27NkDx48fL119RUVFBxiUfI9gdXUVbrrpJvjd734H\nc3Nz8KUvfQl+85vfwOc///lSl6ioqOgARRXB4uIi7N69GxYWFmBmZgbuu+8+OHnyZMlLVFRUdICi\nRLC8vAw7d+4cns/Pz8Py8nLJS1RUVHSA6ZKVDQaDIjYVFRXdQdoNKKoI5ubm4Ny5c8Pzc+fOwfz8\nfMlLVFRUdIGmIK5cudLccMMNzdLSUvPf//63OXDgQHPmzJl1NgBQP/VTP2P8SCi6NJienoaf/exn\n8JWvfAVWV1fhwQcfrE8MKiomAEUfH4YuWPcIKirGis73CCoqKiYTlQgqKioqEVRUVFQiqKiogEoE\nFRUVUImgoqICKhFUVFRAJYKKigqoRFBRUQGVCCoqKqASQUVFBVQiqKiogEoEFRUVUImgoqICKhFU\nVFRAJYKKigqoRFBRUQGVCCoqKqDwz5lX9BN9/3m4Ef9aXoWASgSbAH13dA+R9ley6BaVCCYIk+7w\nbSD1vZJDOVQi6CG2ssOnQBunShDpqETQA1THLws+npUYfFQiGAP65vijaM84nbEuK3xUIhgBxuH4\nk0o2o3JQ2p5KCpUIOsOoHLFvDt8WXn+6cNpKCpUIiqJLp9xsDp+LrmX+Vt1fqG8WFkIlgfGhjn17\nVEXQEl1MlEmpsxRKRN0uIznWvZnVQSWCRJR2qFL19dnRPXSxL9AFMWzmvYRKBEGUdLS2dfWVjCy0\ncZwS+wKliWGzqYRBM+KeTFrk6kPEzi07aWMN0M6xcsuWcIFJIgSprZUIFJRoZ04dqWX6QFQeSk2x\n1HoqMcioRBDAqCN3147f9/GmyJmK0TJd1t11HaVRiUDBKNfsXTnyKEioBEbhkF2QwziWLF2hEgFD\n3wiglE0b+xJl206pko5cyibHtmTZkpDasWVfKKok0M+yKeXHMa7jHJcuseUUwahuZIkJWHISj3vc\nPYwqupdUCJOqDrb00mAUBDAK5y9BHil2XaKUw2n5bUlhsxLCliWCrp/Dd+W8VrlJJYSunX8ceSk2\nJcvlYssRQdc76bmOmpPXBVmk2rVFCRJIzctRC6NcXrQtk4MtRQRdkcConL8UIXSxCdkGpSN9qqOn\n1NEmL8WmjX0OtgQRTAIBtHX8ErapNqVQcr0ftR0nWaTYtLFvW/emIoIuHq+NOpJH6hzH8qEUcoig\njcPm2qS2xcuL5Le1b1PvpiCCLlRAlwRQyvm7JIgu0YX875IUNhshbEoiqCTQTm1E8kqiy+heySC/\nrokmglEuBUoSQFtHTz1PScuxkZCzbs5x0hQiSKkr2h4tzUqP5re1T6knmwgWFhbg05/+NExNTcHM\nzAwsLi7CpUuX4Otf/zr861//goWFBXjhhRfgqquuWn/BQkRQmgRSomVXUVzLT3H8LpYQpZDrWFHn\nL0UKJfcZvLxIfq5tSh3ZRHD99dfDn//8Z7jmmmuGaY899hh87nOfg8ceewyOHz8O77//Phw7dmz9\nBQtMtlLP+r38HAcq5eAl7LyykfSSKLUsiOTlkIeX59mnpqXk59pGy7cigj/96U/w2c9+dpi2d+9e\nePXVV2F2dhYuXLgAX/7yl+Ef//jH+gu2mHCjUAEl5L5lm+PgJct7aVZ6Lto85kuN/l0f57bdSovk\ndV0mmwhuuOEG+MxnPgNTU1Pw3e9+F77zne/A1VdfDe+///7wYtdcc83wfHjBzElWkgRKbu6VjPAR\ne8k251raeTQvihLLgdxjTx1ElEGb5UZKWkp+F/bZP176+uuvw44dO+A///kPHDp0CPbu3bsufzAY\njHwZELHP3UiLnkcc1CvbNj/aDqs/WloKmqYR64g4FJbzji0nwHzPLtJunob10bq1fN4/yU5qdxSp\n9hKyiWDHjh0AAHDttdfCPffcA4uLi8MlwXXXXQfnz5+H7du3t2rcJJBAbhQvkablR9uVmpYKbYJG\nHThSR66za9DIhju0Rk5aH1PHYtRkkPXDJB9//DF8+OGHAADw0UcfwenTp2H//v1w+PBhOHHiBAAA\nnDhxAu6+++7shpUiAVQmkqN4afSc15NyXPKzbdu24Tc9pvmazTg+U1NTSfZaP3iaNzZdjL02D7Q5\naM2n6NxNQRvyztojWFpagnvuuQcAAFZWVuAb3/gG/OhHP4JLly7BvffeC//+979hYaHd48OUTlm2\nuRFQO0+JxpEypW28tJS+ckTuSXQzLHVPwFr3N01jrum9siVsUvonnWtpVrqGnD2IXr5QtJVIQMv3\n6sklDutY6odnxxGZ8FKe5XhSGv3WSEGzS/2O2lh9ipxraVa6hui4I3r3n47GSQLW5I86V6oDe+Ui\n5JCqDqS2SNi2rd1PWnKnlPKtY/qNbeWOPRgMNthpTop51CbSB6ksHztt8zC6F0LTtLZL6aXQK0Uw\nShKIOL123EbG5zp+rnLw8iL5JRCN+FJ+RA1EFIKnCiJ1Wd/RYy8vNz1q2+ulQVckkEIKKUsCKV9y\nTC3dUwc5dlIbMaprfZTq1exylwc8TVMKKY4o1ZeT7xGCt0yIkpk0LuMig94SwbhIILIsSI24kcgt\nObYX5duQhJavjRFCWxpYZbTptLa2Ztrjt2RHHQ7zIw5cyob3yyMSrW/S+ESctDQZ9I4IUuVniprI\nWQpEyQCP20R5LV+zSSGSSB518ig5S+TC4Tm8lBaR4Fqal2eda/VoNtG2SXZaWuRcS4vkSTaSfe82\nCzWMkgQ0Z5bsrSjrOTe30Zw46vySc1ttsPqkwSMCLV+b7BH5LzntYDAQ02me5ey0PLWnbaE2vO2e\nKuL5tF28Xu/caksUXvmJIIJxkECKGvDyLIJIUQEWeWi2Xjukdmtjkgor0nkEoEVgvjRYW1vb4MjU\nwXmdFiHk9M9yeFq/9K2V0drVJRmMjQiik6tPJGBFdFpH1Imtc4D1G32pxMHVgdZ2r+/SOKYg4vz0\neG1tTSyjObdEApKq0Ow8VWD1S3JmeuyRAcJLp+iCIADGRARtSSDH1pvYbUnAi9aR/AgBSDb4zctJ\ndVt9kPK18ZLStcmoLQOkPOpMNN2S+fScKoaIw0cJQXM2iQDocZQMtOuMigx6uzSwHNuL/DxNinJS\nuhXleXqKs0dtvDwp6lsEQ9MjCkHKj4wvheYs+G0RAd1otNKtj+XoKYTA+2o5mbREkPpokYF2HLl+\nqp2E3hKBhnGQgOXI9LiNkwPElUAkX+qT1jdtjLwnCxHQicmfKHCCwOvxaM/TadTHpYSmFLT8FMdK\ncS5LIdA625KB5vS5ZNBLItAmXWQyepGMpueQgOdkqc4fcfbIXoFEELydlCS0PnmEmQo6KaempkR1\nIDkndWJuwx1NWxqsra2JJOIpBmtpoOVrqqAkGXCUJIPeEUHKhLOcXsqz0qIkoDlSKglwBweQCUBz\nZGobzbecnrZHG58ccGeRvqWXhFAJ0LTV1dV1+ZwI0PHxmOdhn1dXV1XH5oRC2+6NAXduqWyEDCg8\nYshxegm9IgJroCOObuXlkEAbIgCQZTy/nub8loNrCsFyfu2a2jhIY2qNM4JPSklaR5YEUuSmzmTl\nr66uDtUH2qHDrK2twfT0NKytra0jC+74SCicHKJkYKV7ZMCdO4cMUgmiV0SgITIBpbyuSEBzLMme\nOyq30Zw88gMckh2/Bs+X2s/7
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 26
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# sample distribution in 3D\n",
|
||
|
|
"from mpl_toolkits.mplot3d import Axes3D\n",
|
||
|
|
"subplots(1,1,figsize=(8,8))\n",
|
||
|
|
"ax = gcf().add_subplot(111,projection='3d')\n",
|
||
|
|
"ax.plot_wireframe(xs[::10,::10],ys[::10,::10],p_x[::10,::10])\n",
|
||
|
|
"ax.set_zlim3d(0.0,0.1)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 33,
|
||
|
|
"text": [
|
||
|
|
"(0.0, 0.1)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAHMCAYAAABY25iGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXecVOXZ//85bdrO7lIF2aUsvUhvIiCPHVDXEhULQogi\naox+1fyMpj958sToE2MjEDAxigqCGkFNJIoNAelNigLL7rILAiKwZeppvz+O99kzM2dmZ3bamZn7\n/Xr5Stidcs7smftzruu+rs/FqKqqgkKhUCgUSkzYbB8AhUKhUCi5ABVMCoVCoVDigAomhUKhUChx\nQAWTQqFQKJQ4oIJJoVAoFEocUMGkUCgUCiUOqGBSKBQKhRIHVDApFAqFQokDKpgUCoVCocQBFUwK\nhUKhUOKACiaFQqFQKHFABZNCoVAolDiggkmhUCgUShxQwaRQKBQKJQ6oYFIoFAqFEgdUMCkUCoVC\niQMqmBQKhUKhxAEVTAqFQqFQ4oAKJoVCoVAocUAFk0KhUCiUOKCCSaFQKBRKHFDBpFAoFAolDqhg\nUigUCoUSB1QwKRQKhUKJAyqYFAqFQqHEARVMCoVCoVDigAomhUKhUChxQAWTQqFQKJQ4oIJJoVAo\nFEocUMGkUCgUCiUOqGBSKBQKhRIHVDApFAqFQokDKpgUCoVCocQBFUxKwaKqarYPgUKh5BB8tg+A\nQskGqqrC5/NBFEXY7XbwPA+O48AwTLYPjUKhWBQqmJSCQ1EUBINBqKoKRVHg9/v1nzMMA6fTSQWU\nQqFEQAWTUjCoqgpZltHQ0ACXywWWZfX/AE0wRVEMEUmO4yAIAhVQCoVCBZNSGKiqimAwqEeXRUVF\nEXuYRAw5jtOfY4xAye+ogFIohQkVTEreY0zBGgVOkiSIogie5/Uo0wjDMCGPpwJKoRQ2VDApeYuq\nqpAkCZIkgWGYEFH0+XwIBAJgWRZ+vx+qqoJlWT1ty7JshPhRAaVQChtGpbX1lDyE7EeSQh4iXGQP\nk+d5uFwuSJIElmX1x0uSpL8Gy7Ihwtea+KmqqouoMb1LBZRCyQ9ohEnJK0iEKIoigNCoMBgMwuPx\nAACKiopCxItlWXAcB0VR4HQ6oSgKZFmGLMsIBoMANPEj/5kJKPkZiWSJePp8vhAB5XkegiBQAaVQ\ncgwqmJS8ITwFS8RIVVV4vV6Iooji4mI0NTW1KlSkelYQBABImYBKkgSPxwOHwwEAIeJJBZRCsTZU\nMCl5QbQULBEojuNQUlISUdxjFKhYYmUUUJJ6jSWg0YqIyM85jguJhsl7UwGlUKwLFUxKThMuOsZo\nLhAIwOfzweVywWazxSU+8WzpGyPHZAU0/JhlWQ7ZRyUCSip5qYBSKNmDCiYlZ1FVFaIoQpblkKhS\nURR4vV7IsoySkhK9rzL8uakSn0QElFTixnod4zGGCygRTyqgFErmoYJJyUnCeyuNKdjm5mYIgoCS\nkhJTQUm3yJgJqHEPFICeJo4nAiWQPVBjQRMRTyqgFEr6oYJJySmIY4/X64XT6QxJZ/r9fvj9fhQV\nFcFms2X5SFtgGEYXRhL9OhwOPXoMBAL6Y2K1n1ABpVCyCxVMSs5AxJKIjMvlAqBFm83NzQAQNQVr\nJYwCCiAkAhVFEX6/X29ziVX8E0tAiYORzWajAkqhpAgqmJScIDySIpDeSrvdDqfTmZOCkA4BJelq\nIp7k9zQCpVDaDhVMiqUx661UFEXvrQwGg3C73Xq/ZCKva7ZvaAVSKaDGaDuagJI2FiqgFEpsqGBS\nLEssezvyv2a9la2Ra6LQFgGN9ToEKqAUSmJQwaRYjnB7O6MgBgIBeL1eAIDb7U75gp4L1srxCKiR\nZCJQow8uFVBKoUMFk2IpovVWqqoKj8cDSZLgdrvjsrdLlFwVAzMB9fl8AKDPAG1rCjcYDOpVvEYB\n5Xk+LkN6CiWfoIJJsQyt9VbyPI/S0tIsH6X1CRe3cBMFIqCtRY6xBNT4e4fDEfdEFwoll6GCSck6\n0eZWhtvb2e12/eeU+DFWxwIIEdBAIABFUSJMFOIRUNLOY/y7hU9ioQJKySeoYFKyColawgt7FEWB\nx+OBoigp761kGKagRTeVAgpAj2IBLQVMrACpgFLyDSqYlKxBosrwFKwoivB4PLDZbKaFPcZ9TboA\nJ08qBNQ475O8BhBbQK3a1kOhRIMKJiXjEBs7WZb1Bnrjz61ob1dIJCKg5PdUQCmFABVMSkYhvZU+\nny9iQDOxtystLaWLp4WIJqBkioqZkTwVUEo+QgWTkhHCeyvN7O0cDgccDgdNs6aIdH2OREA5joMk\nSXC5XHoEKooiVFVtk4CS/WyjgIb3gVIo2YQKJiXttNZbKYpiwvZ2pHCnLaIQq+gnX4qBMnkeLMtG\nZAuiCWi0/k0zI3lSJe33+yGKIhwOBxVQSlahgklJK9F6K0k0wfN8m+zt0kGmItt8EeVoxBJQYqgQ\nbqIQS0DJ3jYRUNIHSiNQSqahgklJC7F6K0nzO8/zabG3szL5dK7xRvixBJSkX1sTUPI6xvcOF1CO\n4/T9TyqglHRABZOScqL1Vhrt7ex2O+3LK1CMAkqEL5aAApE3GmYpXEVRdB9dVVVDCohIKphCSQYq\nmJSUYty3MrO3EwQBpaWl8Pv9UBSlze9T6OYD+QK5RloTULIPHi1yjCWg5FokEShJ4VIBpSQKFUxK\nSoiVgiW9lUZ7u2wuVlRsrYuZgMqyrPftmkWgiQoogQooJVGoYFKSJtrcynTa21EKAyKeDMPA4XCE\njDKTJEmfpEIFlJIJqGBS2oxZb6XR3q65uRl2uz1qYU+yUR6NEguPWLNAUyGgDQ0NEARBH4dGBZRi\nhAompU3E6q30+XwIBAIxeyuTXXzo4kUB4hPQeGeBkp+ToiQagVLCoYJJSRhFUeD1euH3+0OiR1mW\n4fF4wDBMTtvbUVN3a5HI3yOWgIqiCL/fH1NASdYiWgTq8/lCXIqogBYWVDApcWNMwZJqRrJI5JK9\nXbSiHysfM6VtJCqg0cTZWIxkfB0qoIUFFUxKXMTqrfR6vRBFEcXFxbpBd2skW6lKK10pbaE1AQUA\nv98fdwo3loASw3oqoPkDFUxKq5jZ2zEMA0VR0NjYCI7jLGNvR8kc+ZC6DhdQ0itM9uhbS+EaXydc\nQIkIGwXU6ESU659dIUIFkxKV1uztFEXR51bm6pc/V4+bkj6MlbUkcpQkSb/mUyGgqqqCZVk4HA4q\noDkEFUyKKbF6K71ery6ixIggUWhKlWJVjMJllsI1uhC1VUAlSdJdiAgkAiVD1amAWg8qmJQQYvVW\nGu3t3G63PvA5WyQjuFSscwcrpX4ZxnyYtlFAW5sFanytcCEmQ7mN70MF1DpQwaTohKdgw8cr+f1+\nPQUry3JWjzWZxYMuPJRUEUtAA4FA3AJq1sYiSVLIjSsV0OxDBZMCIPrcSkVR9EjSaG9Hq1wp+UYq\nrsd4BdT4nlRAcwcqmAVOeDGCsdKV9Fba7XY4nc68/EJaKd1HsQapvB6iCShxyfJ4PDQCzSGoYBYw\nsXorfT4fgsFgTHu7bEaIqYhQRVGEz+fTizbIzEQqopR0QYSNXGN2uz1lKVxRFKmAphkqmAWK0WfT\n+OWTZRnNzc1gWTZmb2UqvGCzJbjk7tzv98Nms+n/DgQCALTImjabU9KJcevDGIEqiqK3sRBHrXgF\nNDzVSwSU1Bs4HA69D5QKaNuggllgEHEgBuklJSX67wKBALxeL5xOJ+x2e15+oUi7jKqqKCkpCSle\nUlVV98INb1qnd+nZo5AifmL8bhRQEoG2VUBlWdav+/AIlApoYlDBLCDCU7DGn3s8HkiSlJC9HXlu\nrnzRRFGEx+MJ6ZszCia54yfjndpa8Vgo5NLfPh6seD5EQMm2SLiAAqHDtMNTtYRYESj5vdEHt9Cv\n7WhQwSwQwgsEjA3Uzc3N4HkepaWlcX9Jsp2STeT54W0xsizH9dxYFY+k6ZxEn7EWKwrFjLZcK7EE\nNBgMAogUULMbATMBDQaD+jxR
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 33
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# conditional distributions\n",
|
||
|
|
"p_1_given_x = p_x_given_1 * p_1 / p_x\n",
|
||
|
|
"p_2_given_x = p_x_given_2 * p_2 / p_x\n",
|
||
|
|
"imshow(p_1_given_x,cmap=cm.gray)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 32,
|
||
|
|
"text": [
|
||
|
|
"<matplotlib.image.AxesImage at 0x79ebc90>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAQIAAAD9CAYAAABX/HkoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfV+MXVX1/7p3Zl5VEDo0M00G20KtlqaxEnwgIYHikxVe\nKo1Rwh9jMPEFIuqDBk2U9sEnlMQYjH2S8oLlQZrGB4rhZYLii42h0am0pW2ElojwMDK934ffb1/X\nrK7PWmvvc86de2f2J5ncc/a/s8/ee33WZ/+5d3qDwWBAFRUVGxr9ta5ARUXF2qMSQUVFRSWCioqK\nSgQVFRVUiaCiooIqEVRUVFAHRHD8+HHasWMHbd++nQ4fPtx28RUVFR2g1+Y5gpWVFbr11lvpD3/4\nA83NzdHnP/95+u1vf0uf/vSn23pERUVFB2hVESwuLtK2bdtoYWGBZmZm6IEHHqBjx461+YiKiooO\n0CoRnD9/nrZs2TK8n5+fp/Pnz7f5iIqKig4w3WZhvV6vlTQVFRXdQVsNaJUI5ubm6OzZs8P7s2fP\n0vz8fJuPqGgJTQh5MBgM85eWM+p8UVy9epX6/e4205osyZXm5flQGa2+8d69e+n06dN05swZWl5e\npqNHj9L+/fvbfERFC2jLmEZpzL1eb12oySbv0WW+VhXB9PQ0/fznP6cvfvGLtLKyQo888kjdMVin\nGNVgXgvSiqRtutmWnpFbTpN8Vp5Wtw8jWA+sPslYSzUwCSSQMzVoy3TakPxRXL16tfs1gorxRhuG\nNQkE0CR/zvoAes64ePkcVCLYIFjPJDBuKpPXJ8dQS/K1RQaVCDYAJoEE1mra0DWakkIkT6mi4KhE\nUOGiSxKYhHUDonbWA0oMNsfjN1EHlQjWOUY53+7KqMfhzIFVVumaQDRvrjooIYNKBOsY40gCXRJL\naZ6mkM/scgrQFRlUIlin2EgkMG7rBSXEECWEnHQ5ZFCJoOIarBUJjNL4S/KXzr+7WPiLGHoOGdRf\nKKpYhUnwxJP63Lbbtk1yrYpgHWIUi2ujHqi5aUvSl5bZ1VmBtrx+JE0lgnWGUXjLUZLApCqUtqYB\n0TRNyaASQQURjXauvxYkMWrken4vnfuloQxS0VCJYB2h6ynBKEigCwLoijByDvpE8rRl7PUcwQbG\npJNAmwQwKqVQMiXwSKEtdVDPEWxAjDsJjKNK6Ao5Zwgso48QRptkUIlgg6ItEuhSBTRVEE3SR5Cz\nIGil91QAMurSfBoqEUw4uvSEa0UCo5pCNEXu1CBiuCjei2tKBvVA0QSjyylBJYEyrNVaStN2q4pg\ng2EtSaDtAZ6Tpmm+tg8NNZH11lShVBlUIphQlAz+NrzlqDx9GwuIOemalDNO0wGPDBAqEUwg1gMJ\ndDU9WIupQWStwNtNKDV6VFbdPqwoRtvG2XW4F9ckrYeuzgugfLlEUbcP1zm6UgOjIIG2DH0cFg1z\ndgzalP85U4K6fbhOMQkk0Ia3H3dlEJ2D58r/aJ6cKUFk4ZKoEsG6xqhX3JsaextkkZumBFE14Ml8\nKy5q5CVko6ESwYSgi0Hd5jZgE8NuiwC6VgXRb/aVTAFkXNTIS3YPNFQiqFBRIs+9dG2HRerTJoGi\nsnLkuBdXog7aIIN6snACMOrtwqbz/PVIAqXPaWONJRLWlLirIhhzjHqBcBQk0BUBNFErUZQsFOau\nCeRMFdpSBpUIxhijXOxqI7wtEmhKFNH4ErQ5PchdE4iQRs5iI0clgnWGNhcAc8JzDbwLuTvKnYSc\nk4Pe7kGuEZeqg7prMIHoYkowChJow+M3IZ9IXBuwDF/Ge6SACMFLUzpV0FCJYAwxriRQYsCe8bY1\ndbDCvbhSWOsAPL6JIrCMGxFG9GgyRyWCdYC2B3kXJNAFieSElaThiB7QyfX+WphnzJ4aiKgFiUoE\nY4ZRLnBFn9W2kUfjIvel4bmwyonsEOTKf08dROLrGsEGQtdTgiZG7t23rRhQWCQuF6Xbg7mKQBJG\nRD1YZIBQDxRNMEa1OIjSbFQSKHlWU4LNSZtTTkJVBGOELqYF0WfkDt625H7uIC5dNyhJIxGR2hHp\nn8I8ddDE+3vTCIlKBBOKUjUQTdsFCbRFAE3XDUqBymu6QJgz3y8lA6v+RJUIxgZNDbfNstaKBJoo\nCSu8K6WleXoehxSAFoYUgJUOxZWcJahEMAZYqylB03lrE0PvatqBwiJxOdAMG8XlkoCWN0cRWISC\nUIlgwlCiBkrnzE1JoK1rL85Lr6HpGoFm2Fpc7loAWheIrh14pIFQiWCN0aYayCGBtubkEVnvpWlD\ndXh5ESJprClAio9MD6wyLc8fTRMhA4RKBGuIXBJoizRyvGXbBj7KaYMXHoWVH8l+mS86z0dhnAQs\nYkDh9YjxOkFbUnctSaCLsqIqoE3lZaGNRcGcKYK1QJhDBvVA0Rqh6ymBFp4zBeiCBLpUCtq9F94G\numjjSHqZtpSQE6oimABYHTiuJJATHyWLSF299G3CWzvw0ljqQKbx1IOnIjqbGiwsLNDHPvYxmpqa\nopmZGVpcXKTLly/TV77yFfrnP/9JCwsL9MILL9AnPvGJ0kesW+QMzK4G8ahIIOrFupp2WGFdwFog\nRPERI47EW4uFHhn0BtFvJQjcfPPN9Kc//Ymuv/76YdiTTz5JN9xwAz355JN0+PBhunLlCh06dGj1\nA0fUIeOMaBt46UrVQMTA5H0bsr8tgrCutXsU1gTIbGQ4v/euUV70WRL2wQcfqHVvRASvv/46ffKT\nnxyG7dixg06ePEmzs7N08eJFuuuuu+hvf/vb6gducCJoSw10PSVoQgKRuLZIJnqPwkqhmU2UBFBc\nxKhzSUOGtU4En/rUp+jjH/84TU1N0Te/+U36xje+Qddddx1duXJl+NDrr79+eD98YCWCVtKVzo3H\nnQSaThvQPQorRRMiiHpvLb6ULNL1f/7zH7XuxWsEr732Gm3evJn+9a9/0b59+2jHjh2r4nu93oY3\neom22qO0nKi0luGlawNdEsBaE4E35y4Fmu9H1gS0z//+97+0srLiPreYCDZv3kxERDfeeCPdf//9\ntLi4OJwS3HTTTXThwgXatGlTafHrDms9JcidX+eQQI6Bt6UYou+B7rsGWjS0DhbllMPjiTAZzMzM\n0PT0/8x8eXlZLafoHMGHH35I77//PhH9v8WHEydO0K5du2j//v105MgRIiI6cuQI3XfffSXFV2Si\n1AiakAD/42Eov5YW/UXSWH/9fr9R/nH8i7ZrpP/UsVCyRrC0tET3338/ERF99NFH9NWvfpW+//3v\n0+XLl+nAgQP01ltv0cKCvn04amYeB+S+c9Rwo2GWh0Xhlue10ljEkBvv1QfVUbvXkNMvETOJLA5G\n1gei6wFeOi3+/fffb3exsBSVCMrSRsMjJICuI/GTRgJW2+eOxS63DEdFBv/+97/bXSysaB9tKQct\n3iOSSHxTEohK2CZEkEOMueB5u/KfkTWBXq+nptMWEbVwDZUIOkYbCihX/st7bxrg5Svx8NZnLlFY\n5cm6e+TXhSLVjEwaYxtAJKAtQCIyQKhEMCbInRLkprHSNvG80oDbSqM9zzP+6JRAg1b+1atXzfRW\nHCeGVE60TtHdBF4mUgjR51Yi6BBdeJ9IuRHZnOt9Swy8KRFo6YmI+v2+2g4WmZZMEaampobXSFYj\nI+dkkOrL5+w5YyNCDCi93F5EqETQEZp6aRReOiVogwSihm7lLykjQSMAlAbFl5KzNKyEqakpuEDH\n30NK9pTOqo+1DsDroqkCa/qgoRLBmKLplMAjl65JoA2S8PJb+aw2kO1hpZPTA0TOmuf1FEBKaxlp\nk0NH6CSihkoEHaANNVCSNmr8WvomJNA0Xl6j53tE4b1jTpoEqTAii4LS+Dgx5CoAz4DR0WIUj1CJ\nYA1RKlO9siwD154bMbh0HTHypkQRiYvW13rfCDSj0u7l9IA/W1MI2jOuXr0aOpqM4Bl9VQQjRHSQ\nWelyvViONI6qBu15bZGApTos
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 32
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [
|
||
|
|
"# conditional distribution in 3D\n",
|
||
|
|
"from mpl_toolkits.mplot3d import Axes3D\n",
|
||
|
|
"subplots(1,1,figsize=(8,8))\n",
|
||
|
|
"ax = gcf().add_subplot(111,projection='3d')\n",
|
||
|
|
"ax.plot_wireframe(xs[::10,::10],ys[::10,::10],p_1_given_x[::10,::10])\n",
|
||
|
|
"ax.set_zlim3d(0.0,1.0)"
|
||
|
|
],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"output_type": "pyout",
|
||
|
|
"prompt_number": 31,
|
||
|
|
"text": [
|
||
|
|
"(0.0, 1.0)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"output_type": "display_data",
|
||
|
|
"png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAHMCAYAAABY25iGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXm8VfP6x99rr7XHU50mhQZDIaTIUInETaOISrlCuK5S\n4uJeQ8YrQ7oXUUqhQRoUt0FUojRPCNfQJaJCfmk6Z49r+v2xrH323mdP55x9OtP3/XqdF52zh7X3\nXvv7Wc/zfZ7PI5mmaSIQCAQCgSAtjoo+AIFAIBAIqgJCMAUCgUAgyAIhmAKBQCAQZIEQTIFAIBAI\nskAIpkAgEAgEWSAEUyAQCASCLBCCKRAIBAJBFgjBFAgEAoEgC4RgCgQCgUCQBUIwBQKBQCDIAiGY\nAoFAIBBkgRBMgUAgEAiyQAimQCAQCARZIARTIBAIBIIsEIIpEAgEAkEWCMEUCAQCgSALhGAKBAKB\nQJAFQjAFAoFAIMgCIZgCgUAgEGSBEEyBQCAQCLJACKZAIBAIBFkgBFMgEAgEgiwQgikQCAQCQRYI\nwRQIBAKBIAuEYAoEAoFAkAVCMAUCgUAgyAIhmAKBQCAQZIEQTIFAIBAIskAIpkAgEAgEWSAEUyAQ\nCASCLBCCKRAIBAJBFgjBFAgEAoEgC4RgCgQCgUCQBUIwBTUW0zQr+hAEAkEVQqnoAxAIKgLTNAkG\ng6iqitvtRlEUZFlGkqSKPjSBQFBJEYIpqHEYhkEkEsE0TQzDIBQKRX8vSRJer1cIqEAgKIYQTEGN\nwTRNdF3n0KFD+Hw+HA5H9AcswVRVNU4kZVnG6XQKARUIBEIwBTUD0zSJRCLR6DIvL6/YHqYthrIs\nR+8TG4HafxMCKhDUTIRgCqo9sSnYWIHTNA1VVVEUJRplxiJJUtzthYAKBDUbIZiCaotpmmiahqZp\nSJIUJ4rBYJBwOIzD4SAUCmGaJg6HI5q2dTgcxcRPCKhAULORTFFbL6iG2PuRdiGPLVz2HqaiKPh8\nPjRNw+FwRG+vaVr0MRwOR5zwZRI/0zSjIhqb3hUCKhBUD0SEKahW2BGiqqpAfFQYiUTw+/0A5OXl\nxYmXw+FAlmUMw8Dr9WIYBrquo+s6kUgEsMTP/kkmoPbv7EjWFs9gMBgnoIqi4HQ6hYAKBFUMIZiC\nakNiCtYWI9M0CQQCqKpK7dq1KSgoyChUdvWs0+kEyJmAapqG3+/H4/EAxImnEFCBoHIjBFNQLUiV\ngrUFSpZl6tSpU6y4J1ag0olVrIDaqdd0ApqqiMj+vSzLcdGw/dxCQAWCyosQTEGVJlF0YqO5cDhM\nMBjE5/PhcrmyEp9stvRjI8eyCmjiMeu6HrePaguoXckrBFQgqDiEYAqqLKZpoqoquq7HRZWGYRAI\nBNB1nTp16kT7KhPvmyvxKYmA2pW46R4n9hgTBdQWTyGgAsGRRwimoEqS2FsZm4ItLCzE6XRSp06d\npIJS3iKTTEBj90CBaJo4mwjUxt4DjS1ossVTCKhAUP4IwRRUKWzHnkAggNfrjUtnhkIhQqEQeXl5\nuFyuCj7SIiRJigqjHf16PJ5o9BgOh6O3Sdd+IgRUIKhYhGAKqgy2WNoi4/P5ACvaLCwsBEiZgq1M\nxAooEBeBqqpKKBSKtrmkK/5JJ6C2g5HL5RICKhDkCCGYgipBYiRlY/dWut1uvF5vlRSE8hBQO11t\ni6f9dxGBCgSlRwimoFKTrLfSMIxob2UkEqFWrVrRfsmSPG6yfcPKQC4FNDbaTiWgdhuLEFCBID1C\nMAWVlnT2dvZ/k/VWZqKqiUJpBDTd49gIARUISoYQTEGlI9HeLlYQw+EwgUAAgFq1auV8Qa8K1srZ\nCGgsZYlAY31whYAKajpCMAWVilS9laZp4vf70TSNWrVqZWVvV1KqqhgkE9BgMAgQnQFa2hRuJBKJ\nVvHGCqiiKFkZ0gsE1QkhmIJKQ6beSkVRyM/Pr+CjrPwkiluiiYItoJkix3QCGvt3j8eT9UQXgaAq\nIwRTUOGkmluZaG/ndrujvxdkT2x1LBAnoOFwGMMwipkoZCOgdjtP7OeWOIlFCKigOiEEU1Ch2FFL\nYmGPYRj4/X4Mw8h5b6UkSTVadHMpoEA0igUrBWxbAQoBFVQ3hGAKKgw7qkxMwaqqit/vx+VyJS3s\nid3XFAtw2cmFgMbO+7QfA9ILaGVt6xEIUiEEU3DEsW3sdF2PNtDH/r4y2tvVJEoioPbfhYAKagJC\nMAVHFLu3MhgMFhvQbNvb5efni8WzEpFKQO0pKsmM5IWACqojQjAFR4TE3spk9nYejwePxyPSrDmi\nvN5HW0BlWUbTNHw+XzQCVVUV0zRLJaD2fnasgCb2gQoEFYkQTEG5k6m3UlXVEtvb2YU7pRGFdEU/\n1aUY6Ei+DofDUSxbkEpAU/VvJjOSt6ukQ6EQqqri8XiEgAoqFCGYgnIlVW+lHU0oilIqe7vy4EhF\nttVFlFORTkBtQ4VEE4V0AmrvbdsCaveBighUcKQRgikoF9L1VtrN74qilIu9XWWmOr3WbCP8dAJq\np18zCaj9OLHPnSigsixH9z+FgArKAyGYgpyTqrcy1t7O7XaLvrwaSqyA2sKXTkCh+IVGshSuYRhR\nH13TNOMKiOxUsEBQFoRgCnJK7L5VMns7p9NJfn4+oVAIwzBK/Tw13XygumCfI5kE1N4HTxU5phNQ\n+1y0I1A7hSsEVFBShGAKckK6FKzdWxlrb1eRi5UQ28pLMgHVdT3at5ssAi2pgNoIARWUFCGYgjKT\nam5ledrbCWoGtnhKkoTH44kbZaZpWnSSihBQwZFACKag1CTrrYy1tyssLMTtdqcs7ClrlCeixJpH\nulmguRDQQ4cO4XQ6o+PQhIAKYhGCKSgV6Xorg8Eg4XA4bW9lWRcfsXgJIDsBzXYWqP17uyhJRKCC\nRIRgCkqMYRgEAgFCoVBc9KjrOn6/H0mSqrS9nTB1r1yU5PNIJ6CqqhIKhdIKqJ21SBWBBoPBOJci\nIaA1CyGYgqyJTcHa1Yz2IlGV7O1SFf1U5mMWlI6SCmgqcY4tRop9HCGgNQshmIKsSNdbGQgEUFWV\n2rVrRw26M1HWSlVR6SooDZkEFCAUCmWdwk0noLZhvRDQ6oMQTEFGktnbSZKEYRgcPnwYWZYrjb2d\n4MhRHVLXiQJq9wrbe/SZUrixj5MooLYIxwporBNRVX/vaiJCMAUpyWRvZxhGdG5lVf3yV9XjFpQf\nsZW1duSoaVr0nM+FgJqmicPhwOPxCAGtQgjBFCQlXW9lIBCIiqhtRFBSREpVUFmJFa5kKdxYF6LS\nCqimaVEXIhs7ArWHqgsBrXwIwRTEka63MtberlatWtGBzxVFWQRXiHXVoTKlfiUp+TDtWAHNNAs0\n9rEShdgeyh37PEJAKw9CMAVRElOwieOVQqFQNAWr63qFHmtZFg+x8AhyRToBDYfDWQtosjYWTdPi\nLlyFgFY8QjAFQOq5lYZhRCPJWHs7UeUqqG7k4nzMVkBjn1MIaNVBCGYNJ7EYIbbS1e6tdLvdeL3e\navmFrEzpPkHlIJfnQyoBtV2y/H6/iECrEEIwazDpeiuDwSCRSCStvV1FRoi5iFBVVSUYDEaLNuyZ\niUJEBeWFLWz2OeZ2u3OWwlVVVQhoOSMEs4YS67MZ++XTdZ3CwkIcDkfa3spceMFWlODaV+ehUAiX\nyxX9dzgcBqzIWjSbC8qT2K2P2AjUMIxoG4vtqJWtgCamem0BtesNPB5PtA9UCGjpEIJZw7DFwTZI\nr1OnTvRv4XCYQCCA1+vF7XZXyy+U3S5jmiZ16tSJK14yTTPqhZvYtC6u0iuOmhTx28bvsQJqR6Cl\nFVBd16PnfWIEKgS0ZAjBrEEkpmBjf+/3+9E0rUT2dvZ9q8oXTVVV/H5/XN9crGDaV/z2eKfSVjzW\nFKrSZ58NlfH12AJqb4skCijED9NOTNXapItA7b/H+uDW9HM7FUIwawiJBQKxDdSFhYUoikJ+fn7W\nX5KKTsmW5P6JbTG6rmd133QV
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"prompt_number": 31
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "heading",
|
||
|
|
"level": 1,
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"Bayesian Decision Theory"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(zero-one loss)\n",
|
||
|
|
"\n",
|
||
|
|
"Above, we used a _zero-one loss_. That is, we used a cost of 1 for an error and\n",
|
||
|
|
"a cost of 0 when there was no error.\n",
|
||
|
|
"\n",
|
||
|
|
"In the general case, we have a 2D table of costs, saying how much of a penalty\n",
|
||
|
|
"we pay if the decision is one thing and the state of nature is another thing.\n",
|
||
|
|
"\n",
|
||
|
|
" state 0 state 1\n",
|
||
|
|
"\n",
|
||
|
|
" action 0 0 1\n",
|
||
|
|
"\n",
|
||
|
|
" action 1 1 0"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(other losses)\n",
|
||
|
|
"\n",
|
||
|
|
"These costs need not be at all like this. In fact, costs frequently are\n",
|
||
|
|
"asymmetric.\n",
|
||
|
|
"\n",
|
||
|
|
" state 0 state 1\n",
|
||
|
|
"\n",
|
||
|
|
" action 0 0 10\n",
|
||
|
|
"\n",
|
||
|
|
" action 1 1000 0\n",
|
||
|
|
"\n",
|
||
|
|
"\n",
|
||
|
|
"We call this matrix the _loss matrix_ and write the elements\n",
|
||
|
|
"as $\\lambda_{ij}$, or a function $\\Lambda(\\alpha,\\omega)$, which\n",
|
||
|
|
"is the cost of taking action $\\alpha$ when the state of nature\n",
|
||
|
|
"is $\\omega$."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(relationship to game theory)\n",
|
||
|
|
"\n",
|
||
|
|
"We will see these kinds of matrices much more later on when we talk about\n",
|
||
|
|
"_game theory_.\n",
|
||
|
|
"\n",
|
||
|
|
"- In Bayesian decision theory, nature picks a state at random and we take an action in response.\n",
|
||
|
|
"- In game theory, we need to make a decision and then someone else is going to make a decision in response."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(risk = expected loss)\n",
|
||
|
|
"\n",
|
||
|
|
"In general, we want to minimize expected loss. \n",
|
||
|
|
"We call expected loss the _risk_ of a decision.\n",
|
||
|
|
"\n",
|
||
|
|
"Just like there are conditional probabilities, there\n",
|
||
|
|
"are conditional risks.\n",
|
||
|
|
"\n",
|
||
|
|
"$$R(\\alpha_1|x) = \\lambda_{11} P(\\omega_1|x) + \\lambda_{12} P(\\omega_2|x)$$"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(minimizing expected loss)\n",
|
||
|
|
"\n",
|
||
|
|
"We want to minimize overall risk, and for that we minimize risk at each point $x$.\n",
|
||
|
|
"To do that, we choose action $\\alpha_1$ if the risk of that action is lower than\n",
|
||
|
|
"action $\\alpha_2$ and vice versa.\n",
|
||
|
|
"\n",
|
||
|
|
"\\begin{eqnarray}\n",
|
||
|
|
"R(\\alpha_1|x) & \\leq & R(\\alpha_2|x) \\\\\\\n",
|
||
|
|
"\\lambda_{11} P(\\omega_1|x) + \\lambda_{12} P(\\omega_2|x) & \\leq & \n",
|
||
|
|
"\\lambda_{21} P(\\omega_1|x) + \\lambda_{22} P(\\omega_2|x)\\\\\\\n",
|
||
|
|
"(\\lambda_{12}-\\lambda_{22}) P(\\omega_2|x) & \\leq &\n",
|
||
|
|
"(\\lambda_{21}-\\lambda_{11}) P(\\omega_1|x) \\\\\\\n",
|
||
|
|
"\\end{eqnarray}"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"(likelihood ratio vs risk)\n",
|
||
|
|
"\n",
|
||
|
|
"With _Bayes Rule_, we can transform this into a decision rule based on the likelihood ratio:\n",
|
||
|
|
"\n",
|
||
|
|
"\\begin{equation}\n",
|
||
|
|
"\\frac{p(x|\\omega_1)}{p(x|\\omega_2)} \\geq\n",
|
||
|
|
"\\frac{\\lambda_{12}-\\lambda_{22}}{\\lambda_{21}-\\lambda_{11}} \n",
|
||
|
|
"\\frac{P(\\omega_2)}{P(\\omega_1)}\n",
|
||
|
|
"\\end{equation}"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"collapsed": false,
|
||
|
|
"input": [],
|
||
|
|
"language": "python",
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": []
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"metadata": {}
|
||
|
|
}
|
||
|
|
]
|
||
|
|
}
|