nvidia-AI.summerschool17-no.../teaching-lw/210-normal-distributions.ipynb

488 lines
281 KiB
Text
Raw Permalink Normal View History

2017-09-22 17:26:46 +00:00
{
"metadata": {
"name": "210-normal-distributions"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Title: Normal Distributions\n",
"Author: Thomas Breuel\n",
"Institution: UniKL"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"\n",
"from pylab import *\n",
"from math import atan2"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 31
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Normal Distributions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Normal distributions have pdfs of the form:\n",
"\n",
"$$ p(x) \\propto e^\\frac{-x^2}{2} $$"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# standard Gaussian\n",
"xs = linspace(-5.0,5.0)\n",
"plot(xs,exp(-xs**2/2))"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 2,
"text": [
"[<matplotlib.lines.Line2D at 0x2d80850>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD9CAYAAAChtfywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVOW+B/DvKJDmJU1Qi0FJQQdUQEXRTCXT8FDSKa2o\ntvbkJY9nW9v208lt5+yUvU9eupxOyZPhftR2WWzaXQ6WiKk1aqbgLTUvhAI6UFLktVAGxnX+eBsU\nhGGANetda8338zw8MLBg/Ubhy8tvve+7LIqiKCAiIsNqI7sAIiJqHQY5EZHBMciJiAyOQU5EZHAM\nciIig2OQExEZXJNBPn36dPTo0QODBg1q9Jinn34akZGRiI2Nxf79+1UtkIiIPGsyyJ944gnk5uY2\n+vGcnBwcP34chYWFWLlyJebMmaNqgURE5FmTQT569Gh07dq10Y+vW7cOjz/+OAAgISEB586dQ3l5\nuXoVEhGRR63ukZeVlSEsLKz2sdVqRWlpaWu/LBEReSlAjS9Sf5W/xWK57piG3kdERE1raieVVo/I\nQ0ND4XA4ah+XlpYiNDS00WLM+rJw4ULpNfD5efcyb56CBx9UsGWLeJk2bSG2bFGwaZOCXr3E+2TX\nyP87Pj/3izdaPSJPSUlBeno6UlNTsWvXLnTp0gU9evRo7Zcl8om9e4H33wcOHwaCg8X7tm0Dxo0T\nb6enA//2b8DBg0C7dvLqJGqOJoP8kUcewdatW1FRUYGwsDCkpaWhuroaADB79mwkJycjJycHERER\n6NChA9asWePzoolaoqYGmDULeOmlqyFe36RJwN//Drz4IvDXv2pbH1FLNRnkmZmZTX6R9PR0VYox\nssTERNkl+JQZnt8bbwBduwLTptV9f/3n9sYbQGws8MgjQHS0dvX5ihn+7zwx+/PzhkXxtgnT2hNZ\nLF73e4jUdvIkMHQosHMnEBnZ9PFvvglkZgJbtwJtuP6ZJPImO/ktSqanKMDcucC8ed6FOCD65DU1\nwKpVvq2NSA0ckZPpffghsHAhsH8/EBTk/ecdOgTcdZe48Nmzp+/qI/LEm+xkkJOpnT8v+txZWcAd\ndzT/8xcsAEpKRJuFSAYGOfm9Z54BKiuBjIyWfX5lJTBoEPC3v12dokikJQY5+bXqatES2b8f6NWr\n5V9nxQrgq6+A995TrzYib/FiJ/m1LVuA/v1bF+IA8MADQE4OcOmSOnURqY1BTqb1wQfAQw+1/uv0\n6AEMGQJs3Nj6r0XkCwxyMiWnE8jOBqZMUefrPfig+MVApEcMcjKlLVsAmw2wWtX5emyvkJ4xyMmU\nPvhAjKLV0r27WBnq4WZZRNIwyMl01G6ruD30ENsrpE8McjKdzZvFIiC12ipuDzwAbNjA9grpD4Oc\nTEfttopbSAgQHy/CnEhPGORkKlVVwLp16rdV3NheIT1ikJOpbN4MDBgANHK3wVa7/35xwbOy0jdf\nn6glGORkKr5qq7iFhADDhrG9QvrCICfTcLdVJk/27XnYXiG9YZCTaXz+OTBwoO/aKm5sr5DeMMjJ\nNP75T3X2VmlKcDCQkCBWehLpAYOcTOHyZeDTT33fVnFje4X0hEFOprBpk7gBxK23anO+++8XuyGy\nvUJ6wCAnU9i0CUhO1u583bqJfvzXX2t3TqLGMMjJFLZuBcaO1facY8eK8xLJxiAnwztzBigqEsvn\ntZSYyCAnfWCQk+Ft3w6MHAkEBmp73ttvB/bt4yZaJB+DnAzPbte+rQIAHTuKPvmuXdqfm+haDHIy\nvK1bRZtDBrZXSA8Y5GRo584BhYVi/xMZeMGT9IBBToa2fTswfDgQFCTn/KNGAbt3iwVJRLIwyMnQ\nZLZVAKBzZ3E3ovx8eTUQMcjJ0GTMH6+P7RWSjUFOhnX+PHD0qGityDR2rJg5QyQLg5wMa8cOcZGz\nXTu5ddxxh2itOJ1y6yD/xSAnw9JDWwUAunQB+vUTFz2JZGCQk2HJWgjUELZXSKYmgzw3Nxc2mw2R\nkZFYtmzZdR+vqKjAxIkTERcXh4EDB+Ltt9/2RZ1EdVy8CBw+DIwYIbsSgQuDSCaLoihKYx90uVzo\n378/Nm/ejNDQUAwbNgyZmZmIioqqPWbRokWoqqrCkiVLUFFRgf79+6O8vBwBAQF1T2SxwMOpiJpl\n40bgxReBbdtkVyKcPQv07g38/LP2e76QuXmTnR5H5Pn5+YiIiEB4eDgCAwORmpqK7OzsOsfccsst\nuHDhAgDgwoUL6Nat23UhTqQ2PbVVAKBrV6BPH2DPHtmVkD/ymLhlZWUICwurfWy1WpGXl1fnmFmz\nZmHcuHG49dZbcfHiRXzg4f5XixYtqn07MTERiTJXcpChbd0K/PWvsquoy91eGTlSdiVkZHa7HfZm\nXnDxGOQWi6XJL7B48WLExcXBbrfjxIkTmDBhAg4cOIBOnTpdd+y1QU7UUr/+Chw4oL/AHDsWWLkS\n+NOfZFdCRlZ/kJuWltbk53hsrYSGhsLhcNQ+djgcsFqtdY75+uuv8eCDDwIA+vbti9tuuw0FBQXN\nqZuoWb7+GoiLA268UXYldY0eLea219TIroT8jccgj4+PR2FhIUpKSuB0OpGVlYWUlJQ6x9hsNmze\nvBkAUF5ejoKCAvTp08d3FZPfk72/SmOCg4HwcHGzCSIteQzygIAApKenIykpCdHR0Xj44YcRFRWF\njIwMZGRkAACef/557NmzB7GxsRg/fjxeeukl3HzzzZoUT/5JLwuBGsJ9V0gGj9MPVT0Rpx+SCi5d\nEiPf8nJxhx69+fBDYM0aYP162ZWQWbR6+iGR3uzdC0RF6TPEAbE/+a5dAMcspCUGORlKfj6QkCC7\nisbdcgvQoQNw4oTsSsifMMjJUPLy9B3kgKiv3nILIp9ikJOhMMiJrscgJ8MoLxc3k4iMlF2JZwxy\n0hqDnAwjL0/cDaiNzr9rhw4Fvv0WqKqSXQn5C53/SBBdZYS2CiBWnPbrB3zzjexKyF8wyMkwjBLk\nANsrpC0GORnClSviVmqyb7TsLQY5aYlBToZw7BjQrRsQEiK7Eu8MH84gJ+0wyMkQjNRWAQCbDfjp\nJ6CiQnYl5A8Y5GQIRgvytm2B+HixEpXI1xjkZAhGC3KAfXLSDoOcdK+yEigoAAYPll1J8yQkcERO\n2mCQk+7t2wcMGAC0aye7kuZxBzl3QiRfY5CT7hmxrQIAPXuK7XaPH5ddCZkdg5x0z6hBDrBPTtpg\nkJPuMciJPGOQk66dPg1cvKj/HQ8bwyAnLTDISdfcOx5aLLIraZkhQ4DDh4HLl2VXQmbGICddM3Jb\nBRA7Ifbvz50QybcY5KRr7hG5kXHfFfI1BjnplssF7Nlj/CBnn5x8jUFOunXsGBAcbJwdDxvDICdf\nY5CTbuXnG7s/7maziV0QuRMi+QqDnHTL6Bc63dq0AYYN474r5DsMctItswQ5wPYK+RaDnHTp0iWx\n42FcnOxK1DFsmLhVHZEvMMhJl775RvSWjbbjYWPcQc6dEMkXGOSkS7t3i/Azi9BQIDAQOHlSdiVk\nRgxy0iWzBTnA9gr5DoOcdIlBTuQ9BjnpzvnzQGmpuCuQmTDIyVcY5KQ7e/cCsbFAQIDsStQVHy9u\nW3fliuxKyGwY5KQ7ZmyrAEC3bmLLgYIC2ZWQ2TQZ5Lm5ubDZbIiMjMSyZcsaPMZut2Pw4MEYOHAg\nEhMT1a6R/IxZgxwQo3K2V0htFkVpfGary+VC//79sXnzZoSGhmLYsGHIzMxEVFRU7THnzp3DqFGj\nsHHjRlitVlRUVCA4OPj6E1ks8HAqolq9ewObNgH9+smuRH2vvCKmIC5fLrsSMgpvstPjiDw/Px8R\nEREIDw9HYGAgUlNTkZ2dXeeY999/H5MnT4bVagWABkOcyFs//igudkZEyK7EN3jBk3zBY5CXlZUh\nLCys9rHVakVZWVmdYwoLC3HmzBnceeediI+Px7vvvuubSskv7N4t2g9tTHr1ZsgQ4NAhwOmUXQmZ\nicd5ARYvbpRYXV2Nffv2YcuW
}
],
"prompt_number": 2
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# shifted and scaled Gaussian\n",
"xs = linspace(-5.0,5.0)\n",
"plot(xs,exp(-(xs-1.0)**2/(2*0.7**2)))"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 3,
"text": [
"[<matplotlib.lines.Line2D at 0x2fffdd0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD9CAYAAAChtfywAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHy1JREFUeJzt3XtwVFWeB/BvQ4cRCYLyTjrQkG5IQhKCG4zo4kZHTCZI\nfKBWQNQFxCxjtHC0ZLV2BuKUPERrSk2puKvM+srG2bUMxUBkFVtQA0F5EwwJJEsnQHjLI0iS5uwf\nxw559iO53af73u+nqitpcvveX5Pw5eR3zr3XJIQQICKisNVLdQFERNQzDHIiojDHICciCnMMciKi\nMMcgJyIKcwxyIqIw5zXI586di2HDhiEpKanLbZ5++mnY7XZMmDABO3bs0LRAIiLyzGuQz5kzByUl\nJV1+fd26daiqqkJlZSXeffddLFiwQNMCiYjIM69BPmXKFFx//fVdfn3NmjV47LHHAABpaWk4e/Ys\n6uvrtauQiIg86nGPvK6uDjExMS3PLRYLamtre7pbIiLykVmLnbQ/y99kMnXYprM/IyIi77xdSaXH\nI/Lo6Gg4nc6W57W1tYiOju6yGL0+Fi9erLwGvr/QfW/TpwtERwsMHy7w+98LbNwo0Nysn/en9++f\nyocvehzk2dnZ+OCDDwAAW7ZswcCBAzFs2LCe7pZIN06fBr75Bti/X360WIBnnwWiooAnnwQaGlRX\nSOHOa2tl5syZ+Oabb3Dy5EnExMQgPz8fTU1NAIDc3FxkZWVh3bp1sNls6NevH1avXh3woonCyf/8\nD3DXXUD//vLxwgvycfAgMHs2sHYt8NBDqqukcOY1yAsLC73upKCgQJNiwll6errqEgJKz+8v0O+t\nsBDIy+v457GxwMMPA+vWBTbI9fy9A/T//nxhEr42YXp6IJPJ534PkV4cOQIkJsqP11zT8euHDgGT\nJwNHjwK9eJ41dcKX7OSPDlEAFRUB99zTeYgDwJgxwA03AD/+GNy6SF8Y5EQBVFgIzJzpeZusLNle\nIeouBjlRgFRVAYcPA3fc4Xm7adMY5NQzDHKiACksBB58EDB7WVLwj/8I/PQTcPx4cOoi/WGQEwWA\nEL61VQCgTx/gt78FPFybjsgjBjlRAOzeLU/0mTzZt+3ZJ6eeYJATBcAnn8jRuK+XGMrKAjZsAJqb\nA1sX6RODnEhjV64A//VfvrVV3KKigFGjgNLSwNVF+sUgJ9JYaSkQGQl4uKlWp9heoe5ikBNprLAQ\nmDXL97aKG5chUnfxFH0iDTU3yzZJaam8loo/XC5g6FBg1y55hUQigKfoEwVdaSkQE+N/iANA795A\nRgZH5eQ/BjmRhrZvB9LSuv96tleoOxjkRBratQuYMKH7r8/IAL7+Grh8WbuaSP8Y5EQa2rkTSEnp\n/usHDwYSEoDNm7WrifSPQU6kkaYmec2UxMSe7WfaNODvf9emJjIGBjmRRn76CRg5EujXr2f7+d3v\ngC++0KYmMgYGOZFGetpWcUtKAqqrgUuXer4vMgYGOZFGejrR6danD2CzAfv393xfZAwMciKNaDUi\nB+SofO9ebfZF+scgJ9KAENqNyAE5Ybpnjzb7Iv1jkBNp4MgReW2VESO02R9H5OQPBjmRBnbulKNx\nfy+U1ZWkJI7IyXcMciIN7NqlXX8ckMsYz50DzpzRbp+kXwxyIg1oOdEJAL16AePHs71CvmGQE2lA\ny4lON054kq8Y5EQ9dPEi4HQC48Zpu19OeJKvGOREPbRnDxAfD0REaLtfjsjJVwxyoh7Suj/u5h6R\n88Za5A2DnKiHAhXkQ4bI0/Xr6rTfN+kLg5yohwIx0enGPjn5gkFO1AMul+xjByrI2ScnXzDIiXrg\n4EHZAhkwIDD754icfMEgJ+qBQPXH3TgiJ18wyIl6wH2NlUAZP17eecjlCtwxKPx5DfKSkhLExcXB\nbrdjxYoVHb5+8uRJZGZmIiUlBYmJifjrX/8aiDqJQpLW11hpLzJSXlGxqipwx6Dw5zHIXS4X8vLy\nUFJSgvLychQWFmJ/u9uWFBQUYOLEidi5cyccDgeeffZZNDc3B7RoolAR6NYKwPYKeecxyMvKymCz\n2WC1WhEREYGcnBwUFxe32WbEiBE4d+4cAODcuXMYNGgQzGZz4ComChEnTsjT80eNCuxxOOFJ3nhM\n3Lq6OsTExLQ8t1gs2Lp1a5tt5s+fjzvuuANRUVE4f/48Pv300y73t2TJkpbP09PTkZ6e3r2qiUKA\ne/24Vtcg70piIvC3vwX2GBQ6HA4HHA6HX6/xGOQmH35Cly5dipSUFDgcDhw8eBBTp07Frl270L9/\n/w7btg5yonAX6IlOt6QkYPHiwB+HQkP7QW5+fr7X13hsrURHR8PpdLY8dzqdsFgsbbb5/vvv8eCD\nDwIAYmNjMXr0aFRUVPhTN1FYCvREp9vYscDhw8ClS4E/FoUnj0GempqKyspK1NTUoLGxEUVFRcjO\nzm6zTVxcHL788ksAQH19PSoqKjBmzJjAVUwUIoI1Io+IAOx2oN06A6IWHlsrZrMZBQUFyMjIgMvl\nwrx58xAfH49Vq1YBAHJzc/Hiiy9izpw5mDBhAq5cuYJXXnkFN9xwQ1CKJ1KluRmorAQSEoJzPPfK\nlRtvDM7xKLyYhAjORTJNJhOCdCiigKuqAu68E6ipCc7xli0DTp8GVq4MzvEodPiSnTyzk6gbDhyQ\nvetg4Vpy8oRBTtQNwQ5yriUnTxjkRN1w4ID29+j0ZORI4Nw54MyZ4B2TwgeDnKgbgj0i79VLXkCL\no3LqDIOcqBsqKoIb5IBsr7BPTp1hkBP56eJFeZ2VkSODe9zx44F9+4J7TAoPDHIiP1VVAbGxQO/e\nwT3u2LFy7TpRewxyIj8Fuz/uNnasPDZRewxyIj8Fe8WK26hRwLFjwC+/BP/YFNoY5ER+UjUiN5sB\nq1Xe8JmoNQY5kZ9UrFhxs9vZXqGOGOREfhBCbZBzwpM6wyAn8sOpUzLMhwxRc3xOeFJnGOREfnD3\nxwN9e7eu2O0ckVNHDHIiP6haseLGETl1hkFO5AeV/XEAiIqSF886d05dDRR6GOREflC19NCtVy/A\nZpNnlxK5MciJ/KA6yAG2V6gjBjmRj65ckSNhu11tHZzwpPYY5EQ+cjqBQYOAyEi1dXBETu0xyIl8\nFAptFYBnd1JHDHIiH6leseLmHpF7ubE6GQiDnMhHoTIiHzxYfjx1Sm0dFDoY5EQ+CpUgN5k44Ult\nMciJfBQqQQ5wwpPaYpAT+eDyZeDIEWD0aNWVSByRU2sMciIfVFXJO/RERKiuROKInFpjkBP5IJTa\nKgCDnNpikBP5INSC3G6XvyVwCSIBDHIin4RakF93nTzD9MgR1ZVQKGCQE/lA9XXIO8MJT3JjkBP5\nINRG5AD75HQVg5zIi7NngYsXgREjVFfSFm/ETG4MciIvVN+nsyu8eBa5MciJvAjFtgrA1gpd5TXI\nS0pKEBcXB7vdjhUrVnS6jcPhwMSJE5GYmIj09HStayRSKlSDPDYWqK4GXC7VlZBqZk9fdLlcyMvL\nw5dffono6GhMmjQJ2dnZiI+Pb9nm7NmzePLJJ/HFF1/AYrHg5MmTAS+aKJgOHADuvlt1FR317QsM\nGwb83/8BY8aoroZU8jgiLysrg81mg9VqRUREBHJyclBcXNxmm08++QQzZsyAxWIBAAx2X2OTSCdC\ndUQOcAkiSR6DvK6uDjExMS3PLRYL6urq2mxTWVmJ06dP4/bbb0dqaio+/PDDwFRKpIAQMshV36ez\nK+yTE+CltWLyYZq+qakJ27dvx1dffYWGhgZMnjwZN998M+yd/OQvWbKk5fP09HT20ynkHT0KXHst\ncP31qivpHINcfxwOBxwOh1+v8Rjk0dHRcDqdLc+dTmdLC8UtJiYGgwcPRt++fdG3b1/cdttt2LVr\nl9cgJwoHodxWAeRvCiUlqqsgLbUf5Obn53t9jcfWSmpqKiorK1FTU4PGxkYUFRUhOzu7zTb33HMP\nvv32W7hcLjQ0NGDr1q1ISEjo
}
],
"prompt_number": 3
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(normalization)\n",
"\n",
"We want $\\int f(x) dx = 1$ for the pdf, so the whole Gaussian looks like:\n",
"\n",
"$$ f(x) = \\frac{1}{\\sqrt{2 \\pi \\sigma^2}} e^{ -\\frac{(x-\\mu)^2}{2\\sigma^2} } $$"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Important Properties"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Important properties of normal distribution:\n",
"\n",
"- all cumulants higher than second order are zero (cumulants are roughly \"excess moments\", higher order moments you wouldn't predict from lower order moments)\n",
"- it's essentially the only (absolutely continuous) distribution for which that is true\n",
"- it is the maximum entropy distribution for a given mean and variance\n",
"- it is the limit distribution for the sum of large numbers of small random variables\n",
"- in problems involving Gaussian error models, the least square solution minimizes the error"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Central Limit Theorem"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(central limit theorem)\n",
"\n",
"The sum of a lot of random variables tends towards a Gaussian distribution (under some conditions)."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# central limit theorem\n",
"samples = [sum(rand(1000)) for i in range(10000)]\n",
"_=hist(samples,bins=100,alpha=0.3)\n",
"_=hist(randn(10000)*var(samples)**.5+mean(samples),bins=100,alpha=0.3)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD9CAYAAABdoNd6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X9QVPe9//HnqtiEqkG3ZU0XWyzq4Io/thK0t2YuGUUn\nN5UxTaTF1nKj3ukk98795upk0tAxV++dXLi3zWSqt5T01tzqZBKVb78GchsItQ0a7G1WVylNSMKP\nYoDlRyIEhU1YBM73D7pb8Ae/4eyyr8eMM+thz/JS4M1nP+dz3h+LYRgGIiIyrc0wO4CIiEw+FXsR\nkTCgYi8iEgZU7EVEwoCKvYhIGFCxFxEJA0MW+66uLtatW8eaNWtwOBw89dRTABw4cICYmBicTidO\np5PCwsLAOVlZWSxdupT4+HiKi4snN72IiIyIZbh19p988gmRkZH09PSwYcMGfvSjH/Gb3/yGuXPn\nsnfv3kHPraioYMeOHZw/fx6Px8OmTZuorKxkxgy9gRARMdOwVTgyMhKA7u5uent7mT9/PgC3+h2R\nn59Peno6ERERxMbGsmTJElwu1wRHFhGR0Rq22Pf19bFmzRpsNhv33XcfK1asAODw4cOsXr2a3bt3\n097eDkBjYyMxMTGBc2NiYvB4PJMUXURERmrWcE+YMWMGZWVlXL16lS1btlBSUsKjjz7K008/DcD+\n/fvZt28fR44cueX5FotlRMdERGR4Y+1wM+LJ9LvuuosHHniACxcuEB0djcViwWKxsGfPnsBUjd1u\np76+PnBOQ0MDdrv9toGD/c8///M/m55huuQMhYzKqZzB/mc8hiz2V65cCUzRfPrpp/z617/G6XTS\n3NwceM6pU6dYuXIlAKmpqRw/fpzu7m5qa2upqqoiKSlpXAFFRGT8hpzGaWpqIiMjg76+Pvr6+ti5\ncycbN27ku9/9LmVlZVgsFhYvXszzzz8PgMPhIC0tDYfDwaxZs8jJydGUjYhIEBiy2K9cuZKLFy/e\ndPzYsWO3PSczM5PMzMzxJwsCycnJZkcYkVDIGQoZQTknmnIGj2HX2U/KJ7VYxj3/JCISbsZTO3W3\nk4hIGFCxFxEJAyr2IiJhQMVeRCQMqNiLiIQBFXsRkTCgYi8iEgZU7EVEwoCKvYhIGFCxFxEJAyr2\nIiJhQMVeRCQMqNiLiIQBFXsRkTCgYi8iEgZU7EVEwoCKvYhIGFCxFxEJAyr2IiJhQMVeRCQMqNiL\niIQBFXsRkTAwZLHv6upi3bp1rFmzBofDwVNPPQVAW1sbKSkpLFu2jM2bN9Pe3h44Jysri6VLlxIf\nH09xcfHkphcZhbyCPHJfyiX3pVzyCvLMjiMypYYs9nfccQdvvPEGZWVllJeX88Ybb1BaWkp2djYp\nKSlUVlayceNGsrOzAaioqODEiRNUVFRQVFTEY489Rl9f35T8Q0SG09rZin2tHftaO62drWbHEZlS\nw07jREZGAtDd3U1vby/z58+noKCAjIwMADIyMnjllVcAyM/PJz09nYiICGJjY1myZAkul2sS44uM\nn3/Er9G+TGezhntCX18fX/nKV6ipqeHRRx9lxYoVtLS0YLPZALDZbLS0tADQ2NjI+vXrA+fGxMTg\n8Xhu+boHDhwIPE5OTiY5OXkc/wyRsfOP+D3uW3+vipilpKSEkpKSCXmtYYv9jBkzKCsr4+rVq2zZ\nsoU33nhj0MctFgsWi+W259/uYwOLvYiI3OzGgfDBgwfH/FrDFnu/u+66iwceeAC3243NZqO5uZmF\nCxfS1NREdHQ0AHa7nfr6+sA5DQ0N2O32MYcTmUruMje55AJgnWNle+p2kxOJTJwh5+yvXLkSWGnz\n6aef8utf/xqn00lqaipHjx4F4OjRo2zbtg2A1NRUjh8/Tnd3N7W1tVRVVZGUlDTJ/wSR2xu4Asf9\nB/eQz/V2e3UBV6atIUf2TU1NZGRk0NfXR19fHzt37mTjxo04nU7S0tI4cuQIsbGxnDx5EgCHw0Fa\nWhoOh4NZs2aRk5Mz5BSPyGTzz8cDnHWdHfb5pecu0tHRQ3tFK9aI19m+fQsAeXmv09raDYDVOjtw\nXCRUDFnsV65cycWLF286vmDBAk6fPn3LczIzM8nMzJyYdCKTyF/Az14qZ1FX/yKDjo4erAuSIMoT\nKO4Ara3d2O1bAfB4XjUlr8h4jHjOXmQ6cZe58V4tJypqFVeuXieqo8fsSCKTSsVewpK320uUIw7r\nAjvv1Aw/vTOcvII8WjtbdWFXgpZ644hMAP+1AV3YlWClYi8iEgY0jSNhxb/apqrSw/zeCKxaGixh\nQsVeph3//DmA+w9uartaAgX+Sm8E65IeJjKyBl/X7Zv01dS6aW8vh5c8WOdYgTumKL3I5NA0jkw7\nA7tben3ewHLKyEj7kAV+oK4+L1EOq+bhZdpQsRcRCQOaxhEZRmBNfq2HuZ+xsvgLmtKR0KORvcgw\n+tfkW7Em2OnwaUpHQpNG9iITwH2hgvKPPJQXu3BfqGBtokM3WElQUbEXGYUbV+n4i7nX20PcgiRm\n3llDxJfu0mYoEnRU7CWkTXWbgv5VOnEjLubqlinBQsVeQlprZyu1XS2cPVNOa+MdWK2zzY40iLpl\nSrBQsZeQ19HRQ1TUKuz2rf0Fdd7oX6O6+jIf14zurtqBO1vVNVYTN/pPKzJlVOxFAF8Xo7rpCv6y\nsxWA70Vf4Hh19WUKi6C9opXP9rUHRvYiZlKxl2mhptZNIbm0t5fz2ajrgSJsBl8XgQ1Q2uu1VFOC\ng4q9BL2RXOTs6vNiTbBDm4f22poJ+bz+Ebqapsl0oGIvQc+si5z+EfpwTdNEQoHuoBURCQMa2UtI\nGrhZ+JXZXrPjiAQ9jewlJPmndqKiVmmKRWQEVOxFRMLAkMW+vr6e++67jxUrVpCQkMChQ4cAOHDg\nADExMTidTpxOJ4WFhYFzsrKyWLp0KfHx8RQXF09uehERGZEh5+wjIiJ47rnnWLNmDZ2dnaxdu5aU\nlBQsFgt79+5l7969g55fUVHBiRMnqKiowOPxsGnTJiorK5kxQ28gZHxKz+fR4Wulvb0ca0EXI9km\nsPTcRS2bFPmzIavwwoULWbNmDQBz5sxh+fLleDz9zZ8Mw7jp+fn5+aSnpxMREUFsbCxLlizB5XJN\nQmwJNx2+VqwJdqIc1hFvE9jR0TPqu2JFpqsRr8a5fPkyly5dYv369Zw7d47Dhw9z7NgxEhMTefbZ\nZ4mKiqKxsZH169cHzomJiQn8crjRgQMHAo+Tk5NJTk4e8z9CJBjV1Lppu/IhhaW5g3a4Grghunre\ny1BKSkooKSmZkNcaUbHv7Ozk4Ycf5sc//jFz5szh0Ucf5emnnwZg//797Nu3jyNHjtzyXIvFcsvj\nA4u9yGgM3Caw+gM3zDU70a119XmJjJuHNcFO69seoL+Fg39DdEA972VINw6EDx48OObXGnYy/fr1\n6zz00EN85zvfYdu2bQBER0djsViwWCzs2bMnMFVjt9upr68PnNvQ0IDdbl6PEpmeBm4T6OsJnTX2\nbnc5ubmv4r5QYXYUCUNDFnvDMNi9ezcOh4PHH388cLypqSnw+NSpU6xcuRKA1NRUjh8/Tnd3N7W1\ntVRVVZGkC2MyzfT3zHFR90HT8E8ewOsFu30rXm/PJCUTub0hp3HOnTvHiy++yKpVq3A6nQD827/9\nGy+//DJlZWVYLBYWL17M888/D4DD4SAtLQ2Hw8GsWbPIycm57TSOSKjy98xpaiqgsKj/XW1bW7vJ\nqUSGNmSx37BhA319N69kuP/++297TmZmJpmZmeNPJhLkenr+3MoY6OkpMDmNyNC0AF5EJAyo2IuI\nhAEVexGRMKBiLyGluvoyVZUeqqvrzI4iElJU7CWkjGVjcBFRsRcRCQvaqUqCxsCNxSsrK1i2zAGA\n2/0OEQuizIwmEvI0speg4d99ym7fSnNzT+Cx19trdrRxq6l1U9XsorA0l7rGarPjSBjSyF5kCgxs\niub7oy9w3H2hgtxrrwJgtc5m
}
],
"prompt_number": 4
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Proof Sketch of CLT"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(proof sketch of CLT)\n",
"\n",
"- addition of random variables = convolution of their pdfs\n",
"- convolution of pdfs = product of Fourier transforms\n",
"- take Fourier transform of pdf $g(x)$, call it $G(t)$ (the *characteristic function*)\n",
"- sum of $n$ random variables with pdf $g(x)$ has Fourier transform $G^n(t)$\n",
"- take Taylor series and limit"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(characteristic function)\n",
"\n",
"The *characteristic function* of a pdf is the inverse Fourier transform of the pdf. Let $X$ be a random variable.\n",
"\n",
"$$\\varphi_X(t) = E[e^{itX}]$$\n",
"\n",
"From the Taylor series, you can infer that:\n",
"\n",
"$$\\varphi_X(0) = 1$$\n",
"\n",
"$$\\varphi'_X(0) = i E[X]$$\n",
"\n",
"$$\\varphi''_X(0) = -E[X^2]$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(proof sketch of CLT)\n",
"\n",
"Assume the mean of the r.v. is zero.\n",
"\n",
"Then, the Taylor series expansion of $G(t)$ becomes:\n",
"\n",
"$$ G(t) = 1 - \\frac{1}{2} t^2 + o(t^3) $$\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(proof of CLT)\n",
"\n",
"We need to rescale each r.v. by $\\frac{1}{\\sqrt{n}}$ to keep the second order moment constant, then:\n",
"\n",
"$$ G^n(t) = \\left(1 - \\frac{t^2}{2n} + o(\\frac{t^3}{n^{3/2}})\\right)^n $$\n",
"\n",
"Compare with:\n",
"\n",
"$$ e^x = \\lim_{n\\rightarrow\\infty} \\left( 1 + \\frac{x}{n} \\right)^n $$\n",
"\n",
"Hence:\n",
"\n",
"$$ G^n(t) \\rightarrow e^{-\\frac{t^2}{2}} $$\n",
"\n",
"But you already know that that is the Fourier transform of a Gaussian (you can also try to show it)."
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Normal Distributions in Higher Dimensions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(formula and analogy)\n",
"\n",
"In higher dimensions, normal distributions use vectors and quadratic forms instead of scalars, but they look otherwise the same:\n",
"\n",
"$$ p(x) = \\frac{1}{\\sqrt{2 \\pi \\sigma^2}} e^{ -\\frac{(x-\\mu)^2}{2\\sigma^2} } $$\n",
"\n",
"$$ p(x) = \\frac{1}{(2 \\pi)^{k/2} |\\Sigma|^{1/2}} e^{ -\\frac{(x-\\mu)^T\\Sigma^{-1}(x-\\mu)}2} $$\n",
"\n",
"The division by $\\sigma^2$ has been replaced by a multiplication by $\\Sigma^{-1}$.\n",
"\n",
"Note that, confusingly, $\\Sigma$ is a matrix of variances and covariances, while $\\sigma$ is a standard deviation (square root of variance)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(log domain computations and norms)\n",
"\n",
"Because the exponential values become very small, we often work with logarithmic values:\n",
"\n",
"$$\\log p(x) = \\hbox{const} - \\frac{1}{2} (x-\\mu)^T \\Sigma^{-1} (x-\\mu) $$\n",
"\n",
"But this is just the distance of the vectors $x$ and $\\mu$ in a norm given by $\\Sigma^{-1}$:\n",
"\n",
"$$\\log p(x) = \\hbox{const} - \\frac{1}{2} || x - \\mu ||^2_{\\Sigma^{-1}}$$\n"
]
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Generating Normal Samples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"How do we sample from a normal distribution?\n",
"\n",
"It's pretty easy to get uniformly distributed samples, but we need to transform that to get Gaussian random variables.\n",
"\n",
"You can look up the details here: http://en.wikipedia.org/wiki/Marsaglia_polar_method"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def marsaglia():\n",
" while 1:\n",
" v1 = 2*rand()-1\n",
" v2 = 2*rand()-1\n",
" s = v1**2+v2**2\n",
" if s<1: break\n",
" if s==0.0: return 0.0\n",
" return v1*sqrt(-2*log(s)/s)\n",
"\n",
"example = [marsaglia() for i in range(10000)]\n",
"_=hist(example,bins=100)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGndJREFUeJzt3X9MVff9x/HXRWgaIyik5dbc2+aagsGryL3TAH+0y2WK\nrG1ktC6s0FrS4rLYbElrs7Z20+ESBdv6R3Ux6R+0wywpdckEsyjBNLtd1y3DVsyy0FW24Hq5/Ngc\n/sBav1fhfP9AryA/7gXu5VwOz0dicrycw3lfvfd1P/dzPp/PsRmGYQgAYAlJZhcAAIgdQh0ALIRQ\nBwALIdQBwEIIdQCwEEIdACxkylC/fv26CgoK5PF45Ha7tXPnTklSTU2NnE6nvF6vvF6vTp48GT6m\ntrZW2dnZysnJUWtra3yrBwCMYYs0Tv3atWtavHixbt68qUceeURvv/22PvroI6WmpmrHjh1j9u3o\n6FBlZaVOnz6tYDCojRs36ty5c0pK4gsBAMyFiGm7ePFiSVIoFNLQ0JDS09MlSRN9FjQ3N6uiokIp\nKSlyuVzKyspSW1tbjEsGAEwmYqgPDw/L4/HIbrerqKhIq1evliQdOnRIeXl5qq6u1qVLlyRJPT09\ncjqd4WOdTqeCwWCcSgcA3C050g5JSUk6e/asLl++rJKSEvn9fm3fvl27d++WJO3atUuvvPKK6uvr\nJzzeZrNF9RgAILJIK7tE3dm9dOlSPfHEE/rss8+UmZkpm80mm82mbdu2hbtYHA6HAoFA+Jju7m45\nHI5JC0ukP7/4xS9Mr4GarFUXNVFTrP9EY8pQv3DhQrhr5ZtvvtGpU6fk9XrV19cX3ufYsWPKzc2V\nJJWWlqqxsVGhUEhdXV3q7OxUfn5+VIUAAGZvyu6X3t5eVVVVaXh4WMPDw9q6das2bNig5557TmfP\nnpXNZtOKFSv07rvvSpLcbrfKy8vldruVnJysw4cP09WCBSEtLUODgxclSamp6bpyZcDkirBQTRnq\nubm5OnPmzLjHjxw5Mukxb7zxht54443ZVzbHfD6f2SWMQ03RM7uukUA3bm2PNGTMrmki1BSdRKwp\nWhHHqcflpDZb1P1DwHww8o309mua1zfiI5rsZFYQAFgIoQ4AFkKoA4CFEOoAYCGEOgBYCKEOABZC\nqAOAhRDqAGAhhDoAWAihDgAWQqgDgIUQ6sAMpaVlhO8rMN3909Iy4lwdFioW9AJm6O5FvCIt6MWi\nX5gtFvQCYmy6rXNgrhHqwDTcWTd9qtZSMt0sME3EG08DmK6buvuGGcBcoaUOABZCqAOAhRDqAGAh\nhDoAWAihDgAWQqgDgIVMGerXr19XQUGBPB6P3G63du7cKUkaGBhQcXGxVq5cqU2bNunSpUvhY2pr\na5Wdna2cnBy1trbGt3og4SUzWQlzKuIyAdeuXdPixYt18+ZNPfLII3r77bd1/Phx3XfffXr11Ve1\nf/9+Xbx4UXV1dero6FBlZaVOnz6tYDCojRs36ty5c0pKGvvZwTIBmK+mWhpgutu8BzBdMVkmYPHi\nxZKkUCikoaEhpaen6/jx46qqqpIkVVVVqampSZLU3NysiooKpaSkyOVyKSsrS21tbbN9HgCAKEWc\nUTo8PKxvfetb+te//qXt27dr9erV6u/vl91ulyTZ7Xb19/dLknp6elRYWBg+1ul0KhgMTvh7a2pq\nwts+n08+n28WTwOYvbS0jFvLAEipqem6cmXA0udF4vP7/fL7/dM6JmKoJyUl6ezZs7p8+bJKSkr0\nhz/8YczPI/UXTvaz0aEOJII767rM7fR+s86LxHd3g3fPnj0Rj4l69MvSpUv1xBNP6PPPP5fdbldf\nX58kqbe3V5mZmZIkh8OhQCAQPqa7u1sOhyPaUwAJhEW5MD9NGeoXLlwIj2z55ptvdOrUKXm9XpWW\nlqqhoUGS1NDQoLKyMklSaWmpGhsbFQqF1NXVpc7OTuXn58f5KQDxcHtRLiPcNQLMB1N2v/T29qqq\nqkrDw8MaHh7W1q1btWHDBnm9XpWXl6u+vl4ul0tHjx6VJLndbpWXl8vtdis5OVmHDx9mKBcwoWTe\nG4gL7nwE3BLNnYxiOaSRoY6YLu58BAALDDfJACKiqwTzBy11IKI7F02BREeoA4CFEOoAYCGEOgBY\nCKEOABZCqAOAhRDqAGAhhDoAWAihjgUhLS1j3q26OB9rhvlY+wULwt1rtkz0+puLdV2ms/ZLNDVj\nYWHtFyCC0a1hwAoIdSxod+46RCsY1kCoA4CFsEojkFBYERKzQ0sdSCisCInZIdQBwEIIdQCwEEId\nACyEC6VYgLgYCeuipY4FaD5ejExmyQBEhZY6MC/c/iCSBgf5loHJTdlSDwQCKioq0urVq7VmzRod\nPHhQklRTUyOn0ymv1yuv16uTJ0+Gj6mtrVV2drZycnLU2toa3+oBAGNMuaBXX1+f+vr65PF4dPXq\nVa1bt05NTU06evSoUlNTtWPHjjH7d3R0qLKyUqdPn1YwGNTGjRt17tw5JSWN/exgQS/MtURZrCvW\ni35hYZn1gl4PPPCAPB6PJGnJkiVatWqVgsGgJE34i5ubm1VRUaGUlBS5XC5lZWWpra1tpvUDAKYp\n6j718+fPq729XYWFhfr000916NAhHTlyROvXr9eBAwe0bNky9fT0qLCwMHyM0+kMfwjcraamJrzt\n8/nk8/lm/CQAwIr8fr/8fv+0jolqPfWrV6/K5/Pp5z//ucrKyvSf//xH999/vyRp165d6u3tVX19\nvX7yk5+osLBQzzzzjCRp27Ztevzxx/XUU0+NPSndL5hjdL/ACmKynvqNGze0ZcsWPfvssyorK5Mk\nZWZmhodXbdu2LdzF4nA4FAgEwsd2d3fL4XDM5jkAAKZhylA3DEPV1dVyu9166aWXwo/39vaGt48d\nO6bc3FxJUmlpqRobGxUKhdTV1aXOzk7l5+fHqXQAwN2m7FP/9NNP9Zvf/EZr166V1+uVJO3bt08f\nfPCBzp49K5vNphUrVujdd9+VJLndbpWXl8vtdis5OVmHDx9m5h4AzCHuUYoFgT51WAH3KAWABYZQ\nBwALIdQBwEIIdQCwEEIdACyEUIdlpaVlhCfJWQtrq2NyhDosa3DwoubfzTCicecmH4ODgwQ8xuAm\nGcC8xs0zMBYtdQCwEEIdsAz62kH3C2AhdMWAljosxrojXoDoEOqwFOuOeAGiQ6gDgIUQ6gBgIYQ6\nAFgIoQ4AFkKoA5bEmPWFinHqgCUxZn2hoqUOABZCqAOAhRDqAGAhhDoAWMiUoR4IBFRUVKTVq1dr\nzZo1OnjwoCRpYGBAxcXFWrlypTZt2qRLly6Fj6mtrVV2drZycnLU2toa3+oBAGPYDMOYdJGMvr4+\n9fX1yePx6OrVq1q3bp2ampr0/vvv67777tOrr76q/fv36+LFi6qrq1NHR4cqKyt1+vRpBYNBbdy4\nUefOnVNS0tjPDpvNpilOC8zYyEJet19bbN/e5v1mDdFk55Qt9QceeEAej0eStGTJEq1atUrBYFDH\njx9XVVWVJKmqqkpNTU2SpObmZlVUVCglJUUul0tZWVlqa2uLxXMBAEQh6nHq58+fV3t7uwoKCtTf\n3y+73S5Jstvt6u/vlyT19PSosLAwfIzT6VQwGJzw99XU1IS3fT6ffD7fDMoHAOvy+/3y+/3TOiaq\nUL969aq2bNmid955R6mpqWN+Fmnt6sl+NjrUgdlIS8u4teQuYC13N3j37NkT8ZiIo19u3LihLVu2\naOvWrSorK5M00jrv6+uTJPX29iozM1OS5HA4FAgEwsd2d3fL4XBM60kA08Ua6sAdU4a6YRiqrq6W\n2+3WSy+9FH68tLRUDQ0NkqSGhoZw2JeWlqqxsVGhUEhdXV3q7OxUfn5+HMsHAIw25eiXP/3pT/r2\nt7+ttWvXhrtRamtrlZ+fr/Lycn311VdyuVw6evSoli1bJknat2+f3nvvPSUnJ+udd95RSUnJ+JMy\n+gUxxIgXRr8sFNFk55ShHi+E
}
],
"prompt_number": 5
},
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Generating Multidimensional Normal Samples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The simplest multidimensional density is one in which each dimension is just a standard normal random variable.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"ps = randn(10000,2)\n",
"xlim((-4,4))\n",
"ylim((-4,4))\n",
"figsize(5,5)\n",
"plot(ps[:,0],ps[:,1],'.',alpha=0.1)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "pyout",
"prompt_number": 40,
"text": [
"[<matplotlib.lines.Line2D at 0x3c101d0>]"
]
},
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAE1CAYAAACcI5eKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsfVlsXNl55qm9WAtZC6tY3ElREkVqb0ut3twtb3HS48QJ\nEATxDBJPEj9mAAcIEOQpHQRIHhLAcJK3PAQIAjgBEmAmYxvtLG257W53q7sttRaK4iLuS7GKtbFW\n1jYP33z+L0tVZFGkSIq6P0CQrLr33HOX891//X5DtVqtKl100UWXYyrGw56ALrroosvTFB3kdNFF\nl2MtOsjpoosux1p0kNNFF12Oteggp4suuhxr0UFOF110OdayLyBXLpfV5cuX1S//8i/vx3C66KKL\nLvsm+wJy3/72t9Xo6KgyGAz7MZwuuuiiy77JnkFucXFRff/731ff+MY3lJ5XrIsuuhw12TPI/cEf\n/IH6y7/8S2U06u49XXTR5eiJeS87f/e731XBYFBdvnxZ3bhxo+42ugmriy66PC1pxnrck/r1/vvv\nq3/7t39Tg4OD6mtf+5p655131G//9m/XnchR+PmTP/mTQ5+DPhd9Ls/qPI7aXJqVPYHcn//5n6uF\nhQU1MzOj/umf/kl9/vOfV//wD/+wlyF10UUXXfZV9tWRppumuuiiy1GTPfnktPLGG2+oN954Y7+G\neypy/fr1w57Cz0WfS33R5/K4HJV5KHW05tKsGKq7MW6f5AAGw67sZ1100UWXZqRZbNHzPnTRRZdj\nLTrI6aKLLsda9s0np4suz5rEYkrl80qZTEoFg0rpcbPjKbomp8tzK/k8fm9uKrW+frhz0eXpiQ5y\nujy3YjIpVS5Dg/P7D3s2ujwt0aOrujy3Uq1Cg/P7dVP1WZRmsUUHOV100eWZFD2FRBdddNFF6SCn\niy66HHPRQU4XXXQ51qLnyelSV/QcMl2Oi+ianC515aByyGIxpZaXlQqHEe3URZf9Fh3kdKkrB5VD\npifk6vK0RQc5XepKMKhUS4tS3d1P11TVE3J1edqi58npcqiiJ+Tq8qSiJwProosux1r0ZGBddNFF\nF6WDnC666HLMRQc5XXTR5ViLngysy76Injysy1EVXZPTZV9Ez3fT5aiKrsnpsi9iMgHgnvd8N12j\nPXqia3K67IscVPLwURddoz16omtyhyTH7Y1vMCjV3r6/Yz6L10jXaI+e6JrcIYn+xt9eYjGlZmeV\nikSUKhSenWuka7RHT/YEcvl8Xl27dk1dunRJjY6Oqj/+4z/er3kdeznKNZtHgRkkn1fKaMTvROLo\nXaNGQo1WB7ijI3su68pms8rhcKhSqaRee+019Vd/9VfqtddekwPoZV11pVpVampKKYdDKbP5aJlj\ny8v4XS5DK9lvM3QnicWUWljA8Y1GpS5cwO+jIM+iCX1c5cDKuhwOh1JKqc3NTVUul5XP59vrkM+F\nGAxKOZ34fdRM1sPWMvN5AKvJBLPvqACcUrqb4VmUPT8+lUpFXbp0SXV0dKjPfe5zanR0dD/m9VzI\nYYNJI3kSv9J+mrgmk1KVCq7JQWuRO8lRvWe6NJY9R1eNRqO6ffu2SiaT6stf/rK6ceOGun79+pZt\n3nrrrZ//ff369ce+f14lGDyaNEP0KzVjmnGbtTWlAgHRcPYCTkf1uih1tOd23OXGjRvqxo0bu95v\nX6mW/uzP/ky1tLSoP/zDP5QD6D65Z1aa8c1xm9VVmJXt7XpkUZeDkQPxyUWjUZVIJJRSSuVyOfUf\n//Ef6vLly3sZUpcjJM2YZtwmEAC46QCny1GTPZmrKysr6utf/7qqVCqqUqmo3/qt31Jf+MIX9mtu\nuhyyNGOaBYMSJa5UDnZ+uujSjOjMwLrsSZi0azQq5fMB7LRmrZ5yocvTkmaxRS/rOgZymECysqJU\nNKpUqYT/e3u3fs+Ui3AY1QuBwOGA3UFdIx3Uj54coQwkXZ5UDjN3q1hUyuPBou7qenxR02dXLCrl\n9R5eflk+r1Q8rtT8vFIPHjy9So6VFQRhlpYA/rocvuggdwxkL7lb2+W3NZP7Fgrh+0oFi7vWL6fN\nuatUDj6/jOewvq5UJgOzulhU6v793eX0NZsHWCziXlCz1eXwRQe5YyDNJu/WW6jbaYHNaIgdHajc\nCAaVyuWUmpzc+j1z7jo6ZI7x+MHVxvIcKhUc02AQrTIcVurevebm0ay2HAopZbEo1dl59BKZn1fR\nQe4YSLNF4fUW6nZaYDMaosEAP1upBC3p1Kmd56idx9TU/gJeLZDzHEolpa5cwf8tLQC9YhHBkmZM\n6Ga15Y4OpXp68KP7446G6NHV50jCYfwUi9CoOjrweaM0kWYbP1cq0OBOnWquzjQcFs41/uwXGUBt\nArPfj3MolXDeBgN8h7HY1s920oKbuRZ60OFgRW8urctjUq3CPPP5AEyHwTDCeRAw1tYE8PYjkVgL\noNrx6oFUsyDerBw2e8vzJjrI6VJX6mlzXOCHoYnsN9Ds93i7kfFxpdJpUGcdJXqo4yo6yB1j2QsY\nVauILHq9os2RnDISgfbxJFpe7Zzi8eaL+/cKqkfFTFxawnm3tj6eFK3L/suB8cnpcvCyl7w4Bgq0\n6RzaCGQ0+mRpHrVz2mmO+0lvflQ43sxmpdra8NLQaZiOjugg9wzKXjnNalNOOF57O5zyT+Ibq53T\ndnMkwMXjSDvZK7259lgGw+FRt+v9HY6m6ObqIcqTmln77XdaX0emPvPZnmTM2jltN8flZWhwuRzO\n//XXAXRPanJOTSk1NwcN1eeDJrVb5/9eTN6jYi4/b6LXrh6Q7OUBrzWzml2Qe2n/V2++hQLAoVjc\n3Txqx9Lut90cTSb4BA0GpV58UXyCSu3+WsRiSk1PY6xsVqmNDaXcbgmsVKvN3ZMnPf5e992N6GD6\nZKKD3B5lLw/40+zR2WhB1Jvvk86jdiyC1U6LkBROvb2yzV7mYLEA3JRS6gtfQG1qKIT0lKkppVwu\n/J9OK5VKYfuzZ7dGP02mrVHnarW54Mle5r5bOSgwPW6im6t7lEZ5Wc3IbszO3b7Fl5exSLNZlF2N\njGCfevPdbecwzmV9HY72ZBKaIFlGniQ6W3steIxUCpoZ51ULPGtr+H92VqnXXhOw2tzEd0phPhYL\nxvR4hFRgeHjr8Wujzpub+G4n0/eg0lb28qwdR9Gjqwcke3E2N1uOpdTuI4gmEwDOaIQmw33qzbe2\nc9hOpVacS1sb/Go+H/atVAB0T6LRxOMCTNWqHGNjA+DUKGprNmOb4WHRzGrPsVrFfCwWAf+TJ7eW\ngCn1eNS52QDPbu7jXuQoBzaOQq/eRqJrcs+I7PYtXq3CbHO5sPh32me7UqtaM7S2SoH/K6WU3Y7t\nC4WtWmc9TVSrrWUyACyvFxpluSxAxmCC9lg89soKjltP26pWhe6ovR15bDMzSvX1AdRrNTWWgDUK\nnuicdI3lMKo99GTgZ0SafaCbNYm04wUC+L8ZM6pRqVVLCyKXDBY4HDuDQb0HfrvPxsYQXXU6kcJy\n4QI+n5rCtqmUUmfOAOhqj7Ud+MdiAMFSSep0tbWqjUrKGpnK2wHqbmW7+/4slocdhimtm6s7yFFR\nr5s1Q5s1ibTjxWLNm1Ha8c1m+NnMZoxHTS6RwPdkA9aaetrjpFLC4cZe4/VMP35WqUC7MhpBUURN\n0unEZ243zoXzNBoxh3AYQN7IhMvnoVEWi9iWDCTctpH5x2uYTosZvb6+vz1Xt7vvz2Jv16NsSj+3\nIHdUsuT3+4E2mQBAa2vQYJ4EwLUpJQwq2O1KnTuH75R63FemFbcb2/t8Ak71FgE/O39eKZtNqdFR\ngJb2XOpdGy2l+v372KbRtaDZ7fMBiLVgTGCv5bfjcc1mlGgRdEslnHc9BuRaqX2JNqKAqnffjzJg\nNJKD8ks+iTy35upRiVQ9jQL1vTKNaK8NaYlqzcNaX5l27jtd21pTTan616DRtdFGT0OhxudZ65Nr\ndH0b0TMRpP3+3ZuqtWPu5P/T
}
],
"prompt_number": 40
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When the variances differ, we get distributions that are elongated along one dimension or another.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"sps = randn(10000,2)\n",
"sps[:,1] *= 2.0\n",
"xlim((-4,4))\n",
"ylim((-4,4))\n",
"plot(sps[:,0],sps[:,1],'.',alpha=0.1)\n",
"figsize(5,5)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAE1CAYAAACcI5eKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsfVlsXNl55ql9YbHIKtbCVeIiiovWVqv3brfiTiZ2knYw\nk8xg8hAHWRAgDwkSIPOcBgLED8FgYGDmOUAQYPKY8TieOPbEit3tsd2LpFaLLYqbuNfG2ousvebh\n85f/1mVVsShSbIqqHxBEsu4999xT93z3X7/fUKvVaqojHelIR86oGL/oCXSkIx3pyNOUDsh1pCMd\nOdPSAbmOdKQjZ1o6INeRjnTkTEsH5DrSkY6caemAXEc60pEzLccCcpVKRb3wwgvq3XffPY7hOtKR\njnTk2ORYQO6b3/ymmp2dVQaD4TiG60hHOtKRY5Mjg9zGxob6zne+o/7gD/5AdfKKO9KRjpw2OTLI\n/dmf/Zn667/+a2U0dtx7HelIR06fmI9y8re//W0VCATUCy+8oG7fvt3wmI4J25GOdORpSTvW45FA\n7sc//rH61re+pb7zne+ofD6v0um0+vrXv67+9m//9tATOQl577331Hvvvdf28bWaUjs7SvX1KdUO\nVm9t4f9KRSmHQymfb/8x4bBSxaJS/+2/vaf+6399r25cfmYwKDU0hP/jcaXyeaVMJqUCgfbmcdh5\n/vf/jnUJh5VaX8dngYBSTieO0c9pa0upREKpvT0cMzMj8/r8c6U2NnDvIyNKra1hDXt7lcpksJZ9\nfbhuoYDj3W6lqlWlxseV+h//o/V3FAop9eCBUi4XxujrU8pslrU56Dt78ADXNRiUGh5Wyu9vvk7/\n5b+8p/7Tf3pPGY04zuls/J0q9XS+J8phn9unKadpLu0qUEeyMf/qr/5Kra+vq5WVFfX3f//36stf\n/vI+gHuWxWDAQ93uA2syARQMBmyyRhIIYHO63fvH5WcEE6WwcZQC0OzsPNl9tDtPAlsgoJTRiM8a\nzclkUmp7W6lYDEAXi8kYbrdS/f04d3sb48TjSi0uAugGBjBWoYDjPR6lcjlct1jEz63eicGgUlev\nAhAJZNq1Oeg7K5cxn60tpSKR+mvx7+Ew/t7VhX9+v6xHM3ka31NHjkeOpMnp5ThM06f5Rjxu0c81\nEDhY8+MmbPczk0k0qVab7DDSbJ4GA7Qy/Wf6OQUCmJfbjTWwWqGJmc1KRaP42WAAOBSLOJZjGAzQ\nAnkcNb5iUalkUqlUCtqdVjvUrxG1L63m287axOOi6Y2PK9XdjZ85t0ZA1Wg9GsnT+J46cjxybNGC\nt99+W33rW9868jhP841469atYx1PP9fDaH7tzqWRJtWu6DUTin6e2rk0ugf9OARDqxWamNeLNdje\nFq3HbofW5XQqdeGCUj09SpVKGC+fxzXSaQBbtQqQ2NtTanr6lsrlAHSNNDrtXPz++rVpdr887/Fj\ngGitBtNZD0h6DffWrVvKYMA5d+4odf8+5tpIjvI9NVtnynE/t0eR0zSXdsXwtPnkDAbDoXxyjfxS\np1VO+1xb+c4OEmqp6bRSu7sALq8X4xiNGDOVAsiVyxjXbAaQ6dcjFMJaeb3wg0UiWLdIBKZttQpQ\njEZhrhoMYjpTy+J8IhGAW7W63+9JX2MsJmPQGqB5+uGHmKPfr9SXvgQg5thGo2id2nX65BP8XyrB\n5J6aqp+T1uqIxwH2pRLuLRjEd9COddKOT7cjIu1iy6nL+ziON+JJyUnPtZWm0kioHSkFR327mjG1\nnkgEGk86DR9aIgENJ5/H/XZ3A9j4u1IAK/16WCyYi8WC37XrVq1Cm+PLIpvF/3oti1pztQowbGQW\nptNKbW4CZJRC0IMaockEkK1WlTp3TimbDfejHTsSwdrqfXUWC0CLfjl+B1xbrdWRz2OtSiWA+85O\n+9ZJOz7djhxeTh3IHdbZ/0XKk8z1sECllcOa8vpAAjWag65NraNQAMiNjeHnvj4AgNEom9Hng3Pe\nYBAtTm/qrq5KlFZr1geDADuvF3/L5fB3gvP2tsyVAOD3Axz1QBqPQ+Pc3oZm+eABPne5EPQolwGg\nr7yCsfr7lbp4Eedy7FIJmql+fS9dggb34ov4jN9BKrUflEwm/F+rSSS5XfB6ll7wz5KcOpA763IU\nn+Nh3/SJBDSX5WWJiM7PK3X3rlJzc82Bjhs1k1FqehoA53LBDCwU8Dk3I4MI4TDO0c+LZmAopNSj\nRwAbXpdgZzbjvsxm+O4InFynjz5CWsr77+M6odD+uROYDQZoaVYrgC8ex1wNBlzbaETA5MUX8bNS\n+zVL/foyhYSaGc3z6en9oGQ2Q/Pr6pK/twtez9IL/lmSDsidsLQLVI00Pm4Wh6Ney2km+TxAwuWC\nBhaNQkMql6G1NANZRk9nZgBs6+viV0oksBGNRsxhbQ33wWCDXruKRHC/JhNAodF1zWaYrNTshoYE\n+AwGAa94HAC9sQHA1K+rx4N7nZyESdrfDy0xnRZNra8Pn9FUVWq/ZtkokEHTtKcHPw8N4Z71oESN\nl5Fn7fgd8Ppi5FhTSDpysGjTN1o5pPUaHzeJzycJrUqJltFICA6VCjZwIAANzuUSU6qRaNM/CAz0\nNb38Mj7X+shiMcxL7yjP5zFONIr8OEZmeV19MKFchhZkMECL/Owz+P2UgmZkNOIeSiWAV6N1fftt\nMV1zOaWWlhDdzWSUGhzENZJJAN/GRn1wgAGDchl/034HmQzmYjTCzG0GWMedSvIspVSdVumA3AmL\nNheuEZBRWm2WUgnA1ShNQp+3R5OMY5vN8nurDUPQGBqC1rS9DU1MPz+/Hxocx9POwWjEXH0+AEw8\nXp9vpg8m+Hxyr+k0tKrdXYByrabU176m1MICctz0wK5dV0Z/s1n4JFdXMcdaDffl9UI7q1RggprN\nuJdCof5v2u9genr//FutW7tVMgdJq2ekI+1JB+S+QGkFZIEAHOZOp1I//Sl8TBYLnOD9/diIfX31\nKRaPH0uqBzeEHgxalTFR9GA5NwdtKJ8HyFDros9Oy82wvQ2wWF8HYCil1Kuvtk50JlDSBDaZxISN\nxZSamMB1gkH42w4CEM5dC5JKQRNjqotS9cGBSGT/35SqB6x2AKbd49qVTpLx0aUDcl+gtHrrGwzi\nfE8ksLm7ugAyFy9io+q1IpNJ/GYjI62v3coM0moP8ThAiLlyk5MAWKaQxOP1m3pnB2bi2hrG7eqC\nJmi1Nq6cWFwEKLIagWZyfz9Ajv6tR48AeIHAwetK/1wyqdTNmwi2uFz1pVlaDZfBAf3f+HO76/Y0\n5Lg1w6ctp9G87oDcCYv+IWj11udb3GqVJNzJyeZaETf25csHP1yHMZW9XoDr5CTmkE7DFDSbUUeq\nFbcboGWzSTT2ypXGWkgiAW1QW8CfSgGQhoYA5ltb+LxUgtO/UGhutmmTeh0OAH2zUrVG5m47Wu5x\nm49MHqYfMBjcX2r3LJmop9G87oDcCcthHgK+xW/dgsZDkGl1LDe2XvTg2swMisfrTVHmvTHLXylo\ncOWyRBC199Dfj+N7eqD99fQAoBvNifNh0fzMjGhvNB9tNsy3WsXfYjGAOIMYjda2VML8SyUAMpOW\n43FJgWlXy2h33Z5UGNCpVkVb9fsx33bmeto0p9NoXndA7oTlMA+B9i2uBZmDjm0kenA1m6GduVz4\nmRqEtppBD2CUTAbHlcso09JKMIjxmKbSal7UPpVC9NXvB6iFw9Dg7HaYu3t7Sp0/DwDQpqFox43H\n64v+nU6ZK8uxmMi8vQ0AZpqJXntqtW7HbT5qI+B8GaytweSfnDz4ZXjaNKfTaF6futrVZ10OerMe\nlqPuuK798CFSJlIpRCcNBgBCtYqAxtAQNkg79bibmzA13e7mHGvt3GetBg2VqSr9/QDITz/FvPb2\n8PfuboDS6CjO29mBCa1N/2DQRSnMe2cHALexgciuzYZjHzyQxGWCqteL9WqkPT3t+uRaTaiqKhUE\nbFidYjJh/Vpd97TXTz9NaRdb
}
],
"prompt_number": 41
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The most general normal density combines rotation and elongation.\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"R = array( [[cos(0.7),-sin(0.7)],[sin(0.7),cos(0.7)]] )\n",
"rps = dot(sps,R)\n",
"xlim((-4,4))\n",
"ylim((-4,4))\n",
"plot(rps[:,0],rps[:,1],'.',alpha=0.1)\n",
"figsize(5,5)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAE1CAYAAACcI5eKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvWlsXNl5JnxqJatYrOJSrOIqUqTERaKkVqtbvdjdLacT\nLwM7DpwMYGMmzmSZDDBAggRIfttJgPhHfhmY34NBkEGCWTAZj5OBPbFb3rrd3eqWWhtFiRT3rYpV\nRRZZG2v7fjzf4/fU5a2NLErqdr0AQbLq3nPPOfee577nXZ7XUiwWi6opTWlKUz6hYn3aHWhKU5rS\nlJOUJsg1pSlN+URLE+Sa0pSmfKKlCXJNaUpTPtHSBLmmNKUpn2hpglxTmtKUT7Q0BOTy+by6fPmy\n+tKXvtSI5prSlKY0pWHSEJD79re/rc6dO6csFksjmmtKU5rSlIbJsUFudXVV/fM//7P6gz/4A9WM\nK25KU5ryrMmxQe5P//RP1d/8zd8oq7Vp3mtKU5ry7In9OCd/97vfVYFAQF2+fFldv37d9JjmFrYp\nTWnKSUktu8djqV9vv/22+s53vqNOnz6tvva1r6kf/vCH6utf/7ppR56Fn2984xtPvQ/NvtTXl7U1\n/CwvF1U4/OT7Ynb9SASfbW4WVaHw5OZE78vDh0X1wQdFdfNmUS0tFVUohP4VCofnbHMTf6+sVO5v\nLXNd77PCNpeWiurBg8bOV61yLJD767/+a7WysqIWFhbUP/zDP6hf+ZVfUX/7t397nCab0pQSsdmU\nyueVsliU6u5+Nq6fTuP3wYFSkUhjrxeNKrW+rtTWllLGdcy+7OwotbenVCyGvsRiSvn9+LFYDvc5\nEFDK5VJqYACf1TPW4wrbtFqVGh+vfP2TkmNtV43S3Jo2pdESCABIuruPt0CiUQCCzYY2q7UVjQJI\nikWlWlsFQJRCGwcHJwO8OoDOzSnV1qZUIoF+cC6sVqXCYXyXySh19WrpeMzmzO+vPg+NmmtdTqLN\neqVh3oI33nhDfec732lUcyci165de9pd+IU0+2Iuxr5YLKUAc1SpV/tKp5V65ZVrKpvFtY1gUItm\nVE3MtDZdm3K78dmLL15TkYjMhd2uVFcX+vDaa9Ds9HYqzVmleag219GoUtPT10y1zHLSqPt3HLEU\n69ncHuUCFktd++emNOUkZGtLtK9awInHLy0BbBwOpc6fB6BsbCiVyykVDOLnqAv43j1oYhaLUoOD\nSvX0ADwIaCsrShUK+HxwUK7DY6gdra/j8+1t/B8IlNdW650HXXidfB4AS+3waUmt2NKM+2jKL4XU\nq33xeGpTyaRSjx5BE8pklMpmARjHscnlcgCxgwP5jJpPJgNws1qxXdb7bNSOqP1ls9DwKmmr5eah\nki2QYmaz43kPHii1tlb5/Hqllj7VIk2Q+yWXRj1IJyGN7Fu92yYe73AAPKxWpc6exUK3WLDYu7qq\n2+QqjSEYRPv9/Ye1IpsNAEiHQiXRgatQqGwrLDcPtWzneR2XC9rs1pZSqRS+29vDWBvpjGmUg6eh\njoemfPzE+CDVugWp15D/JPp2En06fx4a3NmzALpAAL+Vqg00NzZkS2q1QjujOBzoq8Nx+Lx6DPYE\nLuM2tlaJRpUKhWRrXA0guW09OAC4tbdjDD5fY50x8bhSq6sY1/S02BvrlSbI/ZLLUT2FRwXHk+yb\nsU9Wa3nQ0wHRbgcQmR1ntSo1MSH/WyylQFVNuCXN5w9/l8lAG8xmD88hAaUeOco5SmEeenrgsTVu\njc1Evy+Tk5jLixfxu5Fe1PZ2tOXzAYQdjqONr7ld/SWXo3oKn0T8Wr19M/ap0nZH/07XTOrdFlXb\nUlfbkj6pGMBq8XdmW+Ny5+j3xWrFOfzdSI2eXuRCoTbTQDlpelebciQ56tboJMXYp0qeRP07u139\nIlSkHrCPRpVaXMQC7+qCk8IYj2a1AkTMAMBsDk/KDFDJM1ruXp6UN7XWMRaL8BgrZT5/tWJLc7va\nlCNJLAaQCIVOziZXq5gtmmgUW8W9PWypjP3TbV5KHQ2weU1mHQwNyXe6LU4P/9DFbHtZz5bbbPx2\nu1KbmwDt3l4Jcam09S+3zT2poOdaTR31mgbKSXO72pQjSbmt4NPw1pr1JZ3GImlvR5+MooO0Ukfb\natlsSnV2Qsu5cKH0fD08ZGFBqQ8/VOrOHXxWSeJxzF8kAu2wFg+jfgzBNZtVanZWqbt3cS96euo3\nSzQq6Nkox9mm689XrdLU5JpyJCn3ln8SDola+lJNC2mEV7mnB/8PDR0GgWAQCzEYhHc2m8XCdjig\nWZaT9nYApNeLtuNxaKMOB4z7Zn2JRGCcp11sfR0vmLY2iZubn8fvtTWlnE60b7eXar5GjfGojoxq\ncpxUL/2+1SpNkGvKkaTcg3qSeZ3lbDl6X5i0ziDachpaJfCoJPoii0bLg0AwCBDp7lbq4UOAXLEI\n0KkkdntpKAa1Mp/v8PXYF58P8Wrj4/jfZsNvBggzRWxvD5pkKKRUXx/OI8BXCnVppOj38CjC52tn\np/ZzmiDXlCNJubf8SSZkmyWvE/DYFy5Wpcw1LEp7e3nwqCS1grg+P+fOYes4OlodPIzzZ7cr1dFh\nfj32xcjwwWvoDoXZWRjxcznY6gh2AwM4Tg91icUwN+XsgMdxjhxX09dJCmqVpne1KR8b0T2i/DF6\n/m7dEq2po0Mpj8d8MZbzvJZzYhi3qPWA+HE80ZXOrafdtTXYBlMpaHGFAs4rFDB/uRzmpKsLAGI2\nt5TjeF2PkztrbKe3t+ldbconTHQtJxQy16h6e+Fd7O6Wt72Z1mC3I/i1mnfT7699i1pOw6lm24pG\noYEaPaLGc83arwVgolGMNZHAvHi9Su3vH04B4/a63NxSjmOSqKbp16olBgK1X7MJck352Ii+qMst\nFt0WVmmxbm5KIDA1NKWO5sRQ6nDMHAGylkXLpP98XraVdju21LpzoNxWr9o1mNEQiaA9pqdtbkob\nlebW2P5xTBLVgLme8JJapRlC0pQnJk8i4V7/3G5XancXv43Xprczlys93yxsopZQinQa2tHqKjyZ\ndDBUCwGhlhWJQLNqa0NYilnCuzG8RL92pWswo2F8HA6InR2lbt7E2GvhlWP7W1ughwqFqgPcUe/1\nSWSBNEGuKU9MTpI23Ez03NCNjdJr9/bCs9rais8qEU6WA1RdbDZcL5HAdvDGDcTGzc6Kh7NcKIvf\nD03L50PgcKFgnvCezwO08/nS+asGDATpwUHYKA8OcI3NTWifmUwpGBkBSqdy6uys7f4d9V5Xe6E0\n4+Sa8kzLSYaXmIkeJhIMQnMx2qAYNlGPt29+HmBDIk1u/xwOeFBzOWh1DBReX1fq5ZfNFy3nxO/H\nsZub0JQmJtCe1QqAttkAMl7v4WT/QADeZrfbPANF3yKSKsrnKw2xMdogYzE4KSIRScIfGKgM2MZx\nbW3heHpwa9li1rOdrVWaINeUJya12HKOUouh3PEMEykUsCW020tTvFgroRrFkFF2d/GbRJoTE2hz\nclI8lAQJm02pK1dK+2XmrbVYwEK8vQ2vcCgEzYvhMAcHuF4yiX7qfY3FAOaJRKk90EyMVFFmdst4\nXKnl5dL+1UvlFAig7d5ec5aVo4r+oqxVmtvVpjwxqWXbV882h8b+uTksypmZUvsPY8xyOSzMXE6p\n+/dlG0aDvBn7biUxEmlSgkFoLYOD4D/r6ADAEVS41VpZwfXprbVaAXCMT4vFBMj0reipUwCN7m5J\nVeMc6JW7qsXvUSsMhczTvdrb8ZnXC0Bne7XcP/06gUB1Es96Rd/O1irNOLmmPBEpp3EZP9c1i2px\nVOvrOH51FSBFwzqT2skAQnsSNQvGhuXzR4vZKhRKiTQrjc/YX6UAslYrFv7AAIB3fR19cbuxBe7p\nQRu69qTPjcsFLS8cFsDOZFDYplygLPtIcOM8GL3ABOBYDDm59QTe6tepxMDSCGmykDTlxOQoEe/l\nQgOMn1ezL+lis2F7Fo0qNTYm
}
],
"prompt_number": 42
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(coordinate transformation)\n",
"\n",
"Let's start with a normal density with arbitrary covariance matrix $\\Sigma$:\n",
"\n",
"$$ f(x) \\propto \\exp(\\frac{1}{2}x^T\\Sigma^{-1}x) $$\n",
"\n",
"Now let's decompose the $\\Sigma^{-1}$ matrix into an orthogonal matrix and the eigenvalues:\n",
"\n",
"$$ \\Sigma^{-1} = R^T \\Lambda R $$\n",
"\n",
"With that, we can represent the normal density as:\n",
"\n",
"$$ f(x) \\propto \\exp(\\frac{1}{2}(x^T R^T \\Lambda^{1/2} ~ \\Lambda^{1/2} R x) $$\n",
"\n",
"And that means that our normal density is related to the standard density by a coordinate transformation:\n",
"\n",
"$$ x \\rightarrow \\Lambda^{1/2} R x $$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(recovering rotation and scale)\n",
"\n",
"Let's try to recover the rotation and scale from these samples. Note that these values are close to the parameters we set above (the `pi+` is needed because the direction of the eigenvector is arbitrary up to reflection).\n",
"\n"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"M = dot(rps.T,rps)/len(rps)\n",
"ls,vs = eig(M)\n",
"print \"eigenvalues\",ls\n",
"print \"angle\",pi+atan2(vs[0,1],vs[0,0])"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"eigenvalues [ 1.02305825 3.9323627 ]\n",
"angle 0.705607128771\n"
]
}
],
"prompt_number": 39
}
],
"metadata": {}
}
]
}