{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "UuhJeQeAakNd" }, "source": [ "# Tutorial: Basic ridge and LASSO models\n", "\n", "This lab dives deeper into ridge regression and LASSO and how to implement these tehcniques in R.\n", "\n", "## Goals:\n", "* Learn to use the `glmnet` function.\n", "* Understand that hyperparameter selection should also be validated.\n", "\n", "This lab draws from the practice sets at the end of Chapter 6 in James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). \"An introduction to statistical learning: with applications in r.\" " ] }, { "cell_type": "markdown", "metadata": { "id": "iGlxmzMHakNx" }, "source": [ "---\n", "# Ridge Regression\n", "\n", "For this tutorial we will load the [Hitters (baseball) dataset](https://vincentarelbundock.github.io/Rdatasets/doc/ISLR/Hitters.html) included in ISLR.\n", "\n", "Ridge regression is made easy with the [`glmnet` package](https://cran.r-project.org/web/packages/glmnet/glmnet.pdf), so we'll install that to start." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 85 }, "id": "FwHAN34kC4TA", "outputId": "3800ec7a-6b22-4237-c007-9d30087afb55" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Installing package into ‘/usr/local/lib/R/site-library’\n", "(as ‘lib’ is unspecified)\n", "\n" ] }, { "data": { "text/html": [ "\n", "
  1. 'AtBat'
  2. 'Hits'
  3. 'HmRun'
  4. 'Runs'
  5. 'RBI'
  6. 'Walks'
  7. 'Years'
  8. 'CAtBat'
  9. 'CHits'
  10. 'CHmRun'
  11. 'CRuns'
  12. 'CRBI'
  13. 'CWalks'
  14. 'League'
  15. 'Division'
  16. 'PutOuts'
  17. 'Assists'
  18. 'Errors'
  19. 'Salary'
  20. 'NewLeague'
\n" ], "text/latex": [ "\\begin{enumerate*}\n", "\\item 'AtBat'\n", "\\item 'Hits'\n", "\\item 'HmRun'\n", "\\item 'Runs'\n", "\\item 'RBI'\n", "\\item 'Walks'\n", "\\item 'Years'\n", "\\item 'CAtBat'\n", "\\item 'CHits'\n", "\\item 'CHmRun'\n", "\\item 'CRuns'\n", "\\item 'CRBI'\n", "\\item 'CWalks'\n", "\\item 'League'\n", "\\item 'Division'\n", "\\item 'PutOuts'\n", "\\item 'Assists'\n", "\\item 'Errors'\n", "\\item 'Salary'\n", "\\item 'NewLeague'\n", "\\end{enumerate*}\n" ], "text/markdown": [ "1. 'AtBat'\n", "2. 'Hits'\n", "3. 'HmRun'\n", "4. 'Runs'\n", "5. 'RBI'\n", "6. 'Walks'\n", "7. 'Years'\n", "8. 'CAtBat'\n", "9. 'CHits'\n", "10. 'CHmRun'\n", "11. 'CRuns'\n", "12. 'CRBI'\n", "13. 'CWalks'\n", "14. 'League'\n", "15. 'Division'\n", "16. 'PutOuts'\n", "17. 'Assists'\n", "18. 'Errors'\n", "19. 'Salary'\n", "20. 'NewLeague'\n", "\n", "\n" ], "text/plain": [ " [1] \"AtBat\" \"Hits\" \"HmRun\" \"Runs\" \"RBI\" \"Walks\" \n", " [7] \"Years\" \"CAtBat\" \"CHits\" \"CHmRun\" \"CRuns\" \"CRBI\" \n", "[13] \"CWalks\" \"League\" \"Division\" \"PutOuts\" \"Assists\" \"Errors\" \n", "[19] \"Salary\" \"NewLeague\"" ] }, "metadata": { "tags": [] }, "output_type": "display_data" } ], "source": [ "install.packages(\"ISLR\") # uncomment if you haven't installed this library\n", "library(ISLR)\n", "names(Hitters)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "jGoWwja6akNx", "outputId": "48b17787-d11b-4572-b4f8-f5e9c55cc696" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Installing package into ‘/usr/local/lib/R/site-library’\n", "(as ‘lib’ is unspecified)\n", "\n", "also installing the dependencies ‘iterators’, ‘foreach’, ‘shape’, ‘lars’\n", "\n", "\n", "Loading required package: Matrix\n", "\n", "Loaded glmnet 4.1-1\n", "\n" ] } ], "source": [ "install.packages(\"glmnet\", dependencies = TRUE) # uncomment if you haven't installed this library\n", "library(glmnet)" ] }, { "cell_type": "markdown", "metadata": { "id": "zUN_c-zuakNx" }, "source": [ "Remember that, for ridge regression, you need to find the best tuning parameter ($\\lambda$) to use. This tuning parameter determines the sparsity of the model, or the impact of each predictor, using the L2 norm. This contrasts with LASSO, which conducts *feature selection* using the L1 norm. In both cases, the larger the $\\lambda$ value, the more pressure there is on the $\\beta$ coefficients to become very small (ridge regression) or to become 0 (LASSO). \n", "\n", "Let's start by establishing the range of $\\lambda$ values we'll be considering." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 437 }, "id": "zF0lOi1uakNx", "outputId": "8bfd1b34-9d80-41ae-dcd3-3c0934f1c125" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAADAFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////isF19AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO3dB3gU1cLG8bM1PSG9ZxOlhpIQEAJI74QYeoeEhGJoAYMEBOlFQVGUJkiRIihNRQWlX3qJBRDUgEiREJCEEiB1z3dmE4qfOAvZk9nM7vt7nrszuTnsHjfz35nthAKAyYi5JwBgCRASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMCBKSHpz+/YvHnXJW5zAZCtkoeUmexFDIKm3uc4IQA5KnFIV0NIhbhJs2dP6OlHwjJ5TglAfkocUoLm8+K1ggWKJE6zAZCpEofkE/94vXug2MifTgDIyk/P30OJQ9LMeLw+WSsy8DgBkJnjz91DiUPSdXu8HhMsMvAgyS3pZQCYQy45+Nz/psQhJSnm5BStZU8kKSIDERLIjKQhZUUQp+Zxw4bGNrEnDe+KDERIIDOShkRz54arhMNJTeSSArFxCAlkRtqQmAe/p6amGcsEIYHMSB7SQ3+nifwSIYHMmC2kFLFzQUggMwgJgAOEBMCBpCHVeoIPQgILImlISqXNIyqEBBZE0pBSnB4/VIdDO7AkkoaUV7N23sN1hASWRNoHG87YjX64ipDAkkj8qN3tmw/X9s4SGYaQQGbM9vC3KIQEMoOQADiQPqT0mENGx/xXSD+ntGuX8rNJFw9QGqQPKY1sMDrmP0KapWr0+uuNVGL3rgDMQk4hbdBuFhabtRtNmgAAf3IKqWbxY+ejI0yaAAB/EoeUnJycQKLZafK/flW4Z8cj7z8tpLvkcNHKIUV2yWcAUBokDkmn0/kTT3aq+9ev/vB0fcSePCWVq+S3opVfydWSzwCgNJTNQ7vF5CkfjZJn+03Ryte2ef/+LYA5ySgk2rmVXlgUtuxs0gQA+JNTSL+69EqnNL2Xy28mTQCAPzmFRFOrkeBgUi3VpMsHKAVmeIlQvt7okP8IiRamrlx5otC0iwcoBWYI6c7pLGND/iskgDJK8pD21iJkG6XRO8UGISSQGalDOqp1as1Cuu6jPSEyCiGBzEgdUlTQ5XRhj5QRFCMyCiGBzEgdkvssagiJznQVGYWQQGakDkm9pjikFRqRUQgJZEbqkALGF4fUXycyCiGBzEgd0iDXVCGkzDfIEJFRCAlkRuqQ0gPVESQ83IYEXRMZhZBAZiR/Hikj0Z0Q4pGYITYIIYHMmOGVDfpraWJ7IwFCApkpm9/Yh5BAZsrm9yMhJJAZhATAAUIC4KBsfmMfQgKZKZvf2IeQQGbK5jf2ISSQmbL5jX0ICWSmbH5jH0ICmSmb39iHkEBmyuYXjSEkkBmEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXAgeUj5l8/+ccvYIIQEMiNtSLnzI7WE8Ys9IjoOIYHMSBrS7TrErkakpnzvmgryuthAhAQyI2lIo8hIdlR3VjeP/tmBrBAZiJBAZiQNKSDKsFjjnE0LakWIDERIIDOShqSZalicJQconWInMhAhgcxIGpJvN8NiI9lH6Qh3kYEICWRG0pBiFUv1lJ7UOWTTY+XaiAxESCAzkoZ0wY34NQpVkPm0QONwTGQgQgKZkfZ5pPPdnYi64W62NuaU2DiEBDIj+SsbsgufYRBCApmROiT9+R2bN++6ZGQUQgKZkTakzGQvYhA09b7YOIQEMiNpSFdDSIW4SbNnT+jpR8IyRQYiJJAZSUNK0HxevFawQJEkMhAhgcxIGpJP/OP17oEiAxESyIy0LxGa8Xh9slZkIEICmZE0JF23x+sxwSIDERLIjKQhJSnm5BStZU8kKSIDERLIjKQhZUUQp+Zxw4bGNrEnDcVSQUggMxK/1XxuuEp4GkkTuaRAbBxCApmR/CVCD35PTU3LNTIIIYHM4OO4ADiQPqT0mENGxyAkkBnpQ0ojG4yOQUggMwgJgAOEBMCBxCElJycnkGh2mvyvX2UOGfRIQ4QE8iJxSDqdzp94slPdv36FkEDGcGgHwAFCAuAAIQFwgJAAODDDS4Ty9UaHICSQGTOEdOd0lrEhCAlkRvKQ9tYiZBul0TvFBiEkkBmpQzqqdWrNQrruoz0hMgohgcxIHVJU0OV0YY+UERQjMgohgcxIHZL7LGoIic50FRmFkEBmpA5JvaY4pBUakVEICWRG6pACxheH1F8nMgohgcxIHdIg11QhpMw3yBCRUQgJZEbqkNID1REkPNyGBF0TGYWQQGYkfx4pI9GdEOKRmCE2CCGBzJjhlQ36a2lieyMBQgKZMcvHceWdPpEjOgAhgcxIG9KuJsFtj9DtfoQ4LxAbh5BAZiQN6ZCaOCsdDjkH9uvmangM/L8YCynvYn5J5wBQGiQNKdrnZ3q9aVDYfUozg9uIDBQP6ft6GqJtcqCkswDgT9KQ3Kexk+NkpbA+3U1koGhIS1WJuy/s6KfeWNJpAHAnaUjqVezkKvlGWF+mFhkoFtJlu0WG5XTXmyWdBwBvkobkPYmd7CXzhPU3vEUGioU0p1LRO2zzfZaVdB4AvEkaUg+33bknq1cJukLpGdcuIgPFQorvV7wSNbqk8wDgTdKQzjoRQtzO6Oyb1lOrjooMFAspoU/xStsxJZ0HAG/SPo90qmfduF/pqToK8sIXYuPEQpoXUvRlfznuq0s8DwDOzPNFY3evi/9eLKQM55mG5WjvOybPA4ATGX5j3wZN1w1H10fZfl/q0wB4VjIMiR6PdidenU+V+iwAnpkcQ2LulfoUAJ6HTEMCKFsQEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABxIHlL+5bN/3DI2CCGBzEgbUu78SC1h/GKPiI5DSCAzkoZ0uw6xqxGpKd+7poK8LjYQIYHMSBrSKDKSHdWd1c2jf3YgK0QGIiSQGUlDCogyLNY4Z9OCWhEiAxESyIykIWmmGhZnyQFKp9iJDERIIDOShuTbzbDYSPZROsJdZCBCApmRNKRYxVI9pSd1Dtn0WLk2IgMREsiMpCFdcCN+jUIVZD4t0DgcExmIkEBmpH0e6Xx3J6JuuJutjTklNg4hgcxI/sqG7MJnGISQQGakDkl/fsfmzbsuGRmFkEBmpA0pM9mLGARNvS82DiGBzEga0tUQUiFu0uzZE3r6kbBMkYEICWRG0pASNJ8XrxUsUCSJDERIIDOShuQT/3i9e6DIQIQEMiPtS4RmPF6frBUZiJBAZiQNSdft8XpMsMhAhAQyI2lISYo5OUVr2RNJishAhAQyI2lIWRHEqXncsKGxTexJQ7FUEBLIjMRvNZ8brhKeRtJELikQG4eQQGYkf4nQg99TU9NyjQxCSCAz+DguAA6kDyk95pDRMQgJZEb6kNLIBqNjEBLIDEIC4AAhAXAgcUjJyckJJJqdJv/rV394uj5iT+6U/DIApCdxSDqdzp94slPdv35VuGfHI0nYI4G84NAOgAOEBMABQgLgACEBcGCGlwjl640OQUggM2YI6c7pLGNDEBLIjOQh7a1FyDZKo3eKDUJIIDNSh3RU69SahXTdR3tCZBRCApmROqSooMvpwh4pIyhGZBRCApmROiT3WdQQEp3pKjLqmUK69/Vbc3fkmzAXAG6kDkm9pjikFRqRUc8S0lYvxzo1bcofNWEyALxIHVLA+OKQ+utERj1DSPs0E+5Tmhnncs6E2QBwInVIg1xThZAy3yBDREY9Q0iRAwwLfdM+JswGgBOpQ0oPVEeQ8HAbEnRNZJTxkG4ojhetrBO7rwUgEcmfR8pIdCeEeCRmiA0yHtIpcqNo5TB5YMp0ALgwwysb9NfSxPZGAuMh/UV+KVr5wt602QDwYI6P4yo4edDIV/Y9w32kShOKlj2iTJwNAAfShnRwKDtZ7c2O7cL2iY17hpDWadezU/1szZESzwaAG0lD2qN11NMNxLHrkJZKG1NfIjRHXWNA3/KO60s6GQCOJA2piVcapSG6q2z1iF20yMBnemVD2lt9Et5LL+lcAHiSNCTn0ZTeIvMM6wPLiQzEa+1AZiQNyeFNSnMUmwzrU2xFBiIkkBlJQ2pQ4R6l9UcLqzlhYSIDERLIDLeQsp/h3spWEvFdfqrvJ/fyjjQjH4kMREggM9xCWuv7DP9yqQOxC9URlYooXhP75AaEBDJjckg3PkxOYgb7Oz3LP702p7XOyca91ohU0WEICWTG1JAueJIi6in8JoWQQG5MDam30/xd5OPtY/2385sTQgLZMTWkoLH0ATlM6Y9uB/hNCiGB3JgakmYJOwvhdXNvNuc2J4QEsmNqSG7TKHVcwVbWufCaEkVIIDumhhTjv4fWq802+4Fe/CaFkEBuTA3pqG0tupwEdgwnvflNCiGB3Jj8PNKJhVQ/zo4oXrnBbU4ICWSHzysbHly4z2MyjyAkkBlzvNXcOIQEMmNKSHX/oSbHWSEkkBlTQlIJNIQQBfufSyDHWSEkkBlTD+0yGw796QG9c6BHs1v8JoWQQG5MDSm+W/FKVAKX+RRBSCAzpobkuax4ZY4nl/kUQUggM6aGZPNW8cpYGy7zKYKQQGZMDamm/zHD8qiX2GcwPC+EBDJjakhbVaR8y+iW5Ynic36TQkggNyY/Ibu/jS0hRNsEb+wDa8bhlQ2FV36/zPmrXBESyAyHkC7t33GQ8ycHIySQGZNDWhpi+OyTUK4fZo+QQGZMDWk+0TZNGJ7QWEs+4TcphARyY2pIFV7407C8XCmU04wECAlkxuQPP5lUvPIOnpAFK2ZqSH6zilcW+XOZTxGEBDJjakivNSwsWnklmc+EDBASyIwpIV1mzkc1/vTUxV+3RLfO4jgrhAQyY0pI5J84zgohgcyYElL3f4jhOCuEBDKDDz8B4AAhAXBgckgFBzeuK8JtTggJZMfUkE4E48EGAJNDiiyXtHBpEX6TQkggN6aG5LCF31weQ0ggM6aG5C3+rcolhJBAZkwNafg4fnN5DCGBzJga0r3ormt27zfgNymEBHJj8heNBeJROwCTQ6pj2238pCLc5oSQQHZMDcl2Nb+5PIaQQGZMDcn9R35zeQwhgcyYGtKAafzm8hhCApkxNaSsFkN2nEkz4DcphARyY2pIeGMfADU9pJ6xCQ/xmxRCArnh9n6kbJ6fWoyQQGa4hbTW1+S5PIaQQGZMDunGh8lJzGB/J25zQkggO6aGdMGz+KEG9RR+k0JIIDemhtTbaf4u8vH2sf74ojGwZqaGFDSWPiCHKf3R7QC/SSEkkBuTP0R/CTuLfWzlzebc5oSQQHZMDcltGqWOK9jKOhdeU6IICWTH1JBi/PfQerXZZj/Qi9+kEBLIjclv7LOtRZeTwI7hpDe/SSEkkBuTn0c6sZDqx9kRxSs3uM0JIYHs8Hllw4ML93lM5hGEBDJj6vcjPeHZ/nH+5bN/3DI2CCGBzEj7/Ui58yO1wlC/2COi4xASyAy/70fqbvwf3q5D7GpEasr3rqkgr4sNREggM5J+rcsoMpId1Z3VzaN/diArRAYiJJAZSUMKiDIs1jhn04JaESIDnyOkwu3TBs85UdIZAfAhaUiaqYbFWXKA0il2IgOfPaQrkTYNe0YoeueUdE4APEgakm83w2Kj8OK8Ee4iA585pPywBn+xxYnA+JLOCYAHSUOKVSzVU3pS55BNj5VrIzLwmUNaU+5vw/Kg4reSTgqAA0lDuuBG/BqFKsh8WqBxOCYy8JlDin34wqTyC0o6KQAOpP0y5vPdnYi64W62NuaU2LhnDinq4aPojXm+QRfgeUn+rebZhc8w6JlDiutZvPLCwpJOCIADqUPSn9+xefOuS0ZGPXNI65wzDMt9ynMmTArAVNKGlJnsVfRyoqCpoq9yfeaQCmrXucAWB30Hl3hOABxIGtLVEFIhbtLs2RN6+pGwTJGBz/480rXGmtodqioScks6JwAeJA0pQfN58VrBAkWSyMDneYnQvtkj5os+cgFQ+iQNyeeJp027B4oMxGvtQGakfYnQjMfrk7UiAxESyIykIem6PV6PCRYZiJBAZiQNKUkxp/jFpdkTSYrIQIQEMiNpSFkRxKl53LChsU3sSUOxVBASyIy0zyPlzg1XCU8jaSKXFIiNQ0ggM5K/ROjB76mpacae9UFIIDOSh/RMEBLIjPQhpcccMjoGIYHMSB9SGtlgdAxCAplBSAAcICQADiQOKTk5OYFEs9Pkf/2qcM+OR5IQEsiLxCHpdDp/4slOdf/61R+ero/YkzslvwwA6eHQDoADhATAAUIC4AAhAXBghpcI5euNDkFIIDNmCOnO6SxjQxASyIzkIe2tRcg2SqN3ig1CSCAzUod0VOvUmoV03Ucr9p1GCAlkRuqQooIupwt7pIygGJFRCAlkRuqQ3GdRQ0h0pqvIKIQEMiN1SOo1xSGt0IiMQkggM1KHFDC+OKT+OpFRCAlkRuqQBrmmCiFlvkGGiIxCSCAzUoeUHqiOIOHhNiTomsgohAQyI/nzSBmJ7oQQj8QMsUEICWTGDK9s0F9LE9sbCRASyIx5Po7rdspZ0d8jJJAZ84R0mWwV/T1CApmR9ovGHupJWiUkiAxESCAzkoZE/kFkIEICmZE0pFGq8O1Zgl/I+iyx91IgJJAZae8jHQ9XJN6iuI8EFkfiBxvy37Lz24iQwOJI/qjdueYk+hJCAgtjhoe/V7g5TkJIYFnM8TxSRg+CkMCymOcJ2W+Tz4j+HiGBzOAb+wA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwYDEhFZ4z9kEQAKXHQkL6q6c9Id6Tc0tnOgDGWEZIF/wiv7x45iOf1vmlNCEAcZYRUnTjPGFxwXVBqUwHwBiLCOm66kDRyoQ6pTEbAKMsIqSDipyilS9dSmM2AEZZREhHyb2ilY1upTEbAKMsIqTbNt8UrQxvWhqzATDKIkKi/atmCosjtp+VynQAjLGMkDLDgubs/CLZbrC+lCYEIM4yQqL3JoVpXRp+WjqzATDKQkJiCkphHgDPyHJCAjAjhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwIGFhZT22bzvsvjOBeAZWFRImV0VntXtnD7gPR0AYywppIIGVU9Qmv+RLUoCqVlSSKudrxqWSxxv8Z0OgDGWFFKXhKJlnvMWrrMBMErykPIvn/3D6A6jZCHVm1m8Uu3DEvxrABNIG1Lu/EgtYfxij4iOK1lIbUYXr/iuKsG/BjCBpCHdrkPsakRqyveuqSCviw0sWUgzy+cZlgcVF0rwrwFMIGlIo8hIdlR3VjeP/tmBrBAZWLKQbnrE57LF+fJ9SzY9gBKTNKSAKMNijXM2LagVITKwhM8jHfUNih/X0bZNdkn+MYAJJA1JM9WwOEsOUDrFTmRgSV/ZkDWvX+thX+pL9G8BTCBpSL7dDIuNZB+lI9xFBuK1diAzkoYUq1jK9hYndQ7Z9Fi5NiIDERLIjKQhXXAjfo1CFWQ+LdA4HBMZiJBAZqR9Hul8dyeibribrY05JTYOIYHMSP7KhuzCZxiEkEBmpA5Jf37H5s27LhkZhZBAZqQNKTPZixgETb0vNg4hgcxIGtLVEFIhbtLs2RN6+pGwTJGBCAlkRtKQEjSfF68VLFAkiQxESCAzkobkE/94vXugyECEBDIj7UuEZjxen6wVGYiQQGYkDUnX7fF6TLDIQIQEMiNpSEmKOTlFa9kTSYrIQIQEMiNpSFkRxKl53LChsU3sSUOxVBASyIzEbzWfG64SnkbSRC4pEBuHkEBmJH+J0IPfU1PTco0MQkggM5b0cVwAZiN9SOkxh4yOQUggM9KHlEY2GB2DkEBmEBIABwgJgAOJQ0pOTk4g0ew0+V+/yhwy6JGGCAnkReKQdDqdP/Fkp7p//QohgYzh0A6AA0sMqWBFl6oNhpw24RwAnpMFhnSvmfPg+TOaaZeX/CwAnpMFhjToxYuG81D9WPLzAHg+ZniJUL7xz+Y2JaSbmm+LVtrFlvg8AJ6TGUK6czrL2BBTQtqhzS9aWVC5xOcB8JwkD2lvLUK2URq9U2yQKSF96Vy88klQic8D4DlJHdJRrVNrFtJ1H+0JkVGmhPQzuVy0ktK4xOcB8JykDikq6HK6sEfKCIoRGWVKSPoqiYblNY/5JT4PgOckdUjus6ghJDrTVWSUSY/a7dG++gfN/b5SpLG3DwJwI3VI6jXFIa3QiIwy7QnZfaHESa2Ov2XCWQA8H6lDChhfHFJ/ncgoE18iVHjuq/+JfSIyAG9ShzTINVUIKfMNMkRkFF5rBzIjdUjpgeoIEh5uQ4KuiYxCSCAzkj+PlJHoTgjxSMwQG4SQQGbM8MoG/bU0sb2RACGBzJjt47gyL4j8EiGBzEgb0s/tdC8vKPqM1RSxc0FIIDOShnTAhthrSGPDI9MICSyJpCFFabboc+ZqXsqmCAksi6QhBfYRTndp2xUgJLAs0n5j30TDYhUZgZDAskgaUsArRctxZDZCAosiaUgjFB/mCUt9LBk5HCGBBZE0pL+DSAvDin4EIQgJLIi0zyPdGDKyeG3TiwgJLAi+aAyAAwsO6faiQZ3HP/9/HUAJWG5IB338ewxvoozL5zAfACMsNqT0coOEz2w47jWGw3wAjLDYkMbWKDQsN9kY/ThKAJNZbEh1pxUt8+y+NfWsAIyy2JAqLS5e8Vtr6lkBGGWxITUZW7S8q9lt6lkBGGWxIb3jf8ewfN8tx+TpABhjsSHdq9TwPKUFH2mX8pgQgDiLDYlebqQqX6+cIz4AHKRguSFRemLJrC03OZwPgFGWHBKAZBASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4sPqT777YMrDXwFK+zA3gqSw/peg3fMaveaaVdzen8AJ7K0kOKrmX4Epl52l85nSHA01h4SOdIatFKwxF8zhDgqSw8pPVexSvT6/E5Q4CnsvCQVgYXr7wXzucMAZ7KwkP6nyazaCW+M58zBHgqCw8pX5dsWP5mt4HPGQI8lYWHRLdrEn8rzFzn94qe0xkCPI2lh0T3VCM2xD4FHyUEpcriQ6L08s4fkRGUMisICaD0ISQADhASAAcICYADawlpX7eKvs3ez+V8rgDFrCSkmaqeH60b6xl5m+/ZAhSzjpD2qL4QFtcqxXM9W4CHrCOkTj2LltsevvQOgC/rCEm3omiZp9zH9XwBillHSD7ri5Z6m++4ni9AMesIqcG4ouUZcp7r+QIUs46QFrhdERb6rpFczxbgIesIKa9x8Oc3co52dPqR69kCPGQdIdF7I+0IIY3w8XZQSqwkJLZTOn0oi/d5AjxkNSEV+Wlu4tRvC0vnvMGaWVVIeQmKsO6NbGv/WRpnDlbNqkIa5nOInaY3qYJ3zAJn1hTSRdUOw/KW59JSOHewatYU0gr/4pUB3Urh3MGqWVNIs18qXpnYtBTOHayaNYW00q/4w+3iu5fCuYNVs6aQLqu3G5aZ7stK4dzBqllTSHSkl/AuiisvV8stxJvOgSurCik/URHaoa5N5Jw69prQSfdL4yLASllVSJT+smDEWzt6O47ftmduUM1bpXMZYI2sLCTBcsefhcXNyoNK7zLA2lhhSHVTipZb7O6V3oWAlZE6JP35HZs377pkZFSphmT3bdHyJvm59C4ErIy0IWUmexGDoKmid/VLNaSHn9twi/xQehcCVkbSkK6GkApxk2bPntDTj4SJfTBWqYYUNq1o+b3m/eR5R0rvcsCaSBpSgubz4rWCBYokkYGlGtJ7HheFxS1PUrl9mKrtzdK7JLAekobk88QHnXYPFBlYqiHltvBddPLcZ26qr9gPv9ZohC/FBNNJGpJmxuP1yVqRgaX7bRS5k/0IcSJFjzlcsv+yFC8KrIWkIemeePtCTLDIwFL/Wpe/L70X+nAiQ0r3osAqSBpSkmJO8XtTsyeSFJGBEnw/0rjWxSvx4YMHvGfs8XgAcZKGlBVBnJrHDRsa28SeNBRLRYKQ3qlRtFyndlTTcowAACAASURBVOreu5LN4tK+PLBs0j6PlDs3XCU8jaSJXFIgNk6CkH5SGD4s8oDa5jO2+Fi9tbQvECya5C8RevB7amqasTcxSPHVl13L/8JOG3nWMiQ9qmapXyBYMit8iVCx7A6q+v2akMpXDT99TcJcI4ZeLPVLBQtlhS8ReuTAzP4pJNWwesiZjNsyt47L818ZAAIrfInQkwrtDM8iZfu/orzNfhrsd/faDWkuGSyLFb5E6B86thdOV7n3biQsr9k5EeI9Fu+dhedlhS8R+odT9sPYRQ2rqBGuhqzqji3Pnv5YV/ceFX1QEeD/s8aXCP3D3gDHyJe1NoZHv4dX6vQqW2R4VwlQhMTiy/3g2VnpS4SekLN11vT+FYS1/HJrK7/DlhtUdmv+t6yR0/5jK9b/Itk8QNas9iVC/3DRZhU7vUTetfmT0nTHYeQOpfr2akWIN3n5HL3+U7akswEZstqXCP3TXO2k33P3E827bH1OhcOElXPEVrOC0rRWrgGEKCIP0ow9J/AgBPwXq32J0P/zaQiblmqgsNon/p0X2aJ+3yaT2GKUssHpzMNx6spES7Sv3vpmxvBFv7H/+x6+rQyeZL0vEfr//jpwZVzQX2ylZw/Ptym9pkhtMZ7So8pRQez/S9XaHc27vTXY3q5Bh4rKIa8GKxzqb7i7bHifmT/SvNR1u/5mY25flXzOUGZY4cdx/bd7DX1m797ZXN02j9IT5KbHakqHtdqtZDuful3st1J6N0grfN7DSqXPx0e+fU3j6N05oY6irS/x1WoGz2C7NNf+vyTXcqrc79TxxJdf6r8tb92rbQYue3B13sAB712h+94aMvsQvbV+/JTN9+kPiyesPE/vfDXrg72F9NSKtzZdp3e2v7fsB0pPr35v+y16+7v3Pz1D6ck183fdp1nffbDhD0p/Wr14fy69uf3DL65QferKpUcKaMY3H269RguPLV95Qk+vfvXh9r9p/uGlq3+i9PKWBd9n0dz9i9f9QukfGxfuukvv7130GduZ/v754n33afbuRZvY/cEz6z86mEtv7Vj4xWVKf17z8ZF8+vd3C75OZxfwyYrjhTTj2/nbrrMLWLbqRz3966sF32fSvMNLPj3F7lBuXrjzDs3Z/9Fnv1J6fuPiPffovT2LN5yn9LfPPvrfA3pn16LNF9l/zrolh/Jo1vcLvvyL6n9avexoAb2xff43bNInVqxM1dP0r+cLkz768ZqfKb3yxcIdt2juwSXCpP/ctIhN+sG+xZ//zo6wP1+81zDpjRcMkz6QQ2/vXLiFTfrk2qVs0je/W7D1KtX/8MnyJya9fNUP7FrZuuC7m8K1svYkm/SWhTtvC5Nef1a4VhbtvseulcUbzj2c9N1dizYVTzrXMOkrwtX+8ROTXrnyxNOPRaQPKT3mkNExZgqJ5s6qptFWUa1hq7+R0W63KG01bpMLu87JmVrvUPpO4Ni67FfNqtlmsj2Yo0L4TP7xpDnbuL6118xN/W19VVWlOV8taqNWtp0+u7vGw7nruJ7uvrble/WuaFNVXafHS8raLh4tGrv41FNWah6kauPmHFlDU6UxCY50te3malczRFG3EQkMt3fp52JXw480q08CQrVecY521d0V7euQwIrqwDh7+2ouqq41iO5FZfneNo7VnLS9KitCgkn1HmrnqvZ2sSHKFwNI7U7KclVtHBP8VRV9ycvtFW6h2nIDPTWVPUmLlsSjstprkKumipsiuiHxqaAKGOBoE+qi7PIS8X9R+UKcnV2ok6ZXdaILVlTpo3UItbeJragICSTh3dTOobYO8TpleT9Sr4OyXKjWeaCvuqIXadpW4V5Z4z7IQ1PZnbRtSrwqqX0GumiruCpj6hPf8qqgBAfbUGdVt5ok8AVFhVgb+1BHbZ8qJFhHqvfSOIba2cWxy2WT7qpyCbVxHBCgquBDXo5WuFbRlhvkpa7kSVq2EibtWTTpqMbEu6LKf4CTNrScsnMdYdIh/YVJq3uGkaAQRaW+bNIONv3YpINIWHe1U6itfXyw8kU/UrdT8aRVFb1Jk3YKtyoat6JJt25OPCupvQc9MenABAcbYdIRJOAFRfk4WzZpTe+qJDiYVEt96qYjeUhpZIPRMeYKicnLp++qxxxL32ev3Mh+jBrdJ4rSHRp99XmsquQNnuwmnfzk9AWlyTW79aD0vns3x3uUfuxo9xmlBdXdo9m/+UZly27J9DVVk9kPXyl82IFsYSXFCvbDMkUo29nd9VMKL0yaThrkUHrVTb2H/X4Eacd+OOtgc4xNoTfpw/7NYRtHdsN8rw0Zlk/pNrUbu7G/XY+8yW4Q1yr92DHo36GKuXqqX6CokMHOJki5nJ3NNBLBGr/gqdpMaX4yaXqHnaezdhf7O8eTjmyeJ+zsjrPz7EDi2aXt1Dqzvd7tJuQ1NqmNKk+217sZQaYVUP3HSh27gGsVFAv0tPAdRdXrbBfnp2S3LwUTSGQW+xu6qb9m5zmUtGF/p5OONvvZIXsv0uM+pQdtHdj+8G478ir7L/hW48r2Jrfqk3Hsv2CdyvcSpTeqKWYXCpMun84mHaxcyi5guqImOzT+00vFrvD80aTxbTZpF80OSnMSSEw2pcft7Y6ySXcmsWzSu7VO7Fq504yMZJPeovZg10pmbTKF3eVeoQxkV3tGJcW8Qlr4nqIKu1auBChXsUlPJHXYtXLeXc2u9rzhpNUdw6T/xybdh3Rjkz5k68COBbKjyCA26W2acmy/fetlksImvV7lw/ZQN2oo3mKTXqx4gU06vZfLb0/ZbhDSU2yqoSCqYAe2edAJQWp2hR9QXBQWNd/7JJDSza60whL2w9vzq7Jo7G8KITQYE9eN/ZE1X6rYNtfk1arsCHCPZpIv2+gb9nNZR+n3Nn0j2bm91EPFdsebHZp2ZT9U7qJim+4n5WoMYwUEdLS7Sek87+Cp7M/rFuPONpkpOo8FbCtxiArRUzqyiiMr9ZKmZS32L2MjtDvZAYiycUv2Q3QDJbuJPKCqI3zpU4NmKnac8pVdtUT2Q5XWtmxjWunyAruvV+DfthzbJt/18XmXbTLl2vqzLW98iMsnbMuzbSW86z6xmu1XlJ5TNX2Z/dD9JdUBdsdQWf8V9kOrhkq26e7UhMexH2q30LLjqc8dKr3GbipebOPIqlrsHjidbUhebT3YpGcGeCxkm6FTmxD23z66giP7b/9L20J4j0p8mPY7dlimbNyc/RBTT3mCbcCqOsIzi42aqtIo/dq2mvBseLVWttcoXeX8whts0gFtXdiG8L63LzscuO/a1pdt2hODy7FbjBt2rSuxwcNC7baw2w11s3rsh1611PvYkZiyAbvlo21fVp4U/gYRfdkPdZtr2GHsJvvKSWzS5dvYs2t6iVsQu6bzvNu6P6D0LX/P+ewWxamNjk16THmntaxxm5Zh7F8OCLPZRumvqsbCV9N1qqtiQdPCVp2fss0gpKe6eyZHP1zVbvzocMJ2O/SOTWQYu45bju7Zid12u+c7sz+fbuXCKpQuqkLdN1HqtuktdtD3bkSugm2Att8M78g6qP8HOU9zlHs6DmGbVMsDimx6g5yoN4X9dbpsdGLbLEkLWURpl4RFIWxPobhajl0tLUZPfUmoLkO7m9KIGUltKP3M9ZLwibAhC/v2YteL7hRh27HL+qjhbJutuU91hz7QfldvEruA5lvsC+l1xfGK77NttusyP+HgNM2L3R53GDynGtu9Ka/as51Iw3HjGrFt1uEv5WFKq84ZxOa5yvt3wm5ifZd1SWDbbKVjiuu0wH5Ls9fZNltvu/YBva3aFz6LbbNR68qx/RI5FfQR22b7LHiB3aciF4VJtxkxrbawzaYLe746k0e1YtusyyXh0zfLfxDLrr2lAacJ247d10SzK+KtsP3KWzTHZlv9N9k22/RLO3b3Q3G08nvsWum0wkfYOn7zYXV3GvBuqPBIz18O7K5pkzHjWd3b7K4Kddd4K5HVvdbjHGF704Al3VndH5ZPVVyjeseNLVndU+rs0Nyj2erdtVjdSW0+d2b37chPIazuvj0X6yg9Rf50W09pu2EzIijdp75qw/Z8kROTW7Ddm/MVBau74vv9Wd3L/M8QdjPnuSqG1T2n2kElO363/brhG8LGsdU2799bjMQhJScnJ5Bodpr8r1/94en6iD0pC0+B7h/RvP242epOy7d/4K7aw/6POR6qvexGlcyzYUcKdaf2Y7dMq/xvKdk16LFhHLuhnVUvR3GQ5in2j2Y3iq9F/01O0mvkzEBWQFzcOXKJ/kIyurDdT3TyUXKP7if5wuPrDaZvs2X7EGdagx07Vlm4lhWwMpgGrKbUe/2HrIC5Nan9N5Rqv5/CCpjQ/AE5TO+SE0kdKB3S9S9WwEXyR994SnsMPkVu0B/JrajRwh27fYoCukulrzeT7UPmbGEFbPCglRazHpcvZwUsrsQmTGm5L+awbGfW06t20QLF/4RPsRgdlUV+ZMGfHswKiO97nlxkPV7tygrokHSc3bodJg+aT2D7kCnfaYW9Ma05l+1DPlzHClgdwG5aKPVbu5AVMK8GdWL7N7tt01kBk5rkkQP0Pjn6GjvuHdaZXSOsx3OxrIBeA9k1wnrMjGYFtEnZr8ije5SFDVgBdd7+0pn16MauEdbjUnaN0KXl2TXCetw0lxXwdh12jVC9cs8EVkBKmzvkBM0kPw9hO/qBvf5kBZwjl3sMprTzsB/ILXqM3Gs1jtKmE3eq9JRd27XnsB7nfebFevRj1wjrcfXiyqzHatSFHbI7fDOzPuuxEbtGaA45PLo967HDdXKaXiW/xQv7N/orecrjsxKHpNPp/IknO9X961eFe3Y88j4pO18EdrxriLZ6Yj23kSs+iCFVhMe5q2jYtkSnBGi+Fm5BR3my26emw2uw/2+jyzYNSyxgafORbPdU9YAqi+Zptws37q+33KV+wKr6OZLtkWL7fGFPWVV/VWHtRI38hLWzT3Xfl90Q15v2PtsMt7jkCTfEFRdMYpvhMt09YS/nsX4E2wxn1/yLbYaF6u/7xFI6tvkpkk6zyIl2oygd1HWfIoeeJ3/UY0eVXQZvcWQTJ7cqsWPDZuOWBVH6nU2eJzvMCntnTjiln3rlCZ/bHLhcqH9+5SzC7jc5fjGYbYZT659nu9EHiv91ZZvhyPbHSRZNJ6ebs82wX7/v1IVs0ldrss2w/ch1HuyGRnlft4zS+lMWVKT0S6d8F3bHrPKH09gx7IrAB8JNjvfa19qxKyLsGrtF0Wu3xfWh9I1mZ8gVeosci2aHWa922a+4Ty+Qcw3YtdJt0JfsWkklmVU+ZLvmsSsC2F1TbZ43u2MW8fa7Ndiu2SNP+NT24KUTmrADgQp3yBG2a940hB0izKj7J/md5in39BjAbqrb/kD+ptfJz63GUNq/905VPtvwL9d+ix1ODv/cld2JU9x7gR2YvzxpUXm2b3HML8fumIXOm1GHHWT756jZgYDv6tHsQOC96tfJT1Rv8008uxl8s/Gv7GbwLjncYZiwRRxSPOVmvmwe2h0sQyEVyf+ora56n/XV7Jt0qaq0bbT59yPjFYHshqkwQsFulOmnant27/yuu29P9kNSsJJtmb8p67ODHBrVXL1fuF8fJdyfqRGnYn/+T9yi2dFJoe9rilOUvhPcuje7J+0wQcEOf1LCGrA7N9fVkzUs2AFNqr8hPLQxxZHdH+7UKYRtwEeUb3qzOzeN470+ZnflbUezDZhWH+nINuA1rq+yY0t9wHgN24A/COjVlv1pXaYKB4WTQ9uxC7ilna5g9/WH1a/HDgovKWZob7EdQlToJHaAppjmwq7tNr0D5gkHaOMD2D2yuomubJe4yXFkdXYBFcbYsV3iUq+Exuxei9dk4aDw7Rc6dxQmPV04KEwJbxYvTHqGit25Gdi01mjhbzzdjm1snTuWZ7vEY8rJHuzOTdP+PmyXuM02he0SadhIJ7YVrHUdwnaJ+sA3NOyg8EP/PmyXmFtuqvBhGlMqt2fX5G2bGcK1klT3ZbZLvKKcIdxK9W1Tjd1knSbTnNg9sqgeQeza/596gnBvtP5gd7ZL/MLhNeEuX+XR9myXuNxjILstKvSZKBwUvhPcjV3t9x2nCweF48JasF3i3+oZynRKBzeuzXaJ5xUzbNk9iq4dKk4XjrWnuLFJN4/zXSgca48VXhkaMdxlvbAtjI54ygaCkJ5L/leTh35w9s/uzkRZeVotTVhjD7tg14HvpYRrbMd/d2C+ryL+x/tpUxQBP+jpuYqKeWyb/EhRiW1S94IVwmOAHyhqsv/rnqeSbTh0kkJ4D9RNR+GxAjpUKbwx6pKN+ne26K4SDiF+UdkKhxAtVMI7t/YrnNj9elpLxe4i068UnqytwvJKtsOhy5TB7C+e46sUPsTlbVV1dgF3yil2sh9eVzVkpxl2hguIV8ew0zS15hxbvKIW3tByQmmXwRYvq9htN92hKCe8wLCa6m32w3qlL9tM83XCI4F0vrIC20zveyiET6adrBIe7ch0VAqPxAxXswMretlGxW4QaC91d8OktcL79VurhU8LPKR0ED4Quo5qIjvdqnBnV0RhReUHVHiELYjtx3P9FGwvSeeoqgqTdlUI326QomIHVvS6nXBbRAeohWvlD43hWumojmWnPyoN10pjFdv06W6FM7tB0NdQsV7pBqX3A5Z6sJLtcOgi5QvsFue+p4LdyNBpKuHRjixnJbuRoUlq4YGDKzaqk2zRRy087HNWrWW90rZq4dGOw0oH4bCjrmo8O/1W4SZMurKS3Ymjq5QBwp2jzdqNT9kyEFLJXLnH/oD75k3+7Hre8h41W0+4tOIlO2X5178KJ4T4zWxDHFxJzf5al3A3VbSvY2Q9J686qlqdaipr2r7YtaOfc4hT24FN1BWULw3qHeLk5d19ZCuNTtlsZP8XnFyC+ye3UQdoo8bEB7s4VBmc0kbla9d5fHygq03NYSktlN7OPSb293VXR456vYnSy63PxH4enqpGr49uoHL36T+pdzkvZYuxI1/SugYOmNTd0UfV7o1hYbbOLw6a2NnORxszIbGKg0PokDc7aL3tukwaXN7ZNnzE+Ci1t1OPyQODymnqjBrbUunl2ndyvI+76uXXxzRWeXjFTe7n7qlslpJcT+3mnzC5t4uXss24pFpal5BBk7o5eGuiJwyrbudY6dWJnWy8bTpNTKzkaF9t2IRXNN4O3SYNCnHRRiS90Vbl7dxr8gB/V3Vkckpzpad7v8lxXh6qxmNeb8gmHT+5r6unsuXYUXU0roEDJ/dw8lZFvTEinE168KQudt7amDeHhNo7VHk06QpOtmHDx7dXezt2nzxQV0770qhxrZRe5fpMjvd1VzUYPaaJytMzdnKsh6eyacro+io3v6JJtx43srbWJXggu1a81WzSNeycKr46sbOtj03HiYmVHe2qDRUmbd910qAXXGweTzqATfq1sS2Unm59J/f39mDX9OuNVLOetkEgJG4KHginmUcus9PL325mN6LXvnxv4yX6YOusmV/ep4fmDH/3GP1rYWLSx5n5G8f0mribnp7Rc+AHN7I/Hhwz5nt6YOwr/d69fvP92KjRO+j2Ue36vnfz6lvd2ybvLdwypGXfebcuTOncZvSh/LWDmsUuuPvr+Jh2Y37IWT6gafzSBz+ltGs/7nT2otgmAz7JPTK6dYcJabfn9W46aH3B3pEtO0+5+Pc7PZolfqHfPrxF15npV2d0bTZ0W+GWV5t2n/P3n5M7tUjak79uUJNe79/6fUJMq9cO565MaNxvYfapsVFtx/x4/6O4xv0/zjkxuk30uDN3PuzTeMDqvP2jWnZ483zm3J5NB39euGN4885TL2e81a1Z4leFW4c06zYr48q0Ls2Hf1+4YXDTHu9m/jGxY8uR/8tfM7BJnw/unH0juvXoYznL4hvHfnT/pzHtolJOZi/q1zhhRe6R5FavjP+dTbrJwE/z9ya16DTpz7/f6d701c2F24Y17zL9avpMNulv9F8kNu0++8bFKZ2bj9hVsJ5N+r1bacKkD+V+MqBx3wXZp8e1b/P6Dw+WsEkvfZD6epv24365O79v4wGr8g6+xiZ9Luu9nk0GfVawcwSb9KXrb7NJf1n49ZBmXWdd+2t6l+bDvivcKEz65oVJHVuM3Je/dmCT3h/c+XX8K62Tj+UsZ5NefO/nlHbtUp7+rVpmeIlQvvFPrZdlSGDNzBDSndNZxoYgJJAZyUPaW4uQbZRG7xQbhJBAZqQO6ajWqTUL6bqP9oTIKIQEMiN1SFFBl9OFPVJGUIzIKIQEMiN1SO6zqCEkOtNVZBRCApmROiT1muKQVmhERiEkkBmpQwoYXxxSf53IKIQEMiN1SINcU4WQMt8gYt84iZBAZqQOKT1QHUHCw21I0DWRUQgJZEby55EyEt0JIR6JGWKDEBLIjBle2aC/lia2NxIgJJCZsvmNfQgJZKZsfmMfQgKZKZvf2IeQQGbK5jf2ISSQmbL5jX0ICWSmbH5j33ECIDPHn7uH0v/GPvrTif/ve/L2avN6oaeZJ9Ap1MwTmEKWmXkGzsPNPIFGbf61ZRb76fl7KP1v7HsK4XMFzUv4XEGzmtTUzBMQPtfSvDw/Nz6mVMXFcTyz0v/GvqdASAgJIT30rN/Y9xQICSEhpCc80zf2PQVCQkgIiQOEhJAQEgcICSEhJA4QEkJCSBwgJISEkDhASAgJIXGAkBASQuLgpuKcOS72CfXnmXkC01ubeQI/qnLMPAP/L8w8gUGDOJ6ZWUKi581yqU/464GZJ5Bt7B36pc7sf4M/n/d5fN4yxd5G97zMExKAhUFIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXBghpCyknQa34Sr0l8wFb5mMEgbHHPYvNMYRRLMOIFvGzm6NN1jxgmc7eOj9uhw1DwzyBurrFW09sRl85iG9CHlRpDOM+I1ITzfnvisbgaTqDd7q21PmnMax1WGkMw0geXkxQmjPbUHzTaB005uE1dN81HvMscMzkQ4FYf0xGVzmYb0Ic0lb7PTz0iy5JdM6VDyITvdRNqZcRr54WGGkMwzgQzHmtmUpjkOMds10IvsZqc/kyZmmMFtu9ppNkUhPXHZXKYhfUjhToZP3SjvpZf8ounI5nnsVG+nM+M03lJsM4RkngnMIduFhd5sE6B1ifA3oM7BZpjBzeQ8WhzSE5fNZRqSh/RA1dywjCNm+/SNHE0D803jnF1ilhCSmSbQ2i6P5twW1sx1DcSSU+z0hrKtmWZQFNITl81nGpKH9Dsp+jSxSWSH1Bf90Dx2gGe2aTT3vWUIyUwT0IX+0EBBXlxhvj/EGdew/ek/NLc/YqYZFIX0xGXzmYbkIaWSoYblHLJZ6osutlf7cr7ZprGCbKSGkMw0ASedb/LGeUFkrfn+EL+GEkKCDpnrKigK6YnL5jMNM4Q0zLCcTbZIfdFFPrWJuGm2aWS4tacPQzLLBGzIJ+z0qqNPgbn+EGdCAt/duqyqyw4zXQUPQ3p02XymIXlIaSTWsJxAdkp90QL9RNLmjvmm0cPxYnFIZpqAu8rwScVdyUlz/SEi7a+w03v+/nnmmUFRSE9cNp9pSB5SrrqJYdmTXJT6ohl9PBleYL5pfEvevHz58i+k5+XbZroeaqkMj5kNIQfNNIG7iqJPPe9HTptnBkUhPXHZfKYh/cPfde2F28RCv0DJL5lJIjPNOY1k8lCKma6HYeSIsGhFLplpAtdJPcOyGzlhnhkUP/z9xGVzmYb0IS0hk9npIjJF8ksWnopNMus0zmwVrCettp410/VwQtEsh9Ljyhpm+0OEaH5jp1luzjnmmUFxSE9cNpdpSB9SQUMSM6WHoro5vlXkRTI8xSDTnNMw3Ecy1wRGkvApA+20e8w2gc1K9/HLZ4SQBWaYwV72p1f5sJO/n7xsLtMww4tW747WafyH3pT+gtl/7UMXzDmNopDMNAH94jBbl3bHzDcBeqiDp9q1xTfmmMGsh3//tH9cNo9p4G0UABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADhASAAcICSL0p1cNvcUrBRCsigIyVwQkkVBSOaCkCwKQjIXhFSW5cyu4exYfXYhWz3awV2j63OBrfUkWYO87OoevZfk51Avlf0fHcjVBC9tpYW0OKRrQ4I0HjHHzDpzq4OQyrL+pNeixR3JUEpP2PpNXTLWyetvSmNJiyk/rLQNap9yYmM57zyhnjopB/e3JEuLQrquc0lZPTPAZq+5Z29VEFJZZl9POB3VuYAujNjD1j4kH1KaQBLZajfShZ0mkYNCPT3Z6i2b4KKQEtXH2Y+XnGqbb95WCCGVZS5+GU/8lPdgF0kWQtrBfhhPVrPThWSjUM+Xwq9bkKtCSHqPiHRBa3LXLHO2UgipLJtHnPsuv2JYXdWoHGGShJDOsJ8nkd3sdClZJ4R0VhgRS34QQrpGHvrFjDO3OgipTNvVwYEo2v1J6ThSe8Xewx8XhZRGhZD200chXRTGDmFpsZDSSPi2Ilnmnbt1QUhlXM6OWEX53Ad2gcKB2vb/CEnYRdHe5OeiPVK4WSdspRBS2ZdIjl4gHYW1cf8R0ibhl3XIdcODDR62hl3RdfNN2BohpDLssN8nwmIo+eG+oiZb+dGfDH5qSFFs9TdFpeJH7cgb7MfrPu3NOHPrg5DKsPxq2oELFsYrX9bT9mTwujddv1UHfJr9lJBatF+8MJisLQopI4j0XzkzSPO9uadvVRBSWXZz5Iv2LmEz2d2j6708XZrtp1McfdKfElLaSD9t6Epa/MqG9MRAdblX4cUrkAAAAI9JREFUjpp57lYGIckfXmBXBiAk+UNIZQBCkj+EVAYgJPlDSGUAQgLgACEBcICQADhASAAcICQADhASAAcICYADhATAAUIC4AAhAXCAkAA4QEgAHCAkAA4QEgAHCAmAA4QEwAFCAuAAIQFwgJAAOEBIABwgJAAOEBIABwgJgAOEBMABQgLgACEBcICQADj4PynfEbM6HjEsAAAAAElFTkSuQmCC", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 }, "tags": [] }, "output_type": "display_data" } ], "source": [ "# Use a non-linear search on lambda\n", "lambda_search_space = 10^seq(10, -2, length=100) #create a search space from (10^10):(10^-2), 100 samples long\n", "plot(lambda_search_space, xlab=\"sample\", ylab=\"lambda\")\n", "\n", "#this will span a range of models, from the null model containing only the intercept (lambda = 10^10, extremely sparse)\n", "#to the least squares fit (lambda = 0, lenient)" ] }, { "cell_type": "markdown", "metadata": { "id": "L8l9MliGCDK2" }, "source": [ "The `glmnet` package requires that you use matrices instead of dataframes, and that x and y are specified separately. So let's select x and y. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1vwgRbcIakNy" }, "outputs": [], "source": [ "# Define x without the first column\n", "x = model.matrix(Salary~., Hitters)[,-1] #without first intercept column\n", "#the model.matrix function also automatically transforms qualitative variables\n", "#into dummy variables and gets rid of NAs\n", "\n", "y = Hitters$Salary[!(is.na(Hitters$Salary))] #selecting y from the dataframe - get rid of NAs so that rows are matched with those in x" ] }, { "cell_type": "markdown", "metadata": { "id": "_WjdKQR6akNy" }, "source": [ "Now, technically you are using an *elastic net* algorithm when using the `glmnet` function. So we have to set $\\alpha$ to zero in order to run pure ridge regression. If $\\alpha$ is 0, then a ridge regression model is fit, and if $\\alpha$ is 1, then a LASSO model is fit." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3WuVVpEWakNy" }, "outputs": [], "source": [ "ridge.mod = glmnet(x, y, alpha=0, lambda=lambda_search_space)" ] }, { "cell_type": "markdown", "metadata": { "id": "5j4CAuBWakNy" }, "source": [ "Ridge regression returns all _p_ variables for each value of $\\lambda$. Thus we get a matrix of regression coefficients instead of a vector." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "id": "0l5WFr-yakNy", "outputId": "c0ed6f12-48f2-44fe-bd37-02aad65f3c22" }, "outputs": [ { "data": { "text/html": [ "\n", "
  1. 20
  2. 100
\n" ], "text/latex": [ "\\begin{enumerate*}\n", "\\item 20\n", "\\item 100\n", "\\end{enumerate*}\n" ], "text/markdown": [ "1. 20\n", "2. 100\n", "\n", "\n" ], "text/plain": [ "[1] 20 100" ] }, "metadata": { "tags": [] }, "output_type": "display_data" } ], "source": [ "dim(coef(ridge.mod)) #this gives 20 rows (one for each predictor, plus a new intercept)\n", "#and 100 columns, one for each value of lambda in the search space" ] }, { "cell_type": "markdown", "metadata": { "id": "MeSkZU_6akNz" }, "source": [ "Let's look at all 20 coefficients (including the intercept) when $\\lambda = 11497.57$ (i.e., the 50th entry in the $\\lambda$ vector)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 454 }, "id": "XUHrsA-GakNz", "outputId": "07635c50-ffa8-4d5b-fac2-e974cd1baaa3" }, "outputs": [ { "data": { "text/html": [ "11497.5699539774" ], "text/latex": [ "11497.5699539774" ], "text/markdown": [ "11497.5699539774" ], "text/plain": [ "[1] 11497.57" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAADAFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////isF19AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO3dCXxU1aHH8ROSSVgFBELYAlRweWpB0GIrIIqolVVtQdQnEKwVEFMLGhckFKuxUKnPPtTS51JtxVpFra3aUpQWgSpEsVJFAkVRFkETBZQt5L6ZuROimHsn5P6ZmZP8vp+PmQNzMnMc5peZ3DnJGAdAYCbZCwDqAkICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAIAEhrVoJWGXV4d/Lj3xIKwxgmRWHfTc/8iEtNXuP+HUAQnvN0sP+HEICDkFIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKJDqli/cIFCxZtjDOLkGCZxIZUOiXb/WnC3Jlf+M3zCunNggsuKHiz1lcPHCkJDWlzV9N9bOGsWdNGtzc9Sn0meoRUlN7/+uv7pxfV9vqBIyWhIY0PPREblc9Ny/eZWH1If8hcEDlZkPlkbRcAHCEJDSknr2o8qpPPxOpDOmWqezq1V20XABwhCQ0pdHvVeEamz8RqQ9pplruDZWm7arsC4MhIaEidR1aNh3fxmVhtSJvNu+5gjdlc2xUAR0ZCQ8pPm73HHe2abgp8JlYb0r6Gf3YHf2q4r7YrAI6MhIZU1ss0Gzj2mkljBjQ2/Xb6TKz+e6SLz62InBwYdHFtFwAcIYl9HWnvnJ7pkZeRQqfPK/ebV31Ia5pfusVxtlza/N1aLwA4MhK+RWj32uLiknj7FjxeRyo+yXTpYk4qDnL9wJFg1xahA8UPP7zyQICrB44Mu7YIASnKqi1CQKqyaYsQkLJSaIvQZ6UHvUhIsEvqbBFal/bld236vLbXASRDCm0Ren/9QT81fi/XAiknNbcI3U9IsEtqbhEiJFgmNbcIERIsk5pbhAgJlknOr+P6rOAd3/MJCZZJTkgfmOd8zyckWCaxOxsqjTbnjh/vM5GQYJmEhmS+wmciIcEyCQ3puvSeL5ZF/Ns8XlbmM5GQYJnEfo+0omfahE8dvkdCnZPggw3772zU/klCQp2T8KN26waaoRsJCXVMEg5/P3R000JCQt2SjNeRPrrEEBLqluS8IPv8lLd9zyckWCY137GPkGAZQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQSHVLF+oULFizaGGcWIcEyiQ2pdEq2icqd+YXfPEKCZRIa0uaupvvYwlmzpo1ub3qU+kwkJFgmoSGNDz0RG5XPTcv3mUhIsExCQ8rJqxqP6uQzkZBgmYSGFLq9ajwj02ciIcEyCQ2p88iq8fAuPhMJCZZJaEj5abP3uKNd002Bz0RCgmUSGlJZL9Ns4NhrJo0Z0Nj080uFkGCZxL6OtHdOz/TIy0ih0+eV+80jJFgm4VuEdq8tLi7ZG2cSIcEybBECBNgiBAiwRQgQYIsQIJA6W4S+mHPnQRcSEuySOluENn2n90G5ZkdtrwNIBrYIAQJsEQIE2CIECLBFCBBgixAgkJRfx1X+9ordvhMICZZJbEhLv99jRLFTcpIxzeb6zSMkWCahIf0zZELmqPVnNLnsoqbmjz4TCQmWSWhIQ0ILyj88+fL0JY7zbpNzfCYSEiyT0JBaXR7+sMj0j4zHtvSZSEiwTGK3CBWGP+wyV0fGN2f4TCQkWCahIXW9IvKx+Y2Rj6Pa+kwkJFgmsT9GkbWkcrg8dLHPREKCZRIaUknLtJvc0eWhjNd8JhISLJPY15HePmeaOzi507N+8wgJlknSG41t8j+bkGAZ3rEPECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAIEtKGrxOtipBgmSAhma8TrYqQYJlAIQ3L/6oRhIR6KlBIzx1yxguEhHoqSEjHvXzIGUuOC7qcGEKCZYIetatYv3DBgkUbdQuKIiRYJlhIpVOy3YMMuTO/UK6KkGCZQCFt7mq6jy2cNWva6PamR6lwVYQEywQKaXzoidiofG5avmpJDiHBOoFCysmr+stRnSTrcRESLBMopNDtVX85I1OyHhchwTKBQuo8suovh3eRrMdFSLBMoJDy02bvcUe7ppsC2ZoICdYJFFJZL9Ns4NhrJo0Z0Nj0U971CQmWCfY60t45PdMjLyOFTp9XrlwVIcEygX8eaffa4uKSvbL1uAgJllH8YN/e1176j2Y1lQgJlgkU0m0vRT7e3zL85K73G8JFERJsEyik6JG6P5msC394hmm+TrgqQoJlgofUvfnb4Y9PpY0TroqQYJnAIW0zN0fHIzroFkVIsE3gkDaaR6PjaSHdoggJtgkcUnnzoug472jdoggJtgkW0ugVJdtv6vZ5ePhOk6HCVRESLBMsJNeTjvO7Jg1eE66KkGCZQCE99IvC/DEjBixynLkdDv2NQoEQEiwj+pXFOw8I1lKFkGAZfvc3IEBIgAAhAQKEBAgQEiBASIAAIQEChAQIqEIqOaP/8NllwdfjIiRYRhXSG8Y8cNGM4OtxERIsowppb0mJ48je2oWQYBm+RwIEgoR03/uHnLHxvsDrcRESLMObMQMCgUJ6YMNXPUxIqKcChfR1olUREiwTJKRJXydaFSHBMhy1AwQICRAIHtKGhfPvn79wg2g9LkKCZQKGtH5yt9hhhm7XCt/ahZBgmUAhbcvLMNmjpt3zyD3TRrUxGXnbVKsiJFgmSEgvZ6dfVlwR+0NF8WXp2YtFqyIkWCZISKH+X31PpJL+ql+kT0iwTJCQbj30DZjLbw28HhchwTIc/gYEgoTU5ytOEa6KkGCZICGlR4SMMWnh/5p3Eq6KkGCZoE/tSvtNWrXb2fHKJWd/qlsUIcE2QUPKGxkbDB4vWY+LkGCZoCG1eSA2mN1Gsh4XIcEyQUPKujM2uDFLsh4XIcEyQUM6pYP7lpevZvcQrSiCkGCZoCE9l266DRo6qJtJe0K3KEKCbQK/ILvk/IbGmMwBL8qW5BASrCPY2XDgw7Uf7BctJ4aQYBlBSNuXLfyn7Ld+uwgJlgn+1K5P5Mf60ga+JVuSQ0iwTtCQXs1K7zv+mnF90o5ao1sUIcE2QUMa2vGd6Onr2aNFK4ogJFgmaEit7ogNZrSVrMdFSLBM0JAyHokNHlb9dGwEIcEyQUNqf1NscEMHyXpchATLBA1pbNNnIr/+pGJBkytlayIkWCdoSBuyTc7ZQ8/OMe0+0C2KkGCbwK8jbRzT3Bhz9JWbZUtyCAnWEexsqNhcskW0mkqEBMvwW4QAgcAhlS99cr5LtiZCgnWChrSyy+G9W1/F+oULFizaGGcWIcEyQUM6vUX+vb921eAzS6dku9HlzvzCbx4hwTJBQ2ry9GF84uaupvvYwlmzpo1ub3qU+kwkJFgmaEhtiw/jE8eHKn8evXxuWr7PREKCZYKGNPkmz2lfl5NXNR7l95tZCQmWCRrS50O//9uXlkTF/8TQ7VXjGZk+EwkJlgn8g32dDuOoXeeRVePhXXwmEhIsEzSkbzUceUuhK/4n5qfN3uOOdk03BT4TCQmWCRpSw0cP4xPLeplmA8deM2nMgMamn18qhATLBP4J2TcO69rm9EyPPAsMnT7v0Hf7+wpCgmWChnTlbYf5ybvXFheX7I0ziZBgmaAhlZ0zceHbJVE1+ly2CKFOChqSMYez144tQqijgoY0esz4SvE/kS1CqKsS+vNIbBFCXZXQkNgihLoqoSH5bhH64MRvHNSakGAXVUjrBg6M/4m+W4T2PPirgy4jJNhFFdIbNTlqxxYh1FWqkHa/VYP3dWGLEOqqxP4WIbYIoY4ShLRj9eG8Xx9bhFAXBQ5pcW9jXnCcoX87zAsp3eBzJiHBMoF/sC+z2XnhkLblZK6swWe+eUHnvnPdJ3UFfk8QCQmWCRrS4NwPtkQekT7KHR7/E1/JMo1D5szo5iBCQl0S+OeRipxoSM4dLeN/4uDQ0xV75oRO2+UQEuqWwO/Y99tYSA/V4B37Ol0e+bgo84JyQkLdEjSkjrfEQhrXOf4nhqZHTx4x1xIS6pagIV3VsjgSUunNZmL8T+w4zD29ycwiJNQpQUPa0imjl+nZM8vkbo3/idem/XJf5LRijPnRZEJCHRL4daSPJrQyxrSe8FENPvHjXHNOdFBxrf9P1BISLKN4x76tJTV4NIraPvFHsdFTxxAS6hDesQ8QCBrSKX0qfWfYrMPZc+eLkGCZwIe/I+9pHtnRnZVpTOdNolUREiwT/N0ozn5xh/P5onPH7P9sTnoNfpNQjRASLBM0pElnHYieHjh7uuNc1VG0KkKCZYKGlD03Nri/i+PMq8E2oRohJFgm8LtR/CQ2+FmW4xS2k6yJkGCdoCH1ynHfRPadLsc7K7KHiFZFSLBM0JD+mG6OHzJy2DfTzANO/6zDv6zqERIsE/xHzQc1jBwA7/OU4zz4mmpVhATLKHY2lK57P95vMzlMhATLsEUIEAgSUp+vOEW4KkKCZYKElB4RCn+HlBb+r7nfu0scLkKCZYI+tSvtN2nVbmfHK5ec/aluUYQE2wQNKa/yDSYGq/bZRRASLBM0pDYPxAaz20jW4yIkWCZoSFl3xgY3ZknW4yIkWCbwD/Z1cF+FfTW7h2hFEYQEywQN6bl0023Q0EHdTNoTnvMPHyHBMoFfkF1yfmSLUOaAF2VLcggJ1hHsbDjw4doP9ouWE0NIsEyQkLaUhv+rIlwVIcEyQUIy54X/qyJcFSHBMkFCGlUU/q+KcFWEBMuw+xsQCBrSs6t1a6lCSLBM4F9+cqfntAAICZYJGtI53z2gW8xBhATLBA1p6+jzH1tZEqVbFCHBNkFD4vA34AQPadR/542P0S2KkGAbDn8DAkFCWvm1c77+N7VDSLBMkJAy7znkjHsyA6/HRUiwTJCQrjHffv5Lf/38t80kyZoICdYJ9D3SY63MCfnPrN66e+vqZ/JPMK0eU62KkGCZYAcbdha1PXj0O+dO3Z2fkGCZoEftDrxWlDdswLC8ohXKHQ6EBMtw+BsQCBLSjxc5zg9XadfjIiRYJkhIDYrC46e163EREiwTJKR2LSYWmIsKKglXRUiwTJCQHm1oDJtWASfgwYayFUvMHUsqCVdFSLBM0KN25y3TraUKIcEygsPfO1aXiRZzECHBMsHf1by3MS84ztC/yZbkEBKsEzSkVzObnRcOaVtOpupHKCIICZYJGtLg3A+2RB6RPsodrlsUIcE2QUNqVeREQ3LuaClbEyHBOkFDyvhtLKSHQrI1ERKsEzSkjrfEQhrXWbUkh5BgnaAhXdWyOBJS6c1mom5RhATbBA1pS6eMXqZnzyyTu1W3KEKCbQK/jvTRhFbGmNYTPpItySEkWEews6Fia4ny0SiCkGAZQUjbly38p3iTECHBMsHf1bxP5Eco0ga+JVuSQ0iwTuAtQlnpfcdfM65P2lFrdIsiJNgmaEhDO74TPX09e7RoRRGEBMsE3iJ0R2wwo61kPS5CgmUCbxF6JDZ4mC1CqMeChtT+ptjghg6S9bgICZYJGtLYps9UhE8qFjS5UrYmQoJ1goa0IdvknD307BzT7gPdoggJtgn8OtLGMc2NMUdfuVm2JIeQYB3FFqHNJVtEq6lESLBM8JBWb498eF20HhchwTJBQ9qXZ14On/zSjC1XLckhJFgnaEh3mcH/CZ+sGWXulq2JkGCdoCGdPCQ2uKCbZD0uQoJlgobU6K7YYBY7G1CPBQ2p7eTYYCJ77VCPBQ0pr/GfIyf75mX8t2pJDiHBOkFD2tzO5A4a0vdo0+593aIICbYJ/DrS1qsjv/ykzQ8+lC3JISRYR7GzYdO6XaLVVCIkWEYQ0hFASLAMIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAQKJDqli/cMGCRRvjzCIkWCaxIZVOyTZRuTO/8JtHSLBMQkPa3NV0H1s4a9a00e1Nj1KfiYQEyyQ0pPGhJ2Kj8rlp+T4TCQmWSWhIOXlV41GdfCYSEiyT0JBCt1eNZ2T6TCQkWCahIXUeWTUe3sVnIiHBMgkNKT9t9h53tGu6KfCZSEiwTEJDKutlmg0ce82kMQMam35+qRASLJPY15H2zumZHnkZKXT6vHK/eYQEyyR8i9DutcXFJXvjTCIkWIYtQoAAW4QAAbYIAQJsEQIEUmiL0L9WHnQzIcEuqbNFaF0D8yU7ansdQDKk0BahXaUHzeERCXZhixAgwBYhQIAtQoAAW4QAgaT9Oq6PS3zOJCRYJmkhFfhdCiHBMoQECBASIJDQkHp/SQ4hoQ5JaEgNGmQdlE5IqEMSGlJBs6pDdTy1Q12S0JD2nXLqvsoxIaEuSezBhrcbTa0cEhLqkgQftfvsk8rR4iKfaYQEy/BGY4AAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIgAAhAQKEBAgQEiBASIAAIQEChAQIEBIgQEiAACEBAoQECBASIEBIqWrjr3982x/3JXsVqKG6F9L2/XEmvLfsE9/zV994wbk/fs1vxv435i/c5nsZZS8/snyX34SK4ofu+8denwlFoc5D+zU57l++V/OF77lh5bvjTDiwzv9/JKws3oT9O+LN2OX3fxqx913fW6tG9sS7kvg3l1Me5+wl99+/pPo5iQ6pYv3CBQsWbYwzq9Yhrb+klcnqs8Dn+u9pa4zpsch7xt0ZZ1x/4znpN3nPeDbX5GSlj/vMc8K+goahjg1a3ON9EWtONV2Oy+j0V88Jcxs+Hv5Y9v223vfz4hHZJud7q72vpOLBUxs2OGaK9zqdD0c3NqbdbT6Pe28Ma2lanP9P7wnOb3pnmo5X++S44/quaaGT7jngPeOVvhmmwTd9/tXi2114bHroxLt8vohuuCLXtDx3sc9lLB7cNuO4yVu9JyzvnnH88Rndl1d3XmJDKp2SbaJyZ/p+dahtSK+3GPD7txZOCc3wnDG52d3r96yamPG014TnMx6LnPyl0YNeM57OuHm7s/+l4/p6/qtd0fapfc7n9zYu8pqwOWdI+GvJZ1MyX/GYsPdot8J9J3oGvSDzot8vf+yChn/xmlAxrsktf10299jjPvKa8Z+c7zz7/ur7sgd7fiH+U+aFT616enTG770mOBMa3fK34od6dHrPa8LHJ3a777XFt7e4yPNKnskY8/dNr16f8XPPK4lrx7c63b10cVHr73p+TVjZou9vlj81Nv0+z8u4J/2Kx1++t0fOu14TVjcd/3H4/yev2b+rOTOhIW3uarqPLZw1a9ro9qZHqc/EWoZ04KTRFZHT5xp4PTNb1uAf0dPCbK+nEv0muKczu3lM2N/h1ujpphYPeMx4JX1l9HR+1iaPGZN6uf/ged/ymLC0QexxpKiXx4xtzW+Lnk7N9nrEmd+oOHKys9clHhOcC86KLmNd83keEz5tfUv09M5mXjH+MTP69XnPmed5XUneSdEFrml+v8eEz1q7X/fmhzzvwnFd943oQ+J/WnvFuP+4y6MPiQ9kel3JW+m/jZzsG/ytCo8ZwwdHz6kYPLyaMxMa0vjQE7FR+dy0fJ+JtQxpafoWd3D+1R4zJp3vnu5u+kz1E8ozYk+3Vpst1c/4R0bsm4bJ3/W4kusGuacVnX7lMaNj7J67ymyufsLTLWKDRzt6XMQvO7tf4fe0/o3HjLOudU8XZXh8T7ilQexJSsEZHhfxULb7fceBLv/jMWPYOPe0OO396ifsavhHdzCtt8dFPHZ07Jub3oUeM+La1+JRd/CzYz1m/C3zY3fwrRs9Zkwe4J6+l7ai+gl7s553B89nVfO4l9CQcvKqxqM6HXLmp/lXHdSvdiHN6x4bzOznMeO8ypvx1NnVT9hp3EcTZ5Px+NL128q79tz/8riSiyfHBufcUv2EivSXKq/N4x/tlQaxG+Bnp3hcyVWjY4PBUz1mtIl91dqT5vEE8h8NYk9On2rlcRFThsQGl/7AY8Y3Kh+WGz1f/YRVJvbU44WGHhcx/azYYMJIjxlxbTAb3MHStD3Vz5jTMzaYOtjjMvpXfj/wDY9nGgfvEWuq+/KX0JBCt1eNZ2Qecub2S79/UF8T72hTteKHNPS62OBkry+xLea7p4szPI5FPd089tB/h9fzsjFXxAaneX2T1OIP7ul7Zm31E/a0dJ/Ll3/zBo+LGH95bDD0xx4zWj3pnu5rsKT6CQfvdb/P9riIHw+NDS670mNG18pvJRv/ufoJq0zsAfzFLI+nTD/pHxv84FKPK4lrvYk9Hi7zuuP8vPLx8Aav5xFn/DQ2ONbjecSutGXuYGlaNd8XJDSkzl/6kjO8i8/EpSbuoczqLIv71K7wm+6/5qZ0jzuXM/Y70S/TFSO8nvRvafCyO/i215PTX+W4R1I2ZvzdY8awUe7pXe29jmXd3fjZ8Medl7X2Ooh017Hu/0l5+197zOh3vXv6SrrHIbXSUOxp7MRBHhfxf+3cx6yKY37hMWNI7KFqVeVjwqF2ZsUKK/R6bH2mqfslq+L4Oz1mxLX3qNjRkDnHeMz4c6PY18Uzvb7ujB/hnn7s+a922o/c0x+dVt0SEhlSftrs2NfAXdNNgc/EWoYU/2DDxkbRf6w9g3t53YPfbzMi/NVta17TN72u5YpjI0fvK2Y2XOcxYWeHyyPLL+v/ba9vW5dn/DJysqjp/3pdiVOYftz3BzXvWux1/oeN3MesO5t/7DHjwWZrIid7+g7zuozLv/lp5GRZ1pMeEz5p6R7RuLuJx/dyzoKs6FPhfecM9LqSK3pGn6Wua+n1/7q7yw+it9PsJh96XUZcE4+P/p9savdTjwl7ct3vGJ9NX+Ux4+/p7nGoq7p5HY19NhQ9oPu7jGerOTOhIZX1Ms0Gjr1m0pgBjU0/v++CahlS5PD3E6t9D38/kXX+vc/ceUIHj6dUYf8+1XToYo6v9sWCqJ0Djsqbc9NpTTyPoDvF7Y7Jv+uH2f/lfcf4TcOek64/q8FUr9LC1v3vpJv/4PGMP+L/0icufv+lcemeR6YPXHj0XSvXzO/dyfNVu49P6jLnpWemNJzgeSVPZVz2wpq/5qV7HdBwnLymt72y+rHT2q33mrDt2BMefvPVu1oP9ny1YFmzMx9aMv/i0HzPKwk/efvD/S9/7n126cndH/zX63M79Pf8luClhhf9ZcOyGzNnel7GtU2K3ti66KJGXs9VHOcXGadNmnRaRrWPzol9HWnvnJ7pkZeRQqfP830NubYhOetHx3lB1nlrzAktTivw+ioe9ebvHl7p8/KhU/7opb0G3bjBZ8YnRSN6j7zX76WyDTO/N+QG3+0Tcf319JAJ9fuH94Tynx9jTMtxni8jhZ8Y3HpyZvN+fnfg5ec0Ng3P9HkZs+JXJ6abVmO9HrHCyia3M2ndZvm8VrpuTJe0diM8jrtEbB9hWh0fav2o94wd1+UYk1vo84Vn1XkNTXqPJ7wnVPwqfHOFzvF8JhK2pvB73ytcU+1ZCd8itHttcXFJvExqHZJTgy1CdcfeDfH24u3wOIZfJe6tdWBTnH0zzm6fVF2fxN0A5Ps/srdXz9cd54tZGb/zm/Xxp3Guo/y9eHuEyt6p9ebG1NxrFyQk1Dlz27jPIIqyU/duQUhIeYNiL1p8FvLbKpdchISUd8K9sUFHn++SkoyQkPL6xA5qH2ha3YHn1EBISHlTe7svE7yQEfeHpxkLLHAAAAZtSURBVJKGkJDyNjadGjlyuCbX+wWvpCMkpL6FLY+fMP2irOG12oGZGIQEC3xUNPLsic/57ANJOkICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgRSM6QVBrCMz69c9nDkQ3JWrUxtba5+1AojuyV7BTXzazMz2Uuomf7ne90lvN7lwkcCQkp1nR5J9gpqpuj0ZK+gZjzfujDVjB0rvDBCIiQxQqqnCEmLkOopQtIipHqKkLQIqZ4iJC1CqqcISYuQ6ilC0iKkeoqQtAipniIkLUKqp455PNkrqJm7+id7BTWzu8GbyV5CzVx1lfDCCMl5f3+yV1AzX2xO9gpqaH2yF1BDpaXCCyMkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRCo5yE9FHv3gduSvRA/+25s0NsdleV3DrUbn7I/4Hdwoal9s5ZOyc3sMnx5ZKi7Qet5SL8wowsiXkr2Qny83atZ7P65t5e5+Pa8UFflT3YKVS00pW/WT7qYwbdeltHwX9IbtJ6HVJj6v6jjs0anlmS598855mfhj783U5K7Ig9fWmhK36yTzC/DH58yF0hv0HoeUr4pSfYS4vlkyj4ndv/s2WxP5KRbdkVSV+ThSwtN6Zv1RwP3hT9WNOosvUHreUhjzPbyD7YnexVxuffP3ekDo38aa1L214vEQrLgZt0TOkN6g9bzkEaYW1oac+zvkr2OONz751rj/iK2QrMwqavxEQvJgpv1f8JP8JQ3aD0PaYD5RtEjNx1l7k/2Qvy5989iMyn6p9lmQVJX4yMWUurfrIsz++6X3qD1PKRFT+4Kf/x31tGp/c7rlSFdE/3TLPN0UlfjIxZSyt+sj2X1+kR7g9bzkGIuNK8lewm+3PtniRkT/dM087dkLsZPLKSYVL1ZK6ab83c42huUkCJ+aFLyFY+D3Pvn3owB0T+NNu8ndTU+vhpSit6sFXlmcnlkoLxB63dIO+99LHraN3WPg0XF7p99Gn8e/nigfafkrsaHu9AUv1nzzR2xkfAGrd8hHejQ9J3wyTPmlGSvxF8spHlmRvjjfeYnyV2ND3ehqX2zPmXyK4fCG7R+h+Q8m9Zk/K0Xph1VnOyFeFtcUFCQnhP+8LFT3s8M/8klaSd/nuw1VetLC03pm/UYMzm6f6mgVHmD1vOQnGXfbZHR/ooUfh3eKYptAI1sFtg5tXOow6RPkr2k6n15oal8s1Yu02xQ3qD1PSRAgpAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkGyW3ifZK0AMIdngHXNetX9PSCmDkGxASCmPkGxASCmPkGzghjTa7Lyhc2bHORXh8Z97NWwzviwa0taJuaHWw19znIVpoyOTv9tgSVIXWz8Rkg3ckMaY865evvRc86DjLElvf8evL+8XCoe0rXPzgkfv6Ji12HGuNgsd50lzXbKXWx8Rkg3ckMabyCPOejPEcc434UcgZ6IJhzQhY0V4uLHZqY6zs0v3Pbs6HftFchdbPxGSDSpDejHyh8Y9nQONjomM3giHVNG615aI88xOx3kprXBqg2VJXWp9RUg2qAzp7cgfmp/ofGgGRUa7wyFtNZX+Hf6biVmh65O50PqLkGxQGVJJ5A/hkNaaodG/T+vjlJieL7jKwn9RbMxbSVxnPUZINjg0pA/cR6Sd0UeknlXzDny7bat+FUlZYn1HSDY4NKT9md0io6WRgw2tG0YeipxtkQ+zzeMPmbuTtsz6jJBscGhIzoDoUbtLo0ftzM3h4bacIY7zbqMLHOesxmuTutZ6ipBs8LWQnk/LvnH2kLObh0P6KNeMe/iO3NBfw0/smrwXrinrjANJXm59REg2+FpIzuMnZ7bJK+t0Sni4ZUKnjBbDXnWcn5s5kfNnmruSuNT6ipAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRAgJECAkAABQgIECAkQICRAgJAAAUICBAgJECAkQICQAAFCAgQICRD4f0kMlYCpBs8FAAAAAElFTkSuQmCC", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 }, "tags": [] }, "output_type": "display_data" } ], "source": [ "lambda_search_space[50] #getting particular value of lambda\n", "plot(coef(ridge.mod)[,50]) #getting coeff. for that value of lamba, all coeff." ] }, { "cell_type": "markdown", "metadata": { "id": "A42v1D1AakNz" }, "source": [ "Here you can see that many of them are near zero in value while a few terms retain strong, non-zero values. \n", "\n", "Now what happens if we make $\\lambda$ really *really* large?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 454 }, "id": "PUrSvXlKakNz", "outputId": "a2b75499-e442-41c3-9056-2582b521fb61" }, "outputs": [ { "data": { "text/html": [ "5722367659.35022" ], "text/latex": [ "5722367659.35022" ], "text/markdown": [ "5722367659.35022" ], "text/plain": [ "[1] 5722367659" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAADAFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////isF19AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO3dC3xT5f348ae0abkVLJVSbi04kLmpFHQDx0UGIiogOhXsdAOLgoDKNpgV5aKooKB42Q91booKKmOC100dgu7PvAB24kTdKIiCQgFtlYtAbXv+Sc4pRWhOE55vc/Ikn/frRXLaPmm+q/0saXKSoywA2pTXAwDxgJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIERCGkde8CRlkX+W95/Ye0VgGGWRvxr3n9h/SmOljv1wEIOqjejPgyhAQcgZAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEJLnvrr70p+P/Wul12NACyF57e1WHcfMuKRx/91eDwIdhOSxL4+/stx/9ukPR3g9CXQQksdmdfoueL5aFXs8CXQQksfO/Z2z0e4xT+eAHkLyWK/bnI1T7/N0DughJI9dOso+L2+2zNtBoIWQPPZ00y3B8z+mf+3xJNBBSB6rPLPLW/7/DPPT/s/rSaCDkLz2zWUNMk5Kbf6A13NACyF577NlD67g6VjDERIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECEjMkLa9+mxxVb1eAxJMIob0xdCkRsepbu/W41Ug0SRgSKWdz1hTYW28rOm6+rsOJJoEDKnwxL2Bs6qL+tXfdSDRJGBIP/iDfb4maUf9XQkSTOKFVJXymr2xR62ttytBokm8kKxmz9rn29RH9XclSDAJGNKgAvv8j5nf1d+VIMEkYEj/SFkSOHv/+Fvr7zqQaBIwJOuu5HNuv+uytMsq6vE6kGASMSSraFyv7r96vj6vAYkmIUMCpBESIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggpPp1cM3jL3zm9RCof4RUr55vm5TTLOmiXV7PgfpGSPXphZSpZZa1Oi9vv9eToJ5FO6SqTcuXLVuxpY5VcRJSZe4NwfOvsu/xeBLUt+iGVDopSwXlzPzWbZ0ZIX375O9H3/Wxy4JDb/g1pW9UBoJ3ohrSto6q86gZc+ZMzW+jupa6LIyBkEr+etuf33ddsTYn87zLT21QGPpNxJe2cDYWdBCcDLEoqiGN9i1xtirmJ010Weh5SFW3pmX2OqHBeTtDL9me+et9/rNX0u8MueQfac77FN1ziux4iDlRDSm7oGZ7RHuXhZ6HNKvp0/4bmo+6/ST0O3Zdf6r95il/Tg95L7Us1XljiDOvlh0PMSeqIflur9m+OdVlodchfdnoyeD5jozHQq7pfod9vi/ljZBrrm33P/9p1W1p/5OcDjEoqiHlDq/ZHub2Z4PXIf0lw3mrrtEXh1yTW91Y5tKQaw6c33DEbZO7NVkmOBtiUlRDmpg094C9tXe6KnRZ6HVI93V1Nm7rFXLNT26zz3cnrwr9jaqevarXkKns2hD/ohpSWXeVPmDUNRNG9mus+uxxWeh1SI+3cTauGxJyzfQu9oz3tfD8EUZ4L7rPIx2cl5cceBrJ1/Nh17c59Tqkzxr8M3i+P+fukGu+anv+Lv9NzqKGD0ZrKsSwqO8itH9DUVFxXZlohfTp6xsrj/3StoLcD/2ney9p+03oNR+f3Oin57RNm6t7XYgHcbeL0F86qGTV6kHNQy3v/0XKWdde3LLjf9wWVbx65w2Pf6F3RYgT8baL0AO+6RurPru78U3HePlDVk65aPwC1yGBGnG2i1BJ44eD539LXn9s3wA4JnG2i9Af2zn36XpMP7ZvEOeqPvlE804vahdnuwhdf66zcdUvj+0bxLWvxzVVKn3c117PEY/ibBeh6f2cjcsKXNclpLIfn7R48+bFJ51c5vUkcSjOdhF6obH9p9fBtg8c2zeIZxO7BG+Lvj7xN15PEofibBeh8hOHl/vPqq5ryf/rHqky83F74/FM7efZcCSzdhEqeWDcuAdK3K7iP61Ovn3xnB7NQu+RnbB2qA/tjQ/VDm8niUdG7SK0sEnH4cM7NlnodtEdhWe0Ov3aT495xPj1lXJe8btOfeWybOff73+ujmeZK1bed8eLe93XFN05+sZnQ7+aK+6YtIvQipR7/fdJKu9NWakzQNz6cvmjr7vs0WRV5dxrb9ybE3pR5bSGTU5unjLhgMs3eu+k1K490l1ePeK/i12QdPqvzko/udhljZQDC3+TP/3/ReGKXJn0dly9r7LPr+pT7wPEmk/vGzPxz64PW5dfn5Z2gq/pHS5PE83K2hg425g1K/SayRlL/P9n9Wrb/NBLtmReusv/6zszZXnoNddmv+M//ercE/a5zSxifacWF4w9M3mEW/rWricKb15ar++JFv2Qtg97q841tYb0rbNLtvVGg0TbdWeer0v+ha0zX3FZMirLf0fq4IL0aaGXHDwvY9pLL03NGFwecklxsn0d61JCv8hqTE/7bvm1p4Zc8rnzbfa2ud9lZBG721+423/2fruxLosebdL6nH7N2v/L/Tu99fSaY//Fin5Ixeqvda6pNaQvlPOC7f+pBNtT9MnURf7T8sJGofd7ejt5TfD8Od/m0N+nYn7P9PSe813+PL2ni7MxYHLINW0fsc/Xq62hlixq5dwwThgW+rpkzGtv39CsbBB6T+jnUv7gv5ndO7bZxtDfp2JGk+TspIxjDt+gkA6kvGZvLE9xvRmPO1Udb7E3zgl9h+v6nzsbndx/Farc9xCaPNjZGHtpyO+QvMLe2KfWhFpzbxivMBYy5DpnIzv0Y1Bd7KdaqvqNDP19rm6x6Ftrz/81vj30ElcGhWSd7fzHHXG21gDG2ag22RuLskKuya++ZzP49zpXNfMMZ+Pi0PeUMhfb55+okA8lPH2880zV2F/oTBOOXs4r/q1T7wu1ZHP1XZnHskN+m7UN7Pt9f0mt6yU+IUQ5pEmTJo1WQ/2nk4760mcnnnBIK1XbH4Zr0m7wf3p/YdraYx/ARKuV8yf7iuSQtydXjXA2fjZT56re8NlPHJRlLAq55hLn7tqsnJDT7Ei13+6lrOXDOtOE46Ix9nlFi7+EWhLOz++G6vfCPWH+sc0R5ZByc3Pbqpb+09yjvlS+dMkhV6lan659Oat5377Ns14+9us30qfqv/bGgtYh1zyWuTt4/nnqazpXVdW7Z+A9MXcP7hL6GYr3Um8N3N481/CR0N/nxozAf6QtvX9c73fCH21hH+rj6YZfhlqySTl/Gy10uUV3erTOO8Zb9Ni8a/dQ7SFZe5fOnLm0jmcC49CP7L/8q/qE3hN3f8eLA484lZ3ZQ2/3n5LuzfOn/Tqr8waXNcua/eDyMd2T3W76KiclnzD4NN/PQj4cIaa820/8s1Y9k+4yzg+c19ScfVnIJVdW//nZ+2aXK6t6/6mn3q/9Vs2skBLUiyn3fOe/mSho/knoNetzcsbNvrLlj3V/dcsXjul/xR/dn/7Zcd+V+bPcjh7gt+lPv7/rn9F47VPJWcldzsxOne5yXYt9j/m/evB3jUPP/EiW/T95e9qrob/PujzVpo3KW1fb1wjJBE+ktzzrZ007vO225pt5w3v+8qFEPBDT2odmPP2564r70064ZHBWlsszyPvaXxa4M7tnYPfQTw5syBi+xX9/dXhGbQ+yEJIRShffNOsF3j/vmG19cPzvn9jttuLfrTtPvu+6tp1cbvQvGRC821w54JJavujBLkLf1X2DT0iIti9vG3LKhfNc/gAvb/iivfFCo1p2xvUgpN3r63ytECEh5myrfvD0v2rb0V+NekhvnKbUy5Y11PVRWkJCzNmj3rE33q7ttzPaIa1OTR/kD2lnduq7LqsICbEnz3lRd2FeLV+MdkiDc7ZuD9wi7chx252RkBB7FqcF/0h6IbW2fSiiHVLmbCsYkjUrw2UVISEGzUweMGXKgNqfiY52SCmLnJAW+FxWERJiUdFvBw78bVGtX4p2SO1uckK6ItdlFSHBMNEOaUxGUSCk0hvVeJdVhATDRDuk7e1Tuqu8vDSV4/auWoQEw0T9eaQd4zKVUsePc31rNUKCYTzYs6GqpNj1PR4tQoJxYvOIfYQEw8TmEfsICYaJzSP2ERIME5tH7CMkGCY2j9hHSDBMbB6xj5BgmNg8Yh8hwTCxecQ+QoJhYvOIfYQEw8TmEfsICYaJzSP2ERIME5tH7CMkGIaQ4Jntrz69zvUuvkEICR75Kr9Bo2zVMU6OLUJI8MaB00/+V4W1Y3JKfJRESPDG/Vn2gY0mnaB3IJoYQUjwRl/nGfntSbW/LY9hCAne6LDA2chY5uUYUggJ3jjZOXjyd2n/8HYQGYQEb1w1wD5/yef2Ek9jEBK88aFvbuBsU67bGxyag5DgkcWNek6569dNznV99w5jEBK8svH6gaf/anE0DtgcBYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIE6IS0+WhCUxESDKMTkjqa0FSEBMNohXT+xO+7gJCQoLRCevGIL7xMSEhQOiF1ef2IL6zqojuOg5BgGJFH7So+WrtfZJpqhATD6IX05iVdLyiyik9WKn2+5FSEBMNohfSOT/lUs029mlz2i6bqBcGpCAmG0QppiG9ZxeenXJ68yrL+1+QswakICYbRCinzcv/JCtU3sD0qQ24oQoJptELyzfCf7FVXB7ZvTBGbiZBgHK2QOv46cNr8hsDpiFZiMxESjKMV0ui0VdWbb/suEpoogJBgGK2QijOSpthbl/tS1ojNREgwjt7zSB+dNdXeOKX981ITBRASDCP0eqQvBEY5DCHBMLywDxBASIAAQgIEEBIggJAAAYQECCAkQAAhAQKkQiru1XfY3DL9eWyEBMNIhfSeUo/84mb9eWyEBMNIhXSwuNiyvtWfx0ZIMAx/IwECdEOq2rR82bIVW+QGCiIkGEYvpNJJWfZ7fufMFLtbF0BIMIxWSNs6qs6jZsyZMzW/jepaKjgVIcEwei819y1xtirmJ02UGskiJBhHK6TsgppPjmgvMo+NkGAYvbfjur3mkzenisxjIyQYRiuk3OE1nxzWQWQeGyHBMFohTUyae8De2jtdFYrNREgwjlZIZd1V+oBR10wY2a+x6iP5q09IMIze80gH5+UlB55G8vV8uEJyKkKCYbR3Edq/oaio+KDYPDZCgmHE9rUr3aw7ymEICYbRC+n983J7z7fv1BVK7spKSDCMVkj/SlONferM4M5BhIREphXSYN+zVQfm+X6y1yIkJDatkNoHjthnrUg9r4KQkNj0dhGaHjx7Ql1HSEhsWiG1O98+n6LmEBISmlZI1yX9oTxwXjVS/eZaQkIC0wrpyxx1VnCj6jqlCAkJTO95pF3jf+NsLf0BISGB8S5CgABCAgQQEiBAN6QBA/YGz/cOGCA0UQAhwTC6ISllv3N+GY/aIZHphvTBB5XB88oPPhCaKICQYBj+RgIE8MI+QAAv7AME8MI+QAAv7AME8MI+QAAv7AME8MI+QAAv7AME8MI+QIAXL+yr+M+bdRy9mZBgmOjuIvTmBP/Jwlb+m6+u/3RbR0gwjE5ID352xBe2POh+wddTm1ZZf1VNLxk/sEHauy4LCQmG0QlJvXjEF16u47aqX1axZXXM3ebffKfRUJeFhATDaIX0yObve6yOkJpNtqyv1X3B7auOc1lISDCMVkhHc79gk2mWdSBpaXD7loYuCwkJhtEJacLR3C/Yq/M+y/rZ5MDmga5dXRYSEgwT1UftXlTdX/2uqPXj+8rf6a/+6LKQkGCY6D78/acmqtGPclVyskr6XZXLOkKCYfRD2rz86YeeXr45vIuWzB2Um56Wedp1Ra7LCAmG0Qxp07WdnIcZOl33idxUhATDaIW0syBFZY2Yev8T908d0VKlFOys+7JVm5YvW7aijj2ECAmm0Qnp9azky4qq/9SpKrosOeuNOi5ZOinLvv3Kmfmt2zpCgmF0QvL13fi9zxf39blfcFtH1XnUjDlzpua3UV1LXRYSEgyjE9K0iiO+UDHN/YKjfUuqV85PmuiykJBgmKg+/J1dULM9or3LQkKCYXRC6vE93eq+oO/2mu2bU10WEhIMoxNScoBPKZXk/9fc7RbGkTu8ZntYB5eFhATD6N61K+0zYd1+a/e/Lu3/dd0XnJg094C9tXe6KnRZSEgwjG5IBdU3MoNH133Bsu4qfcCoayaM7NdY9XFLhZBgGN2QWj7ibMxtGc61zctLDjyN5Ov58JGP+H0PIcEwuiGl3eFs3JAW3oX3bygqKj5YxyJCgmF0Q+rWdk3wfHWW2+uLIkVIMIxuSC8mq04Dhw7spJKWhFx/pLm96lpBSDCM9hOyq85p6P+jJ7XfK+Fffmydz+gSEgwjsGdD5ecbtn4XyeUJCXFHIKRdby1/pyySyxMS4o7+XbsegcezkwZEcFBzQkLc0Q1pdVpy79HXXNEjqdl/w7582dbaPlv50pJDriIkmEU3pKHtPg6e/zsrP9yL715f6x3BT1pmHNKYkGAW3ZAyZzkbN7cK78JvnKbUy/4AX3NbxF07GEY3pJQnnI3H6nh1rGN1avogf0g7s1N5E33EEd2Q2kxxNq5vG9ZlB+ds3R64RdqRM8xlFSHBMLohjWr6XODtT6qWNbkyrMtmzraCIVmzMlxWERIMoxvS5iyV3X9o/2zVutaH4o6SssgJaYHbXUFCgmG0n0faMrK5UqrFldvCu2y7m5yQrsh1WUVIMIzAng1V24q3h33ZMRlFgZBKb1TjXVYREgwT3TfRt6zt7VO6q7y8NJVT4rKKkGAY7ZAq3nzmaVt4F94xLtN/V/D4cTvcFhESDKMb0rsdwjxaX42qkmK3W6MAQoJhdEPqedzEB/5kC/8616ys48gVhATD6IbU5NkILnjrysDpQxn+m6/T3nNbSEgwjG5IrdyPGHbEBQPvZfeSSrtwbC/VfKPLQkKCYXRDunZKyGW1XDAQUufmH/lPlyZd4bKQkGAY3ZD2Db1k0cpVQWFc0B/STnVjcPsCt33zCAmG0X5hX/sIHrULhLRFLQxuT2UXIcQR3ZB+2nD4TTNsYVzQH1JF89nB7YIWLgsJCYbRDanhwkgumL+2eNeUTvv8mx83GeqykJBgGO1XyLo+jH3kBW3PWNaTTRqscVlISDCMbkhX3hrBBRfcM2PiyAv6rbCs+W1fdFtISDCMbkhlZ41f/lFxUCTfYk+l65cJCYbRDUmpiPe1CwMhwTC6IeWPHF0t7MvzJvqIO9F+PVIA77SKuENIgABCAgRIhbRxwICwL09IiDtSIb0XwaN2tb+J/uEICYaRCmn/BxEc16VOhATDePE3Ut0ICYYRCCnEYVp0EBIMox1SWIdpiRQhwTDaL+wL6zAtkSIkGEY3pPAO0xIpQoJhtF+PFNZhWiJFSDCM9hH7wjpMS6QICYbRDSm8w7REipBgGN2QwjtMS6QICYbRDSm8w7REipBgGO3nkcI6TEukCAmGkThiX92HaYkUIcEw7GsHCNANqVuPaj87f47YPneEBMNoP/wdOKZ5sv9fWqpSuV8ITUVIMIz+0Sj6v7Lb2rfi7JHffTMvOfx3EnJHSDCMbkgTfm6/1WNl/+mWNaad0FSEBMPohpQ139l4qINlPSy1mxAhwTDaR6O4xdm4M82yZrQWmYmQYBzdkLpn2weR/bjDD621WUOEpiIkGEY3pBeS1Q+HDD//1CT1iNU3LfLvVTtCgmH0X2o+sGHgAfAeSy3rUbdDHkWEkGAYiT0bSjd+dlBmmmqEBMOwixAgQCekHt/TTXAqQoJhdEJKDvD5/0JK8v9r3l5wKkKCYXTv2pX2mbBuv7X7X5f2/1puKEKCaXRDKhjubAyW2s8ugJBgGN2QWj7ibMxtKTKPjZBgGN2Q0u5wNm5IE5nHRkgwjPYL+9raz8KuzuoqNFEAIcEwuiG9mKw6DRw6sJNKWiI3FCHBNNpPyK46J7CLUGq/V8RGsggJxhHYs6Hy8w1bvxMax0FIMIxOSNtL/f9qCE5FSDCMTkhqkP9fDcGpCAmG0QlpxGz/vxqCUxESDMPe34AA3ZCeXy83Sw1CgmG03/zkjpDLNBASDKMb0lnnVsoNcwghwTC6IZXkn/PUu8VBckMREkyjGxIPfwOWfkgjflUw2iE3FCHBNDz8DQjQCendo75y9GeODSHBMDohpd5/xBfuT9Wex0ZIMIxOSNeoM/5+2Kf/foaaIDITIcE4Wn8jPZWpTpr43PqS/SXrn5t4ksp8SmoqQoJh9B5s2DO71aFHv7PvkPvlJyQYRvdRu8o1swvO73d+wey1kns4EBIMw8PfgACdkH63wrLGrpOdx0ZIMIxOSA1m+7eflZ3HRkgwjE5IrY8bX6h+UVhNcCpCgmF0QlrYUCl2WgUszQcbytauUrNWVROcipBgGN1H7Qa9JTdLDUKCYQQe/t69vkxomEMICYbRP6r5aUq9bFlDXxMbySIkGEc3pNWp6YP8Ie3MTpV6CUUAIcEwuiENztm6PXCLtCNnmNxQhATT6IaUOdsKhmTNyhCbiZBgHN2QUhY5IS3wic1ESDCObkjtbnJCuiJXaiSLkGAc3ZDGZBQFQiq9UY2XG4qQYBrdkLa3T+mu8vLSVE6J3FCEBNNoP4+0Y1ymUur4cTvERrIICcYR2LOhqqRY8tYogJBgGIGQdr21/B3hnYQICYbRP6p5j8BLKJIGfCA2kkVIMI72LkJpyb1HX3NFj6Rm/5UbipBgGt2Qhrb7OHj+76x8oYkCCAmG0d5FaJazcXMrkXlshATDaO8i9ISz8Ri7CCGB6YbUZoqzcX1bkXlshATD6IY0qulzVf6zqmVNrhSbiZBgHN2QNmep7P5D+2er1lvlhiIkmEb7eaQtI5srpVpcuU1sJIuQYByJXYS2FW8XmqYaIcEw+iGt3xU4+bfQPDZCgmF0QyovUK/7z/6gRlVIjWQREoyjG9LdavAn/rP/jlD3is1ESDCObkinDHE2zuskMo+NkGAY3ZAa3e1szGHPBiQw3ZBaXetsjGdfOyQw3ZAKGv8tcFb+cMqvpEayCAnG0Q1pW2uVM3BI7xaq9WdyQxESTKP9PFLJ1YE3P2l51ediI1mEBONI7Nnwxca9QtNUIyQYRiCkekBIMAwhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECPAipINrVn7ivoKQYJiohnTrysDpQxlKqdPec1tISDBMVENShf6Tl1TahWN7qeYbXRYSEgwT/ZA6N//If7o06QqXhYQEw0Q9pJ3qxuD2BW1dFhISDBP1kLaohcHtqW6HyiQkGCbqIVU0nx3cLmjhspCQYJjohpS/tnjXlE77/JsfNxnqspCQYJjohmR7xrKebNJgjctCQoJhohrSgntmTBx5Qb8VljW/7YtuCwkJhvFoF6E9la5fJiQYJtohVW1avmzZii11rCIkGCa6IZVOyrL/TMqZ+a3bOkKCYaIa0raOqvOoGXPmTM1vo7qWuiwkJBgmqiGN9i1xtirmJ010WUhIMExUQ8ouqNke0d5lISHBMFENyXd7zfbNqS4LCQmGiWpIucNrtod1cFlISDBMVEOamDT3gL21d3rwJRWhEBIME9WQyrqr9AGjrpkwsl9j1cctFUKCYaL7PNLBeXnJgaeRfD0frnBbR0gwTNR3Edq/oaio+GAdiwgJhuHtuAABXoQ0t1ddKwgJhvEipLF1fgNCgmEICRBASIAAQgIEeBFS2dbaPlsy+KxDTlK7Na8DiCoPQtq9vqy2T++ZVnjICFXXM01ATIl6SG+cptTLljX0NbdFbxISzBLtkFanpg/yh7QzO/Vdl1WEBMNEO6TBOVu3B26RduQMc1lFSDBMtEPKnG0FQ7JmZbisIiQYJtohpSxyQlrg9ib6hATDRDukdjc5IV2R67KKkGCYaIc0JqMoEFLpjWq8yypCgmGiHdL29indVV5emsopcVlFSDBM1J9H2jEuUyl1/LgdbosICYbxYM+GqpJit1ujAEKCYTx7heyXxS5fJCQYxrOQCt2+CyHBMIQECCAkQEBUQzrtMNmEhDgS1ZAaNEg7JJmQEEeiGlJhes1Dddy1QzyJakjl3U4vr94mJMST6D7Y8FGjydWbhIR4EuVH7b75qnrrjdkuywgJhonN9/4mJBiGkAABsfkm+oQEw8TmO60SEgxDSIAAQgIEEBIgIHbeRP9whATD8PA3IICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJDyTRPgAAAdRSURBVEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJMSy/f98aMkGr4cIByEhhi3JSunSUg3e7vUcdSMkxK6lKbfttaz//PTH33o9SZ0ICTGrot204PnXbe/yeJK6ERJi1uoGu+yNm3p7O0gYCAkxa2kLZ+OxDp7OEQ5CQsxanlpub9x9qreDhIGQELO+SVtqb/Qe7+0gYSAkxK5JrT/wn1ZNbbTR60nqREiIXeXDU4dNG/+jZi95PUjdCAmx7OVrBlx82zavpwgDIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECYjOktQowzNqIf83rPyRr3btGuL71QsP07ev1BJFqfb3X/5XDsy7y3/IohGSIh7p4PUGkRo3yeoJIdXnI6wnqDSFVI6T6R0gJgJDqHyElAEKqf4SUAAip/hFSAiCk+kdICYCQ6h8hJQBCqn+ElAAIqf4RUgIgpPpHSAng0VO8niBSY8Z4PUGkTnnU6wnqDSFVO7jV6wkiVVrq9QSR2hq/LwQgJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggpKAFzlEIbvV6kLCU39DgNHurbGKur/Xobd6OE4ZDE5v1g44AIQXdo/ILA1Z6PUg4Puqe7vxaHuyuLrq9wNcx1l8qWzOxUT/oSBBS0IxjOCKOV75pdHpxmv1rOU/d6T/9i5rk7UR1OWxik37QESGkoImq2OsRwvbVpHLL+bXMSz8QOOuUVeXpRHU5bGKTftARIaSgkWpXxdZdXk8RPvvXcn/ygOBHo9QmT6cJhxOSaT/osBFS0AXqpgylTnzS6znCZf9ablD2O9vNUMs9nSYcTkim/aDDRkhB/dQJs5+Y0kyZ8gaG9q9lkZoQ/GiuWubpNOFwQjLtBx02Qgpa8cxe/+mHaS0MeeO16pCuCX40Rz3r6TThcEIy7QcdNkI63IVqjdcjhMf+tSxWI4MfTVWveTlMWJyQHMb8oMNGSIcbqwx5fsP+tTyY0i/4Ub76zNNpwvH9kIz5QYeNkAL2PPBU8Ly3AQ9/BTm/lj0a7/OfVrZp7+004bAnNu4HHTZCCqhs2/Rj/9lzqpvXk4TJCelhdbP/9EF1i7fThMOe2LgfdNgIKej5pCajp12Y1KzI60HC8EZhYWFytv/kS6uijxp2y6VJp+zzeiZ3h01s0g86IoRke+vc41La/NqIZ91nO/t9BvYR2DM519d2wldej1SHwyc26AcdEUICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBCSyZJ7eD0BHIRkgo/VoFo/T0gxg5BMQEgxj5BMQEgxj5BMYIeUr/Zcn5vabl6Vf/tv3Ru2HF0WDKlkfI7v+GFrLGt5Un5g8bkNVnk6bGIiJBPYIY1Ug65++82z1aOWtSq5zaw/Xd7H5w9pZ27zwoWz2qW9YVlXq+WW9Yz6rdfjJiJCMoEd0mgVuMXZpIZY1jnKfwtkjVf+kMalrPVvbkk/3bL2dOh8YG/7E7/1dtjEREgmqA7plcAHjfOsykY/CGy95w+p6vju2wMGqT2WtTJpxuQGb3k6aqIiJBNUh/RR4IPmP7Y+VwMDW/v9IZWoah/6PzM+zfd7LwdNXIRkguqQigMf+EPaoIYGP5/UwypWeS/byvyfKFLqAw/nTGCEZIIjQ9pq3yLtCd4i5dWsqzyjVWafKk9GTHSEZIIjQ/outVNg683Agw3HNwzcFFk7Aydz1eIF6l7PxkxkhGSCI0Oy+gUftftl8FE7daN/c2f2EMv6X6PzLOvnjTd4OmuCIiQTHBXS35Oybpg7pH9zf0g7ctQVj83K8f3Df8euyaf+mtJ6VXo8biIiJBMcFZK1+JTUlgVl7bv5N7ePa59y3PmrLesuNS/w9Znqbg9HTVSEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIOD/A/n9KLVHOyOKAAAAAElFTkSuQmCC", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 }, "tags": [] }, "output_type": "display_data" } ], "source": [ "lambda_search_space[3]\n", "plot(coef(ridge.mod)[2:20,3])" ] }, { "cell_type": "markdown", "metadata": { "id": "OidY41dTakN4" }, "source": [ "Notice a difference between when $\\lambda = 11498$ and when $\\lambda = 5722367659$? \n", "\n", "The coefficients are much smaller. The larger the value of $\\lambda$, the more sparse the model. In other words, as $\\lambda$ increases, the flexibility of the fit *decreases*. This decreases the variance of the model and increases bias. " ] }, { "cell_type": "markdown", "metadata": { "id": "DMe0yoEMakN4" }, "source": [ "You can test any value of $\\lambda$ using the `predict()` function." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 437 }, "id": "Z3kpNZgkakN4", "outputId": "d0099162-90c6-4080-a652-4728fa07465b" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAADAFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////isF19AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO3dCXhU5bnA8S/bBAIhIhD2BCuitloDtIKXpcgiyiK4FIzUSwi4EFBsURFUQIuEC3W5KhVpKS5VexVRaxV7EeReFMoS0MoFJSAVEAJIUHZCknNn5pwQxJzJMN+bmfky/9/zmHMgZ5K30/kzM2fOnFEWAG0q0gMAtQEhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECAhDSJ+sBYzyydnfyms+pDUKMMyas76Z13xIH6sTNf47AEEn1MdnfRlCAs5ASIAAQgIEEBIggJAAAYQECAh3SOVbFy9cuGR7NVsREgwT3pCKx6fbL15lPHI00HaEBMOENaRd56kLcqbMnPlgdgt1WXGADQkJhglrSCOTXnPWSmfHjQuwISHBMGENqVlu5frQ1gE2JCQYJqwhJT1auT7VE2BDQoJhwhpS5pDK9UFtAmxISDBMWEMaFzfruL12eLKaEGBDQoJhwhrSgQ4qtVfO2DHDe6SobocCbEhIMEx4X0c68XhWgu9lpKTOc0sDbUdIMEzYDxE6trmgoLCqTA49NOGUoS4hfTqhX78Jn+r8eqBGROJYuxOrl375w78t6t/7lIvVwaoumJ/Q/d57uyfka/5+QFxYQ/rtUt/XOQ29D+46rg+04RxV1TOo1z0LfYuFngWhDgDUkLCG5N9T9zeVfN3tXVTalgAbVh1S+3vs5T0dQh0AqCHhD+mCtI3er2/EjQiwYZUhHVIr7ZUVcYdDnQCoGWEPaa+a5F8f3DLAhlWGtEt9Ya98rnaFOgFQM8Ie0nb1kn/9waQAG1YZUkmdd+2Vv9UpCXUCoGaEPaTSNHunW+65ATas+jnSDVeV+xZlfW4IdQCghoQ3pOw1hfsmtj3iXd1Ub2CADasO6fO0m3db1u6b074IdQCghoQ3JNsCy3q5XvzqABtWHZJVcIlq00ZdUhDq70fUKo/0ALrCGtL8J6aMGz64xxLLmt3ynUAbuoRklRU8//zaslB/PaLUtlFtEzJ/GfCVxagXobMIHQpcg1tIqI1WpXV57sMXBnlej/QgOqLzdFyEFEOOnzfC/8/q9Pomv6pBSIiwt+t951+WtfuPCE+ig5AQYQ93c1ZG3RzROfQQEiJsck9nZfSQgNtFN0JChL3cyDlQ5ecPRXYQLYSECPu20W/9y9cTN0V4Eh2EhEhbkDhqxZ6CBzxGv2GTkBBxyzrFK3Xxf0V6DC2EhChwdMO3kR5BEyEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI0Alp2w8JTUVIMIxOSOqHhKYiJBhGK6Rrx33fYEJCjNIK6Z0zvrGIkBCjdEK68MMzvrH8Qt1xHIQEw7DXDhBASIAAQgIEEBIggJAAAYQECCAkQAAhAQKkQirs0n3QrAP689gICYaRCmm9UvOun6o/j42QYBipkE4UFlrWUf15bIQEw/AcCRCgG1L51sULFy7ZLjeQHyHBMHohFY9Pt9/Ql/GI2MM6H0KCYbRC2nWeuiBnysyZD2a3UJcVC05FSDCMVkgjk15z1kpnx42TGskiJBhHK6RmuZV/ObS1yDw2QoJhtEJKerTyL6d6ROaxERIMoxVS5pDKvxzURmQeGyHBMFohjYubddxeOzxZTRCbiZBgHK2QDnRQqb1yxo4Z3iNFdZO86RMSDKP3OtKJx7MSfC8jJXWeWyo5FSHBMNqHCB3bXFBQeEJsHhshwTBix9oVb9Md5TSEBMPohfRpv8yus+0HdRMkD2UlJBhGK6SPklVKkvqF/+AgQkIs0wqpf9Kb5ccfT/r5YYuQENu0Qmr9K9/XJZ5+pYSE2KZ3iNBk/+JFdRchIbZphdTqWns5Uc0kJMQ0rZDuinu6xLcsH67uvpOQEMO0QvomQ/X2r5TfJfexlz6EBMPovY60L+9uZ+2N8wkJMYyzCAECCAkQQEiAAN2QevU67F8e7tVLaCIfQoJhdENSyj5z/gH22iGW6Yb02Wdl/mXZZ58JTeRDSDAMz5EAAWIhfVOoPUslQoJhxELiWDvEMkICBBASIEArpI6naUZIiGFaIcXHJ5+SQEiIYVohTUit3FXHQzvEMq2QStr/rKRinZAQy/R2Nmyse0/FKiEhlmnutftuf8XasnyReWyEBMNwiBAgQCekZ7864xvbn9Wex0ZIMIxOSOqdM76xSOq+ipBgGK2Q5m37vucJCTFKK6QfEpqKkGAYnZDG/JDQVIQEw7DXDhBASIAA/ZC2LX51zquLtwnNYyMkGEYzpK13tnV2M7S960u5qQgJhtEKaW9uokof+uBTLz714NAmKjF3r9RUhATD6IT0YXrCsIJy5w/lBcMS0pcJTUVIMIxOSEndt3zv7wu7JwlM5ENIMIxOSA+VnvGN0oe057EREgyj9RxpxWfbvjkpOo6DkGAYrZB8e+ueFh3HQUgwjFZIE6c9+ccvRMdxEBIMw5ENgACtkEbcOfHR9aLjOAgJhtF9jpT4nOg4DkKCYbRC2rr3qOgwpxASDKP/HMn3YtLxf6wrr3rb0BASDKMbUmnejZa17UdKdZW86RMSDKMbUr76jWX1ixudF8957RDDdEO65HrL2hk30rJys+SGIiSYRjek+nMsa576wLJmnyM3FCHBNLohpXpDyq53wrKeqSc3FCHBNNoP7W62iupf51259UKxmQgJxtENabq6ooVaZlkveO6VG4qQYBrdkI7l1E17yrtsfmmx2EyEBONIHbS6UvR9SYQEw+iGtLziA5JWLRCZx0ZIMIxuSOpNZ+V3DUXmsRESDKMVUuGiRWryIr+Fl6cITkVIMIxWSPmnfxDFjYJTERIMo/fQbtfb6pZ8v5kLSgJc4mwREgyj+xyp/0q5WSoREgzDORsAAbohlb82IOsnNrmhCAmm0Q1pllIpaTa5oQgJptENqVXfrXLDnEJIMIxuSEn/kJulEiHBMNr3SOy1A/RDujdPbpZKhATD6IZ0qO/N728s9JMbipBgGu2DVivJDUVIMI1uSNnDR1aQG4qQYBqObAAECIR0cMMBoWFOISQYRjukZR2VWmRZAz8QG8kiJBhHN6RVntS+3pD2NvOslRuKkGAa7bdRZOzY7btH2pMxSG6oGAjp23l33/7UV5GeAmJ0Q2qUb/lDsqZzzoaz8H6j5oNvaud5MtJzQIpuSIl/dkKanyQ2Uw2HVH6w5n52kDbUneB7Q/FLSa9GehII0T7W7gEnpBGZUiNZNRrSf/dIVU2zt9TUjw/OTdfYyynnR3YOiNEN6baGBb6QiicpyYPuai6kpxJue6fglSsbrKqhnx+cxq/Yy0L1ZUTngBjdkHa3TuygsrKSVUaR3FA1F9KmxJd8i/KcC7TO1XJgRWGpxsXL45faK4fVap0xED20X0faM7qRUqrx6D1iI1k1GNJ9V9jLYs/i0H/I6s7e/8VpUwKnGPCJWLMX7eVGxY67WkLgyIbyokLJeyOfGgupX8VnZrR/POSf8b91hq09vnN++vXunz+9/Ko01eSXG12/n/ML+7LjfxzyFDKOLXnm+XURnqF20Alpd7H3v0qCU9VYSNfc56x0eCzUH1F24W3+5ca6r7ttMj/h398q+MvVKcvcNtjSYJT3HuvkE4nvhDqFjDeben7SRv1b4LMFVLuTs+yo2EA6tu2M5G/XCUn1Pfu3UZRvXbxw4ZLt1WxVYyGN724vv6uzKNQfsSre+Sdj5GCXLb6q+4x/ObaV6y1sRWZql95NUl8MdQi/4+ve+uREoA2Kn73jV9M3uH9/UeKUw94bYJ+M/a6bfDK4kWrYP9AzuVc6p8Sdd+c31Q1bw/bfmqrUueMPu29R9vbd/W59LsAGenRCGprv/a9SEJcsHp9uR5fxSMB/xXRC+rYswDf/mfCGf3lHm4C3wEBebuGsPH2JyxbTfmI/cDuS+obrTzn+9qOTXnW//Qah/HfnqDTVeLb7Fn9v1HroqI7xD7hucNGv/YtjF9/vtsV7yYNeX/fG0ETXO1/rrjr3vr/yuUsyd7jPsXT0L6661/1xroR9F1z62peFL5x3+RG3LQ72rnPtPTc3a/NZDU0Q1rdR7DpPXZAzZebMB7NbqMsCfTBZyCH961dNVd0uf3XfID9x/IdfvDMg5X9D+/leCxo5K7M6umxx0+3OSvepIf+W6t2b+twBa/+TKdPcNvgi5V7fh1a9V+9plw0+r9j5/pjbU7Vvm0z0L6c3cNuX9G7Sct/iWNd+bmOU3Z547dT7uyb93m0DCaN+6r+r2dt6stsWN160zfv1yC9b19BjHf2QNuzzfQnqGevIpNectdLZceMCbBhqSJ807PrK+vfvSnS9cXmfFvw8SdUfEODxTnW+VM7huX1vddli6GhnpceU0H9NdTYk/N2/fN2zzWWLET3t5X82cdlVvzTB2V3ydgOXHzE/3b7jLm3zlMsWg3Ls5Zo4t4frs87xn2hqfoLrM0Z9x+s79/1PZ7pssVEV+JdHW7r9s6JJN6SSXPWhd/G0ygnihZVmuZXrQ1sH2DDEkMp+OsT/uO6t+ECHopfsdN/dFowBl3/nWzyfsN5lg4ez7OWxtP/S+kUBPfxzZ6XdEy5bZP7BXu5VLoMWKOdRwbwMlx8xfoCzkn2byxbn/9Feltd9r+oNStOdW+4trvdZ+rYoJ+N/KJenDM+2dVZul/zUlNPohvSY6u97fPD5UBXEAZhJj1auT/Wc8c0dP/nRKY1VSMfDrYj/2l65anTgDQN65/q25w142T22oh9nTF0w54ZE18cqWzzz/csJTWvwiMGc4c7KdXe5bNHgbXtZnrC06g1KGj5nr1zzK5cf8ZtrnZVho1y2OO9Pzkq9d6ve4HPlPHt6Q/JUvGf4V8WD1OVxLs99Z3R2Vh7oXTMj6IZ0acW/Wf3aVrXp92UOqVwf1OaMbx7/03OnDAvtHukPFTM80i2Ui/uVj/GMmPunvHo3un8o7uFpXRu3yw5wZsxnEsYu2fzeLz0h7xkMQt4NzkrviS5btHMejn2l3J7ozzjH939++aOef7ps8Mfm9rVQ3tbtZbf+zvPBf7od7LSm4v/KJQl6DwQCOdlovr0yze3Z3otNnX1Q2cNrZgTdkOpWvB4zM4ijv8fFzTpurx2erCYE2DDEh3ZzL3BWNEJ6vt4K32Jj4+kh/wjLWtTZo1Kuknyr4w+82Mjek7u/3lsuW/w6y65g0o/cbsFlY+J73XP7xfVddy7uP8d+DPFUva9dtlhQx/+48WTfK102KIqzn5xYz575L6ek+zP8LyJtauj2XG5Psv0w+6t6kp91fBrdkJre6azkNa3+ggc6qNReOWPHDO+RoroFSiXEkD5KcA6xuOb2wBsG0H6SvXy6eaDd6NUq2al18eodzfiV7xilY4MvcjtUaXfT6/d6B3k8caH7T1lx38Cb8t0i8VqQeMvfN38wKuF5tw3Kb2mQv/qL169I3+y2RZeb/Ytjl/za/ddoO9I9fdrf//ZAgxtcn6k/Uv9l7/8jay66sob+f9ENKTfF/9i4ZG7iLcH8tsezEnwvIyV1nhtw30SoOxt+PMz/j+978SGfkbwk/n/slc0qwCsj0WBd04snzp1wfmv3F2g2XOLJ6nJO2gs6v+XjnnWUp5vLkyyf8mfaxam0Ye5X1uq6t+323sP3zNynM0d1SmZ1qFu/03PulZRPq9ugY/O4od/W0AC6Ie1qrjL6DOh6rmoe5OGXxzYXFBRW92poqLu/1zbo/camDyd4Hgrp0j5HKo7H3qEkTx1bE/ZO6XNh32mBXo4rW/rktDd0bzmlO92fLdoOB7hL81p+oWp5rrpym+Yc1Sqt5jnYvr8+9qrr/aY27deRiu7wHf3d5FbRA51CfkG28MZzVFLH16rf0FVzZ3/ue3Wi4wiyWqBs/csLau4WLMr1wIhqSRz9/fUW6SOYdA4R2q33qdC/vsj/q0v+7SatHwPzvH9lmsocGeI9Qowd/V294nYdFx86urxH8+oOrEUtMzPxjrdW/unyxqEd9RLuo7+DE8mzCO3NTohLiBvwr4gNgIhYn+A/Mrf0uvYhvd4V3qO/gxXZ03EdXrOypvbtIGqNcQ552BEf0vk8OIk+4NP9YWel4vDBs6MT0m+WWNbtn4TyW6tDSAi3rhVvGWg3N5SL64QU731op94M5bdWh5AQbrcOtJd7E5eHcnGdkJqfkzdBXT+hQii/3gUhIdw+jrc/UCXnopDOtKYT0kt1lKp9e+0Qo+6r89DKre9eUz+0w8u0djYcWLNcTV9eIaTfXzVCQvi9dEm8ShkY4skldPfa9V0R2u8NjJCsozO6nnv+DTX49mz80NHtIR8bzl676LQ/q+WUhX8cljAj0oMgOOy1i043/dR/rq6F8ZKPmFFz2GsXlYrinQd1N3LsrBnYaxeV3k92HqzPaRfZQRAk9tpFpb+mOivPZ0ZyDASNvXZRaZNyPlPw7ho6exSECRy0enDDAaFhTon5kKyO2f6D+bc2mB/hQRAc7ZCWdVS+z5Ad+IHYSBYhWdba+tetPLLzpRZX63wyIMJHN6RVntS+3pD2NvNInsWNkKzPesQplTrpeKTnQHB0Q+qfscP/qeZ7MgbJDUVIPgdXaX1QLcJKN6RG+ZY/JGt6Q7GZCAnR6csZw4bNqPrczLohJf7ZCWl+EKcsDhohIQo967l01KhLPc9W9T3dkFo94IQ0IjO04apESIg+7yX6P3pjXmJVn2CjG9JtDQt8IRVPUnkhjlcVQkL06TTWXo7tXMU3dUPa3Tqxg8rKSlYZRSENVzVCQtQ5EveRvbI8rooTsmq/jrRntO+UxY1Hu33IaEgICVHna/WFvfKFquJk5xKnLC4qlLw38iEkRJ0THvtTe633PVV8CIRASPtWLP6H8EFChIToM2CwvRw0oIpvaoe0vJPvLRRxvT4LZTQ3hITo82lK3neW9V1eyqdVfFP7EKHkhK4jx47oFNfg85AH/CFCQhT6n8zk9u2TM/+nqu/phjSw1Sb/cl16dkizVY2QEI1OfPDEE4ur/pQ87UOEKj6zeGoQnyEbNEKCYbQPEXrRWXmeQ4QQw3RDajHRWbmvpcg8NkKCYXRDyqn/lu+tnOUL640Sm4mQYBzdkLalq2Y9B/Zsppq7f0D82SMkGEb7daTtw9OUUueO2iU2kkVIMI7EIUK7CiU/iNmHkGAY/ZA27PN9WSc0j42QYBjdkEpy1YfexdMqR/L8AoQEw+iG9Jjq73sP++dD1ZNiMxESjKMb0qUVR8L2aysyj42QYBjdkOo+5qzM5MgGxDDdkJre6azkcawdYphuSLkp7/oWJXMTb5EaySIkGEc3pF3NVUafAV3PVc2/khuKkGAa7deRiu7wnfykya07xUayCAnGkTiy4esth4WmqUBIMIxWSCs+2/bNSdFxHIQEw2iF5DvtydOi4zgICYbRCmnitCf/+IXoOA5CgmEEniPVAEKCYbRCGnHnxEfXi47jICQYRvc5UuJzouM4CAmG0Qpp696josOcQkgwDM+RAAGEBAggJEAAIQECCAkQQEiAAEICBGh/qvmgFc4XSYQEw+iGVKhed75IIiQYhpAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIgQPsQoZPlFV8EERIMw7F2gABCAgQQEiCAkAABhAQIICRAgEBIBzccEBrmFEKCYbRDWtZRqUWWNfADsZEsQoJxdENa5Unt6w1pbzPPWrmhCAmm0Q2pf8aO3b57pD0Zg+SGIiSYRjekRvmWPyRrekOxmQgJxtENKfHPTkjzk8RmIiQYRzekVg84IY3IlBrJIiQYRzek2xoW+EIqnqTy5IYiJJhG+wSRrRM7qKysZJVRJDcUIcE02q8j7RndSCnVePQesZEsQoJxBI5sKC8qlLw38iEkGEYgpF3rln66V2gcByHBMNohzW2jfC56VWwki5BgHN2Qfq+Sew/PG3Z5nHpBbihCgml0Q2rX91v/8su2PxaayIeQYBjdkDwfOSuzk0XmsRESDKMbUpOVzsqcliLzOD+MkGAW3ZByJzkrA+4WmcdGSDCMbki7Lr/5r5u+2rigX+/CHV5CUxESDKMbkvo+oakICYbRDWnw0O8RmoqQYBjOIgQI0A2p85xv5YY5hZBgGO13yKq62f9dJjePjZBgGN2QvnmuV4Jq/UCh3EQ+hATDCDxH2vvslfGq6x8PCk3kQ0gwjMzOhl1PXKZS7vhCYiA/QoJhREI6+voNdVVGUtJUqQ/uIyQYRiCkj0Y1UHWHfWhtv0FNkRmKkGAa3ZC2T7tAqfbP+E+jX947XWgqQoJhdEOKV2l3nDrr9zNxEiNZhATj6IbU7fmjlX8oXKg/kB8hwTC6IS3f76ysWiAyj42QYBjto7/fdFZ+x0n0EcO0QipctEhNXuS38PIUwakICYbRCin/9Lci3Sg4FSHBMHoP7Xa9rW7J95u5oERwKkKCYbQ/sW+l62YaCAmG4Y19gABCAgQQEiCAkAABhAQIICRAACEBAggJECAVUmGX7oNmHdCfx0ZIMIxUSOuVmnf9VP15bIQEw0iFdKKw0LKOVvGNkBASDMNzJECAbkjlWxcvXLhku9xAfoQEw+iFVDw+3X4zUsYjYg/rfAgJhtEKadd56oKcKTNnPpjdQl1WLDgVIcEwWiGNTHrNWSudHTdOaiSLkGAcrZCa5Vb+5dDWIvPYCAmG0Qop6dHKv5zqEZnHRkgwjFZImUMq/3JQG5F5bIQEw2iFNC5u1nF77fBkNUFsJkKCcbRCOtBBpfbKGTtmeI8U1U3ypk9IMIze60gnHs9K8L2MlNR5bqnkVIQEw2gfInRsc0FB4QmxeWyEBMNIHGt3YvXSL2WmqUBIMIxWSL9d6vs6p6H3wV3H9YJDERJMoxWSf0/d31Tydbd3UWlbBKciJBhGP6QL0jZ6v74RN0JwKkKCYbRD2qsm+dcHt5QbipBgGu2QtquX/OsPJskNRUgwjXZIpWn5/vXcc+WGIiSYRi+k7DWF+ya2PeJd3VRvoOBUhATD6IVkW2BZL9eLXy04FSHBMFohzX9iyrjhg3sssazZLd+RnIqQYBihswgdKgvyssGdLIWQYJjwno4r2JOlEBIME9aQgj5ZCiHBMGENKeiTpRASDBPWkII+WQohwTBhDSnok6UQEgwT1pCCPlkKIcEwYQ0p6JOlEBIMoxvS7kErnC9BCPpkKYQEw+iGVKhed74E9duCPFkKIcEw4Q3JCvJkKYQEw4Q9JCuYk6UQEgwT1pCCPlkKIcEwYQ0p8MlStm89ZRohwSzhD8ntZClb4tRpCAlGCXtI7idL2cU9EowV9pCCOlkKz5FgmLCHFNTJUggJhtE+ROhkecWXIC4Y7MlSCAmGCeuxdkGfLIWQYJiwhhT0yVIICYYJ7zkbTqnmZCmEBMNEKKRqEBIME4mQZnWpbgtCgmEiEdLt1f4AQoJh9EPyvbHo+D/WBbcD3I+QUOvohlSad6NlbfuRUl2Dv+kTEmod3ZDy1W8sq1/c6Lz4/KAvT0iodXRDuuR6y9oZN9KycrOCvvyBHdVtQUgwjG5I9edY1jz1gWXNPkduKEKCaXRDSvWGlF3vhGU9U09uKEKCabQf2t1sFdW/zrty64ViMxESjKMb0nR1RQu1zLJe8NwrNxQhwTS6IR3LqZv2lHfZ/NJAH9NytggJhpE6smHlSf1ZKhESDMNBq4AAQgIEEBIggJAAAYQECCAkQAAhAQIICRAgFVJhl+6DZh3Qn8dGSDCMVEjrlZp3/VT9eWyEBMNIhXSisNCyjurPYyMkGIbnSIAAQgIE6ITU6XvaC05FSDCMTkgJPklKKd+HVqa1FpyKkGAY3Yd2xd3GfHLMOvjRTT2/lRuKkGAa3ZByhzgr/UeKzGMjJBhGN6Qm85yVWU1E5rEREgyjG1LyDGfl/mSReWyEBMPohtS+pf0RlqvSLxOayIeQYBjdkN5JUG37DOzTVsW9JjcUIcE02i/ILr+6jlLK0+N9sZEsQoJxBI5sKNu5eYfoybgICcYRCGnfisX/EHsDhY2QYBj9h3adlO/Yhl6fiY1kERKMoxvSquSEriPHjugU1+BzuaEICabRDWlgq03+5br0bKGJfAgJhtENqdF0Z2VqU5F5bIQEw+iGlPiis/J8ksg8NkKCYXRDajHRWbmvpcg8NkKCYXRDyqn/Vrl3Ub6w3iixmQgJxtENaVu6atZzYM9mqnm1H1V+FggJhtF+HWn78DSl1LmjdomNZBESjCNwZEP5rsLdQtNUICQYhrMIAQK0Qyr9eMGrNrGZCAnG0Q1pbRtVQW4oQoJpdEPqfM643//BJjcUIcE0uiHVe1NulkqEBMPohtS0QG6WSoQEw+iGdOdE1800EBIMoxvSkYG//PPS5X5yQxESTKP9xr7W7LUDtEO6vM6QB6bYxGYiJBhHN6Q6L8nNUomQYBjtd8iul5ulEiHBMLohjfqt3CyVCAmG0Q3pQO+8xRsL/eSGIiSYRjckpdhrB2iHlD18ZAW5oQgJpuH9SIAAQgIEEBIgQCGIuLsAAA7hSURBVCqkLb166Q9zCiHBMFIhrWevHWKZVEjHPpP8XBdCgmF4jgQIEAjp4Abhz+sjJBhHO6RlHZVaZFkDPxAbySIkGEf7jX2e1L7ekPY286yVG4qQYBrdkPpn7Njtu0fakzFIbihCgmm034+Ub/lDsqY3FJuJkGAc7U/s+7MT0nw+sQ8xTDekVg84IY3IlBrJIiQYRzek2xoW+EIqnqTy5IYiJJhGN6TdrRM7qKysZJVRJDcUIcE02q8j7RndSCnVePQesZEsQoJxJD6xr6hQ8t7Ih5BgGI61AwTohtS+U4V/u3am2DF3hATDaO/+9n2meYL3v2SPUplfC01FSDCM/qdR9Hz/oHVkyVXDT373eILUmYQICYbRDWnMlWX+ZVnPyZZ1WyuhqQgJhtENKX22szKnjWXNlTpMiJBgGO1Po3jYWfmPZMua0lxkJkKCcXRD6tDM/hDZTW0ustakDxCaipBgGN2Q/pqgLhow5Nqfxql5Vvfks/9ZVSMkGEb/reZ96vh2gHd6w7L+tFpqKkKCYSSObCje8tUJmWkqEBIMwyFCgACdkDp9T3vBqQgJhtEJKcEnyfsMKc77X1prwakICYbRfWhX3G3MJ8esgx/d1PNbuaEICabRDSl3iLPSn0/sQwzTDanJPGdlVhOReWyEBMPohpQ8w1m5P1lkHhshwTDab+xrab8Kuyr9MqGJfAgJhtEN6Z0E1bbPwD5tVdxrckMREkyj/YLs8qt9hwh5erwvNpJFSDCOwJENZTs37zgpNI6DkGAYnZB2F3v/qyQ4FSHBMDohqb7e/yoJTkVIMIxOSEPzvf9VEpyKkGAYjv4GBOiG9PYGuVkqERIMo33ykxmum2kgJBhGN6Te15TJDXMKIcEwuiEVZV/9ytpCP7mhCAmm0Q2J3d+ApR/S0FtyRzrkhiIkmIbd34AAnZDW/uA7P/yb0BASDKMTkuepM77xlEd7HhshwTA6IY1VV7x32l+/d4UaIzITIcE4Ws+RXmmkLh731oaiY0Ub3hp3sWr0itRUhATD6O1sOJTf9NTe72Yz5G78hATD6O61K1udn3ttj2tz89dIHuFASDAMu78BAbohLd/vrKxaIDKPjZBgGO1DhN50Vn7XUGQeGyHBMFohFS5apCYv8lt4eYrgVIQEw2iFlH/aIavqRsGpCAmG0Xtot+ttdUu+38wFJYJTERIMo/scqf9KuVkqERIMo7/7e8M+35d1QvPYCAmG0Q2pJFd96F08rXJKpUayCAnG0Q3pMdX/S+/i86HqSbGZCAnG0Q3p0gHOSr+2IvPYCAmG0Q2p7mPOyswkkXlshATD6IbU9E5nJa+pyDw2QoJhtD+MOeVd36JkbuItUiNZhATj6Ia0q7nK6DOg67mq+VdyQxESTKP9OlLRHY2UUk1u3Sk2kkVIMI7A+5HKv95yWGiaCoQEw/CJfYAAPrEPEMAn9gECOGcDIICQAAE6IXX6nvaCUxESDKMTUoJPklIqzvtfWmvBqQgJhtF9aFfcbcwnx6yDH93U89ugLlu+dfHChUu2V7MVIcEw2sfaDXFW+gfzQWPF49PtXeUZjxwNtB0hwTC6ITWZ56zMalL9BXedpy7ImTJz5oPZLdRlxQE2JCQYRjek5BnOyv3J1V9wZNJrzlrp7LhxATYkJBhGN6T2LVf7l6vSL6v+gs1yK9eHBto5QUgwjG5I7ySotn0G9mmr4l5z3f6UpEcr16cG+nQ/QoJhtF+QXX51HaWUp8f7QVwwc0jl+qA2ATYkJBhG4MiGsp2bd5wM6oLj4mYdt9cOT1YTAmxISDCMQEgHNxwI8oIHOqjUXjljxwzvkaK6BUqFkGAY7ZCWdVRqkWUN/CCo3/Z4VoLvZaSkznMDnk+SkGAY3ZBWeVL7ekPa28yzNrgLH9tcUFB4opqNCAmG0T6JfsaO3b57pD0Zg4K6LIcIoVbSDalRvuUPyZoezCf2cYgQaindkBL/7IQ0P4gzrXKIEGor3ZBaPeCENCKz+gtyiBBqK92QbmtY4AupeJLKq/6CHCKE2ko3pN2tEzuorKxklVFU/QU5RAi1lfbrSHtG+8602nj0niAuyCFCqK0kzrRaVBjEvZEPhwihttIN6e0NZ3HBgIcIHX1ixinXERLMohtSnRmum1X12wIcIvT1FR1PaauOnfVUQATphtT7mrKzu3BQhwh9rKrbAogquiEVZV/9ytpCv7P7Id8E2p6QYBjdkEI+if6EQNsTEgyjG9LQW3JHOs7uhxASapOInfubkFCbCIS0a93ST/cGdcGOp2lGSKhFtEOa28b/BOmiV4O4YHx88ikJhIRaRDek36vk3sPzhl0ep16o/oITUit31fHQDrWJbkjt+tonz/+y7Y+rv2BJ+5+VVKwTEmoT3ZA8Hzkrs4M4ZbG1se49FauEhNpE+yT6K52VOS2Dueh3+yvWluUH2IyQYBjtj3WZ5KwMuFtkHhshwTC6Ie26/Oa/bvpq44J+vQt3eAlNRUgwjOAhQkEfJjSrS3VbEBIMoxvS4KHfE9Tlb6+2N0KCYSJxiBAhodYhJEAAIQECIhHSgWp37hESDBOxt1EEREgwDCEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEkyw9fU5Hx6J9BCBEBKi377BqtFFSY1fivQcARASot6JDlnrLOvozMSXIz2JO0JC1Jvd5Bv/Mj89em8WhISo1+fX9vK7pGWRHSQAQkLUu/j3zkqr6H2WREiIep2m2cuy+m9HdpAACAlR756O5f7losS9EZ7EHSEh6m2vf0+pd/F5xuhIT+KOkBD9Fje8aPTk65MHHYv0IO4ICQbYkz+kZ9475ZEeIwBCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgREZ0hrFGCYNWd9M6/5kKxP1ka3Jne8ZIQhbSM9QXD+oB6J9AjB6X61203ik7O/lYchpGjX+sVITxCc/M6RniA4h0L49zwicnIEfxghEZIwQopRhCSLkGIUIckipBhFSLIIKUYRkixCilGEJIuQYhQhySKkGEVIsggpRhGSLEKKUef/JdITBOex7pGeIDjH4j+N9AjBue02wR9GSNZXJyM9QXCO7or0BEHaGukBglRcLPjDCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECYjyk+c6nD/w20oMEUnJ/fEd77cC4zKTmI6P2DX6nBo3uq7V4fIanzaCVvlW5KzTGQ3pCZU/wWRrpQQLY2CHVuX2e6KBueDQ36TzJd3YKqhw0qq/W/W1U/4eGJdb5p+gVGuMhTYn+E3V8V/dnhcn27fNx9R/er/+lxkd2IhenDRrVV+sY9bT36xuqn+gVGuMhjVOFkR6hOvvHl1jO7TMr9bhv0Ta9PKITuTht0Ki+Wu/uVeL9Wl43U/QKjfGQhqt9pTv2RXqKatm3z2MJvfx/ylFRe3oRJyQDrtbjSV1Er9AYD2mweqChUu1ejvQc1bBvn5uVfSK2KWpxRKcJwAnJgKv1P70P8CSv0BgPqYf6Uf6LExuoOZEeJDD79lmgxvj/NEstjOg0ATghRf/VuszT9aToFRrjIS1ZcNj79f+Sz43uT16vCGms/08z1ZsRnSYAJ6Sov1pfSe6wX/YKjfGQHNep1ZEeISD79lmohvv/9KD6IJLDBOKE5IjWq7V8srr6oCV7hRKSz+0qKl/xOMW+fZ5I7OH/U7b6KqLTBPD9kKL0ai3PVXeW+lYkr9DYDunQ71/xL7tG734wP+f22SnliPdrWYvWkZ0mAHvQKL9ax6npzprgFRrbIZW1rL/Ju3hLtY/0JIE5Ic1VU71fn1UPR3aaAOxBo/tqfUONq1gVvEJjOyTr7bh6Ix+6Lq5BQaQHcbdswoQJCc28X76xSrupQQ/fFHfpkUjPVKXTBo3qq/V8daf/+KUJxZJXaIyHZK245pzEFv8exa/DW/nOAaC+gwUO3ZOZ1HLM/kiPVLXTB43mq7ViTLVN8gqN9ZAAEYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYRksoROkZ4ADkIywSbVt8q/J6SoQUgmIKSoR0gmIKSoR0gmsEPKVofuy/S0erzcu/5uhzpNRh7wh1SUl5HUeNBqy1ocl+3b+Jr45REdNjYRkgnskIarvnes/Pgq9SfLWp7QYvofftUtyRvS3sy0CS9Nb5W8zLLuUIsta4H6daTHjUWEZAI7pJHKd4+zVQ2wrKuV9x7IylPekEYnrvGubk/9mWUdanPB8cOt2x2N7LCxiZBMUBHS+74/pGRZZXXP962t94ZU3rjDbp++6pBlLY2bck/8ioiOGqsIyQQVIW30/SHtJ9ZO1ce3dswbUpGq8H/ev8lLTro3koPGLkIyQUVIhb4/eEParAb6/z6uk1WoshbZDnj/okCpzyI4ZwwjJBOcGdIO+x7pkP8eKatyu7IrmjbqVh6REWMdIZngzJBOetr61j727WxoXMd3V2Tt9X2Zpf4yXz0ZsTFjGSGZ4MyQrB7+vXY3+/faqUne1b3NBljWF3X7WdaVKZsjOmuMIiQT/CCk9+LS7581oGeaN6Q9GWrE89Mzkv7b+8Cu3r+8NSV3KYvwuLGIkEzwg5Csv1zqaZJ7oHV77+ru0a0Tz7l2lWX9Tj3u+/4j6rEIjhqrCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEDA/wONeqS/GfzsewAAAABJRU5ErkJggg==", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 }, "tags": [] }, "output_type": "display_data" } ], "source": [ "plot(predict(ridge.mod, s=50, type=\"coefficients\")[1:20,]) #s=lambda=50" ] }, { "cell_type": "markdown", "metadata": { "id": "dzyK9fXiakN4" }, "source": [ "Let's create a simple 50/50 split for the training and test set for the ridge regression. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "y1rx5budakN4" }, "outputs": [], "source": [ "set.seed(1) # Use the same seed so we get the same results\n", "\n", "# Create your validation sets\n", "train=sample(1:nrow(x), nrow(x)/2) #50/50 split into training and test sets\n", "test=(-train) #get test indices (not training indices)\n", "y.test = y[test]\n", "\n", "# Make a training model using the training set (for all values of lambda)\n", "ridge.mod = glmnet(x[train,], y[train], alpha=0, lambda=lambda_search_space, thresh=1e-12) #threshold specifies the convergence criterion" ] }, { "cell_type": "markdown", "metadata": { "id": "GLG0n8i8akN5" }, "source": [ "Let's now test different fits using different $\\lambda$." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 68 }, "id": "vTUDMj2eakN5", "outputId": "ffed29d7-3c7c-4bbe-e7c2-b4999a727d56" }, "outputs": [ { "data": { "text/html": [ "224669.833069663" ], "text/latex": [ "224669.833069663" ], "text/markdown": [ "224669.833069663" ], "text/plain": [ "[1] 224669.8" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "text/html": [ "142199.150722761" ], "text/latex": [ "142199.150722761" ], "text/markdown": [ "142199.150722761" ], "text/plain": [ "[1] 142199.2" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "text/html": [ "167789.778381119" ], "text/latex": [ "167789.778381119" ], "text/markdown": [ "167789.778381119" ], "text/plain": [ "[1] 167789.8" ] }, "metadata": { "tags": [] }, "output_type": "display_data" } ], "source": [ "# With lambda = 1e10 (10^10)\n", "#note that this is like a null, intercept-only model\n", "ridge.pred = predict(ridge.mod, s=1e10, newx=x[test,])\n", "mean((ridge.pred - y.test)^2)\n", "\n", "# With lambda = 4 (s=lambda)\n", "ridge.pred = predict(ridge.mod, s=4, newx=x[test,])\n", "mean((ridge.pred - y.test)^2)\n", "\n", "# With lambda = 0 (note that this is OLS regression)\n", "ridge.pred = predict(ridge.mod, s=0, newx=x[test,])\n", "mean((ridge.pred - y.test)^2)" ] }, { "cell_type": "markdown", "metadata": { "id": "XRUqC8aPakN5" }, "source": [ "When $\\lambda$ was large, the MSE was worse than OLS. When it's smaller (but not 0), it improves upon the OLS MSE. Hence we need to maximize the bias-variance tradeoff." ] }, { "cell_type": "markdown", "metadata": { "id": "aNnFpRsWakN6" }, "source": [ "---\n", "# An important note about validation and test sets" ] }, { "cell_type": "markdown", "metadata": { "id": "I_Zvj-c_akN6" }, "source": [ "Remember, if we want to test the predictive utility of a model, it's important to separate estimates from training and test data. We did this above at a basic level, for the glm regression variables. But this separation should also include the model and variable selection step (i.e., in this context, $\\lambda$). \n", "So, the determination of which model is best must be made using training and test observations separate from the estimation of the general linear model. If we use the same data to perform best subset selection as we do to estimate and/or test our final model, then the estimate of test error will be contaminated. \n", "\n", "**Note: You will need sufficient power to split your data into the required number of training and validation sets.** \n", "\n", "Here, we need to use a separate training and validation set to find and test the best $\\lambda$. Let's find the best $\\lambda$ using 10-fold cross-validation." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 68 }, "id": "RXKTnPyNakN_", "outputId": "cff585f1-f5f9-4373-c9ab-8cb2f55cae6b" }, "outputs": [ { "data": { "text/html": [ "0" ], "text/latex": [ "0" ], "text/markdown": [ "0" ], "text/plain": [ "[1] 0" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "text/html": [ "0" ], "text/latex": [ "0" ], "text/markdown": [ "0" ], "text/plain": [ "[1] 0" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "text/html": [ "0" ], "text/latex": [ "0" ], "text/markdown": [ "0" ], "text/plain": [ "[1] 0" ] }, "metadata": { "tags": [] }, "output_type": "display_data" } ], "source": [ "set.seed(2) # Use the same seed so we get the same results\n", "\n", "# Create your validation sets\n", "train=sample(1:nrow(x), nrow(x)/2) #50/50 split into training and test sets\n", "test=(-train) #get test indices (not training indices)\n", "\n", "#split each of the training and test phases into two \n", "#need to have separate validation sets for lambda and for beta estimates\n", "\n", "train_hyperparameter = sample(train, length(train)/2)\n", "train_glm = (-train_hyperparameter)\n", "\n", "test_hyperparameter = sample(test, length(test)/2)\n", "test_glm = (-test_hyperparameter)\n", "\n", "#make sure that the samples do not overlap\n", "sum(test_hyperparameter == train_hyperparameter) \n", "sum(test_glm == train_glm) \n", "sum(train_hyperparameter == train_glm) " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 454 }, "id": "39yls-dZakOA", "outputId": "f95affa7-7bab-4509-cf6b-2d9dc5f7f99d" }, "outputs": [ { "data": { "text/html": [ "135.145590162423" ], "text/latex": [ "135.145590162423" ], "text/markdown": [ "135.145590162423" ], "text/plain": [ "[1] 135.1456" ] }, "metadata": { "tags": [] }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAIAAAByhViMAAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nOzdd2AU5do28HuTLekbSgothJIoJaB0KaIiUoKAHAUb4MGGAiogItI98mE7vhyacBTEjihEuoIIhA4JBAgJLQkhIZX0sn3m+2NzlskkYQNkZ6dcvz/el9w7iTeemL0z81zPo2JZlgAAAABA+jzc3QAAAAAANAwMdgAAAAAygcEOAAAAQCYw2AEAAADIBAY7AAAAAJnAYAcAAAAgExjsAAAAAGQCgx0AAACATGCwAwAAAJAJDHYAAAAAMoHBDgAAAEAmMNgBAAAAyAQGOwAAAACZwGAHAAAAIBMY7AAAAABkAoMdAAAAgExgsAMAAACQCQx2AAAAADKBwQ4AAABAJjDYAQAAAMgEBjsAAAAAmcBgBwAAACATGOwAAAAAZAKDHQAAAIBMYLADAAAAkAkMdgAAAAAygcEOAAAAQCYw2AEAAADIBAY7AAAAAJnAYAcAAAAgExjsAAAAAGQCgx0AAACATGCwAwAAAJAJDHYAAAAAMoHBDgAAAEAmMNgBAAAAyAQGOwAAAACZwGAHAAAAIBMY7AAAAABkAoMdAAAAgExgsAMAAACQCQx2AAAAADKBwQ4AAABAJjDYAQAAAMgEBjsAAAAAmcBgBwAAACATGOwAAAAAZAKDHQAAAIBMYLADAAAAkAkMdgAAAAAygcEOAAAAQCYw2AEAAADIBAY7AAAAAJnAYAcAAAAgExjsAAAAAGQCg50LWSyWOXPmeHp69ujRo+ar6enpL7/8cosWLbRabevWrWfOnFlWVibjNsTTCdoQYRvi6QRtiLAN8XSCNkTYhqg6EQUWXCMpKalbt27+/v4eHh7du3fnvZqamtq0aVOVSvXMM898+OGHQ4cOJaI+ffqYzWZZtiGeTtCGCNsQTydoQ4RtiKcTtCHCNkTViUhgsHOJkpISb2/vHj16XLlyRafT1fxWe/bZZ4noq6++clTefvttIlq1apX82hBPJ2hDhG2IpxO0IcI2xNMJ2hBhG6LqRDww2LlEQUHBzJkz7b8Q1PqtFhAQ0Lx5c4ZhHJWioiJvb+8+ffrIrw3xdII2RNiGeDpBGyJsQzydoA0RtiGqTsQDg53L1fxWKy8vJ6KHH36Yd2WXLl20Wq3VapVxG+LpBG2IsA3xdII2RNiGeDpBGyJsQ1SduBfCE27g7e2tVqtv3rzJq/v4+JjN5uzsbEW1IZ5O0IYI2xBPJ2hDhG2IpxO0IcI2RNWJkDDYuYGHh8dDDz2UnJx8/vx5R/HSpUvx8fFEZP8NQzltiKcTtCHCNsTTCdoQYRvi6QRtiLANUXUiKHffMpS/Wp/6//3330QUHh4eExNz8eLFjRs3tm3btl27dkSUmpoq4zbE0wnaEGEb4ukEbYiwDfF0gjZE2IaoOnEvDHYuV+u3GsuyK1as8PHxsY/Xfn5+//d///fCCy8QUVFRkYzbEE8naEOEbYinE7QhwjbE0wnaEGEbourEvVQsy9Z5Nw8agpeXV+fOnePi4mq+VFZWdvr0aQ8PjwceeMDf37979+7Z2dlZWVkybkM8naANEbYhnk7QhgjbEE8naEOEbYiqEzdz92Qpf3X9DsHL46Snp6tUqgkTJsi7DfF0gjZE2IZ4OkEbImxDPJ2gDRG2IapO3AvhCfeYPXu2t7f3qVOn7B8yDDN9+nSWZd944w0FtiGeTtCGCNsQTydoQ4RtiKcTtCHCNkTViWDwKNYlDh48uHv3bvufP//886CgoIkTJ9o/nDVrVpMmTc6dO/fQQw9ptdqJEyc2btx4+/btcXFxs2bN+vTTT+XXhng6QRsibEM8naANEbYhnk7QhgjbEFUnIiL8TUIlWLp0aV3/wq9cuWK/5tixY0OGDGncuLGXl1e3bt3Wr18v1zbE0wnaEGEb4ukEbYiwDfF0gjZE2IaoOhEP3LEDAAAAkAmssQMAAACQCQx2AAAAADKBwQ4AAABAJjDYAQAAAMgEBjsAAAAAmcBgBwAAACATGOwAAAAAZAKDHQAAAIBMYLADAAAAkAkMdgAAAAAygcEOAAAAQCYw2AEAAADIBAY7AAAAAJnAYAcAAAAgExjsAAAAAGQCgx0AAACATGCwAwAAAJAJDHYAAAAAMoHBDgAAAEAmMNgBAAAAyAQGOwAAAACZwGAHAAAAIBMY7AAAAABkAoMdAAAAgExgsAMAAACQCQx2AAAAADKhdncD0nD27Fmr1eruLgAAAEAU1Gp1165d3d1FLTDYORcXF9ezZ093dwEAAAAicurUqR49eri7Cz4Mds6ZzWYiMplMWq3W3b0AAACAm5nNZp1OZx8PxAZr7EA+GIZJTEx0dxcAACBGubm5ubm57u7C5TDYgXwcOnRIhHfFAQBADObNmzdv3jx3d+FyeBQL8uHh4eHhgd9VAACgFgp5g8BgB/LRv3//CxcuuLsLAAAQo48//tjdLQgBgx3Ih0qlatOmjbu7AAAAMWrUqJG7WxCCIm5LgkIgPAEAAHVBeAJAYhCeAACAuiA8ASAxCE8AAEBdFPIGgcEO5APhCQAAqAvCEwASg/AEAADUBeEJAIlBeAIAAOqC8ASAxCA8AQAAdUF4AkBiEJ4AAIC6KOQNAoMdyAfCEwAAUBeEJwAkBuEJAACoC8ITABKD8AQAANQF4QkAiUF4AgAA6oLwBIDEIDwBAAB1UcgbBAY7kA+EJwAAoC4ITwBIDMITAABQF4WEJzDYgXwwDJOUlNS5c2d3NwIAAG5WWFhosVi4laysLCJq3rw5t6jRaBo3bixoZy6GwQ7k49ChQ0OGDDEaje5uBAAA3Oz48eNms5lbWblyJRFNnTqVW9RqtcOHDxe0MxfDYAfygfAEAADY1RzXvv32WyIaPXq0O9oRDgY7kA+EJwAAFKi0tPTatWu8YmFhYaNGjVQqlaPy4osvent7C9qZO2CwA/lAeAIAQE4YhikpKeEVk5KSrFYrt1JZWWkwGNTqWyMNy7Imk8lms3GLWq3Wx8fHpQ2LAQY7kA+EJwAAJMFqtebk5LAsyy0mJyebzWbuPTaz2cwLQNRFrVZzUxE2my09Pb1Hjx56vd5R/OOPP5SwCBuDHcgHwhMAAO7FMEx2djbDMNzipUuXysvLuRObzWazWCy8VdEsy6rVam6RYRiNRtOhQwfuNefPn3/00UeDgoIcxYSEhPLycu7JQ0ajMT09nftPJKI1a9YQ0dChQ+/1LyluGOxAPhCeAABwHaPRaLPZuJWrV68WFBRwK2azuaysjDdRMQzD+/lsn9geeOAB7mVnzpzp3bt3ixYtHJWkpKScnJz777+f+4nnz5/nff16urvPkhwMdiAfCE8AADSI4uLiwsJCbsVoNCYmJtbnc1UqVWRkJLeSkpISFRXVvn17R+Xq1atXr15t27Yt97KEhASXzl68jU7kCoMdyAfCEwAAd8pms50+fZoXRygoKDCZTNwxy74ejhc+MBqNYWFhnTp1clSysrKSk5O7dOnCvez69evcEIO7+Pv7u7sFIbj/XzRAQ0F4AgDg9lJTU1NSUrgVhmHKyso0Gg23aLFYfHx8goODHRWDwZCTkzNixAjuZXv37g0MDPT19XVUdDqdaxpvALx7kHKFwQ7kA+EJAAAHlmV37drFS5XabDaWZbnL3ViWZRimU6dO3OLly5dDQkIefPBBRyU3Nzc3N1eAtl0H4QkAiUF4AgAUKzU19dy5c7yixWLx9PTkVliW9ff379ixo6NSUVFx7ty58PBw7tPSjIwM3j08GUB4AkBiEJ4AAIXYt28f7+mExWJhGIa3SS/Lst27d/fy8nIUk5KS/P39W7Vq5agUFRUJ0LAYIDwhUizLpqWlpaamlpWVEZFer4+IiOB+j4JiITwBAPKTl5eXnJzMK5aVlXl5eXFvQVksFm9vb+4iY5PJdPr06aCgIG7iITU1VbFPNhCeEJ2ioqIlS5Z8//33eXl5vJfCwsJeeeWVd999VwnHwEFdEJ4AAKlLS0uz37ZwKC8vNxqNWq3WUWFZ1mw2t27dmvuWd+PGDY1Gw73NUVFRIUDDEoLwhLhkZ2f369cvLS0tIiJi+PDhrVu3tsdwSktLU1JSDh48uGDBgs2bN+/fv79Ro0bubhbcA+EJAJAQlmVLSkp4x2plZWXxfohVVlbabDbu41T7p4SFhTVp0sRRrKioqOfpW4qF8IS4zJ8/PzMzc9OmTc8880zNV20229q1a6dOnbp48eJly5YJ3x6IAcITACAhOTk5hw4dqs+VPj4+AwcOdHxosVhiYmLw4+5OITwhLjt37hw/fnytUx0ReXp6vvnmm7GxsVu2bMFgp1gITwCAaJ0/f573KNBisfj5+fFWEN28ebNXr17NmjVzVJKSkkpLSwXqUtYQnhCXgoKCdu3a3f6aDh06xMTECNMPiBDCEwAgEjXXt6nVaj8/P26lpKTEYrG0bNmSW7x586ZWq+WuqOPtVwJ3DeEJcWnevPnZs2dvf82ZM2eaN28uTD8gQghPAIAYFBQU7Nu3rz5XarVa3ulbly5dwjNWF0F4QlxGjx69fPnynj17Tps2reaJJRUVFZ9++unWrVtnz57tlvZADBCeAADh5eTkFBcXcytWqzUsLIx3f+jixYsPPPAAN7V69erVGzduCNQlIDwhNosWLTp06NCsWbM+/PDDXr16tWrVys/Pj2XZ8vLy9PT0kydPVlZWDhgwYN68ee7uFNwG4QkAEN7Nmzezs7O5FZPJZDQa9Xo9t2iPsvKesSpkOb9IKOTftmQGu8DAwGPHjq1ateq77747cOCAzWZzvKTRaLp37z5p0qRJkyZhLYKSITwBAC5lMpnOnDnD26DEvlcw9wAulUpltVoHDx7MvWzr1q3cqQ6Eh/CE6Gi12unTp0+fPt1oNGZkZNi3cAwICAgLC8N/LUAITwCAi6lUKp1Ox72zQEQVFRVqtZp7ugNuxYkTwhMixbJsVlZWenq640gxnU6HI8WAEJ4AgAZVcw2czWarrKzkzQcsy4aEhHTq1MlRSUtLS0pKEqhLqDeEJ0QHR4rB7SE8AQANyM/Pj3eUUVlZWWFhYVhYGLdYVFSEVUCSgPCEuOBIMXAK4QkAuDtWq/X06dM1n7FqNBruUh+TyUREvA1K7Ie0CtMn3AuFPB+XzGCHI8XAKYQnAOCuaTQa3m+GFRUVvr6+9psIdmazWSHDgSwhPCEuOFIMnEJ4AgDqo6SkpKCggFthGIZl2caNG3OLeXl5oaGh3GW7N27cUMg6LVlCeEJccKQYOIXwBADUR15e3rVr17gVq9VaVlbGW8ljNBqxZldOFDKUS2aww5Fi4BTCEwBQk81m4y2eCwsLa926Nbdy8+bNw4cPP/7449wnrfv27eM+hwWpQ3hCXHCkGDiF8AQA8Nhstt9//5032IEyKWR9pGQGOxwpBk4hPAEAPJ6enk888YTVauUWz5w54+vrGxkZ6agUFRXFxcUJ3h0ICuEJccGRYuAUwhMACldYWJiZmcmtsCxbVFTES0UYjUZ/f3/uijre5AeyhPCE6OBIMbg9hCcAFM5gMBQVFXErVqu1oKCAZVnuOg2LxYJJToEQnhApHCkGdUF4AkDhWrRo0aJFC26lpKTkzz//7Nu3L3dx9qFDh7hHu4JCIDwhOjhSDG4P4QkARfnrr7/sR0E4WCwWIuKeA8EwjNBtgVghPCEuOFIMnEJ4AkBROnTowBvs0tLSVCpVeHi4o2IwGPBjAewQnhAXHCkGTiE8ASBjFouFZVluJSgoiHdNbm6uWq1u27ato1JSUoLBDuwQnhAXHCkGTiE8ASBXqamp9dyOhBeABXBAeEJccKQYOIXwBIBctW7dmrfMxmQyxcbG9uvXjxuDSExMxCYJUBeEJ8QFR4qBUwhPAMgDwzA1tyPx8fHhLn6371oaEBDAfb6m1WrxQwDqgvCEuOBIMXAK4QkAedi9e3dFRUV9rkToFeoP4QlxwZFi4BTCEwDy8Oijj/LirpcvXzYYDF27dnVU7I9icX8O6g/hCXHBkWLgFMITAPLg4+PD20BYp9NZrVbuMjuDwSB4XyBtCE+IjiuOFGMYJjY29vZny+DpnlQgPAEgOVlZWSdPnuQVLRaLWq3mroiy2Wx+fn7CtgZyg/CESDXskWLp6eljx469/WBnfyJgsVgQthI5hCcAJCcoKKhHjx7cDepYlj1+/Ph9993HfXB27do13iZ2AHcK4QnRccWRYm3atKn51XjWrl07efJk/EwRP4QnACRHo9G0bNmSW7HnIUJCQpo0aeIoFhQU1DNOAVAXhCfEBUeKgVMITwCIXFJSUklJCbdiMpmsVqv957md/bdo7kJqgAaB8IS44EgxcArhCQCRU6vVGo2GWykvLzcYDIGBgY6KfbBTyFMzEBLCE+KCI8XAKYQnAEQuMjKSV0lKSsrNze3Ro4ejwjBMWloa1stCg0N4QlxwpBg4hfAEgHiwLFtcXMwrGgwGLy8v7t04g8GATYZBGAq5DSyZwQ5HioFTCE8AiEd+fv6BAwfqc+Wdht4A7g7CE+KCI8XAKYQnAMQjODh47NixvOKWLVseeuihZs2aOSr2R7HCtgYKhfCEuOBIMXAK4QkAAKgLwhPigiPFwCmEJwDc5dKlS7x3TbPZbDKZePdIbDabxWIRtjWAKghPiI4rjhQDOUF4AsBdtFotbx8Tg8FQWVnZuHFjblGlUilkATuIkEK+96Q02Dl4eXlFREQQkdlsPnv2bEZGRnh4OBZXAcITAO7Spk0b3g/hq1evpqSkcPcxIaLr16+r1ZJ83wEZUEh4QjK3Nz766KP9+/dzK2vXrg0NDe3Vq9djjz3Wtm3bHj16JCQkuKs9EAOEJwAAoC7+/v5KyE9I5jen+fPnz549+9FHH7V/uHPnzsmTJ+t0uqeeeio4ODgxMfHIkSOPPPJIfHy80+3uQK4QngAQQHFx8d69e+tzfDYWyYCoIDwhatOnT9fr9ceOHevQoYO9smXLlqeffnrJkiXr1693b2/gLghPAAhAr9cPHDiQN9gdP368TZs2ISEhjkpWVhb2MQFRQXhCvPLz869cufLBBx84pjoiGjNmzKhRo/bs2ePGxsC9EJ4AEIBKpQoODuYVPT099Xo9d7ArKyvLy8sTtjWA20F4Qrzst2S4U51d586dd+7c6Y6OQBQQngBocCaTqbKyklthWdZsNvM2isexYCB+CglPSHKwa968uV6vz8zM5NWzsrKUsC4S6oLwBECDO336dEZGRn2urKiocHUzAPdCIROClAa769evx8XFBQYGBgYGvvnmm+vWrXvrrbd8fHzsr168ePGXX3557LHH3NskuBHCEwANrk+fPt27d+dWMjMzExMTeQuV9uzZ4+vrK2xrAHcG4QnR+fnnn3/++WduZffu3f/4xz+I6KeffnrttdcMBsP8+fPd1B24H8ITAA1OpVLxwq1qtbpmEUD8EJ4Ql2+++aaYo6SkpLi4uFGjRvZXi4uLAwMDN27c2LNnT/f2CW6E8ATAvWBZNjs7m3tgIxEZDAatVss9rbGgoKA+e50AiA3CE+Ly0ksv3ebVCRMmTJ48GW/qCofwBMC9MBgMp06d4g1tFovF09OT+9OVYRgMdiBFCE9IiZ+fHxEVFRWVlJSEh4e7ux1wD4QnAO6Fj4/PqFGjeMU9e/a0adPGfoqj3fXr18+ePStsawANQCHhCSnd4jp37lx0dHR4ePiAAQNWr17Ne15ARJ988gne15WMYZjExER3dwEAAGJUWFiohPyEZO7YHTlyZNCgQSaTycfHJysr6/Dhw5s2bYqJiXEsswNAeAKg/srLy4uKirgVlmXLy8t5dzUsFgv2qAN5QHhCXJYuXcowTExMzKhRo8xm8+rVq2fPnj1kyJD9+/cjYw92CE8A1N+1a9euXr3KrTAMY7PZNBoNt2i1WnkbFANIFMIT4nLu3Llx48aNHj2aiHQ63fTp07t27Tps2LCxY8du27aNm9gCxUJ4AqD+OnfuzNv0MScn58iRI/Yfsw579uyxL2IGkDqFhCckc3sjJyenbdu23Mpjjz329ddf79q1a8aMGe7qCkQF4QkAAKiLv7+/EvITkrljFxISkpCQwCuOHz8+OTl56dKlLVu2nDVrllsaA/HAyRMAdcnJybFYLNyK0WhUq9Vq9a13geLiYuxjAjKmhOQESWiwGzNmzIoVK1auXPn6669zl4AsWbIkKyvrvffey8rKqpmTBUVBeAKgVmaz+eTJk7wMhNVqValU3HUs2KAO5A3hCXFZsGDB77//Pm3atK1bt+7du9dRV6lU33zzjV6vX7ZsmRvbAzFAeAKgVlqtduTIkbzigQMHgoKCOnXq5KjY19gJ2xqAcBCeEJcmTZrEx8cvXLiw5gGFKpXqP//5z8CBA997772UlBS3tAdigPAEAADURSHhCckMdkTUtGnTVatW1fXqmDFjxowZI2Q/IDYITwAQkcViKS8v5xUNBoO3tze3YrVa8eAVFEUJyQmS1mAHcHsITwAQ0aVLl5KSkupzpY+Pj6ubARAPhCcAJAbhCQCqbYO6wsLCv/766x//+Ac3KnHgwAG9Xi94dwBug/AEgMQgPAEAAHVBeAJAYhCeAAUymUxWq5VbsdlsKpWK+0uOwWAQvC8A0UF4AkBiEJ4ApWEYZseOHdjCE6A+EJ4AkBiEJ0BpPDw8oqOjeYPd6dOnvby8Onbs6KgUFxdjgzoAhCcAJAbhCVAgLy8vXsXT01OtVvv6+joqJpNJ2KYAxAjhCQCJQXgCAADqgvAEgMQgPAHyduXKlTNnztTnypCQEFc3AyA5CE8ASAzCEyBv4eHhAQEB3IrBYDh58mSfPn10Op2jmJSU5OfnJ3h3AGKH8ASAxCA8AfKm0Wh4t+LsR4cFBQVxjwtLSUnBmgSAmhCeAJAYhCcAAKAuCE8ASAzCEyAnf/75J29jYfu2JtxjwViWdfxfALg9hCcAJAbhCZCTbt268W4/p6SkMAwTERHhqBiNxjNnzijk7QrgHiE8ASAxCE+AnAQFBfEqubm5NputVatWjkp5eXk9c7IAoJDwBJ5bgXwwDJOYmOjuLgAAQIwKCwuVkJ/AHTuQD4QnQKJu3LiRkJDAKxoMBi8vL+5jVpPJxNvuBADqD+EJAIlBeAIkqmnTpl26dOFWrFbrqVOn2rdvzz0ZLCUlRavVCt4dgEwoZDUqBjuQD4QnQKJ0Oh135RwRmc3mU6dOhYaGBgYGOor2NXaCdwcgEwhPAEgMwhMAAFAXhYQnMNiBfODkCZCE/fv3l5SUcCsMw7Asy92gzg735wAakBKSE4TBDuQE4QmQhC5dulRWVnIr6enplZWVHTp0cFTsa+xqjnoAcNcQngCQGIQnQBKaNGnSpEkTbqWoqIhlWe4yO/saO8FbA5AzhCcAJAbhCRAhi8Xi9MgvPHIFEADCEwASg/AEiE1WVtbhw4frcyV3WxMAcAWEJwAkBuEJEJvQ0NDBgwfzinv37u3evXvjxo0dlStXrhgMBmFbA1AchCcAJAbhCRAbDw+PRo0a1az7+/tz615eXiaTScC+AJQI4QkAiUF4AtyrrKyMF3e12WwWi8XLy8tdLQGAA8ITABKD8AS4V3x8fF5eXn2uxP05AOEhPAEgMQhPgHs98sgjvMrFixdv3LgxaNAgbnHTpk06nU64tgCAiBQTnsBzK5APhmESExPd3QUAAIhRYWGhEvITuGMH8oHwBAimsrIyOTmZt0FdcXGxn5+fWq3mViwWi+DdAUAtEJ4AkBiEJ0AwLMvW3Hm4uLjYw8PD29vbUbEfAit4dwBQC4QnACQG4QkQjK+vb58+fXjF33//PTIysmXLlo6KfY2dsK0BQO0QngCQGIQnwEUYhrFarTWLvDvEuDkHIGYKCU9gsAP5wMkT4CJHjx7Nysqqz5U4QAJAtJSQnCAMdiAnCE+Ai/Tu3dtsNnMrGRkZKSkpvP1N9uzZw11gBwCigvAEgMQgPAEuotFoNBoNt6LValUqla+vL7eokKXZABKlkP9CMdiBfCA8AffOYDDExcUxDMMtlpeX63Q67mxnMBiwjwmAtCA8ASAxCE/AvVOr1YGBgbwYRGFhob+/f2BgoKOiUql4D2cBQOQQngCQGIQn4N5pNJqoqCheMTMzs2XLlm3btnVUUlNTy8vLhW0NAO4JwhMAEoPwBNypvLw83s96m81WVlbGvTlHRGazueZ2JwAgLQhPAEgMwhNwp4qLi3Nzc7kVk8lUWlpqMpm4RZvNhhV1AFKH8ASAxCA8AXcqMjIyMjKSW8nMzIyLixs4cCC3uGvXLuxjAiB1CE8ASAzCE3AbDMOkp6fzUhFlZWVeXl7cuGtRUREOkACQJYQnACQG4Qm4DYvFkpqaytvHpKysTKPReHl5OSpms9lmswneHQC4HMITABKD8ATchk6nGzRoEK+4b9++Fi1a3H///Y6K/VGssK0BgBAQngCQGIQnwCE/Pz87O5tbYRimtLSUF3etrKxEKgJAIRCeAJAYhCfAoby8vKioiFsxm80118/ZbDbsYwKgEAhPAEgMwhPg0KZNG943Q15e3oEDB3hx13379iHuCqAQCE8ASAzCE8pks9kOHz7MO+DLaDR6enpy4664MwegcAhPAEgMwhPK5OHh0aJFC97clpaW5uXl1axZM0elvLy8rKxM8O4AQCwQngCQGIQnlMBms9Wc3Zs0aaLVarmVnJycRo0aceOueXl5qampQrQIAKKE8ASAxCA8oQSXLl1KTEysz5Wenp6ubgYAJAThCQCJQXhCCTp06NC+fXtupaSkZP/+/cOHD+fetDt27Jherxe8OwAQL4QnACQG4QmZYVk2Li6Ot3iusrJSpVJxo6z2jei0Wi13sMNDeQDgQXgCQGIQnpAZlUrl7+/Pi7uWlJR4eHj4+vo6KiaTSfDWAEB6EJ4AkBiEJyTNYDDwzoogIqPRGBAQwK1oNBo/P78uXbo4KsXFxWlpaUK0CABShvAEgMQgPCFp+fn5yVS0Jr8AACAASURBVMnJ3ArLspWVlT4+PtwfxyaTCeM7ANwFhCcAJAbhCQkxGAy8h+a+vr69e/fmRllNJlNsbOwjjzzi5+fnKJ44cQJxVwC4CwhPAEgMwhMSsm/fvsrKyvpcyTCMq5sBACVAeAJAYhCeEC2LxcKyLLfy6KOP8ioXLlwwm829e/d2VIxG4x9//IEHrwDQIBCeAJAYhCfE6fr168ePH6/PlQEBAdwtS2w2m8uaAgDFQXgCQGIQnhADhmF4O88FBwcPHDiQ+yOVYZjY2NjevXtzE6+XLl3CJAcAroPwBIDEIDwhBvv27SsqKqrPlV5eXo0aNXJ8qNPpDAaDy/oCAKVDeAJAYhCeEJh9OxJesUuXLrwbbxkZGaWlpT169HBUGIbZt2+fRqMRoksAACJCeAJAchCeEFhubm5sbGx9rvT29ubenMMjVwAQHsITABKD8IRLlZSUlJaWcis2my0qKsrLy4tbjI+P7927d0hIiKNy8eLFgoICgboEAKgDwhMAEoPwhEtduHAhLy+PW2EYxmaz8Z6oMgzDMAw33Orp6amQn6cAIGYITwBIDMITDcVsNl++fJm3MzDDMGFhYWr1rR8aRUVFFRUVw4cP5162efNmnU4nUKMAAPWG8ASAxCA80VAsFkthYSFvA+H8/Hx/f3/ug9fKykqslgMAqUB4AkBiEJ64O8nJybxnrFar1WAw1Pzttk2bNpGRkY4Pr1y5kpaWJkSLAAD3DOEJAIlBeKI+rFYr7xmrv78/r1JcXFxaWsoNQBBRYWEhNigBAOlSyGJfDHYgHwhPOFVeXr5r1676XOnh4XH//fdzK1euXPH09HRNXwAALofwBIDEIDzBU1xczFtTYg9A+Pn5cYspKSlhYWHh4eGOSnZ2dkpKijBNAgAIA+EJkWJZNi0tLTU1taysjIj0en1ERESrVq3c3Re4H8ITPNnZ2ampqdyKzWYzGo2+vr7cosVi8fT05G4gXFpaqpBnFgCgHAhPiE5RUdGSJUu+//573kJvIgoLC3vllVfeffddb29vt/QGYqDk8ITFYjly5IjFYuEWjUajp6cnd2GcxWJRqVTR0dHcy/78808fHx+BGgUAcBOEJ8QlOzu7X79+aWlpERERw4cPb926tf2uQ2lpaUpKysGDBxcsWLB58+b9+/dzbzyAoignPGGz2XiHQDAMExAQwLvNlpmZ6efnFxwc7KiUlZUZDAaBugQAEBOFPIiQzGA3f/78zMzMTZs2PfPMMzVftdlsa9eunTp16uLFi5ctWyZ8eyAGyglPXLhw4eLFi/W5smnTptwMRHZ2dkZGhsv6AgAQL4QnxGXnzp3jx4+vdaojIk9PzzfffDM2NnbLli0Y7BRLluEJlmXPnz9vNpu5xYqKitDQUO5ewUajMScnZ+TIkdx7locOHdLr9cL1CgAgYghPiEtBQUG7du1uf02HDh1iYmKE6QdESB7hCYvFwj3ygWVZg8FgtVq515SWlqrVau7iOfunaLVa7mCnkAfTAAD1gfCEuDRv3vzs2bO3v+bMmTPNmzcXph8QIRmEJy5duuT0+9wuJCTkoYcecnxYUFCQm5vrsr4AACQP4QlxGT169PLly3v27Dlt2rSaR4xXVFR8+umnW7dunT17tlvaAzGQXHji8uXLvDG05jNWi8Vy48aNhx9+mPttf/bs2YCAAOEaBQCQPoQnxGXRokWHDh2aNWvWhx9+2KtXr1atWvn5+bEsW15enp6efvLkycrKygEDBsybN8/dnVaze/duk8nErdgPTeft4K/T6YYNGyZoZ3Ik8vAE7xkrERUUFPAGO8fujI6K/RtGr9dzt/LRaDQK+QkFANBQEJ4Ql8DAwGPHjq1ateq77747cOCA/d3OTqPRdO/efdKkSZMmTRLbkUc9e/bk7S5x+fJli8USGhrKLVqtVl5WUaVSBQUF1bw3Cbch5vBEWlraqVOn6nNlYGDgwIEDHR+WlZXt3r3bZX0BACgFwhOio9Vqp0+fPn36dKPRmJGRYb+3ERAQEBYWptVq3d1d7Zo2bcqrXL58uby8/Nq1a44Ky7IWiyUzM5N3Zbdu3cLCwlzdoZyIJzyRlJRUWVnJrRgMhsaNG3PvujEMk52d3adPH+7PmqSkJGk9TQYAkAqEJ0SKZdmsrKz09HTHQyudTiehI8X8/f39/f179erlqBQXF+/Zs0ev13Pf0Q0Gw8WLF9PS0rif6+fn1717d+F6lRq3hCdsNltBQQHvGWtRURHDMNxKaWkpwzCBgYHcTyQif39/7pbaWq2W94kAANAgEJ4QHbkeKWZfLKXX69XqW/9zVFRUEBHvr+Ph4VFUVMT7XL1ej+VWdm4JT+Tm5h4+fLg+V/r5+fXo0cPxoclkSk9Px/05AABhKOS9UjKDnYyPFLN/q3Xs2JGbhczIyCgtLeUdG0VEV65c4VX69u3bsmVLVzcpCQKEJw4dOlRQUMCtMAzj4eHBHcqJyGw29+/fn7v5ztmzZ2v+rwkAAIJBeEJclHakmF6vDw0NjYqKclTy8vKOHj3atGlT7u8cBoMhOTk5JSWF+7kBAQEPPvigcL2KRsOGJ8rKyhITE3nF0tJSX19f3kNzhmG4/8JZlj1+/Dh392AAAHA7hCfERYFHinl6enJDIfY/N2vWjDvYXblyRaVScW/1MQxTXFzMO0jUPvGINmLSUO4lPHHjxg3erTij0VhUVMQLJldWVur1eu4ecvYtS7irPO2D3V30AAAAroPwhLjgSDH7XaL77ruPe7soJSXFarVy4wIWi6WiooK3xwrLsp6enrxfVnx9ff38/FzctaDqGZ6w2WypqancHXOIKCMjw2w2c4dmq9VqsVjsT/y5WrRowb0vmJycnJWVdc+9AwCAayE8IS44UqxW3t7eoaGhHTt2dFRSU1Pj4uLKy8t5V54+fZpXadKkSbdu3XhFya1Q5Ko1PHHz5s2bN29yK0ajsWZqwWQyeXl5cW9qMgzj6enJ3VKOiGJiYmR/4xMAQJYQnhAXHClWT15eXmq1esyYMdzili1beKfIE1FBQcHevXu5FZVK1blzZ95tvMDAQKmsS+jTp8+WLVvOnTvHLd64ccNkMnHHOJvNZrVauTuPEJHJZGrbti13RE5JSbl8+bKrewYAAGEgPCEuEj1STCR8fHzat2/fvn17R+XChQtJSUm8Bf4Wi+XChQvc32lYltXpdM2aNeNeplarO3fuzMuBulpZWRlvNr127VpOTg63W5vNZjAYUlNTuZdZrVYfHx/uncjKysqSkpLBgwdzL9u5c6cU98oBAIB6kspNinskmcFOokeKiZavr6+3t/eIESO4xS1btuj1eh8fH0elqKiovLycNyoRUc1bWd7e3rxjNqxWa6tWrbiDF8MwGo2Gd8P15s2bvPny5s2bpaWl3P8pbTZbSUkJbxPgmliWzczMjI6O5havXr3aunXrTp06OSoZGRk1H0wDAIC8ITwhOlI8Ukxa1Gp1REQE9xwz++kXvKV4sbGxvr6+3GnMHtfgHXdLRNnZ2XfXiUql4t4RZFmWZdl27dpxn6jeuHEjODj4gQcecFS2b98+Z86cmTNncr9UZmamwDcXAQBAhBCeECmpHykmLfatd0NCQrhFlUrVvXt3bjExMbGgoICbM7BarVu2bOnfv79er+deZjKZunTp4qgYjcbY2NgnnniCe5swLi5Oq9VyD2koKirau3dvly5duNNkWVmZt7c3d6ZXq9UKWRsLAAB3SiFvEFIa7OR6pJiMeXt7c7cL0Wg0vPNSKysr7XXufObh4XF3//n17t37iy++uId+AQBAthCeEBcZHykGDUWlUgUHB7u7CwAAECOEJ8RFaUeKwV1gGKbmOj8AAABSTHiCv5uraNXnSLGxY8du2bJF4MZAPE6cOPH++++7uwsAABCjNWvW2PMT8iaZwa6eR4rl5uYK0w+IkEqlUsjaWAAAuFMKeY+QzKNYHCkGTiE8AQAAdVFIeEIyd+xGjx7966+/fv755yaTqearFRUVCxcu3Lp167hx44TvDUQC4QkAAKiLv7+/EvITkrljhyPFwCmEJwAAoC4KCU9IZrDDkWLglD08MWPGDHc3AgAAooOTJ0THFUeKlZWVffrppxaL5TbXJCQk3N0XB4EpZGEsAADcBYW8QUhpsLNr2CPFDAbD2bNnDQbDba65ceOG/Z97d/8IEAzCEwAAUJdpr77q7haEIKXBzhVHigUHB2/btu3216xdu3by5MkKmfQlDeEJAACoxblzNG3ao4cPExH1708rVhDn1HKZkcxghyPFwCmEJwAAgC8/n4YOpexs+z2hkNhYGjqUzp6loCA3N+YakhnscKQYOIXwBAAA8P3yC2VnE5F914yviCg7m375hWS6rZ1k9rHDkWLgFMITAACKYzL5JCf7X7hA5eVVlQ0bdA8++PQLL/h160arV9PFi/ayB3fouXRJ+E6FIZk7dvU8UiwmJkaYfkCEEJ4AAJCPwsKAv/6i3Fxq1YoiIoiIzp5VLV36xPHjPhER9M47FB1N+/bRpEmR168TEX3wAS1fTkYjvfKKikhFRCkpNGUKjRxp/3ofc7+4/QvKkWQGOxwpBk4hPAEAIAElJX579rRISaGWLalzZyKio0dp8eLokyc9WrSg116jN9+kXbvopZdaFhUREX3yCc2aRaNG0cMPqyyWQCJKT6e//qL/9//os8/Ifg0RFRbSP/9Zy8q5AwcoOJjy8m4twA8OprFjBfiLuoVkHsXiSDFwCuEJAAB3Yll1ZmbglStUWlpVSU2lSZMenjYt9IUXaN06Yhjat48iIkKnTIn64guKiqLXXqPjx+mRR2jPHl1xsebCBXr7bZo2jcaPvzWx2Wz08cf08svE23R28eJb19hZLJSVxe+qtJQ2bKDevXOJcomod2/atYtCQ13w9xcFydyxw5Fi4BTCEwAAAmFZ2rYtYuNGvxYt6OWXqUMHunaNJk5sHhvbnIjmz6d58+i556hbNyopCSCia9fo6FGKj6fffqP8/Ftf56uv6Ngx/sS2di3V3Dv28mV+pbYbPeTlRUZjtYq3Nw0eTMOGTY6OJqKYnTvv5u8rHc4Hu23btrVr165Tp04CdHMbOFIMnEJ4AgCgQahstmqj1e+/B65Y8XhaGm3dSh98QBERFB1Ne/aE219dvpxWrqSvv6ZTp6quNxho7lz6808qKan2ddesqWViu3qVX6n1RACtlqxW560/9BDt31+tMn48qdVVX0EBnA9248aNW7RokdsHO3LNkWIgJwhPAADcKVVlpX92NpnNVdNPfDzNnDng6FFWq6Unn6R//5s2bqSZM3VEOiJKS6PffqMpU2jPnltfwmKht96q5f7ZmTP8Sq0Tm0bDv8dWq65d6dixapW+fclmoxMnblVat6ZffqHFi2ntWrJaycODJk6k//s/+4tTZbq/CY/zNXb9+/c/ePAgwzACdFMfjiPF7K5fv56bm+vupkAUEJ4AALgdi8UnLc0rPb3q1ldFBb36amBY2LB33lE3bkzz5lFaGj3+OB08qLJYPCoqaONGGjaMPvig2hcxGum77/hfudanopxna7fTrRu/0qcPjR9frdKuHcXE0JNP3qp07Uo//ki7d9Pbb5tatTKFhtKECXToEAUF0cqVxhs39nz2Wdm1a7R+Pfn42D/D39/f39+/Xi1JmfM7dj/88MP06dOjo6MnTJgQGRmp1+t5F7Rv3941vfG54kgxkBOEJwAAHFRlZfrr11VlZVWTzc6dNHly18xMIqJ27WjdOvruO1q/vupqk4mWLKGjR6m4uNpXOXeuli/Ne8Bal8hISkioVmnThkaMoBUrblWaN6dNm+iTT2j58qpxs18/+vFHatmS+vcv//57a2lp4PDh9N571KgRbdvGnDlzcsOGDoMH64cNI/viq2XLkl96qby8vH///re+bEBAcVgYW31iKSwsrFfbEud8sAv9X3Lkjz/+qPUCttY7qw0NR4qBUwhPAIBieZjNHo7bZkYjvf22z9dfD2EYev99evVVeuMNGjuWKiurLkhJodGjb+3o63DyZL3+YYGBxLvDolbT6NH022+3Klot/fe/9O239OWXZH/o17YtbdpE3bpRnz4V33xjyslpPHQovfceBQXRv/9Ns2fHrl17/8CBwQMGkH219GuvXe/fPycn57HHHrv1Zbt2vd6vX/s+fejOl9SvWbOGiIYOHXqnnygt9Vpjp9VqNRqNe5el40gxcArhCQBQoqQkmjbtgQMHVETUpw+tXEkbNtB//1v1qs1Ga9bQ2bO3pjo73p05O941dm3aUFpatcpLL9GBA9WmwKVLaepUioqyrl/P5udrevakjz6inj2pZ0+aPj1+3boWUVGhY8aQTkdE9Pzz2b16Xb16tdqMFRx8s1Mna/v25LIf4wp5g3A+2G3cuFGAPpyqz5FisbGxW7ZswWCnWAhPAID8xcaqly4dfuaM13330Tvv0MCBNHQoZWRUzSxHj9KQIbVkEeLja/lSajU/Z9qmDWVlVfv08HD69VeaMIGSkqoqTz9N//oXqVT03Xc3Nm/2ad680aRJ1LcvEdGCBVkvvZSQkDDyf4c9EBG1a5fdt2+Tzp2rpjr3UUh44g72sbt58+aVK1cqKir8/f3vu+++wMBA17VVE44UA6cQngAAuYmLC/7mm8CSEiovp+HD6c8/aehQFZEfEeXmUmwsvfgi8dYWc3eJczCbaykOG0bbt1erzJ9PTZvSlClkP6SrVy9at446d6aEhKIdOy4fPNj75ZcpKqrq4pdfTgoLa926daPIyHv/iwpACckJqudgd/jw4XffffcEJ1GsUqkee+yxZcuWdbYfBuJ6OFIMnEJ4AgCkzWr1y81VOZ6HLl1K8+eH2LOlP/5IzzxDV67wP+XXX2v5Ot7eZDBUqwQHk5dX1bhm17Ej/fADrVrF/Oc/Hnl5bJs2qrlz6aWXiIiio09s2qRv1uz+hx+uulijMfftm2Gz9XZMdRKkkPCE8+1OTp48+fjjj8fFxfXv3//ll1+eOnXqP//5z169ev3999/9+vW7dOmSAF0SjhSDerCHJ9zdBQDAXfnkE2rUaNi0acHt2tH48XT0KC1YUG3HkF9/pfPn+Z9V6z4jgwfzK/Pn0/799MwzVr3e2rgxTZhAe/dSQADNmVOSnPzbjz/aLl2iSZOqLlapjKGh1saNG+pvJhJr1qyx5yfkzfkdu48++igoKGjv3r33338/t37mzJmhQ4cuXrz4p59+cll7t+BIMXAK4QkAkIyjR4PWrfPLyaGMDBo/ntavJ8fvpSxLP/xAiYm1HLSg1fJvxXl4UEgIZWffqjRuTCtX0vDhzOefU1qaqm1b1bvv0quvkkpFmzadOnZMp9N1q751HKOMQ5sU8gbhfLA7evTozJkzeVMdET344INvvvnml19+6ZrG+HCkGDiF8AQASMOnn9Ls2VU3xHbtoq++qiXuwNsBzq51a7p4sVpl+HBasoTeeIOOHiUi6t6dVq2iVq3o9dcNL764c+fOESNG+Pxvh16FQ3iiSklJScuWLWt9KTw8XMgn1jhSDG4P4QkAEKO//6b580fExbHBwTRpEo0dS3PnVrvg2DGq5+76b71F+/bR5s1VHw4cSF9/TSEhdORIQmys1Wzu8fjjDdy8jCA8USU4ODg5ObnWl5KSktzyPurl5RUREVGzXlRUVFJSEh4eLnhHIAoITwCAGKhzcvyvXSOjkby86OhRGjqULBZPIsrMpA8/pP37a3nGqtPxn7GqVDRjBv3737cqQ4fS66/TG29Yk5KOf/ttlxEjAgYMcLxo8/W14h7HbSE8UeWJJ55YsWLF1q1buSdMsCwbExOzatWqYcOGubK9as6dOxcdHR0eHj5gwIDVq1fbahxC98knn7Rp00awfkBsEJ4AADdLTaWBA1v17fvw1KkUHEwrVtDHH5PFUu2aQ4dq+cSaO4a8/jp9/jn9/ffNiROzRo2ijRtp507y8CAiNiIiq3t3m1C7UsgGwhNVFi5cuGvXrtGjR4eGhnbs2NHX17eioiIpKSknJ6dZs2YLFy4UoEsiOnLkyKBBg0wmk4+PT1ZW1uHDhzdt2hQTE4MDxMAB4QkAEFpubtPTp3UmE4WFERE9/TSdOVP1UlkZvfUWhYTU8lkaDX/ae/55evttWryYrlxhGjXymDyZ5s8nInr00eymTYuKipo7dh6Bu6WQNwjnd+zCw8Pj4uImTpxoMBj+/vvv7du3//3332az+ZVXXomPj69r+V2DW7p0KcMwMTEx5eXlZWVlX3zxxdGjR4cMGVJRUSFMAyB+CE8AgKAWLaKwsJ7z5rV86inq0oU2bbo11TnUeLhERDR3Lqk5N1aGDqUpU+j55+nSpc0//JCflERLlpCXlws7V6SpU6cqIT9Rrw2KW7VqtWHDBpZlc3JyKioq/Pz8QkNDXd0Zz7lz58aNGzd69Ggi0ul006dP79q167Bhw8aOHbtt2zaEYYEQngAAl2JZdXp6k4sXqaiIGjWin3+mxYtvvXrxIs2cWctn1Uyk9u1L8+fT00/fXL++/MaN8Oeeo5EjHQekMhqNi9oHhYQnnN+x27Zt24ULF4hIpVI1a9asffv2wk91RJSTk9O2bVtu5bHHHvv666937do1Y8YM4fsBEUJ4AgBc5fp1euSRkH79Hv7gAwoNpblz6Zdf+Nfk5tbyif3709Klt8a7xx+nX34hDw/q1KngtdeuvvIKjRrlumPvgauwsFAJ+Qnng924ceN27NghQCu3FxISklBjU5/x48fPmTNn+fLln332mVu6AlFBeAIAGpCHxUIMQ0TEsjRuHMXGVr1gNtP/+3+17zPXq1e1D3U6mjWL3n+f8vMPLFuWdfIk7d1LQi1hAh6FhCecD3b9+/c/ePAgY//mdp8xY8Zs37595cqVluoLTpcsWTJx4sT33ntv+vTplY7z9UCREJ4AgIZx6BD16vVIdPT9PXrQc8/RsWN0/Dj/mppbChPR6tU0a5a1WTObVksDBtBff9EDDxAR+fiUhIfb3PG8CxwU8h7hfI3dDz/8MH369Ojo6AkTJkRGRur1et4F7du3d01v1SxYsOD333+fNm3a1q1b9+7d66irVKpvvvlGr9cvW7ZMgDZAzBCeAIAGkJxMQ4dSZaWKiIxG2rixlkgEEXl6kl5PJSW3KhMnUvfu1L17xhtvJCUlRUdHC9Ux1IsSkhNUn8HOsaLujz/+qPUC7v52rtOkSZP4+PiFCxfWPGRCpVL95z//GThw4HvvvZeSkiJAMyBOCE8AwN1ISPBbvrz/+fOqhAR66y368kviPf+5dIlUKuK92T34IP3737RgQWVsrGfTproJE+jtt4XsGu6UQsITzge7cePGabVajUbj9huYTZs2XbVqVV2vjhkzZsyYMUL2A2KD8AQA3LHffqNnn/W22byJKC6OvvySOnWq5bIBA26tsSMijYbmzaP77qNffjm4e3dkZGS7du2E6hjukhKSE1SfwW7jxo0C9AFw7+zhCaSkAeB2rFbvjAxtYCB16UIMQ6+/Xm2rueJiysys5bNmzaInn7R+9RWTk6Pt0YMWL6Y+fQRrGRqEPTkxdOhQdzfiWs4Hu23btrVr165Trb/BAIiJQhbGAsDd27qVpkzpceMGEVHnzjR/PtW8i1NURFotmc23Kh060KBBNGJE7nPPnTx58qmnnhKuYWg4CnmDkMx2JwBOITwBALdz/jw9+yzZpzoiSkykWlfT6/X066/UunXVh4MG0bZt5O0tUJPgMgo5eUIy250AOIXwBABUU1qq/fbbzr/84vnDD2Qw0E8/8fcoyc+nZs34n/XEEzRyJF27dviXXy6dOEF//UWCbP4Arubv76+E/IRktjsBcArhCQC4JTGRBg/2zsnpSERbttCSJRQVVctlo0fT779TdnbVh9270+ef2/9oatKEUcAcoBwIT1QRyXYnAE4hPAEAt/zzn5STc+vDtDSqsVsWEdGgQfTxx2U//JB2+HDU2LGqJ58kHD4uUwhPVBHPdicAt4fwBIBy2Wz03XftN24klYqee46efJLi4/nXZGRQcDDl5d2qdO1KI0aQTmd85pmLTZtG4dhWWVPIGwS2OwH5QHgCQKFYlkaPph07mtg//PNPGj6cv58wEdls9NdfNGeObf9+VqNRjxpFH39MOp3Q3YKbKCE5QbcZ7G7evOnt7e3r63ubT967d++JEyfmzZvngsYA7hjCEwAK9dtvxNu9Ydcuat6csrKqFXv3pqgo2rHj6KFDAQEBXbt2FbJHcDslJCfoNqnYoKCguXPnciuLFi3as2cPt7Jz58758+e7qjWAO4TwBIBSpKVpZs/u/+mnuvfeo8uX6dixWq559NFqe5To9bR6tWANgggVFhYqIT/hfLsTh8WLF/MGOwBRsYcn3N0FALjYiRPUsaN6xYrm8fHaNWsoKory82u5LDKSkpJMM2em9+9vnTePLl2q/awwUIw1a9bY8xPy5nyNHYBUIDwBoAhTplTbjs5srnaKq8PgwRQebpw//8Sff44aNUqNtXSKp5A3CAx2IB8ITwDIU3x8yLffWioqyGKh3r0pIYF/wfXrNGsWffFF1amvnp60aBE99JDwnYKYKT08ASA5CE8AyNC//kWLFzezT2zr19OkSaTVksFQ7Rr7JPfii+kbNqg8PMJeeok6d3ZLsyBmCglPYLAD+UB4AkBu4uNpwYJqlfXrqUcPiourVhwwgHx8qEuXnHHjPDw8wjDVQW2UkJygOwpPAIgcwhMAclBc3DglxSM3l4ho//5aLujcme6779aHbdvSunUC9QZSppDwBAY7kA+EJwCkzWaj6dM9QkIe/+CDRp060ZgxVFpay2VaLZ07Z/7xx3PPPWfYsIEuXKC2bQXvFaRHIe8Rt3sUe/z48UWLFnErR48e5VaOHz/umq4A7gbCEwDS9vHHtGzZrQ9jYqjWZ2cDBpBWa3vqqYtqdZthw8jL1KEaRQAAIABJREFUS7AGQdIQnqATJ06cOHGCWzl27NixWveBBBABhCcApO3bb/mVQ4folVfo669vVUaMoOefF7IpkA2lhye+//57IfsAuHcITwBICcvSpk0tv/02uKSERo6kqVPp+nX+NQxD77xDTz1189tvrUZj6LPP0rhx5IFFRHA3FBKeqHOwe/HFF4XsA+De2cMTM2bMcHcjAFAPb75Ja9YE2P989Cj99BN16MDfo06rpYgI6tQps3nzioqK0H793NAnyIU9OTF06FB3N+Ja+L0H5EMhC2MB5CA+nnj5xHPnKCqKf9n775NWK1hTIG8KeY/APnYgHwhPAIictry86nCI6gu4q5SXU0wMLVliO3+eWrXynDaNpkwRuEOQMYQnACQG4QkA8dqwgebNG3HjBqvT0aRJ9OCDtVzj70+jRzMjR27+7bdBgwY1adJE8C5BzpQengCQHIQnAERq82b65z/tf1SZTPTll/Tkk+TjQ5WV1S6T++IncC+FhCewxg7kAydPAIjU8uX8yvbt9Nln5ONzq/LGG/Tcc0I2BUqjkJMncMcO5EMhC2MBJMBs1p0/3+TSJXrwQQoOpsuXa7mmc2e6fDnrm2/K8/MjJ06kbt0E7xKURSFvEBjsQD4QngAQhfh4ev75VpcvtyKiRYto/nyKiKCcHP5lERHUrFnxmDG5ubmRmOrA9RQdnujTp089P99sNp8+fbrh+gG4ewhPALhfRQX94x+Unl71oclE8+bRjBl06FC1y55+mpo1E747UDJFhyfi4uK4H3p4eFgsFvufVSoVy7L2P+v1+oCAAJf2B1B/CE8AuN/x47emOofr1+mrr2jePMrNZTUa1Usv0WefuaM5UDRFhyesHPn5+X369JkyZUpCQoLBYGAYprS09PDhw88++2z37t3Pnz8vcMcAdUF4AsA9Ll9ufviw15EjZDBQVlYtF2Rl0SuvUE7OzvXrc1JS6L//Jb1e8C5B6RQSnnCein333XebNWu2cuXKrl27enl5EZG/v3+/fv1+/vlnb2/vmTNnur5JgHpBeAJAaCxLkyfT/fd3++STkBdeoE6dSF3bg6CuXe3/36TX134BgOsp5D3C+WC3ffv2IUOG1PrSI488sm3btoZuCeAuITwBILRVq2jtWvrf+hxKS6O5c2ns2GrXBAbS7NnCtwbAM3XqVCXkJ5z/5lRaWpqfn1/rSwUFBaWlpQ3dEsBdQngCQGgxMfxKWhp9+y098IDpxx+ZggLvgQPpww+pdWt3NAdQjULCE87v2HXs2HHFihWnTp3i1U+ePLl+/fr777/fNY0B3DGEJwBczmrVJiYGnT5dtZau1l/7i4tpzpyM33+P3bCBNm6kyEiBewSoVWFhoRLyE87v2H344YejR4/u1atX+/bt27Rp4+XlZTQa09LSrl69qlKpVq5cKUCXAPVhD0/MmDHD3Y0AyFRyMj37bOi5c6FE9NFH9M471K0b8SJ0Hh61nwML4G725MRQuZ9c53ywGzFixIEDB5YsWXLgwIGrV6/ai1qt9pFHHnn//ffrWn4HIDyFLIwFcA+LhZ5+mpKSqj60Wunzz2nJEgoMpOLiW5fNnEktW7qlQYDbU8gbRL3SSf3799+9ezfDMNnZ2ZWVld7e3qGhoWokm0BkEJ4AcKHExFtTncO+fXTmDP3rXyUHDuhatPB6+WWaMMEdzQE4p4TkBN3RkWIVFRXFxcUtWrQIDAx0XUMAdw3hCYAGpysq8ggNJaLaN6jLzqbwcFq37tCOHVFRUa0RkgARQ3jiloMHD/bo0SMgIKBz587Hjx+3F0eOHLlv3z5X9gZwZxCeAGhIMTEUHj7w2WfbPfggPfkkNW1KNZ9k/W+DOgDxU0h4wvlgd/LkySeeeOLy5cvc5XT5+fmnTp0aPnx4fHy8K9sDuAM4eQKgwRw7RuPGVR0OxjC0Ywe9+SZNnlztGh8fmj/fLd0B3AWcPFHlww8/DA0NTUpK2rBhg6MYFBR09uzZ0NDQf/3rXy7sDuBOIDwB0GD++1/63xHhVU6fpmeeoeXLzV27VoaE0FNP0fHj1LGjm/oDuGMKeY9wvsbu+PHj7777bsuWLXNycrj14ODgyZMnf4aDnEE0EJ4AaDBXrtRSTEujadNyRo06e/bsk08+KXhPAPdEIeEJ53fsSkpKWrVqVetLzZo1Ky8vb+iWAO4SwhMAd4llaePGxuPGDX7nHXruOTp/nmrdfP6++wTvDKDB+Pv7KyE/4XywCw0NTU5OrvWl2NjY5s2bN3RLAHcJ4QmAu/Txx/Tcc9rY2IDMTNq4kXr2pEGDSKerdk3fvtS7t5v6A2gACE9UGT58+OrVq0+fPs0tFhUVzZ0795tvvomOjnZZbwB3BuEJgLtRXk4LF1armEz0/fe0fTtFRZFKxep09PzztGULYftSkDKFhCec/1e6ePHi3bt39+7du0uXLkQ0Z86cOXPmJCcnm0ymsLCwBQsWuL5JgHpRyMJYgAaWmMjPSRDRmTM0eDCdO7dv+/awiIgIHAsO0qeQNwjng11oaGhcXNyiRYs2bdpERAkJCUTUtGnTSZMmLVq0CEuaQDwQngCor2PHWm3c6BMURBMnUq0/xv9XtOl05OkpaG8ArqGQ8ES97qsHBwevXr161apVeXl5ZWVl/v7+ISEhru4M4E4hPAFQL6+8QuvWtbP/+dNPacUK6taNqq+3oTFj3NAYgCspITlB9Vljt23btgsXLhCRSqUKCQlp3749pjoQJ4QnAJz76Sdat+7Wh2YzvfUWffopRUXdKr74Is2dK3xrAC6F8ESVcePG7dixQ4BWAO4RwhMAzu3Zw6+YTJSeTqdPF2zbdnzWLLp4kb7/HjkJkB+FhCecD3b9+/c/ePAgwzACdANwLxCeAKgdy2orKqr+XFlZywWVlaRWW3r2vNG7NzarA7lSyHuE89/Jfvjhh+nTp0dHR0+YMCEyMlKv1/MuaN++vWt6A7gzCE8A8JlMtHBho1WrRpeX05w5NHcu9e1Lv/7Kv+yhh9zRHICgEJ6oEhoaav/DH3/8UesFLMs2ZEcAdwvhCQC+WbNoxYqqexS5ufTWW7RqFfXqRSdP3rpm2jTq3t097QEISCHhCeeD3bhx47RarUajUcINTJA0hCcAqjEaqeaKopUrKSGB1q7N27JF26hR4Pjx9NRT7mgOQGhKSE5QfQa7jRs31vVSRUVFWVlZg/YDcPfs4YkZM2a4uxEAcUhLq2Xn4atXSa2madOSoqKCgoICO3VyR2cAbmBPTgwdOtTdjbiW8/DEbWzdurVbt24N1QrAPVLIwliAOrEsbd7c9t//bvH55/THHxQeXku4tV078rinn/wAEqWQ94h6Bdpv3ry5cePGa9euWa1WR9FoNO7YsaO8vNxlvQHcGYQnQNFYlp55hjZvrloW/dNPNHUqvfwyrV1b7bIpU9zQG4AIIDxR5dq1a7169crPz6/lk9Xq+fPnu6ArgLuB8AQo2qZNtHlztcrKlbR3L3l5sWvWqEwmCgykDz7AYAeKhfBElXnz5hmNxpUrV3bo0GHQoEFff/11y5YtDxw48P33369bt27IkCECdAlQHwhPgKLFxtZSPHGCli0r+uCDwzEx0S+/7Ilth0HBFBKecL7S4tChQ1OmTJkyZUrfvn2JqFOnTkOGDFm6dOmOHTuef/75I0eOuL5JgHrByROgOEajf3Y2mUxERJ6etVxgL6rVxkaNSAGriwBuAydPVMnOzm7bti0ReXh4EJHZbLbXH3jggSlTpixcuNCl/QHUn0IWxgIQEZlM9Pbbfs2aDXvnHe/gYJo5kwYOrOWyRx8VvDMAkVLIe4Tzwc7f3z83N5eItFqtn59famqq46WOHTvGxcW5sDuAO4HwBCjInDm0fDnZA20WC33xBcXH06RJ1a6ZO5d693ZLdwAiNHXqVCXkJ5wPdgMGDFizZs2BAweIKCoqatWqVY4k7N9//63T6VzaH0D9ITwBSsEw9PXX/OJXX9G6dfTXX5kTJuS8/jodO0YffeSO5gBEyt/fXwn5CecLaT/44IOBAwe+++67cXFxr7766qRJkzp27NijR4+0tLSEhIQXXnhBgC4B6gPhCVCK/HyquTn8zZtUUkKDBl339vb29g598EF3dAYgXgoJTzgf7Hr16nX48OGTJ08S0UsvvXTlypVly5bFxMSoVKqRI0cuW7bM9U0C1AtOngDZYln68cc2P//M2mz0/9m784CoyvUP4M/sAwyLO6CgKCgquSRqiYb7WqZpmrcyf5Zdy3Ipvba4m+WSbW551XIpKzWXksrMVEhxB3EBUcEFFURZB2aY7fz+GO5wOAyLypyZM+f7+ec2D2fs8abMyznv931Gj6axY6luXeJ8SgUGkq+vk/oDEACRTJ6oUfS9U6dOnTp1IiKJRPLxxx/PmTMnMzOzUaNGHh4eDm4P4AGIZGMsiNGoUbRjRz3rP+/bRz//TNOn0wcflLtmxgwnNAYgHCL5gHiYwTJqtbpZs2ZY1YGrQXgC3NPevbRjR7lKTAw1b06ffcY0bkxETFAQffUVTZninPYABEIk4Ynq79j17du3iq8aDIZYu6diAvAO4QlwT0eP2inGx9MXXxRNmPDHL78MGT4cP2kDVEsMyQmqycLuwIEDlX1JJAETEAqEJ8BtSO7caZCcLA0PJz8/8vS0c8X/ihYMkwCoGZGEJ6p/FGusoKio6Pz589OnT+/YsWNycjIPXQLUBCZPgDsoKaFx4xQhIb3mzfNp356efdb+WXT9+vHeGYCwYfJEKXkFnp6ebdu2XbZsWbdu3WbOnMlDlwA1gfAEuIMPP6RNm4hhSl/+8gtt2EALF5ZNDJPJ6IMPMFIC4EGJ5DPike7hP/vssyNGjFizZk1tdQPwKBCeAHfw/ffcys6dtHEjPfvszY0bGbM5eNw46tDBGZ0BCJsYkhP0iAu7wsLCvLy82moF4BEhPAGCZzDQ3bvcotFImZn02GNZY8aYzeZgrOoAHopIUgHVL+zsLt2MRuOFCxf+85//hISEOKArgIeB8AQIT1ERLV0atmOHRaejgQNpzhxq1Yo4e5d9fSk42En9AbgPkYQnql/Y1alTp4qvbtmypfaaqRGGYdLT09PS0goLC4nI19c3LCwsKCiI5zbABWHyBAiMxULPPksHDpQeVbJmDf3xB82fT2PHlrts/nySPsyZowDAhskTpYYMGVKxqFAoAgICRowY0adPHwd0ZV9ubu6iRYu2bNlyt8KjiuDg4Ndee2369Ok4zEnMRLIxFtzHvn3EOU8qPZ1u36ZffmEWLzZcuCBv3lw2dSq9/LKT+gNwKyL5gKh+Ybd3714e+qjWnTt3oqKi0tPTw8LCBg8e3LRpUy8vLyIqKCi4evXq4cOH58yZ8/PPPx88eLDqW4zgxhCeAIFJTLRfnDnTOGDAnt27+/fv7+fnx3tbAO4J4QnXMnv27IyMjG3btj3//PMVv2o2m9euXfvWW2/Nnz//iy++4L89cAUIT4Cr0+tp69b2u3Z5pqTQ66+Tv7+da+wWAeCRITxRqkOHDiqVqoY3MI8dO/bILdkXExPz8ssv213VEZFMJnvzzTdjY2N37tyJhZ1oITwBLu3uXerenS5fbkVEe/fS8uX0zTfk60v5+WXXyOU0YoTTOgRwawhPlMrMzCwoKNDpdNaXEomE+d/JmR4eHgaDwYHdsdy/f79FixZVX9O6detdu3bx0w+4IIQnwKW9+y5dvlz2sqCA3nmHtm6lV1+lzEwiIi8vWrqUund3VoMA7g3hiVLJyclDhw5t3779q6++2rp1a7VaXVBQkJiY+OWXX2q12u3bt/v4+PDQaGBg4NmzZ6u+JiEhITAwkIdmwDUhPAEu7eBBbuXGDQoLoytX0n76qTgvL2LsWKpf3xmdAYiCSD4gqo/Qv/vuu6GhoStXruzYsaNarSYiHx+fp5566ueff5ZKpe+++67jmyQiGjZs2Pbt2z/99NOSkpKKXy0qKpo7d+6ePXtGjx7NTz/gghCeANdSVOR57Fj9o0fp1i0iIrPZzjVmM3l5aSMjczt3xqoOwKHeeustMeQnapSKXbx4sd0v9e3bd+nSpevWravtruyYN29eXFzcjBkzFixY0KVLl6CgII1GwzCMVqu9fv36iRMniouLe/ToMWvWLB6aAdeE8AS4kIMH6eWXg2/dCiaijz6iDz+kHj1o+/Zy1/j7U8uWzmkPQHwQnihVUFCQad3/UcHdu3fz2dt+HcnPzy8+Pn7VqlWbN28+dOiQmfWzr0Kh6NSp0/jx48ePHy+zzckG8UF4AlzF/fs0ejRlZ5e+NBho7lxav55sd++ISK2mb7/FycMAvEF4olSbNm1WrVrVp0+frl27sutHjhz55ptvwsPDHdYbl1KpnDZt2rRp0/R6/c2bN62TJ3x8fIKDg5VKJW9tgMtCeAJcRWxs2arO5vBhSk6mdevSY2IadujgNWkSNW/ujOYARArhiVLz5s177rnnnnjiiZCQkBYtWnh4eOh0urS0tLS0NIlEYv2/iU8Mw9y+ffv69eu2kWIqlQojxYAQngDnysurm5oqb9SImjShCtNxiIiyssjbm95552STJj179vTCtgEAfonkA6L6hd3QoUMPHDjwySefHD58OD093VpUKpW9e/d+//33+/bt6+AOy2CkGFQN4QlwmrlzacmSp6zRruhosnvb+PHHeW4KANjEkJygGk6eiI6Ojo6Otlgsd+7cKS4u9vDwCAgI4Hk3G0aKQbUQngDnWLeOFiwoe3n4MJWU0KhRtG1bWTEw0P5qDwD4gvBEKYvFIpVKiUgqlTZu3Dg7Ozs2Nlar1fbq1Ss4ONjxHZbCSDGoFsIT4BxbtnArx47Rhg3Utavu++/N+fmafv1o9mxq0MAZzQFAKZGEJ6oKZMXGxnbs2PEg61DN/fv3h4aGjhw5cty4cS1atFixYoXjOyxVk5Fio0aN2rlzJ28tgauxhiec3QWIgMWiuHq1TnJy6TSwGzfsXHP7Nr3zzvXvvz++di2tWUM4Ox3A2b7++mv+gwH8q3Rhd+bMmUGDBiUmJt65c8da0Wq1//rXvwwGw/Tp0xcuXNiwYcOpU6eePn2an0ZrOFIsKyuLn37ABSE8AXxITqbOnYP6939y+nQKDKSlS+mxx+xcFhHBe2cAUBWRfEZU+ij2008/NRgMMTExgwcPtla2bt167969FStWWLcfjho1KiIiYs2aNevXr+ehUYwUg2ohPAEOV1JCI0ZQcnLpy+JimjmTFi2iffvIaCy7bOJE8vd3SoMAUBmRhCcqvWN39OjRwYMH21Z1RPTbb78plcqXX37Z+rJly5b9+/ePi4tzeI9EhJFiUAMIT4DDnT5dtqqzOXGC/vqLoqONGo2peXP65BP68ktnNAcAVfH29hZDfqLSO3aZmZnjxo2zvWQYJi4u7oknnvD19bUVW7VqdbDiWGvHwEgxqBbCE+AQSUlN9u/3unGD/vUvsvsHLCODnnqKDh2K2b07MjKySZMmvLcIANUTSXiiqlQse2F78eLFnJycqKgozgUGg8FRrZWHkWJQLUyegFpmsdBrr9G337azvpw3j5YssXOZ3T12AOBixD55wt/fnx1EOHDgABH16NGDfU1WVhb7Bp6jYaQYVE0kG2OBP2vW0Lfflr3MyqL33qORI2nHjrKilxfNnMl/awDwoETyAVHpwi4iIuLnn3/+5JNPZDKZ0Whcv369l5dXz549bReYzeaYmJjWrVvz0SYLRopBZRCegFq2dy+3kplJ48dTRIRx82ZLTo4qKooWLiQeR2YDwEMTSXii0oXdK6+8MmrUqL59+44cOXLfvn3nzp2bPHmybWCXxWJ57733bt68OXnyZL5axUgxqAbCE/Co/vmnzpo1T1y+TGfO0JQpZHdHTnExzZ178+WXU1JS2PEyAHBxYkhOUBULuxEjRowcOXLHjh2HDh0iok6dOn300Ue2rw4dOjQmJiY0NHTixIk8dEkYKQY1gPAEPJKvv6Y33vAi8iKikyfpv/+lQYPoxAnuZZ06OaE3AHhkYg9PSKXSbdu2HTp06OLFi0FBQYMHD5bLyy5u0qTJwIED161bp9FoeOkTI8WgeghPwMMrKKBp08pVsrKouJj8/Skzs6z4n/9Qs2b8dgYAtUPs4QkikkgkvXr16tWrV8UvrVq1iuf8aU1GisXGxu7cuRMLO9FCeAIejNnsceuWXK0mIkpMJL2ee0FiIp09S0uW5OzbpwoM9Hr1VRo1iv82AaBWiOQDoqqFXRX4P1WkhiPFdu3axU8/4IIQnoAH8NNPNHVqN+vduG7dyO6NXi8vatiQli8/1qdPeHh48+bNee4RAGqRSMITlU6esOvTTz/t3r27g1qpGkaKQbUQnoCaio+nl18ue8Z69Ci9/z5V/O4xYADPfQGA44hk8sSDLeyuXLly5MgRB7VSNYwUg2ohPAE19e235Ua7EtHlyzRtGtWrV1bp1YvmzuW5LwBwnJycHDHkJx7yUSz/MFIMqoXwBFRq/37fr77qc+kS/fkn/ec/lJ5u5xpPT0pNzdm06fbZsxH/+hf160fi2JEDIBIIT7gWjBSDaiE8AfatXUsTJ6qIVER0+TJ9/z0NH27nsjZtqG7d4uefvxISEtG/P99NAoCDieQDQjALO3LMSLHbt2+PHDmy6om32dnZRMQwzMP9K4A3CE+AHUYjvftuuYpeTzdukEZDWm1ZMSqKnLSBGAD4IZLwxIMt7BYvXuz0Z521O1Ksbt26o0eP1lc85oDl+PHjN27cEMlKX9AQnoBSMTEhW7cqvL1p/HhSqaioiHvBxYv01180Y4YlPp7x9JSNHEmLF5NcSD/oAsCDEkNygh50Yefn5+fn5+egVqrliJFiarV6ypQpVV+zdu1anKIiCAhPADEMvfACbdvWxPpy7Vr755jUrUtdu1Js7KH9+wODgsIx7BVABMSQnKCaLOwYhtmxY8fmzZszMjKMnBwZERGdP3/eAY1xYaQYVAvhCaDvv6dt28pVvvqKwsMpJaVccehQ6/8y0gc7GQAAhAvhiVLLly+fMWMGEXl6eioUCse3ZB9GikG1EJ4Qo8JC5XfftT18WKrT0Qsv0KFD3AtMJhoxgrZvp9TU0srAgbRoEb9dAoDzieQDovqF3ZdffjlgwIDVq1c799R1jBSDaiE8IToXLlC/fp537rQloh076KOPqFMnO5fVr0/nzuXt3n350KHOr7xCXbvy3ScAuACEJ0plZWXt2LHD6bN0MFIMqoXwhOiMG0d37pS9vHKF7G6O7tGDlEpDz57pRJ2xqgMQK4QnSjVq1MgVTvrASDGoFsIT7m/v3qZbt0pkMnrpJerShU6f5l5w5Qo9/TTt3VtWef99+7fxAEBkRBKeqH7j8JgxY7Zs2cJDK1XDSDGoljU84ewuwGFeeYWeecb/hx8affcdDRxIU6dSxZ85jUbauZO2b7/93HPZ48bRwYP08cfO6BUAXM7XX39tzU+4t+rv2M2ZM2fkyJEvvvji2LFjg4ODK+YnQkNDHdNbORgpBtVCeMKd/fILbd5crrJ5MwUElHsUS0RdupBCQSNHXm3UyNfXt0G7dnz2CACuTCQfENUv7GzPpLdu3Wr3An4e1GKkGFQL4Qm38vffyiVLBp0/77FuHU2fTrGxdq7p25e2bSPbXXwfH1q1is8eAUBAEJ4oNWbMGKVSKXeBM9kdMVIM3AnCE+5j1y567jkZkTcR3b5Nf/1Fzzxj57LmzenChZIVK+6ePBkYHS17+20KCOC7VQAQCIQnSlV2o46IioqKrKsrnqnV6rCwMCIyGAxnz569efNms2bNQkJC+O8EXArCEwKm13tevCjX6ah9e/Lyov/8h3tBfLydd/XsSS1a6ObNi//zz2HDhsnwAx4AVA7hiert2bPn8ccfr61WqvbRRx8dPHiQXVm7dq2/v3+XLl169+7dvHnzyMjIxMREfpoB14TwhFD9+SeFhbV66aXHJkygpk1p40a6epV7zb17NHZsucrUqdSzJ18tAoDgITxR5t69ez/++OO1a9dMJpOtqNfr9+7dq9VqHdZbObNnz545c2avXr2sL2NiYiZOnKhSqYYPH96wYcPz588fOXKkZ8+ep0+frva4O3BXCE8I0o0bNGoU5eeXvrx/n/79b/L0pKKicpep1bR+PY0de3vzZolUGjBuHEVH898sAAiXSD4gql/YXbt2rUuXLtnZ2XbeLJfPnj3bAV1Vb9q0ab6+vvHx8a1bt7ZWdu7cOXLkyEWLFn3zzTdOaQmcDuEJYTh3TrpkSd/4eM3mzTRtGqWmlq3qrAwGCg+npKRyxWefJYWC+vS5qdHIZLKAyEg+WwYAN4DwRKlZs2bp9fqVK1e2bt26T58+69evb9KkyaFDh7Zs2bJhw4YBAwbw0CVHdnb25cuXP/jgA9uqjoiee+65Z5999s8//+S/H3ARCE8IQHw8RUdLjMa6RJSWRjEx9lMR7dpRYCD98Ufpyz59aPVqHrsEADeE8ESpuLi4SZMmTZo0Sa/XE1Hbtm2feOKJAQMGjB49uk+fPr/88ktUVJTj+yzH2gl7VWcVERERExPDczPgOhCecEXp6XW2bVPfu0eentSlC02fTkZjuQv+/tvOuzp1oqlTS06fPvXTTx2HD/d88kl+mgUAN4bwRKk7d+5YB8VKpVIiMhgM1nqHDh0mTZo0d+5ch/ZnV2BgoK+vb0ZGBqd++/ZtV1+Pa7Was2c1Z88SX3sTRQXhCZfzzTfUunXgggUtVq+mrl3p9dcpIYF7TVERRUSUqzRvTuPHE5GlTZtbkZGWxx7jq10AcGciCU9Uv7Dz9vbOysoiIqVSqdFo0tLSbF9q06bNqVOnHNhdeTdu3Dh16tSVK1dyc3PffPPNDRs2FBcX276akpLy008/8X/78AHs3EnNm7d5/fU2r79OzZvTzp3ObsjdIDzhfLdvNzh3Tp6aSgxDaWn05ptw7NDcAAAgAElEQVTEngG4bh2p1dy3SCS0axe99VZJkyYlAQE0bhzFxpKPD59dA4AYiOQzovpHsT169Pj666+7d+/es2fPxx57bNWqVSNHjtRoNET0999/q1QqxzdZ6ocffvjhhx/Yld9//33EiBFEtHXr1tdff12n0zkrzFG9lBR66SXS6UpfZmfTSy/RmTMUHi7JyqqXmiq5d4+aNHFqi4KH8IQzWSw0ZQqtWfOU2Uzz51NUFA0bRhUnOzdqRLm55Srdu1NoKK1YcXH8eJ1O161bN95aBgBRQXii1AcffBAdHT19+vRTp05NmDBh/Pjxbdq0iYyMTE9PT0xMfPHFF3nokoi+/fbbPJb8/Py8vLw6depYv5qXl+fn5/fjjz927tyZn34e2J49Zas6K52Otm2j5GTvH3/sQ0Rz59L//R+tWkU8rpXdDMITPJNnZXnduEFGIykUtHw5rVxZ9rUjR+j2bTvvad2agoJo//7Sl+3acSfAAgA4hqtv1qol1S/sunTp8s8//5w4cYKIxo0bd/ny5S+++GLXrl0SiWTo0KFffPGF45sk67+6iq+OHTt24sSJ1l2ALsruh9yPP1Jycuk/Wyy0YQP5+tLy5USkyM+X8TKE150gPMGf1FT6v/8LOXo0hIhmzqTly6n83XQiovR0O2/s0YOmTbMcP35qy5bW/ft7Dx5MLjCuEADEAOGJMp06dXrjjTeISCKRfPzxxzk5Oenp6UVFRXv27Klfv76DO6wRjUYjlUrv379/5coVZ/dSifbt7RQrdrtlCx0+TG3bdn366ceeeoq6d6fz53nozj0gPOFAxcU+aWky688nOh0NG0ZHj5Z+KSeHJkywv4x77rlyLyMj6c03iYiJjLz21FPG7t2xqgMA3iA8wVVYWHjhwoW8vDy1Wt2sWTMPDw/HtfVwli1bZp0h64rGjOFG/1q14p77QET37tHTT9PFi6UvjxyhIUNse5LknIe5UJ5INsY6wbJl1LBh9NSpjZ98knr3pj17yu40W5lMVPEbgkxG335LO3bkDR2aHR1Nn39O//yDnQYA4Cwi+Yyo0cLu8OHDkZGRPj4+ERERx44dsxaHDh164MABR/bmXjw86OBBmjRJHxysDw6mSZMoLo7q1uVe5ufHPQnlxg3atYvee69eixbPjRsnDQ6mtWt561pYEJ6oHQaD6sQJ/+PH6do1IqIffqD//KdswNfBg/TBB3be1bAhKRTlKjNnko8PjRhx6+OPL8ybR1OnYlUHAE701ltviSE/Uf3C7sSJE/37909NTWUPmcjOzj558uTgwYNPnz7tyPbcS/36tHJl0vbtSdu308qV1KABzZnDvSY83M4b16yhJUskBQVERLdu0cSJtGmT9Svy4mJJxdt+YoXwRC1ITKSICP9RoyI/+ohatKDJk2njRu41dp+6du1Khw7RwIH6unWN7drRmjW0YIHj2wUAqClvb28x5Ceq3+CyYMECf3//I0eOyOXygIAAa7FBgwZnz57t3LnzwoULd+/e7eAmiYgiazAa8tatWzx0UpsmTyZvb/NnnzFpadJWraTvvUepqRQfz73szBlu5fPPKSiIpk596tw5RiajwYNpxQpq2pSfrl0WwhMPjGFo166WP/3kExxMr75KISH0/PNlWz8tFlqxgho1svPGDh0oMbHspacnTZ1KrVvT77/H/PxzVFSUv78/H/0DANSYSMIT1S/sjh07Nn369CZNmmRmZrLrDRs2nDhx4rJlyxzWWzkJCQlEpOA86ynPZDLx00ytkUho/PiikSP/+OOPoUOHqtVqunmTvviC2H/4goPpxg3uG1NS6OmnreenSMxm+vVXun6djh8ntVqan++dkUEGAymVPP5OXII1PPHOO+84uxGBMBppwAA6eDDE+vLLL+m99+wEeuymszdupPXrzT/8INFqpU88QUuWUIURfwAALsWanBg4cKCzG3Gs6h/F5ufnBwUF2f1SQECAlq/RWDNmzPDy8jp//ry+ctOnT+enGQcKCqJ9+6hbN0YqZeRyGjKE9u6lise4aDTcU/GSkmjXLnruOf82bfpNmUJ169LSpbx17SJEsjH2IVks9M039Z59tv+kSTR8OJ05Q599RgcPll1gNNKSJXbe6O3N/SHhpZeofXtasSItPv6vX36hQ4eoa1fHNg8A8MhE8hlR/cLO398/mZOA+5/Y2NjAwMDabsm+hQsXhoaGjhkzxuj2W8oiI+nIkfi//kqKj6e9e+mxx2jsWO41dp+OzZ9Pu3aV/nNREc2cWbo76sqVgNhYzyNHyva/uymEJ8rJza2TkOBx7hzp9URE8+bRq68qT57UZGbS7t305JO0Zw/3LQYDVfyuFxVFf/5J3bubVSpTkyY0axb997989A8AUKsQnig1ePDg1atXnym/zSs3N/fDDz/89ttvhwwZ4rDeylEoFN9///2FCxc+sJvIczsWhYKxHfG1ciW98UbpXRNfX1q2jPr2tfOeS5e4lfXrado0atWq/SefBI4bR61a0T//OLJrJ0N4oszKlRQU1OHdd0NeeIFat6Y//qCPPy53gcFAqal23jh4cLmXPj40Zw5FR1Nc3G/bt98+coQWLrRzsgkAgMtDeKLU/Pnzf//9965du7Zr146I3n///ffffz85ObmkpCQ4OHhOxVynw7Ru3TozM7OKjXSDBg3y8/PjrR/+eHnR6tX3Zs+O37NnyGuvSeVyOneO/vvf0jsxViEhdrKKFy7QkSNlL2/dolGjKCWFLl2q+9//dkhLo7Q0mjDBbT6nxRueOHjQ+8sv+yQnS37/nWbOpFu36O23y7567Rr9619kNnPfZTBwK0olrV5N+/bpN2ww3r7t3bMnzZlDLVo4tnkAAF6IJDxRo0ex1imx169fJ6LExMTExERvb+833njj5MmTjew+E3QYHx+fuhXPfvuf6Ohodx48oFDo6tYt3W/32GP0yy+l29VlMho0iLZts/sWbuXOHZo9m5580mf9+iZ//01TplDHjpSfb/2irOLIdkERy+SJmzfr//2374EDlJ1NRLRpE/Xurdyzp15qqmTjRurQgVat4r7lf2dcl9OiBXXqVPZSJqOlSyk4mCZMyPrpp8OrV9PmzRQa6rDfBgAAr0QyeaJG83waNmy4evXqVatW3b17t7Cw0Nvbm+f1HNjRrx9dvBi3Z0/9oKDWjz9ORDR+PH3zTblrGjQo/exnW7Om3M2bS5do/nzq0YM++GBISorFy4vGjqVPPiFfXwf/Bmqfu26MlbNvzS5fTh9+2Mq6BJ8/n1avpilTyl1dUkKxsXZ+laAg4tzOfOEFeucd2rQpY+dOr8aN60yYQF261HrzAAAuwi0/ICp6gEGNEomkUaNGWNK5FKO3N6NWl75YuZL8/c3r10vv35dERNCCBfTXX2XTycreUyF98ttvtGIFmUxEJC0qojVr6M4d2rWLkpLqbdigun2b8vJoxAg74VwX427hCbOZFi/WfP75c/fvMzNm0IwZ1LkzsaPfhYX02mtU8T6r9SxrjsWLafny0jMRpVJ67TWaPp1kMnrttQtBQS1atKiDm3MA4NbEkJygKhZ2NT8TTo4x3i7Cw4MWLcqaNOl4fPzwESOIiDp3pp9+ort3y6555hn69VfuG7OzifOfe/duWrCAFiyoZzbXI6IdO6hXL9q3jxQKysmpd+mSrFkz+8lcpxJ2eEKrpaVL2+zcSQxDw4bRzJn0+ec0b571B0zJ3bs0YwZFR3PfZffpeYMGZLEQ++DJgQNpzBh64YV7+/enxsZ2e/11HGcNAGIjhuQEVbGwq/ooYDbG7vml4DyM7W5zQACdOUMffVRw4IC8QQPPV16h55+n5s0pL6/6X+Wjj8o9sT14kL74grKy6KuvehmNNGsW9etHmzeTdcAAw0hc4HRoIYUn0tPln3/e48gRj/37aepUatmS+val48e9rF+9eJH27bNzVvDJk3Z+qSZNKCOjXOXZZ2nmTJo9W/fXXxJvb/WoUfTeeySRkERibNcus6gIqzoAECGRhCequtkmk8natWvXqlUrLN0ErHFjWrPm6B9/hIaGhlqftW3YQC+9VHa+cY8e5OFBf/7JfWPFJ7bffkvsEw3376eXXqJNm+idd9r8+qvEaKQnn6TPPqMaDH9zENedPLF9e9B335mLiuj55+nVVykxkZ56SqbTBRDRmTO0cSO98w4dP17uLXanMBcX2ynOnUvLl1NKSunLPn1o6VLSaGjz5mMHDzZq1KhNmza1/NsBABAgkUyeqHRhN3Xq1K1btyYkJOTk5IwaNeqVV15p27Ytn52Bozz3HCUn52zYcP/y5bDRo2noUNq7l7uwi4ig8+e5b6w4ivfAAerfny5eLN18FxdH/fpRQgKdOCFbtGhEcjIFB9OUKfTmmySTEZHEaJQ68nxplwhPXL4s+fTT3ocPe4eH05Qp1KsXTZhA69eXZrkPHKCtW6mkpNzgEJOJ7Aa1lEruiSQNG5K/PyUllVUGDqRXX6VXXinYs+fSgQOdXnxR2r177f+mAACEz/kfELyodDv8559/fuvWrV9//bVz585fffVVREREZGTkihUr7t27x2d/4BBNmxaMH3/pxRdp2DCSSmnoUFq/nho0ICKSSun552nTJjtHpdh92MoJZ+Tl0bRpNHq0JClJZjTKrl6lyZPp44/p0iUaMKBD9+5devWip56yzY+XXroUeOaM1Ha36dE4PDxx61bDxETFhQtksRAR5eXR5MmhvXtHP/ssDR5M587R2bPUvj3997/1L11S7dlDvXvT++/T+vXlfpHYWDt34+zGHSpmVCdNon/+odmzCzp0KOzShZYtoz17SCIhhcLYq1d6r17UrVtt/V4BANwMJk+QXC5/+umnt2/fnpmZ+fXXXyuVysmTJwcGBg4fPnz37t3uP9pLVF59lbKy9n/zzc2LF2nbNnr8cVq2zHqbrVRUVLkzz6zs/vRz4AC38vHHNGAA/fmnxGSSWCwUF0f9+9OlSzRkiPrxx7svWeIRGUmDBlF+PpnNtH592OzZIbNm0ZYtpesnIumNGw2SkyV37lT9m3j48ERmZv2EBNX582WL199+azhjRqf582nhQiooIIuFpkyhpk2jFixoOGAAdelCqan03HO0YoUiM1NeVES//05PPUVvvcWd4fv553Yb5VakUlKpylUUCvrsM5o+naypZ09PmjWLPvyQvL1pwYJzK1deXbuWpk/nTnEFAIBKiGTyRI0OsPDz8/v3v/999OjRy5cvv/fee4mJicOHDw8MDJw8efJpuzuBQIgkEl29eoxtCsWUKXTsWM6//31z6FDasoUOH6Z587hvGTrUzq9TcSKtXk/Xr5erZGfTv/5Fv/1WVvnjD5o0iQYPpgkT6v31V90//qCxY+mFFygnh555xrdDh17z5smbNaPx48lgoEuXaPjwbs88E963L02cSNa7yLt3+4wZ02TqVHrxRbpwgYjozh16882ub7wRNHYsffUVmUyk09GHHwZ27frMyJHUtSvt309ENHs2BQd3nT07cOhQ6tCBzp2jBQtoyBDvXbsaHj9Oc+ZQ+/a0eDF99VVZmuT0aXr6aTp4sNxvKi/PTrjBbmo1KIhbefJJWreOfHxKX2o0tHYtde5My5YV3r796+rVuqwsWriw3FIbAAAeRE5OjhjyEw92UkloaOiCBQvmz58fFxc3e/bsFStWrFixAtEKtxUZec/H5/r160H9+hER9e5N+/fT/PmGhARp48byN96gsWMpMrLcKDO1murWpdu3y/06Uqnt3lsZ9kYxq+3buVvKtm+nO3fK5tsyDH37LXl60vbtdPeunIgKCmjtWkpMpJEjacaME0QDiPRbt9LPP1NMDL38Mt25U/rT2YkTdPIkSSS0ZYvMVhk8mGbOpEWLyv6NFy7Qs8/SjRvl2rh2jVau5HZ7+bKd/8fsHvUnl3OfYs+eTcuXl+1iDAmhjRspNJQGDUr97jvGYmn18sulT8aJSC7X1auHJR0AwCMSe3jCLovFsn///o0bN/76669FRUWhoaFjx451UGfgivr2pb59f9mxo0ePHqVHVf/6K732Gh07RkTUtCmtXk0nT3Lv7XXuzI18kr0dexVHlxKVm3VrtXkzFRaWqxw/TgkJRCS13YIuKaHXXiPOo9vvvrPTw7p13GLFkbtEVMMf8po3L71ZaPP44/TKKzR9elnKeNIkGjeOXnrJuHv3xV9+CevTx3P06NLnrfXr50dFWSyWslUdAADUEpGEJ2q6sLt06dLGjRu3bNly69Ytb2/vUaNG/d///V+PHj0c2hwIQNu2FB+fHBdXcOdO11GjiIj696fcXFq1ikwmkkho9GhatIh69Ch3G8/PjzQa7ulrPj52MgQV7wdrtXbaMBiIqDtR2aqqYobXrpoc6UdEvr7lznkmIqmUGjcuN6RLLqfPPqNly+ivv0orYWH0/fcUHk79+9/auNFQUBAyblxpJEIutzzzzCWLpdmAAWSbHQIAAA4jhuQEVbvHLi8vb+3atU8++WR4ePiSJUvCwsI2bdqUmZn5zTffYFUHNuY6dfS2QRRyOX3xhSkzc/+SJXmpqfTDD9S8Of32Gz3xROkF7dtTTAwtX879VSZP5lbkcvL05Bbr1ausDQlRiO2FbbNg1ez+ahXn5I4dyw03vPsu/fortW9f+rJuXVq3jvr3p/37mbi4k//+d/4PP9D58xQeTkQUHn5vzJiMMWMwjBUAwFnEHp7Yt2/fmDFjAgICJk6cmJWVNW/evLS0tIMHD44dO9az4mctAIevb26zZoxt2dS+PcXHnz106OTvv1NiInXrRqNG0e+/m3v1KmrUyNyrF/32Gy1cSB9+WLaZTC6nRYto4ULurzxzJvcoFn9/6wLLQlR2+F6fPtw3NmhQtri0efdd0mjKVV59lb7/vtzabuRIWryYjhyhZ54pbtDA0L49rVlDn3xC7dvTmTNX9+07sXYt3bpF48aVXh8Vld67t6FPH0RWAQBch9jDEwMHDpTJZF26dBk4cGCnTp0Yhjl//vz5iofWEj399NOO7BDch9nb28Re6wwcWPLUUzF79w4ZMsTLy4uI6KOP6Pnnr23cKFMqg155hdq0IYahevXMX3xhSk9Xtm0r+eADGjKEgoLo7bcpO5uIKDycNm6kunXpuefizp8fQKQnohEjaOtWWriQFi8u3cwXGEjffUetWtGrr9IffxAReXnRvHn07rs0aBDNmaP75x9pgwaqceNo8mRSKCg19e7GjfevXm09dixFRRERdepEv/yyb9euLl26NG7cuPS3IJUagoIKZTI8TgUAcHEIT5DZbI6Pj4+Pj6/6l0AqFmpT+/aZo0crFIog6yAsiYReeaVg6ND9+/cPHz68dITx6NE0bNiZ77/XNGzYctCg0pt8CQlFS5fS3Ll05gw99hgR0cKFNGFC0jff1GnaNOj550vvzP3++62EhNS4uF7//nfp09U2bWjHjr9jYtq0aRMS8r9nuQ0bFo4YcS01tbV1VQcAAAIn9vDEli1b+OwD4MGoVNrQUEWdOuxHt4+PH7/c17d0VWcVHHy3e3dVUBD7eaulfv2Cpk25e+YAAMCtiSQ8UenC7qWXXuKzD4BH9/CTJwAAwN2JITlBD3qOHYArs1gsN9nnjwAAAPyPGJITVMORYgCCcPz48ffee8/ZXQAAgCv6+uuvrfkJ94Y7duA+JBKJSPbGAgDAgxLJBwQWduA+unbt+tlnnzm7CwAAcAkHDhxgL+aGDRtGRLt372Zfo1ar3ewAFCzswH0gPAEAADZt27b1YE0hKikpISJV+SMRVG53QgIWduA+EJ4AAAAbf39/Hx8f28usrCwiamQbgOmmsLAD92ENT7zzzjvObgQAAPhjNBoLCwuTkpJsFZPJRESXLl1i35CbP3++RqPZtGmTE1rkERZ24D4QngAAEDSTyWQwGGwv9Xp9cXHxhQsXbBXrsKvDhw+z32WxWIgoNTWVXZRIJFqttri42FYxGo1ms9lBnbsOLOzAfSA8AQAgCEajUa/Xs++xEZHJZDp27FjFi9kLOytPT0+ptOy8NoZh6tSpExgYyL5GoVD4+/uzK+3atXvUvoUACztwHwhPAAA4l8ViMZvNaWlp7KJerz9x4sSJEyc4F6ekpLBfSiSSOnXqlM4E/x8/P7/g4GB2RSqV+vr6PkRvderUeYh3CQ4WduA+EJ4AAHAo9nNSItLpdCkpKdeuXbNVSkpKjEZjQkIC+zKGYTw8PGS20d5EROTj4xMWFsauyGSy+vXr137T/4PwBIDAIDwBAFArzGZzQUEB+0fle/fuWSwWziFwVnq9nv1SrVa3bNmSXZFIJCEhIUql0kHd1tCsWbOIaN26dc5tw9GwsAP3gfAEAMCDssYRUlNT2bvWCgoKcnJy7ty5w7mY85yUYZh27drVq1ePXfTw8FCr1Q7r9+Gxf4NuDAs7cB8ITwAAVK2kpESr1bJTCzqdjoguX77M/sHYYrF4eHhw7rH5+vp26dKFt1Zr3eLFi53dAh+wsAP3gfAEAABHcnIy+zZbYWGhyWSyHvNmxTCMSqWq+M2zZcuWnFtxQofwBIDAIDwBAKJlMBgKCwvZOVNr0CE7O5v9CFIikWg0Go1Gw36vWq3u2LEjb606C8ITAAKD8AQAiEdWVpZWq7W9LCwsLCoqun79uq3CMIxCoVCr1exnrCqVKiwsjHPkm0ggPAEgMAhPAIBbYhiGfc6I0WgkoosXL3J2xdH/NszZqFSq6OhofGO0QngCQGAQngAA95Ofn5+VlVXxnBH2PjmriIiINm3a8NWX8CA8ASAwCE8AgNAxDHPx4kX25AatVqtSqfz8/DhXRkREcG5BeXt789GiYCE8ASAwCE8AgLCYTKbc3FxOUSqVsoc0WLMOPj4+7GtkMlmdOnVE8myxtiA8ASAwCE8AgIAUFBQUFBTs37+fU7euP9jCwsJEMsDeoRCeABAYhCcAwGVZLJarV6+yF235+flKpZITUJVKpW3atJHLy306c+Y9wMMRyQ1OLOzAfSA8AQAui2EYs9lsDbRaKRQKLy8vs9nMuUyhUGAl5wgITwAIDMITAOAKiouLi4uLY2Ji2EWz2azVavV6PbsYFBSEZ6y8QXgCQGAQngAAV6BSqZRKZevWrdnFwsJCLy8vztNAkSw1XATCEwACg/AEAPBPp9OdPn369OnTnPqpU6fYL6VSab9+/Xx9fXlsDcpBeAJAYBCeAAD+qdXqgIAAdgaCYRi9Xu/h4cG+TCqVco4sAZ4hPAEgMAhPAIBD6XQ6o9G4bds2Tv3y5cuXL19mVxo3bhwVFcVja1A9hCcABAbhCQBwKLVaLZfLOSs2nU6nUqk4d4M0Gg2/rUH1RLKjEQs7cB8ITwBALdLpdFeuXMnIyLBVjEaj2WxOSkpiXyaRSCIjIyuO/AJXg/AEgMAgPAEAtUihUGg0moCAAFvFemRJxQCEl5cXv63Bw0B4AkBgEJ4AgIfDMAwRnTt3jl00GAwGg0Gr1bKLdevWbdq0Ka/NQS1BeAJAYBCeAICHYx3/kJubyy7KZDKTycQpMgyDhZ1AITwBIDAITwBADZnNZoPBYHtpvWP3xBNPsO/6SyQSjPZyJwhPAAgMwhMAUBMFBQV37ty5cuUKp75nzx5OJSoqqnHjxnz1BY6F8ASAwCA8AQA14e3tXb9+/ZYtW7KLer1erVZzrsSgCHeC8ASAwCA8AQB2Xb9+XS4v+7zT6XRUYUedh4eHSB7ViRbCEwACg/AEAHBYLBYiSk1NZf/UZzAY9Hp9YWEh+0ofHx/2ySbgfhCeABAYhCcAgMN6k6Zfv34qlcrZvYCTieSOLBZ24D4QngAQOb1eX1xcfOrUKVvFGn1NTEyUyWS2okQiadGiBWZFiI1IwhOieN4MImENTzi7CwBwGoZhLBaLkYWIvL29OUWDwWA9uA5EZdasWdb8hHvDHTtwHwhPAIich4eHl5dX586dnd0IuCKEJwAEBuEJALE5f/48+wzh+/fvSyQS9qNYIlKr1REREby3Bi4H4QkAgUF4AkBsTCYT+6X1IDrrE1gb9u46EDOEJwAEBuEJALHp0KGDt7e3s7sAYRBJeAILO3AfmDwB4K6MRqNWq01JSWFXiCgtLY1zjknjxo2x1AO7MHkCQGAQngBwVyUlJcXFxexb8gzDKBSKrKwszt96T09PLOzALoQnAAQG4QkAd6XRaCQSSVRUlLMbAQFDeAJAYBCeAHAb1tPmbC/NZrPFYmFXiEgqlbInwAJUDeEJAIFBeALAbcTGxlYs7t69m/1SJpMNHz5cJM/X4NEhPAEgMAhPALiNqKgo9sgv66AIzsElUqkUqzqoOYQnAAQG4QkAt6FWq728vJzdBbgVkfwYgIUduA+EJwAEp6SkpLCwkD0rgmEYIkpOTraeNmwTHByMTbTwKBCeABAYhCcABMdisTAMw5kV4e3tLZFIOEVOcgLgQSE8ASAwCE8ACI6Hh4ePj8+TTz7p7EbA/YkkPCGK580gEtbwhLO7AAAAVzRr1ixrfsK9Ce+OHcMw6enpaWlphYWFROTr6xsWFhYUFOTsvsD5EJ4AcHFms/nkyZPscKvBYDCbzTExMezLZDJZ7969lUol7w2CO0N4wuXk5uYuWrRoy5Ytd+/e5XwpODj4tddemz59uoeHh1N6A1eA8ASAi5NKpUFBQb6+vraK0Wg0Go2enp7sy+RyuUKh4L07cHMIT7iWO3fuREVFpaenh4WFDR48uGnTptYkfEFBwdWrVw8fPjxnzpyff/754MGDItkdCRUhPAHg4iQSib+/f0BAgLMbATESyfJAMAu72bNnZ2RkbNu27fnnn6/4VbPZvHbt2rfeemv+/PlffPEF/+2BK0B4AgAAKiOS8IRgFnYxMTEvv/yy3VUdEclksjfffDM2Nnbnzp1Y2IkWJk8AuI7CwsL8/Pxt27Zx6nFxcZxKaGjo448/zldfIF6YPOFa7h4pdRwAACAASURBVN+/36JFi6qvad269a5du/jpB1wQwhMArkOj0Xh5eUVGRrKLOp1OrVZz/p6yt9wBOA7CE64lMDDw7NmzVV+TkJAQGBjITz/gghCeAHAdEolELpe7/WMvEBCRhCcEs3odNmzY9u3bP/3005KSkopfLSoqmjt37p49e0aPHs1/b+AiEJ4AAIDK1KlTRwz5CcHcsZs3b15cXNyMGTMWLFjQpUuXoKAgjUbDMIxWq71+/fqJEyeKi4t79OghhrMHoTIITwA4S0lJyfHjx48fP86pc/bYyWSygQMHWs80AOAZwhOuxc/PLz4+ftWqVZs3bz506JDZbLZ9SaFQdOrUafz48ePHj2efewlig/AEgLMolcqmTZuyPzItFktJSQnnbFGpVIpVHTgLwhMuR6lUTps2bdq0aXq9/ubNm9bJEz4+PsHBwTigHAjhCQDnkUgkvr6+bn8vBAQN4QkXxTDM7du3r1+/bhspplKpMFIMCOEJAAConEjCE0Ja2GGkGFQN4QkAHuj1+pKSkv3793OK58+fT01NZRcDAwPbtm3Lb3cAlRJDcoIEtLDDSDGoFsITADxQKBQymYzznMTX19fT01MuL/eZgu/G4FIQnnAtGCkG1UJ4AoAHMplMLpeHh4c7uxGAByOS8IRgNhLWZKTYqFGjdu7cyXNj4DoQngAAgMpIpVIx5CcEc8cOI8WgWghPANQ6k8mUlZXFPmEqLy/PZDKlpKSwL5NIJM2bN1coFLw3CFBTCE+4FowUg2ohPAFQ60wmU15enk6ns1XMZrNUKuXsZ5VKpYGBgVjYgSsTyaZPwSzshg0b9tVXX3Xu3Pntt99WqVScrxYVFS1dunTPnj0zZ850SnvgChCeAKh1arU6JCQkLCzM2Y0APCqEJ1wLRopBtRCeAACAyogkPCGYhR1GikG1EJ4AAIDKiCE5QQJa2BFGikF1EJ4AeBR6vd5sNm/bto1TT0hISEhIYFcCAgJ69OjBY2sAtQDhCReFkWJQGYQnAB6FWq2WyWTdu3dnF/V6vVKp5NzqsJ4PDyAsCE+4HIwUg6ohPAHw6Nx+azmIFsITrgUjxaBaCE8AAEBlEJ5wLRgpBtVCeAKg5oxGY25ublpamq1SUFDAMAy7YtWkSRPsYwY3gPCEa6nJSLHY2NidO3diYSdaCE8A1JzRaNTpdJyTh2Uy2dWrVzlX+vn51a1bl9/uAGofwhOuBSPFoFoITwDUnKenZ4MGDdq2bevsRgB4IpKdWoK5LYmRYlAthCcAAKAyWVlZ1vyEexPMwm7YsGHbt2//9NNPS0pKKn61qKho7ty5e/bsGT16NP+9gYuwhiec3QUAALiiWbNmiWE8lWAexWKkGFQL4QkAu6yjeuLi4th/QfLy8oqLi+/du8e+0sfHp2PHjnz3B8ALhCdci+NGimm1WqPRWMUFxcXFD9Mx8A7hCQC7rOu5OnXqsBd2SqVSLperVCr2lb6+vnw3B8AXhCdcjiNGil29ejUsLIxhmGqvxK0g14fwBIBd1hsVERERmKYNYiaS8ISQFnZWtTtSrEWLFklJSXb37dkkJSWNHz9eoVA83L8CeIPwBAAAVAaTJ1yOg0aKRUREVH1B1cs+cB2YPAEAAJXB5AnXgpFiUC2EJwCIqLCwMCMj4/Lly5z6zz//zKl069atSZMmfPUF4GQIT7gWjBSDaiE8AUBEXl5eGo0mNDSUXSwuLvb09GRXJBJJvXr1+G0NwJkQnnAtGCkG1UJ4AoCIpFKph4eH228kAnhQInmgJ5jbkjUcKSaGQ6WhMghPAABAZUQyeUIwd+wwUgyqhfAEiFN+fr5er7e9NBqNUqk0NzeXfY1SqbTuSwYQLYQnXMuwYcO++uqrzp07v/3225wTNYmoqKho6dKle/bsmTlzplPaA1eA8ASIjcViIaLY2NiKX7px4wb7paen59NPP81TWwAuCeEJ14KRYlAthCdAbKwfVAMHDlSr1bYiwzAVf8KRywXz3R7AQRCecC2OGykGbgPhCRAnhULx0NN3AMRDJOEJwSzsyDEjxcCdIDwBAACVweQJF1W7I8XAnSA8Ae7NbDYXFRWxf3qxZiZu377N+eG2QYMG7IezAEAIT7ggB40UA7eB8AS4N51Ol5eXl5SUZKswDCOVSpOTkzl/8h977LHg4GDeGwRwaQhPuBaMFINqITwB7k2j0Xh4eHTt2tXZjQAIEsITrgUjxaBaCE8AAEBlRHLfRzC3JWsyUmzUqFE7d+7kuTFwHQhPAABAZTB5wrXUcKTYrl27+OkHXBDCE+BmLl++zD6PPScnx2KxsPfYEZFKpWrVqhXvrQEID8ITrgUjxaBaCE+AmykoKGAfLMwwDBFVnBVm90RiAOBAeMK1YKQYVAvhCXAznTp18vPzc3YXAG4C4QnXgpFiUC2EJwAAoDIiCU8IZmGHkWJQLYQnAACgMpg84XIwUgyqhvAECJT15OH4+HhbxWKxEFFSUpJCoWBf2bRpU+wkBng4CE+4LrVaHRYWVrF+//793Nzc0NBQ/lsCV4DwBAiUVCqVSCScNVzdunU9PDw4f6TxQyzAQ0N4QniWLVu2ZMkSa3AMRAjhCRAolUrl6+sbGRnp7EYA3BnCEwACg/AEAABURiThCVHclgSRQHgCAAAqg8kTrqUmDylu3brFQyfgshCeAKE4duwYO8JvMBgsFktMTAz7Grlc3qdPH/YBxQDwKBCecC0JCQlExNlczGEymfhqB1wRwhMgFE2bNvX29ra9NBqNJpPJw8ODfY1CocCqDqAWITzhWmbMmLF69eozZ85UEXp97733lixZwmdX4FIQngChCAgIwH5QAJ6JJDwhmNXrwoULQ0NDx4wZYzQand0LuCiEJwAAoDJ16tQRQ35CMAs7hULx/fffX7hw4YMPPnB2L+CiEJ4AV8MwjMViKSqPiPR6PafInqYDAI6A8ITLad26dWZmZhUb6QYNGoSB2WKG8AS4msLCwtzcXE4qgoiOHTvGqbRs2bJDhw589QUgRghPuCIfH58qvhodHR0dHc1bM+BqEJ4AV+Pj41OnTp1u3bqxi2azueJUa05yAgBqHcITAAKD8AS4IKlU6uXl5ewuAEAs4Qks7MB9IDwBAACVEUNygrCwA3eC8AQ4l8ViuXXrllartVWys7N1Ol1KSgr7MqlUGhoaKpKnQgCuw5qcaNSokbMbcSws7MB9IDwBzsUwzL179woKCmwVo9FY8ecNuVzerFkzpVLJe4MAoobwBIDAIDwBziWTydq3b9+kSRNnNwIAdojkNjkWduA+EJ4AAIDKIDwBIDAITwAAQGUQngAQGIQngDfWcRG7d+9mF41G44kTJ06dOsUuNmvWDCcPA7gChCcABAbhCeCNh4eHSqXq2LEju6jVaj09PTn7eDAOB8BFIDwBIDAITwBvpFKpXC4PCgpydiMAUFMITwAIDMITAABQGYQnAAQG4QlwELPZnJmZaTAYbJWcnByj0cg5eVgikbRo0UIux/dVAFeE8ASAwCA8AQ5iMplycnKKiorYFSLi/HmTyWRBQUFY2AG4JoQnAAQG4QlwEJVKFR4e3rx5c2c3AgAPD+EJAIFBeAIAACqD8ASAwCA8AbWF/dSViBiGMRgMnKJSqVQoFPz2BQAPD+EJAIFBeAIeXXFxscFgiImJ4dSTkpKSkpLYlcDAwO7du/PYGgA8EoQnAAQG4Ql4dJ6engqFYsiQIeyixWKp+BAHIQkAYRFJeEIUz5tBJKzhCWd3AYInkUiU5anVamUFItmvA+A2Zs2aZc1PuDf8xAnuA+EJAACojEh+GMPCDtwHwhPwoLRabcXNc0S0bds29kuJRNK7d+969erx2BoA1DKEJwAEBuEJeFCenp6NGzdmj3xlGEan03l6erIvk0gkItl2DeDGRPK3GAs7cB8IT8CDkkqlGo3G7TdTAwAhPAEgOAhPAABAZRCeABAYhCegCiUlJVRh8xwR3b9/n7PHrlGjRtHR0fx1BgC8QHgCQGAQnoAqqFQqIuKs2PR6vUKhkMlk7CJngx0AuAeEJwAEBuEJqJbbb68BgMogPAEgMAhPgA3DMEajkT3dVa/XU4UhsETk6emJJ/gAYiCS8AQWduA+rOGJd955x9mNgPMVFhZmZmZevXqVU684BDYqKqpx48Z89QUATmNNTqxbt87ZjTgWFnbgPhCeABtvb29/f//w8HB20WQyVRzwih11ACKB8ASAwCA8ATYSiUShUHh5eTm7EQBwFQhPAAgMwhPixDAMEV2+fJkdbtVqtSaTKSUlhX2lRqNp0qQJ3/0BgGtAeAJAYBCeECeLxUJEt27dYj+IN5lMxcXFnD8P3t7eWNgBiBbCE1ALTp8+bTAY2JWcnBwiio+Pt1WMRiPfbbkphCfEyXqjrmfPnkql0tm9AIDrQngCaoGXl5dCoWBXrDcV2Pu1rQfiw6NDeAIAACqD8ATUAk4oj4hMJlNhYSG7UlRUlJ6enp+fr9PpbEWj0WjdOQQ1h/CEGBQWFubn52dkZNgq1r8pv/32G+fKyMhIPHgFABuEJ8Ahrly5whlMaXX48GFORSQ/W9QihCfEwNPTUy6Xt2jRgl0sLCz09vZmV/CHAQA4EJ4AhwgPD2/evDmnyDAM5xni8ePHOR9UUC2EJ8RAJpNpNJqgoCBnNwIAAoPwBDgKZ4s3wzBJSUmcCEV+fn5xcfGpU6dsFetAJKgCwhNu6dy5c+yNqvfv35dIJOy/GkTk6enZpk0b3lsDACFBeAJ4Yh1qyVnYeXh4SKVSdtFkMvHemsAgPOFmrJvnOH/yVSqVRCLh/H1BAgkAqiWSDU5Y2DmfVCqNjIys9rJ79+79/fffPPQjXAhPuBnrMr1jx44ajcbZvQCA4CE8Ac507ty55OTkivUdO3ZwKph0aYP98oJmNBqLiorS0tLYFSK6efOmSqViX9moUSPMCgOAB4XwBDhTq1atKp7UoNPpPDw82JWTJ08iY2GD8ISglZSUaLXaq1ev2ioMw8jl8ps3b3KesMvlcizsAOBBITwBzqRUKjkZC4vFcu/evYKCAnbRaDTev3+fPRCzoKDAOmFJhBCeEDSNRiOTybp16+bsRgDAPSE8Aa7FZDJlZGRwNpJbLBatVsveOS7mAWUITwjLrVu38vLybC8LCgoMBgP7USwRyeXy4OBg3lsDADeE8AS4FqVS2atXr2ovu3379rFjx3joxwUhPCEsGRkZcnnZtyCDwWCxWNiPYolILpc3adJEJN+OAcChEJ4AAUhMTLx27Rq7YrFYzGbz7t272UWj0ajVanntzBkQnhCWrl271q1b19ldAIBYIDwBAtCyZcuAgAB2hWGY4uJiztbykydPimGzOcITrkmv1xcWFrKPFLYeUJecnMyJuwYFBbn9vmYAcBaEJ0AAPD09OcedmEymlJQUzv05k8l0+/ZtnU5nq+Tn55vNZp665AvCE66JYRiLxcLZ/WlNc3OK7vdnEgBcB8ITIEgmkyk3N5cTjFUoFEajMTc311bR6XTuF55FeMI1eXh4+Pr6Pvnkk85uBABETSS7dbGwczdqtbpHjx6coslk4sxcysjISE1NLSoqYhetD8iEC+EJV2CxWI4ePcr+Bmo2mxmG4ez7lEqlAwcO5JzpAwDgOAhPgPs4fvz4rVu3KtZjYmI4leLiYl46cgiEJ1yBRCJp3bo1OxVhMplMJpNarWZfJpVKsaoDAD4hPAHuo2vXrhWnpJvNZplMxq4cOHBA0APKEJ7gmfVOMPt8bCutVst5Jt6gQYN69erx2BoAABfCE+A+5HI5+8AwIsrPzz969Chnm11JSUlCQsK5c+dsFaPRKKBNCQhP8MxgMBgMBs5iWi6X5+TksE8eJiKZTIaFHQA4F8IT4M68vLzCw8M5m+oKCgq8vLzYt/GysrI4n9CuDOEJRyspKWHvy7TObOUMAZNIJIK+7wsA7kpA9ykeBRZ2IiWXy0NCQjjFS5cu3bt3j10pLCwsKSmJj49nFy0WC2eymYtAeMKhrDd0ExISOPWKOzWHDBkihnMTAUBYEJ4A0ZHL5QqFgl3RaDQKhYJTdNm7YghPOJRKpQoPDw8MDGQXK+7UlMlknJwEAIArQHgCRKdFixacSlFREWd2p9WNGzfu379ve5mdne0Kp+IhPFFbioqKiouLt23bxqlXvGPXrFmzLl268NgaAMBDQngCgEwmE+esOyLSaDRms5ldNxqNrnAbD+GJ2uLp6alSqZ544gl2sbi42MPDg/MfWqPR8NsaAMBDQngCgOwODLh06VJhYSG7IpVKS0pK2MNArTfweL6Nh/DEw9HpdElJSefPn7dVLBaL2Wzm7K0kop49e/r5+fHbHQBA7UB4AsA+o9HIGfGpVCqlUim76JQhFghP1ATDMJy7sEqlskGDBuztiRaLxWKxcO7GSSQSX19fnroEAKhtCE8A2BcREcGpZGdnX7x40WAw2CrWhV1CQgI7eFFYWOjh4eG4xhCeqJZOpzObzRVzrFqtNj09nV1p1KhRdHQ0j60BADgWwhMANeXp6VlxO6pCofDx8WHf+jYajTKZjL3+49z5e0QIT3CYTKbc3Ny0tDRbJT8/XyaTtWnThn2ZxWIJDAzkDPjCvC8AcDMITwDUlPW4Y07x119/tTugljMMnmrvuS3CExwGg0Gr1bKPmLbuemQv9YhIKpUGBATg5DkAcG8ITwA8kt69e7NvzhFRQUFBZmYmO99gMpkyMjIuXbrEfmKbl5f3cHu5xByeMBqNDMP8/vvv7KJOp+McMmddw3FmRQAAiAHCEwCPxMvLi3MTSK/X6/V6doVhGLVanZeXx16QlZSUcEZX6XS6mvwbxROeMBqNOTk5KSkptkpRUZFEIvHz82P/P+nl5VWvXj3OfwUcUAIA4oTwBEAtCwgICAgIYFcYhtm1axf7rGOru3fvVtzgXy03Dk9kZ2erVCrby+Li4pKSEvYqmWEYuVxeXFzMfpdEIvH392/QoAF/jQIAuCqEJwAcTiKRDB482Gw2s4vp6enXrl1jVywWi16v37dvH/t2lF6v58w6c4PwBMMwnEOhrf987NixihdzoicajaZfv36O7hAAQKAQngDgQ8W5oiEhIZ6enuyK2WxOSUmpuH8uOzt7//79tpdnzpyZPXu2sMITBQUF7GOcc3Nz8/Pz7YZOODp16lRxBBwAAFQG4QkA5/Dy8mrevDm7wjAM5wBkIsrIyMjPz2cXrZPNdu3axb7MZDJ5enrm5ubaKhWHpPHAeh+OvSvOZDIRUcXpDhUpFIr+/ftzipy1LwAAVA3hCQBXIZFIKt6datKkCXu5RkTt2rXz8/OreDZeRkZGRkYG5xc8cuQIu1JSUlJQUMD+BYuLi+2ew2IymdhpX4PBoNfrU1NTbRXru+Lj49mJVL1eb7FYLly4wGnD29ub872mRYsWQUFB7IpcLhfJ9yMAAMcRSXhC4pTRT8Jy9OjRqKiokpISHNnq+kwmE2dA7fXr1+/evcuu6PX6nJwcR//Jr/inpU6dOpwVm0wma9q0qUPbAACAWmcwGFQq1ZEjR1zw9CjcsQP3YbFYUlJSOBPPwsLCwsLCqn1vUVGRVqtlV0wm0/379znrs6KiIs44DYvFolarOWeIyOVyb2/vh/k9AACAYyA8ASAwcXFxAwYM4ByVV0MVT90josaNG9dGXwAA4HwITwAIjFQqxV40AACwSyQfEFjYgfvo3r07J50AAABgJZLwBBZ24D4kEklISIizuwAAAFckkskTorgtCSJhsVjOnz/v7C4AAMAVZWVlWfMT7g0LO3AfcXFxkZGRzu4CAABc0axZs6z5CfeGR7HgPhCeAACAyojkAwILO3AfCE8AAEBlEJ4AEBiEJwAAoDIITwAIDMITAABQGYQnAAQG4QkAAKgMwhMAAoPwBAAAVEYkHxBY2IH7QHgCAAAqg/AEgMAgPAEAAJVBeAJAYBCeAACAyiA8ASAwCE8AAEBlEJ4AEBiEJwAAoDIi+YDAwg7cB8ITAABQGYQnAAQG4QkAAKgMwhMAAoPwBAAAVAbhCQCBQXgCAAAqg/AElFIqlUSkUqmc3QjUiEQicXYLAADgotavX19bv5R1eeBqJAzDOLsHATh79qzJZHJ2F3w4c+bM66+/vnnzZiyPXNOFCxeWLl26adMmZzcC9iUmJq5cubIWPzmgdp08eXLDhg1ff/21sxsB+44dO7Z169b9+/c7u5HqyeXy9u3bO7sLO7Cwg3JiY2Ojo6PNZrNIYuGC89dffw0aNMhoNDq7EbAvJiZm9OjRWq3W2Y2AfTt37pwwYcL9+/ed3QjY9+OPP06bNu3OnTvObkTA8OENAAAA4CawsAMAAABwE1jYAQAAALgJLOwAAAAA3AQWdgAAAABuAgs7AAAAADeBhR0AAACAm8DCDgAAAMBNYGEHAAAA4CawsINylEqlQqHAPDGXpVQqXXM6IVjhP5CLw38gF4f/QI8OI8WgHIZhrl27FhIS4uxGwD78B3JxFovlxo0bzZr9f3t3H5Xz/fhx/H2pLroXGqHS5P62cldCuRvZkNvmPjknchOlY9bZ3GxjDhk6nWNmc7thzDg4aacQVgqZNmWSpEbMTahcrqur6/fH5dtSUW5+ffS5no+/5v15X+/P6/I5x3ntc3N9WkgdBJXTarW5ubmOjo5SB0HliouLb9686eDgIHWQWoxiBwAAIBNcigUAAJAJih0AAIBMUOwAAABkgmIHAAAgExQ7AAAAmaDYAQAAyATFDgAAQCYodgAAADJBsQMAAJAJih0AAIBMUOwAAABkgmIHAAAgExQ7AAAAmaDYAQAAyATFDgAAQCYodqhESEiIQqGYMWOG1EHwnOjo6H79+llaWtavX79///4nTpyQOhH+c/ny5cmTJ9vZ2ZmYmNja2vr6+iYnJ0sdyqBpNJrFixcbGRl169at4tb8/Pz58+e3aNFCqVQ2bdp0xowZt27dqvmQhuzlB+jBgwcLFy50dHSsW7euk5PTyJEjz5w5U/MhayNjqQPgnXPu3LkNGzZInQLlbdmyZfr06S1btpw/f75Kpdq2bdsHH3xw/PhxDw8PqaNBXLp0yd3d3cTEZM6cOc7OztnZ2VFRUb17946Jienfv7/U6QxRenr6pEmTMjIyKt2qVqsHDBiQkpIyevRoV1fXzMzM7du3Hzt27Pz58zY2NjUc1TC9/ADdv3/fzc3t+vXrw4YNmzp16rVr1/bs2RMTE5OcnNypU6cajlr76IAyNBpN165du3TpIoQICAiQOg6euX37toWFhYuLS0FBgX4kIyPDwsIiKChI2mDQmzBhghDi2LFjpSMXL14UQnh5eUmYymA9fPjQ1NS0W7duGRkZdevWdXNzKzdh7dq1QohVq1aVjuzZs0cIERoaWrNJDVSVB2j27NlCiMjIyNKRX375RQjh4+NTs0lrJS7F4jkREREXL178+uuvpQ6C52zfvr2goGDlypXm5ub6EWdn50ePHkVFRUkbDHqZmZlCCE9Pz9KRzp07W1lZXb9+XbJMBqy4uDgoKCghIcHZ2bnSCdu3b7e0tAwODi4dGTdunLOz844dO3Q6XU3FNFxVHiATE5MBAwYEBgaWjvj6+pqaml66dKmmMtZiFDv8JzMzc9myZTNnzuzVq5fUWfCc2NhYU1NT/UW9p0+fPnr0SAihUCikzoVn2rZtK4T4+++/S0fu3r1bUFDQrl076UIZrgYNGqxZs8bExKTSrSqV6s8//+zRo0fdunXLjnt6et65cycrK6tGMhq0lx8gIcQ333wTGxtbdoJarS4uLm7evHmNBKzdKHb4T2BgYP369VeuXCl1EJR3+fJlJyenv/76y9PT09TU1Nra2tnZeevWrVLnwjOLFi2ysbGZNGnS6dOn8/LyLly44OfnV69evSVLlkgdDeXl5ORotVp7e/ty446OjkKIa9euSREKVfj22281Go2fn5/UQWoBih2e2bp1a1xcXGRkpLW1tdRZUN79+/cLCwuHDRvWq1evvXv3rl+/XqPR+Pv7//TTT1JHgxBCtGvXLjExUaPR9OnTx87OztXVNSMjIzY2tmfPnlJHQ3mPHz8WQpTe1VDKwsKidCveKfHx8WFhYZ6enjNnzpQ6Sy3AU7EQQog7d+6EhoZ++OGHo0ePljoLKqFWq7Ozs7dt2zZlyhT9yNixY1u3bh0aGjp+/HgjIyNp4yE9PX3YsGHFxcURERGtW7e+c+fO2rVrhw4dum/fvoEDB0qdDpWoeCeD/u467nB41+zatcvf379jx44HDx40Nqa0VI2/IwghRHBwsFqt5k78d5aFhUVxcfGYMWNKR+zs7IYOHbp37960tDSe/5fc9OnTb9++feXKlWbNmulH/Pz8WrduPW3atKysrJfcS4SaZ2VlJSo7M6e/ddXS0lKCTKiMTqdbunTp8uXLhwwZ8vPPP3NoqolLsRDR0dG7d+9esGBBnTp1cnNzc3Nzb968KYQoKirKzc3V/2MHabVo0UIIUa4f2NraCq4cvQMKCgqSkpJ69uxZ2uqEEGZmZgMGDPjnn3+uXLkiYTZU5ODgYGxsnJ2dXW5c/2hzq1atpAiF8nQ63YwZM5YvXz537tzDhw/T6qqPYgcRFxcnhPjiiy/s/6dDhw5CiF27dtnb269YsULqgBDu7u5arTYlJaXs4NWrV4UQFe8BRw178uSJTqdTqVTlxvUjFcchLaVS6ebmlpycXFRUVDpYUlISHx9vb2/v4OAgYTaUWrBgwQ8//LBixYoNGzZwt8krodhBBAQEHHre7t27hRCDBw8+dOjQtGnTpA4IMW3aNIVC8emnnz59+lQ/cu7cudjY2M6dO1PsJGdra+vk5HTu3LmyJ+fy8/NjY2OtrKw6duwoYTZUKiAgoKioaPXq1aUjmzZtunnzJu9RfEfs379//fr1wcHBixcvljpL7aPgxxhRUX5+vo2NTUBAwObNm6XOgmcWLFiwbt26rl272Fn9wAAACUlJREFU+vr65ubm7ty5U6vVxsTEeHl5SR0N4tdffx0zZoyNjc3MmTNbtmx569atzZs3Z2VlRUVFBQUFSZ3O4MTHx0dHR+v/e82aNba2tlOnTtX/MSwsrGHDhlqt1tvb+9SpUyNGjHB1dU1PT9+zZ0/Hjh3PnDljZmYmXXBDUeUBcnZ2zszMnDt3bsXDof9poRqNW+tI+doLvKsePHggeKXYO6akpGTjxo1dunSpV6+etbW1j49PcnKy1KHwn4SEhJEjR9ra2hobG9vY2AwcOPDIkSNShzJQL/kxzoyMDP2cx48f698xb2Ji0qxZs9mzZ9+7d0/a2IajygP0ktKSlZUldfx3HWfsAAAAZIJ77AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQAAgExQ7AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQAAgExQ7AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQC8DrVa3aNHD4VCsWzZsjdZJzw8XKFQdO/eXa1Wv61sAAyWQqfTSZ0BAGqfhQsXRkRE9O3b9/jx43XqvP7/JGu12j59+iQmJoaGhq5Zs+YtJgRggCh2APDKkpKSPDw8lEplampqq1at3nC1tLQ0FxeX4uLihISEnj17vpWEAAwTl2IB4JUFBweXlJQEBwe/easTQrRv337WrFklJSXz589/89UAGDKKHQC8miNHjiQlJZmbmy9atKjSCTqdrn379qampv369cvOzq7OmuHh4fXq1Ttz5kx0dPRbDQvAsFDsANR6O3fuVCgUS5curZndRUZGCiEmTZpkY2NT6QSFQhESEhIUFJSQkPDxxx9XZ01bW9vx48cLIaKiot5iVACGhmIHAK/g3r17cXFxQoiXN7YZM2ZERETMmzcvMTHx5MmT1Vl5woQJQojffvvtwYMHbyUqAANEsQOAV3D8+PHi4mILCwtPT88qJ8+ZM0ehUGzevLk6K3t7e5uammo0mhMnTrxpSgCGimIHwCBkZ2f7+/s3a9ZMqVQ2atRo+PDhycnJZSccOXKkR48eZmZmTZo0CQ4OfvLkib29vaura7l1zp49K4RwdXU1MjKqcqdOTk69e/fev39/YWFhlZNNTExcXFyEEOfPn3+FLwYAZVDsAMhfTk5Ojx499u3bN3HixO+++y4kJCQlJaVv376nT5/WTzh58uSIESNycnI++eSTzz//PDU11c/P7/Hjx0qlstxSV69eFULY29tXc9eOjo6FhYUHDhyozmT9M7Z5eXnV/WIA8DxjqQMAwP+7zz777M6dO/v37/f19dWP+Pr6durUKSwsLDExUQjx5ZdfarXaQ4cOdevWTQgRGBg4aNCghw8fVlxK37oaNmxYnf3m5eXt3btXCLFjx46JEydWOb958+ZCiHv37lX3iwHA8zhjB0DmdDrdgQMHGjduPHLkyNLBdu3aubu7nzlzRt+iTp061bZtW32rE0IYGRm96KdMioqKhBCNGjWqzq4jIyM1Gs1HH30UGxtbnfNw5ubmgmIH4A1Q7ADIXF5e3sOHDzt06KBQKMqOt2nTRghx5cqV/Px8lUrl7OxcdquHh8cb7reoqGjjxo0DBw4MDAzUarW7du2q8iP6VwGVywkA1UexAyBz+gcX9CfDyjI1NdVv1Z8hMzMzK7vV0tKy0scj9OvcvXu3yv1u2bLl/v37QUFBgwYNsra23rFjR5Uf0Z8OrOZ1XgCoiGIHQOYsLCzE/+pdWfoRS0tLExMTIYRKpSq7taioSKvVVlzNzs5OVONqaUlJybp161q0aDF8+HClUjlq1KgLFy6kpaW9/FO5ubmCYgfgDVDsAMhckyZNGjRokJ6err/QWSotLU2hULRp06ZJkyZ16tQp9+6vpKSkSlfTP7iak5Pz8p0ePHjw6tWrc+bMqVOnjvjfjw9XedIuIyNDCNG0adMqvhIAvADFDoD8jRo16tatWwcPHiwd+eOPP5KTk/v371+/fn2lUtmtW7fU1NTLly/rt2q12lWrVlW6VPfu3YUQKSkplZ7PKxUREWFubh4QEKD/o7e3d+PGjX/88cdy5bIsjUaTkpIihHBzc3vF7wcAz/BzJwBk4ujRo/n5+eUGR4wY4e3tvWzZssOHD0+ePHnevHlt2rS5fv16VFSUhYXF2rVr9dPCwsLGjh3r4+MTFBRkZWW1c+fO999/v27duhX34u3tbWxsXFBQcPr06X79+lWaJCkp6ffffw8KCqpfv75+xMjIaNy4cZGRkfHx8V5eXpV+6tixYyqVSqlUvmgCAFRNBwC13Esuca5evVo/58aNG/7+/nZ2dsbGxu+9956fn19aWlrZRb7//vs2bdoolUpHR8fw8HC1Wq1UKj08PCruzsfHRwgRGBj4ojxjx45VKBT6i7+lEhIShBDTp09/0aemTp0qhBg+fPirfXkAKEOhe/F1AQAwWI8ePbK2th4+fHjZC7h6R48eHTp0qLm5+Y0bNxo0aPBWdvfvv/86ODioVKqYmJjBgwe/lTUBGCDusQMAsWXLFi8vr7Ivad26dasQwtPTs+LkIUOGeHh4FBYWvug+vNfw1VdfqVQqDw8PWh2AN8EZOwAQSUlJ/fr1s7GxmTVrVtOmTS9cuLBp06amTZtevHix9D65ss6ePevu7m5iYpKamqp/TvZNpKWlubi4aLXaxMRE/cMZAPB6OGMHAKJnz55xcXEuLi5RUVFBQUEHDx6cMmVKYmJipa1OCNG9e/eQkBCVShUQEFBSUvImu9ZqtQEBAWq1OiQkhFYH4A1xxg4AXodGo/H09ExOTl6yZMnSpUtfe53w8PAVK1a4ubklJCQolcq3FxCAIaLYAQAAyASXYgEAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQAAgExQ7AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQAAgExQ7AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMkGxAwAAkAmKHQAAgExQ7AAAAGSCYgcAACATFDsAAACZoNgBAADIBMUOAABAJih2AAAAMvF/O/H59evga8MAAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 }, "tags": [] }, "output_type": "display_data" } ], "source": [ "# Using 10-fold CV, cross-validate on the training data\n", "set.seed(1)\n", "cv.out = cv.glmnet(x[train_hyperparameter,], y[train_hyperparameter], alpha=0) #alpha=0=ridge regression\n", "plot(cv.out) #defaults to 10-fold CV\n", "bestlam = cv.out$lambda.min\n", "bestlam" ] }, { "cell_type": "markdown", "metadata": { "id": "Wk4Fj3VuakOB" }, "source": [ "The $\\lambda$ with the lowest cross-validation MSE is $\\lambda = 390.75$. We can use this value of $\\lambda$ to look at our hold-out test accuracy." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 34 }, "id": "Ocjrc2g-akOB", "outputId": "4ef89ed3-6d1e-4756-f1ba-826c93aaddf3" }, "outputs": [ { "data": { "text/html": [ "119507.530651601" ], "text/latex": [ "119507.530651601" ], "text/markdown": [ "119507.530651601" ], "text/plain": [ "[1] 119507.5" ] }, "metadata": { "tags": [] }, "output_type": "display_data" } ], "source": [ "#passing in the trained model object, the best lambda, and the test data\n", "ridge.pred = predict(ridge.mod, s=bestlam, newx=x[test_hyperparameter,]) \n", "mean((ridge.pred -y[test_hyperparameter])^2) #MSE" ] }, { "cell_type": "markdown", "metadata": { "id": "-JTuKApLakOB" }, "source": [ "Using this best lambda, we can also get the actual regression coefficients for all 20 parameters in the model (including the intercept). Here, we refit the ridge regression model on the training dataset dedicated for this purpose (`train_glm`) and then look at coefficient predictions for the `test_glm` dataset. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "rLiKeMXyakOB", "outputId": "73901ce6-4885-471f-f875-0a682784e8d5" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " (Intercept) AtBat Hits HmRun Runs \n", "-38.812546181 0.007535828 1.517425980 -4.090339763 1.407753980 \n", " RBI Walks Years CAtBat CHits \n", " 1.037274878 2.172859070 -3.143396736 0.014712552 0.109118248 \n", " CHmRun CRuns CRBI CWalks LeagueN \n", " 0.177487099 0.183558581 0.171052370 -0.023204448 10.742320818 \n", " DivisionW PutOuts Assists Errors NewLeagueN \n", "-88.771290300 0.234770575 -0.035088669 -1.551198110 -7.358141646 \n" ] } ], "source": [ "# First setup the model\n", "out = glmnet(x[train_glm,],y[train_glm], alpha=0)\n", "\n", "# # Then predict.\n", "print(predict(out, type=\"coefficients\", s=bestlam, newx=x[test_glm,])[1:20,]) # \"s = bestlam\" picks out the winning value and shows those coefs." ] }, { "cell_type": "markdown", "metadata": { "id": "lMd5OgU7akOE" }, "source": [ "---\n", "# LASSO\n", "\n", "In order to run LASSO, you repeat the same steps as shown in the example above but replace alpha=0 with alpha=1. Try re-running the cells above with LASSO instead of ridge regression and see how it changes!" ] }, { "cell_type": "markdown", "metadata": { "id": "4azkJPDQ_SNh" }, "source": [ "*Notebook authored by Ven Popov and edited by Krista Bond, Charles Wu, Patience Stevens, and Amy Sentis.*" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "regularized-regression.ipynb", "provenance": [] }, "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "4.0.2" } }, "nbformat": 4, "nbformat_minor": 1 }