{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9d2294db-3cb8-4a64-8381-7d7ee26a676a",
   "metadata": {},
   "source": [
    "# Introduction au réseau de neurones : de la simple régression à l'économétrie"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b67e414e-adb1-4d88-ac4f-10f474eef979",
   "metadata": {},
   "source": [
    "Le notebook : https://sergelhomme.fr/notebook/Intro_IA_reg_poly.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db14c0e7-18ab-45da-8653-fece15efd336",
   "metadata": {},
   "source": [
    "Pour s'initier aux réseaux de neurones artificiels, qui sont aujourd'hui au cœur de nombreuses avancées en deep learning et en intelligence artificielle, nous allons utiliser TensorFlow. Plus précisément, nous nous appuierons sur Keras, l'interface (API) de haut niveau intégrée à TensorFlow. On peut voir Keras comme un véritable « panneau de contrôle » permettant de construire, entraîner et évaluer des réseaux de neurones sans avoir à manipuler directement toute la complexité du moteur TensorFlow. En pratique, Keras simplifie considérablement l'écriture des modèles et permet de se concentrer sur les concepts essentiels plutôt que sur les détails techniques de l'implémentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4580f14e-e0d7-4044-82e6-ec3134159e22",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.layers import Input"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7b79a9c-f8dd-4242-80cc-4eff29958d3d",
   "metadata": {},
   "source": [
    "## Régression linéaire simple et réseau de neurones"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac08af74-a798-4287-895a-63a92159dbd1",
   "metadata": {},
   "source": [
    "Une régression linéaire simple constitue un exemple classique de modèle d'apprentissage supervisé. Son objectif est d'apprendre une relation entre une ou plusieurs variables explicatives X et une variable à prédire Y. Pour cela, le modèle est entraîné sur un jeu de données d'apprentissage à partir duquel il estime les coefficients de la régression. Une fois ces coefficients déterminés, ils peuvent être utilisés pour prédire de nouvelles valeurs de Y à partir de nouvelles observations X. Le modèle est ensuite évalué sur un jeu de données distinct afin de vérifier sa capacité à généraliser ses prédictions (voir plus bas l'exemple de modélisation des prix immobiliers à Boston). En deep learning, les variables X sont généralement appelées variables d'entrée (explicatives, d'apprentissages, inputs) ou caractéristiques (features). Les variables Y correspondent aux sorties attendues du modèle. En classification, elles sont souvent appelées étiquettes (labels), tandis qu'en régression on parle plus volontiers de cibles (targets) ou de variable à prédire, de variable à expliquer. Les prédictions du modèle sont quant à elles notées Y' ou Y barre."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4811b2b7-d3fb-43b0-a599-ed6f1127ebf5",
   "metadata": {},
   "source": [
    "<b> D'une certaine manière, les coefficients d'une régression jouent un rôle analogue aux poids d'un réseau de neurones : ce sont les paramètres que le modèle apprend à partir des données afin de réaliser des prédictions. Il est donc intéressant de commencer par un cas très simple : un réseau constitué d'un unique neurone. Nous verrons que ce neurone est capable d'apprendre lui aussi les paramètres nécessaires à la prédiction, exactement comme une régression apprend ses coefficients.<b>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5622f53-9582-4117-af38-2aaf4866ef6e",
   "metadata": {},
   "source": [
    "Avec un réseau de neurones, les coefficients d'une régression linéaire sont appris (calculés itérativement) par descente de gradient, plutôt qu’obtenus par la résolution analytique des équations (normales) de Legendre-Gauss, qui donnent une solution optimale mais ne sont pas adaptées aux modèles complexes (non linéaires) ou de grandes dimensions (dans ce cas, la descente de gradient est un algorithme heuristique permettant de trouver des solutions satisfaisantes en un temps raisonnable). La descente de gradient n'est pas un algorithme de résolution exacte, mais dans le cas d'une régression linéaire il converge vers (s'approche de) la solution optimale, car la fonction de coût est convexe. Plus de détails sur la descente de gradients ici : https://sergelhomme.fr/notebook/Intro_IA.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d5c7c80-5562-4513-9db1-c9361c4724f6",
   "metadata": {},
   "source": [
    "Ainsi, en théorie, une régression linéaire simple est équivalente à l'utilisation d'un réseau de neurones à : 1 neurone ; sans fonction d’activation (plus précisément une fonction d'activation linéaire) ; 1 entrée ; 1 sortie. La solution proposée converge en théorie vers la solution optimale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "31b67f02-6f19-44d6-aaea-3176a82622a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Données (on construit des donnees suivant une régression du type y = 3x + 2)\n",
    "np.random.seed(0)\n",
    "X = np.linspace(0, 10, 100).reshape(-1, 1)\n",
    "y = 3 * X + 2 + np.random.normal(0, 0.5, size=(100, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3bd6bd84-597d-486e-9007-da6c87738e54",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAsrUlEQVR4nO3dfXBU53n38d9BMWuBpXV4k1ZGYJmCaQBTYigGYUBuoaYZngKe1AlJCvEktcfgmGpSkHHb4E4tCTJh2okSMuBnaD0tD/xhwO6kpqZVjaJh7MgYakxiYzvCUCwVy4VdLKtLjM7zB97Nvpx9k3bP2XP2+5nRTPaF5UaTZH9z39d9XYZpmqYAAABsMsLpBQAAgNJC+AAAALYifAAAAFsRPgAAgK0IHwAAwFaEDwAAYCvCBwAAsBXhAwAA2OpzTi8g0eDgoD744ANVVFTIMAynlwMAALJgmqauXr2qmpoajRiRfm+j6MLHBx98oNraWqeXAQAAhuDChQuaOHFi2vcUXfioqKiQdGPxlZWVDq8GAABkIxQKqba2Nvo9nk7RhY/IUUtlZSXhAwAAl8mmZIKCUwAAYCvCBwAAsBXhAwAA2IrwAQAAbEX4AAAAtiJ8AAAAWxE+AACArQgfAADAVoQPAABgK8IHAACwFeEDAACP6AkO6Ph7feoJDji9lLSKbrYLAADI3YGu83ri4GkNmtIIQ2pZM0sPzpuU9L6e4IC6+/pVN260Av5yB1ZK+AAAwPV6ggPR4CFJg6a09eCbWjxtfFzAyDagFBrHLgAAuFx3X380eERcN02d6/sk+jhVQHHiiIbwAQCAy9WNG60RCZPsywxDt48bFX2cTUCxC+EDAACXC/jL1bJmlsqMGwmkzDDUvGZm3JFLNgHFLtR8AADgAQ/Om6TF08brXN8nun3cKAX85UnFpS1rZmnrwTd13TQtA4pdCB8AAHhEwF8eDROpiksTA4oTCB8AAHhMptsvToWOCGo+AADwmGIqLrVC+AAAwGOKqbjUCuEDAACPyeb2i5Oo+QAAwIOKpbjUCuEDAAAXGMpMlmIoLrVC+AAAoMhZXZtdPG284wPihorwAQBAEbO6Ntv03GkZhhwfEDdUFJwCAFDErK7NmlJRDIgbKsIHAABFzOrabKJi6uGRDcIHAABFLPHa7AhJiVmkmHp4ZIOaDwAAHJZ4kyXxceK12Y6zHxbFgLihInwAAOCgxJssq+fcpkMnLyYVk8Zemy3mHh7ZMEzTNDO/zT6hUEh+v1/BYFCVlZVOLwcAgILpCQ6ovrU9qaA0VplhqLOpoegDRi7f39R8AADgEKubLIncVkyaDcIHAAAOyeYmi9uKSbNB+AAAoEB6ggM6/l5fyh4cVgPgHvjibUU7EC5fcqr52LVrl3bt2qVz585JkmbMmKG/+qu/0ooVKyRJpmnqqaee0u7du3X58mXNnz9fP/rRjzRjxoysF0TNBwDAC6xaoqfqQtoTHIgrHk187AYFq/mYOHGiWltb9dprr+m1117Tfffdpz/6oz/SmTNnJEk7duzQzp071dbWpq6uLlVXV2vZsmW6evXq0P81AAC4jFVL9EgXUqvdkIC/XAumjI0GjcTHXjPs2y5jxozR97//fT300EOqqanRpk2btGXLFklSOBxWVVWVtm/frocffjirz2PnAwDgdsff69PaPa8mPf+n996hZzp/5dqZLOnYctvl+vXr2r9/v/r7+7VgwQJ1d3ert7dXy5cvj77H5/NpyZIlOn78eMrPCYfDCoVCcT8AALiZVSHpCCkaPCR3zmTJl5zDx+nTp3XLLbfI5/PpkUce0aFDh/SFL3xBvb29kqSqqqq491dVVUVfs9LS0iK/3x/9qa2tzXVJAAAUFatC0m/dW5d0rdaL12izkXOH0zvvvFOnTp3SlStX9Nxzz2ndunU6duxY9HXDiI96pmkmPRfriSeeUGNjY/RxKBQigAAAXC+xC6kkPdPZHRdAvHiNNhs5h4+RI0fqt37rtyRJc+fOVVdXl/7u7/4uWufR29urQCAQff+lS5eSdkNi+Xw++Xy+XJcBAEDRi22JLt2o8XDzTJZ8GfZsF9M0FQ6HVVdXp+rqah09elRz5syRJF27dk3Hjh3T9u3bh71QAACKSeLwt2y4fSZLvuQUPrZu3aoVK1aotrZWV69e1f79+/Xyyy/ryJEjMgxDmzZtUnNzs6ZOnaqpU6equblZo0aN0tq1awu1fgAAbJdLD49EibshpSin8PHf//3f+sY3vqGenh75/X7dddddOnLkiJYtWyZJ2rx5swYGBvToo49Gm4y99NJLqqioKMjiAQCwW6oeHounjS/5UJEtptoCAJCDVD08/t+379GCKWMdWFFxYKotAAAFYtXDo1RvrQwV4QMAgBxY9fCI3FrJNEgONwz7tgsAAKXG6tbKcIpQSw07HwAADEHs8Ld0g+SQjPABAMAwdff10zo9B4QPAEBJKURdBkWouSF8AABKxoGu86pvbdfaPa+qvrVdB7rOSxp+IElXhIpkFJwCAEpCqrqMKwO/1vYX3xp2oSit07PHzgcAoCSkqsto/Sx4SMMvFI0tQkVqhA8AQEmwqssYISmxzzeFooVH+AAAlASruowtK6ZbFoqOGjmCZmEFRM0HAMDTeoID6u7rV9240ZZ1GbeOuklbD76p66apMsPQqjk1Wv3j40k1ILGfw7HK8BA+AACelarraGx4iA0ko0aOiAYPKf9FqbiBYxcAgCfl0nU0Uijaf+16wYtSQfgAAHjUULqOUpRqD8IHAMCThtJ1NJeiVLqXDh01HwAAT4oEidhi0my6jmZTlEr30uExTDNxM8lZoVBIfr9fwWBQlZWVTi8HAOByPcGBvHQdzdfneFUu39/sfAAAPC3gL89LWMjX54CaDwAAYDPCBwAAsBXhAwAA2IrwAQAAbEX4AAC4Uk9wgOFvLsVtFwCAK8QOdus4+6HlzBa4A+EDAFD0YgfERZqNRppURWatLJ42nquwLsGxCwCgqCUOiDP1m+ARwawVdyF8AACKmtWAuETMWnEXwgcAoKhZDYgzpOhzzFpxH2o+AABFLdWAuMThb3APwgcAoOhZTZqVROhwKcIHAMAVshnsFnsdl2BSvAgfAABPiL2OS++P4kbBKQDA9RKv40Z6f9D9tDgRPgAArmd1HZfeH8WL8AEAcD2r67j0/ihehA8AgOOGOyQuch23zLiRQOj9UdwoOAUAOCpfhaKpruOi+LDzAQBwTL4LRQP+ci2YMpbgUeTY+QAAOCZVoeiJc5c15hb6dXgV4QMA4JhIoWhsADEkfWf/Sfp1eBjHLgAAW8UWlyYWika+lOjX4W3sfAAAbJOquDRSKPpRf1gb952M+zORfh0cv3gH4QMAYItUxaWLp42Pzm3pCQ4kHcPQr8N7OHYBANgimy6k9OsoDex8AABsYVVcarWrQb8O78tp56OlpUXz5s1TRUWFJkyYoFWrVuntt9+Oe8/69etlGEbczz333JPXRQMA3CNSYCop610N+nV4W047H8eOHdOGDRs0b948ffrpp3ryySe1fPly/eIXv9Do0aOj77v//vu1d+/e6OORI0fmb8UAANewKjDtbGpgV6PE5RQ+jhw5Evd47969mjBhgk6cOKHFixdHn/f5fKqurs7PCgEArpSqwLSzqUELpox1dnFw1LAKToPBoCRpzJgxcc+//PLLmjBhgqZNm6Zvf/vbunTpUsrPCIfDCoVCcT8AAPdjzD1SGXL4ME1TjY2NWrRokWbOnBl9fsWKFfqnf/ontbe36wc/+IG6urp03333KRwOW35OS0uL/H5/9Ke2tnaoSwIAFBHG3CMVwzRNM/Pbkm3YsEE//elP1dnZqYkTJ6Z8X09PjyZPnqz9+/drzZo1Sa+Hw+G4YBIKhVRbW6tgMKjKysqhLA0AUAA9wQF196Wft5L4ngNd57X14Ju6bprRAlNapXtTKBSS3+/P6vt7SFdtH3vsMb3wwgvq6OhIGzwkKRAIaPLkyXrnnXcsX/f5fPL5fENZBgDAJtmMvc/UvZQCU0TkdOximqY2btyogwcPqr29XXV1dRn/zEcffaQLFy4oEAgMeZEAAOdkM/Y+3Xu4NotEOe18bNiwQfv27dPzzz+viooK9fb2SpL8fr/Ky8v18ccfa9u2bXrggQcUCAR07tw5bd26VePGjdPq1asL8g8AAAxfuiOVTIWj3X39+p/+aynfQ+hAopzCx65duyRJS5cujXt+7969Wr9+vcrKynT69Gk9++yzunLligKBgBoaGnTgwAFVVFTkbdEAgPzJdKSSqjPpGxev6GvPvKJBUzJ04yc2f1BcilSGXHBaKLkUrAAAhqcnOKD61vakYNHZ1BC3Y5FYOLr5/ju1/chbcX/OkGR8FlIoLi09BS84BQB4Q7ojldjwkVg4avXnTEk//Mocjb3FR3Ep0iJ8AEAJy3bYm6To2PsIqz939+2fJ3Qgo2F1OAUAuNtQR9gP9c8BEjUfAADdqP0YSi+Oof45eA81HwCAnCQeqRT6z6G0cewCAABsRfgAAI/rCQ7o+Ht90Y6kiY8Bu3HsAgAelthAbPWc23To5MW0M1oishkkBwwF4QMAPMpq3spzr1+Mvh6Zv7J42vikcJHNIDlgqDh2AQCPsmoElih2RktENoPkgOEgfACAR0UaiKVj1VAs0yA5YLgIHwDgUVaNwB744m0ZG4NZhRaGxCGfaDIGAB6X2Agsm8ZgiYPkGBKHTHL5/iZ8AICH5POGCt1LkQs6nAJACcr3DRW6l6JQqPkAAA/ghgrchPABAB7ADRW4CeEDAFwiXVt0bqjATQgfAOACB7rOq761XWv3vKr61nYd6Dof97rVtVqra7RAMeC2CwAUuZ7ggOpb2+OOVcoMQ51NDUnhghsqcAq3XQDAQ9LVcyQGDG6owA04dgGAIkc9B7yG8AEARS5dPUe6IlSgWHHsAgAu8OC8SVo8bXxcPQdj7+FW7HwAgEsE/OVaMGVsdMeDpmJwK8IHALgQTcXgZoQPAHAhilDhZoQPAHAhmorBzSg4BQCXsipCBdyA8AEADugJDqi7r19140ZHC0hjH2eLpmJwI8IHANgs8Yrs6jm36dDJi1yZRcmg5gMAbGR1Rfa51y9yZRYlhfABADayuiKbiCuz8DrCBwDYyOqKbKLIlVlap8OrCB8AYCOrK7IPfPG2pCuzHWc/VH1ru9bueVX1re060HXeyWUDeWWYpplhA9BeoVBIfr9fwWBQlZWVTi8HAAqiJzgQd0U29rEk1be2xx3PlBmGOpsauNmCopXL9ze3XQDAAYlXZGMfH3+vL2XrdMIHvIBjFwAoMrROh9cRPgCgyNA6HV7HsQsAFCFap8PLCB8AUKRonQ6v4tgFAADYivABAHlEYzAgM45dAGAYYqfRdpz9MG5gHAPiAGuEDwAYotjptJGbsZH2HJEBcYunjZekaEChhgPI8dilpaVF8+bNU0VFhSZMmKBVq1bp7bffjnuPaZratm2bampqVF5erqVLl+rMmTN5XTQAOC1xOq2p3wSPiOumqb2d52iTDiTIKXwcO3ZMGzZs0CuvvKKjR4/q008/1fLly9Xf3x99z44dO7Rz5061tbWpq6tL1dXVWrZsma5evZr3xQOAU7KZTjtC0jOdv4q+L7IbQj0ISl1O4ePIkSNav369ZsyYodmzZ2vv3r06f/68Tpw4IenGrsff/u3f6sknn9SaNWs0c+ZM/cM//IM++eQT7du3ryD/AABwglUXUkOKPldmGPrWvXUp26QDpWxYNR/BYFCSNGbMGElSd3e3ent7tXz58uh7fD6flixZouPHj+vhhx9O+oxwOKxwOBx9HAqFhrMkACiY2OLSSBfSrQff1HXTjHYhjW0MJknPdHYnDYijTTpK3ZDDh2maamxs1KJFizRz5kxJUm9vrySpqqoq7r1VVVV6//33LT+npaVFTz311FCXAQC2iC0ujb3JYtWFNLao1CqgUHSKUjfk8LFx40a98cYb6uzsTHrNMOL3Ik3TTHou4oknnlBjY2P0cSgUUm1t7VCXBQB5E9npGD2yLK64NPYmS6YupLRJB5INKXw89thjeuGFF9TR0aGJEydGn6+urpZ0YwckEAhEn7906VLSbkiEz+eTz+cbyjIAoGASr9Fa3WTJdsQ9bdKBeDkVnJqmqY0bN+rgwYNqb29XXV1d3Ot1dXWqrq7W0aNHo89du3ZNx44d08KFC/OzYgAoMKtrtImo3QCGLqedjw0bNmjfvn16/vnnVVFREa3x8Pv9Ki8vl2EY2rRpk5qbmzV16lRNnTpVzc3NGjVqlNauXVuQfwAADEVi8WisVNdoRxg3jlyo3QCGJ6fwsWvXLknS0qVL457fu3ev1q9fL0navHmzBgYG9Oijj+ry5cuaP3++XnrpJVVUVORlwQAwXFbFo4unjY+Gkcg12sRbKgcfXaBPrg1SuwEMk2GaZoY2OfYKhULy+/0KBoOqrKx0ejkAPKYnOKD61va4YGFIMj4LG5EwIinplgpzWoDUcvn+ZrYLgJJidaRiSjITbrJ0NjWos6mBWypAARA+AHhebH2H1ZFKoshNlgVTxhI6gAIgfADwNKv6jtjGXyOUPBSOmyxAYVHzAcBzYpuDrf7x8aTC0c6mBkmKHql0nP2Q+g5gmKj5AFCysm0OFnukQhdSwF6EDwCeMZzmYHQhBeyTU4dTAChm6ZqDSTQHA4oFOx8APIPmYIA7sPMBwDMC/nK1rJmlss+maEd2OmbXfp5rs0ARYecDgKslzmiheBQofoQPAK5l1cPjwXmTKB4FihzHLgBcKfFmS6Qtek9wwNmFAciI8AHAlaxutkR6eAAoboQPAK4UudkSi7bogDsQPgC4UqqbLdR6AMWPglMArsXNFsCdCB8AXI2bLYD7cOwCoCj0BAd0/L0+bqsAJYCdDwCOS9WvI7GBGABvIHwAcFSqfh1XBn6t7S++FRdIFk8bTxgBPIDwAcBRqfp1tL74lsyYQNL03GkZnw2Ni90dAeA+1HwAcJRVv44RUjR4RJgS3UwBjyB8AHCUVb+OLSumJwWSRHQzBdyLYxcAjrPq13HrqJu09eCbum6aN3ZCPvuJoJsp4F6EDwBFIbFfR2Ig6Tj7YTSM0M0UcDfCB4CiFRtI6GYKeAfhA4Br0M0U8AYKTgEUnFX3UjqaAqWLnQ8Aw5KpC6lV91JJlh1NAZQGwzQTb9M7KxQKye/3KxgMqrKy0unlAEgjVVv0iJ7ggOpb2+OaiI2QJENxz5UZhjqbGjhSAVwsl+9vjl0ADEmqtuixxyhW3UsHJcuOpvTsAEoH4QPAkKRqix4bIlJ1L018jp4dQGkhfAAYEqtgkRgirLqXtjwwK+k5enYApYWCUwBDEgkWmRp/perPQc8OoHRRcApgWHqCA4QIADl9f7PzAZSoTFdks0XjLwC5InwAJSjTFVkAKCQKToESk80VWQAoJMIHUGKyuSI7XLROB5AOxy5AiYlckU3sMJqvPhsc6QDIhJ0PoMRY9d7IV58NjnQAZIOdD6AEpeq9MVzpjnS4EQMggvABlKhsrsjmeh230Ec6ALyBYxcAlg50nVd9a7vW7nlV9a3tOtB1XlL6YtJCHukA8A52PgAkSVW7cWXg19r+4ltpi0kLdaQDwDty3vno6OjQypUrVVNTI8MwdPjw4bjX169fL8Mw4n7uueeefK0XgA1S1W60fhY8pPTFpAF/uRZMGUvwAGAp5/DR39+v2bNnq62tLeV77r//fvX09ER//uVf/mVYiwRQeLHHKVYTa0dISpwEle/+IABKQ87HLitWrNCKFSvSvsfn86m6unrIiwJgL6veHIkTazfff6e2H3mLYlIAw1aQmo+XX35ZEyZM0K233qolS5bo6aef1oQJEyzfGw6HFQ6Ho49DoVAhlgQghVT1HZ1NDepsaoir3bh11E1xgYRiUgBDkffwsWLFCn35y1/W5MmT1d3drb/8y7/UfffdpxMnTsjn8yW9v6WlRU899VS+lwEgS+l6cyTWbVBMCiAf8h4+Hnzwweh/njlzpubOnavJkyfrpz/9qdasWZP0/ieeeEKNjY3Rx6FQSLW1tfleFoAUcu3NkU1/EABIp+B9PgKBgCZPnqx33nnH8nWfz6fKysq4HwD2oTcHALsVvM/HRx99pAsXLigQCBT6rwIwRBynALBTzuHj448/1rvvvht93N3drVOnTmnMmDEaM2aMtm3bpgceeECBQEDnzp3T1q1bNW7cOK1evTqvCweQXxynALBLzuHjtddeU0NDQ/RxpF5j3bp12rVrl06fPq1nn31WV65cUSAQUENDgw4cOKCKior8rRoAALiWYZqJbYOcFQqF5Pf7FQwGqf8AAMAlcvn+ZrAcAACwFeEDAADYivABAABsRfgASkDs0DgAcFrB+3wAyF5PcEDdff2qGzc6b9derYbGPThvUl4+GwCGgvABFIl8hoRIiBk9ssxyaNziaePp6QHAMYQPoAikmixrFRISd0cSH8eGGENS4l36yNA4wgcApxA+AIfEhoZ0k2VjQ0Li7sjqObfp0MmL0cdb7p+u7Ufein6WVROfdEPjAMAOhA/AAYkhYsv90zNOlrXaHXnu9YvR1wdNafuLb2nQ4u+LfDZD4wAUA8IHYDOrELHjyNvasmK6drz4tq6bZjQkSNLx9/pS7o4kGpRkGJKZEGIOPrpAn1wbZGgcgKJA+ABsluqI5a7bblVnU0N0smzH2Q9V39qednckUZlhaPOKO5NCzOzazxf2HwUAOSB8ADarGzc65RFLZLJstrsjq+bU6PDJD+KCxoPzJun/zK6Jhhh2OgAUG8IHYLOAv1wta2Zp68E340JDbEjIdnck4C/Xd//gzqSgEQkxAFCMCB+AAx6cN0mLp41PuTuRze5IBEEDgNvQXh1wSMBfrgVTxloGh8juSJlhSOKWCgBvYecDKFKZdkcAwK0IH0AR40gFgBdx7ALYgKmyAPAb7HwABcZUWQCIx84HUECpBsaxAwKglBE+gAJKNzAOAEoV4QMooEi/jlhMlQVQ6ggfQAHRrwMAklFwChRAT3BA3X39qhs3mn4dAJCA8AHkWarbLYQOALiBYxcgj7jdAgCZET6APOJ2CwBkRvgA8ojbLQCQGeEDyCNutwBAZhScAmnE3lrJNkBwuwUA0iN8oGRlChbDmcnCNFoASI3wgZKUKVikurWyeNp4QgUADBM1Hyg52VyH5dYKABQOOx8oGZFjlv/pv5YyWER2NSK3VmLfF7m1MpQ6EADAbxA+UBJij1kM3fiJzR9WwaJlzSxtPfimrptm9NZKx9kPh1wHAgC4wTBN08z8NvuEQiH5/X4Fg0FVVlY6vRx4QE9wQPWt7XG7GIYk47OdjUiwkJQULGJvrUhK+pwyw1BnUwM7IABKXi7f3+x8wPOs6jdMST/8yhyNvcVnGSwidSCdTQ1aMGWsJOn4e32WxzUnzl3WmFs4hgGAbBE+4Hmp6jfuvv3z0bCQKlhkqgMxJH1n/0mOYQAgB9x2gedl03U0m7boiZ8T+R8PQ+QAIDfsfKAkZOo6GgkWiQWmie+L/ZyP+sPauO9k3OuJuyUAgGSED5SMTF1Hs22LHvmcnuBAyuu4AIDUOHYBYgT85VowZWxWOxcMkQOAoWHnA0Wt2Bt6MUQOAHJH+EDRGs5gNzsxRA4AcsOxC4pSNvNXAADuRPhAUWKwGwB4V87ho6OjQytXrlRNTY0Mw9Dhw4fjXjdNU9u2bVNNTY3Ky8u1dOlSnTlzJl/rRYnIpu9GOj3BAR1/r4+dEgAoQjmHj/7+fs2ePVttbW2Wr+/YsUM7d+5UW1uburq6VF1drWXLlunq1avDXixKx3BukhzoOq/61nat3fOq6lvbdaDrfKGXCwDIwbAGyxmGoUOHDmnVqlWSbux61NTUaNOmTdqyZYskKRwOq6qqStu3b9fDDz+c8TMZLIdYPcGBjDdJYm/ESAx/AwAnODZYrru7W729vVq+fHn0OZ/PpyVLluj48eOW4SMcDiscDkcfh0KhfC4JLpfpJknijZhvLarLOKMFAOCsvBac9vb2SpKqqqrinq+qqoq+lqilpUV+vz/6U1tbm88lwcOsbsQ887PuYdWKAAAKryC3XQwj/v/9TdNMei7iiSeeUDAYjP5cuHChEEuCB1ndiBmU9K1Fd9B1FACKWF6PXaqrqyXd2AEJBALR5y9dupS0GxLh8/nk8/nyuQyUCKsR92WGoW8uul3fXHQ7XUcBoEjldeejrq5O1dXVOnr0aPS5a9eu6dixY1q4cGE+/yog7Y2YXGa0AADslfPOx8cff6x33303+ri7u1unTp3SmDFjNGnSJG3atEnNzc2aOnWqpk6dqubmZo0aNUpr167N68IBidkqAOBGOYeP1157TQ0NDdHHjY2NkqR169bp7//+77V582YNDAzo0Ucf1eXLlzV//ny99NJLqqioyN+qgRjMVgEAdxlWn49CoM8HAADuk8v3N7Nd4BhaoANAacrrbRcgW4nNwVrWzNKD8yY5vSwAgA3Y+YDtrJqDbT34JjsgAFAiCB+wnVVzsEgL9GxwXAMA7saxC2yXqjlYNi3QOa4BAPdj5wO2iexYSErZHCzTn+e4BgDcj50P2MJqx6KzqSGpOVhPcEDdff2qGzc6KYykO66hzwcAuAfhAwWXaseis6lBC6aMjb4v05HKcI5rAADFg2MXFFw2BabZHKmkm+UCAHAPdj5QcNnsWGR7pMIsFwBwP3Y+UHDZ7FhEAkqsVEcqTKwFAHdj5wO2yLRjEQkoWw++qeumyZEKAHgY4QO2yTR9liMVACgNhA8UlUwBBQDgftR8AAAAWxE+AACArQgfAADAVoQPxMlmYixTZQEAw0HBKaKymRjLVFkAwHCx8wFJ2bU3Z6osACAfCB+QlN38lXTv4SgGAJAtjl0gKbv5K6ne88bFK/raM69wFAMAyAo7H5CU3fwVq/dsvv9ObX/xLY5iAABZY+cDUVbtzXuCA+ru61fduNEK+MuT3pPtNFoAACIIH4gT29481c2WxBbomY5rAACIxbELLGV7syWb4xoAAGKx8wFLuRynZHNcAwBABOGjhKULCNncfomVzXENAAASxy4l60DXedW3tmvtnldV39quA13n414f6nEKjcgAAJmw81GCUgWExdPGx4ULq+OUTLj9AgDIhPBRQiLHLP/Tfy3rgJB4syWTXI9rAAClh/BRImLrMAzd+InNH/kKCJHjmq0H39R10+T2CwAgCeGjBCQes5i6ET4iOxT5DghDOa4BAJQOwkcJsKrDMCX98CtzNPYWX0ECQq7HNQCA0kH4cKFce2ikqsO4+/bPExAAALYjfLhMqh4aiYEk8TF1GACAYkH4cJFUV2SvDPw6Oll2hCGtnnObDp28mBRQqMMAABQDwoeLpOqh0friWzJjAslzr1+Mvp7Yw4PQAQBwGh1OXSRSuxFrhBQNHqlEengAAFAMCB8uYtXyfMuK6UmBJBFNvgAAxYRjF5exqt24ddRNccWkq+bU6PDJDyguBQAUJcM0M23a2ysUCsnv9ysYDKqystLp5bhGT3AgaaQ9xaUAALvk8v3NzodHJBaTUlwKAChW1HwAAABbET4AAICt8h4+tm3bJsMw4n6qq6vz/deUlJ7ggI6/16ee4IDTSwEAYNgKUvMxY8YM/du//Vv0cVlZWSH+mpKQqp06AABuVZDw8bnPfY7djiGKnckiybKdeqRbKQAAblSQ8PHOO++opqZGPp9P8+fPV3Nzs+644w7L94bDYYXD4ejjUChUiCW5QuIux7cW1Vm2Uz/X9wnhAwDgWnmv+Zg/f76effZZ/eu//qv27Nmj3t5eLVy4UB999JHl+1taWuT3+6M/tbW1+V6SK1gNjXvmZ91J3UvpVgoAcLuCNxnr7+/XlClTtHnzZjU2Nia9brXzUVtbW3JNxo6/16e1e15Nev5P771D/7ezO65bKTUfAIBiU1RNxkaPHq1Zs2bpnXfesXzd5/PJ5/MVehlFLzI0LvaYpcww9M1Ft+ubi26nWykAwDMK3ucjHA7rl7/8pQKBQKH/KlezGhoXmckS8JdrwZSxBA8AgCfkfefju9/9rlauXKlJkybp0qVL+pu/+RuFQiGtW7cu33+V68XebAn4yy2HxgEA4DV5Dx//9V//pa9+9avq6+vT+PHjdc899+iVV17R5MmT8/1XFbXEYJEoVf8OZrIAALyOqbYFkKkxWE9wQPWt7Un1HZ1NDQQPAIAr5fL9zWyXIUjX7tzqyuzWg2/Gvbe7rz9l/w4AALyu4LddvCD2CKXj7IdpdzUyBYvuvn6NHllmebOF/h0AgFJA+Mgg9ggl0u8rkhms2p2nujL7xsUr+tozr0RDy+o5t+nwyQ/i+ndw5AIAKAWEjzQSj1CsimMS251HrsxuPfhmNFhsvv9ObX/xrbijmMMnP9DBRxfok2uD3GwBAJQUwkcaVkcoiayOSxKvzKY6ivnk2qAWTBmb51UDAFDcCB9pWB2hGJKMz55Ld1ySeGWWGg8AAG4gfKRhdYTSvGZmzo3AUn0ORy0AgFJEn48s9AQH8tJ1NF+fAwBAsSmqwXJekK+uo3QvBQCAJmMAAMBmhA8AAGArwgcAALAV4SNBurktAABg+Cg4jZFpGm06sfNfKCoFACC1kgof6QJCqmm0sXNbUhlOaAEAoNSUTPjIFBDSTaNNFz6GE1oAAChFJVHzkSogxNZ1RFqpx8qmBXq60AIAAJKVRPjIJiBEWqCXGTcSSLYt0IcaWgAAKFUlcexiNSAuEhBi60ASp9EytwUAgPwrmdkuB7rOJwUESXkrFGVuCwCglOXy/V0y4UOKDwiSVN/anrQb0tnUQHgAACBHDJZLIXaw2/H3+oZ0uwUAAAxPSRScWqFQFAAAZ5Rs+Bjq7RYAADA8JXXskmgot1sAAMDwlHT4kOLrQAAAQOGV7LELAABwBuEDAADYivABAABsRfgAAAC2InwAAABbET4AAICtCB8AAMBWhA8AAGArwgcAALAV4QMAANiK8AEAAGxVdLNdTNOUJIVCIYdXAgAAshX53o58j6dTdOHj6tWrkqTa2lqHVwIAAHJ19epV+f3+tO8xzGwiio0GBwf1wQcfqKKiQoZh5PWzQ6GQamtrdeHCBVVWVub1s/Eb/J7twe/ZPvyu7cHv2R6F+j2bpqmrV6+qpqZGI0akr+ooup2PESNGaOLEiQX9OyorK/kvtg34PduD37N9+F3bg9+zPQrxe8604xFBwSkAALAV4QMAANiqpMKHz+fT9773Pfl8PqeX4mn8nu3B79k+/K7twe/ZHsXwey66glMAAOBtJbXzAQAAnEf4AAAAtiJ8AAAAWxE+AACArUomfPz4xz9WXV2dbr75Zt1999362c9+5vSSPKelpUXz5s1TRUWFJkyYoFWrVuntt992elme19LSIsMwtGnTJqeX4jkXL17U17/+dY0dO1ajRo3S7/zO7+jEiRNOL8tTPv30U/3FX/yF6urqVF5erjvuuEN//dd/rcHBQaeX5nodHR1auXKlampqZBiGDh8+HPe6aZratm2bampqVF5erqVLl+rMmTO2rK0kwseBAwe0adMmPfnkkzp58qTuvfderVixQufPn3d6aZ5y7NgxbdiwQa+88oqOHj2qTz/9VMuXL1d/f7/TS/Osrq4u7d69W3fddZfTS/Gcy5cvq76+XjfddJNefPFF/eIXv9APfvAD3XrrrU4vzVO2b9+un/zkJ2pra9Mvf/lL7dixQ9///vf1wx/+0OmluV5/f79mz56ttrY2y9d37NihnTt3qq2tTV1dXaqurtayZcuiM9YKyiwBv/u7v2s+8sgjcc9Nnz7dbGpqcmhFpeHSpUumJPPYsWNOL8WTrl69ak6dOtU8evSouWTJEvPxxx93ekmesmXLFnPRokVOL8PzvvSlL5kPPfRQ3HNr1qwxv/71rzu0Im+SZB46dCj6eHBw0KyurjZbW1ujz/3v//6v6ff7zZ/85CcFX4/ndz6uXbumEydOaPny5XHPL1++XMePH3doVaUhGAxKksaMGePwSrxpw4YN+tKXvqTf//3fd3opnvTCCy9o7ty5+vKXv6wJEyZozpw52rNnj9PL8pxFixbp3//933X27FlJ0n/+53+qs7NTf/iHf+jwyrytu7tbvb29cd+NPp9PS5YsseW7segGy+VbX1+frl+/rqqqqrjnq6qq1Nvb69CqvM80TTU2NmrRokWaOXOm08vxnP379+v1119XV1eX00vxrF/96lfatWuXGhsbtXXrVv385z/Xd77zHfl8Pv3Jn/yJ08vzjC1btigYDGr69OkqKyvT9evX9fTTT+urX/2q00vztMj3n9V34/vvv1/wv9/z4SPCMIy4x6ZpJj2H/Nm4caPeeOMNdXZ2Or0Uz7lw4YIef/xxvfTSS7r55pudXo5nDQ4Oau7cuWpubpYkzZkzR2fOnNGuXbsIH3l04MAB/eM//qP27dunGTNm6NSpU9q0aZNqamq0bt06p5fneU59N3o+fIwbN05lZWVJuxyXLl1KSnzIj8cee0wvvPCCOjo6NHHiRKeX4zknTpzQpUuXdPfdd0efu379ujo6OtTW1qZwOKyysjIHV+gNgUBAX/jCF+Ke++3f/m0999xzDq3Im/78z/9cTU1N+spXviJJmjVrlt5//321tLQQPgqourpa0o0dkEAgEH3eru9Gz9d8jBw5UnfffbeOHj0a9/zRo0e1cOFCh1blTaZpauPGjTp48KDa29tVV1fn9JI86fd+7/d0+vRpnTp1Kvozd+5cfe1rX9OpU6cIHnlSX1+fdFX87Nmzmjx5skMr8qZPPvlEI0bEfxWVlZVx1bbA6urqVF1dHffdeO3aNR07dsyW70bP73xIUmNjo77xjW9o7ty5WrBggXbv3q3z58/rkUcecXppnrJhwwbt27dPzz//vCoqKqK7TX6/X+Xl5Q6vzjsqKiqS6mhGjx6tsWPHUl+TR3/2Z3+mhQsXqrm5WX/8x3+sn//859q9e7d2797t9NI8ZeXKlXr66ac1adIkzZgxQydPntTOnTv10EMPOb001/v444/17rvvRh93d3fr1KlTGjNmjCZNmqRNmzapublZU6dO1dSpU9Xc3KxRo0Zp7dq1hV9cwe/TFIkf/ehH5uTJk82RI0eaX/ziF7n+WQCSLH/27t3r9NI8j6u2hfHP//zP5syZM02fz2dOnz7d3L17t9NL8pxQKGQ+/vjj5qRJk8ybb77ZvOOOO8wnn3zSDIfDTi/N9f7jP/7D8v+T161bZ5rmjeu23/ve98zq6mrT5/OZixcvNk+fPm3L2gzTNM3CRxwAAIAbPF/zAQAAigvhAwAA2IrwAQAAbEX4AAAAtiJ8AAAAWxE+AACArQgfAADAVoQPAABgK8IHAACwFeEDAADYivABAABsRfgAAAC2+v9TBfa2EOWeqQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X, y, 'o', markersize=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "916a429e-677f-46fa-bc38-5b478a03f423",
   "metadata": {},
   "source": [
    "Après avoir créé nos données, on crée alors un réseau de neurones à l'aide de la fonction Sequential() de Keras. Sequential() crée un modèle de réseau de neurones simple (couche par couche). La fonction Dense() permet de préciser que tous les neurones sont connectés (ici, il n'y en a qu'un...). Le paramètre unit définit le nombre de neurones, tandis que le paramètre input_shape précise que l’entrée du modèle contient 1 variable (1 feature, x est de dimension 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f5083863-1e30-4fc5-bf6b-106d41a93327",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Serge\\anaconda3\\Lib\\site-packages\\keras\\src\\layers\\core\\dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n",
      "  super().__init__(activity_regularizer=activity_regularizer, **kwargs)\n"
     ]
    }
   ],
   "source": [
    "# Modèle Keras : 1 neurone\n",
    "model = Sequential([Dense(units=1, input_shape=(1,), activation='linear')])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cda239a1-d4af-4e10-aeae-cc05ada97635",
   "metadata": {},
   "source": [
    "La méthode compile() permet de déterminer la méthode d'optimisation et la métrique permettant d'évaluer la qualité de nos prédictions. Ici, on utilise la méthode de descente de gradient stochastique (c'est la méthode de descente de gradient, mais on utilise seulement une partie des données pour ajuster le fameux gradient, car prendre tout le jeu de données peut être très couteux). MSE c'est ce que l'on cherche à minimiser ici, l'erreur quadratique moyenne comme dans une régression linéaire classique.<br>\n",
    "La méthode fit() calcule le modèle, l'epoch c'est pour faire simple le nombre d'itération de l'algorithme et l'argument verbose permet de masquer ces epoch.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ef4a5955-017b-4443-b889-3744f8274a83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<keras.src.callbacks.history.History at 0x1aa488b9190>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.compile(optimizer='sgd', loss='mse')\n",
    "# Entraînement\n",
    "model.fit(X, y, epochs=200, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eb1b11d-cbe6-4779-9b52-5650d56134f8",
   "metadata": {},
   "source": [
    "Le méthode layers[] permet d'avoir accès aux couches, donc aux neurones et get_weights plus précisément aux poids finaux du modèle et le biais associé. Ici ce sont les paramètres de notre régression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ab49a75a-f365-4d82-ae58-87aedd601f24",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pente (a) estimée (proche de 3) : 2.9973\n",
      "Intercept (b) estimé (proche de 2) : 2.0662\n"
     ]
    }
   ],
   "source": [
    "# Récupération des coefficients\n",
    "weights, bias = model.layers[0].get_weights()\n",
    "a = weights[0][0]  # pente\n",
    "b = bias[0]        # intercept\n",
    "print(f\"Pente (a) estimée (proche de 3) : {a:.4f}\")\n",
    "print(f\"Intercept (b) estimé (proche de 2) : {b:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ed6eec6-628a-4298-afe2-c325e00d2aeb",
   "metadata": {},
   "source": [
    "La méthode predict() permet d'obtenir des prédictions en se fondant sur les poids calculés. On peut alors reprendre les valeurs de X pour afficher la droite de régression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "08e00712-8b5c-4f88-a457-0ad0ffabd9b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m4/4\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 15ms/step\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIOUlEQVR4nO3de1xUdf7H8deZAUZQwLuMiYamWamtpZmXvNRqalmKlttVs4tuaprbxUsX3TZR29pKy1LL6lemreKlTNOyNNc10XLTbprhJcUQL4MiAjNzfn+MjCAXQWEGZt7Px4PH7pk5M3xkV+bt9/L5GqZpmoiIiIj4iMXfBYiIiEhwUfgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERnwrxdwFnc7vdHDhwgMjISAzD8Hc5IiIiUgKmaXL8+HHq16+PxVL82EaFCx8HDhwgNjbW32WIiIjIedi3bx8NGjQo9p4KFz4iIyMBT/FRUVF+rkZERERKIj09ndjYWO/neHEqXPjInWqJiopS+BAREalkSrJkQgtORURExKcUPkRERMSnFD5ERETEpxQ+RERExKcUPkRERMSnFD5ERETEpxQ+RERExKcUPkRERMSnFD5ERETEpxQ+RERExKcUPkRERAJEiiOTDbvSSHFk+ruUYlW4s11ERESk9BYk7WVc4jbcJlgMSIhvycC2DQvcl+LIJDktg7jaVbFHh/uhUoUPERGRSi/FkekNHgBuE8Ynbqdzszr5AkZJA0p507SLiIhIJZecluENHrlcpsnutJPe66ICij+maBQ+REREKrm42lWxnHWSvdUwuLh2hPc6N6BYcBPOKaBgQPEVhQ8REZFKzh4dTkJ8S6yGJ4FYDYPJ8S3yTbnE1a5KPeMo74dO5l+hMwGzQEDxFa35EBERCQAD2zakc7M67E47ycW1I7BHh+dfXHroP6yNfJoq2UfIMG1cYjnIg/16+GXRqcKHiIhIgLBHh3vDRO7iUovp5PGQfzM05GOqADm1L2dHh1f5vyYttNtFREREykbu4lK7eYjpYdO5yvIrABlXDqbqzVNpHVrFr/VpzYeIiEiASU7LoLuRxKe2cVxl+ZV0M4Jh2aP5vtUz4OfgARr5EBERCSw5p7jy+3/wZthcALa6mzAiZyQp1ONZPywuLYzCh4iISKBI+xUWDqbqwW0AzHLezDTn7ZhGaIHdL/6k8CEiIhII/rcAlo+B7BMQUQv6vUmfup1omWf3S0Wh8CEiIlIJFHkmS3YGfPoEbH3fc92oE/SfA1F27FChQkcuhQ8REZEKrrAzWTo3q8MfO7/lig2jCD2yEwwLdHkSOj8OFqu/Sy6WwoeIiEgFVtiZLGMXfc+d1jU8HfIeoUYOJ211ifjL2xB3nX+LLSFttRUREanAzj40LpKTTA+dzvOhb1HFyOFL15V0SX+OlJpt/FdkKWnkQ0REpALLPTTObUIrYxfTQ6fTyJJKjmllmnMgc1y9MbGwO+1khVzfURiFDxERET8rcjEppw+N69eCX5dN43Hrh4QZLva56zAyZyRbzUuAgifYVnQKHyIiIn5U1GJSbxgJOcnAnY9DyGcAZDa9maS4cWz7eC+5J9NWpB4eJWGYpmme+zbfSU9PJzo6GofDQVRUlL/LERERKTcpjkw6TlmTb02HARinp1naWX5ibtSbRJxKBasNbnwe2j4AhkGKIzPfCbb+VprPb418iIiI+MnZi0kBTMAw3TxiXcKokEVYT5k4a1xCyMB3IKal9768J9hWNgofIiIifpJ3MWmuuhzl5dDX6GD9EYCFrs7E3vga7WIa+qnKsqettiIiIn5ijw4nIb4lVsMAoKvlf3xqG0cH649kmDbGZA/jSedfaWiv4+dKy5ZGPkRERMpJcbtYcg1s25DOTarj/uI5LvphFgA/uRsyPOcR9nBRpVtMWhKlGvmYOXMmrVq1IioqiqioKNq3b8+KFSu8z5umycSJE6lfvz7h4eF07dqVH374ocyLFhERqegWJO2l45Q13Dn7GzpOWcOCpL2F33hsL/bEeG/woO0DVB+1jucfiGf92G4MbBs40y25ShU+GjRowJQpU9i8eTObN2/m+uuv59Zbb/UGjGnTpvHSSy8xY8YMkpKSiImJoXv37hw/frxcihcREamICmuJPj5xOymOTFIcmWzYlUaKIxN++hje6AS/J4EtGm5/D256EXutGrRvUivgRjxyXfBW25o1a/LCCy8wZMgQ6tevz+jRo3nyyScByMrKol69ekydOpWhQ4eW6P201VZERCq7DbvSuHP2NwUef+i6xsxZ/xuhZjYTQj/gXutqzxMXtYEBb0GNi31baBkqzef3eS84dblczJ8/n4yMDNq3b09ycjIHDx6kR48e3ntsNhtdunRhw4YNRb5PVlYW6enp+b5EREQqs9xdLHlZgDnrf6MRKSSGPesNHieufhiGrKzUwaO0Sh0+tm3bRrVq1bDZbAwbNozFixdz+eWXc/DgQQDq1auX7/569ep5nytMQkIC0dHR3q/Y2NjSliQiIlKhnL2LxWoYPHBdHLcY6/kkbDxXWPZw2IxkcPYTbLv8MbCG+rli3yr1bpdLL72UrVu3cuzYMRYtWsSgQYNYu3at93nDyB/1TNMs8Fhe48aNY8yYMd7r9PR0BRAREan0BrZtSOdmddiddpK4aJOoNeOJCFsAwEb3ZTySPYLDRk0SKtGZLGWl1OEjLCyMSy7xHGTTpk0bkpKSeOWVV7zrPA4ePIjdbvfen5qaWmA0JC+bzYbNZittGSIiIn5Vkm209uhw7Jm7YP59kLYDNxZedfbjVWc/DMMakNtoS+KC+3yYpklWVhZxcXHExMSwevVqWrduDUB2djZr165l6tSpF1yoiIhIRVHYYXAFtsSaJmx+G1aOA1cWRNqx9J/DwBpX064CncniD6UKH+PHj6dXr17ExsZy/Phx5s+fz1dffcXKlSsxDIPRo0czefJkmjZtStOmTZk8eTIRERHceeed5VW/iIiITxW1jbZzszpnwsQpByx7BH5c4rlu2gP6zoSqtbFD0IaOXKUKH3/88Qf33HMPKSkpREdH06pVK1auXEn37t0BeOKJJ8jMzOThhx/m6NGjtGvXjlWrVhEZGVkuxYuIiPhaYYfBuUyT3WknPaHi9y2w8D44tgcsIfDniXDtcLDoRJNcF9zno6ypz4eIiFRkKY5MOk5Zky+AWA2D9U92wf7jXPj8WXA7oXojGDAXGlztv2J9yCd9PkRERIJRYdto/3nzRdiX3werJoDbSWbTPjB0XdAEj9LSwXIiIiKllHcbbbNT/6PWyr/A8QNkmaFMct7L/O3Xk9AsnYFtq/u71ApJIx8iIiLnwR4ZRvt9c6i1sD8cP8Cv7vrcmv0c81w34DYN71kuUpBGPkRERErr+EFY9ADs/hqA1Mbx3PLjzZykiveWfItQJR+NfIiISFDJd6rs+dj5Oczs6AkeoVWh7xu4bn2dU0aVfLdZDYOLg7B7aUlo5ENERIJGUc3BStKtFFcOrHkO/vOK57peS7htLtRuih3Pe41P3I7LNLEaRtB2Ly0JhQ8REQkKRTUHO5aZw9QVPxffrfToHlh0P/ye5Llu+wD0eB5Cz4x25F2EGszdS0tC4UNERIJCUc3Bpqz4GbO4bqU/LoNlIzxdS23RcOt0uPzWQr+HPTpcoaMEFD5ERCQoxNWuisUgXwCxQNHdSiMMWPUUJM32PHFRGxjwFtS42FclBywtOBURkaBQWHOwJ3s1x2Lkv89qGNQ4mUzG613PBI+Oo2DISgWPMqKRDxERCWh5F5MWti6jekRovoWiz8VtI3bhi1Q1skgzo/jhmql06a4DUsuSwoeIiASsona35F2XkRtI9qYcoumWidTcuQgM+K/rckblDOfw+uqs7+TZlnvOHTFSIgofIiISkIra3ZJvMelp9sxd2D+/D9J24DINXnH2Z4arL24sYJrMXb+bOet/K35HjJSY1nyIiEhAKmp3y+60k2ceME1Iegvm3OAJHlVjuCvnKV51xXuCB54PytzgAWdCjFqnnz+FDxERCUi5u1vyytd1NPMY/HsQLB8DzlPQtAfWh/9Dv36351uU+sB1cecOMVIqmnYREZGAlLu7pdCuo79vgYWD4dhesITAnyfBtQ+DxcLAtuRblAowZ31yvgCi1ukXRuFDREQCVoHdLZE22DAdPp8IbidUbwQD5kKDq/O97uxmYWqdXrYUPkREJKB5g0TGYfjwXti5yvPE5X2hzysQXv2c76HW6WVL4UNERALf7vWw6AE4ngJWG/RMgDZDwDDO/drT1Dq97Ch8iIhI4HK7YN0/Ye0UMN1Qu5lnmiWmhb8rC2oKHyIiEpjSUyDxQdj9tef6T3dB7xcgrKp/6xKFDxERCUA7P4fFQ+FkGoRWhZtfgiv/4u+q5DSFDxERqZTyntniXYvhyoEv/g4bXvVcx7SEAe9A7Uv8VqcUpPAhIiKVQt6wsW7HoYJntlxiwsIhsH+z5wXXPATdn4PQKv4tXApQ+BARkQov7wFxuftTcnt+uU1Yu+Qtbqv2NpasdKgSDbfMgMtv8Ve5cg4KHyIiUqGdfUBc3k7nNrKZEPIB94ashiygQVvo/xbUaOSPUqWEFD5ERKRCK+yAOIDGxgFmhE7ncsseAE60GU61XpPAGurjCqW0FD5ERKRCyz0gLm8Aibd8zXOhb1PVyOKwGcX2a6bS5aY7/VeklIpOtRURkQot94A4q2EQwSleDH2Dl8JmUtXIwlGvPc6H1il4VDIa+RARkQpvYNuGXF/jEFWXPUhE+i4wLNBlLNGdHyPaYvV3eVJKCh8iIlKxmSZsfps6K8eBKwsi7dB/DlzcqcCthfb+kApH4UNERCquzGPw8SPw41LPddMe0PcNqFqrwK15t+N6e3+0bejbeqVEtOZDREQqpt+3wJudPcHDEgI9noc7FhQaPM7ejus2YXzidlIcmT4uWkpCIx8iIlKxuN2w8TX4fCK4nVC9keck2gZXF/mSwrbjukyT3WknNf1SASl8iIhIxZFxGJYMg52rPNeX94VbXvV0LS1GYdtxrYbBxbUjyq9WOW+adhEREb9LcWSyfcNyXK938ASPkCpw87/gtnfOGTwg/3Zc8ASPyfEtNOpRQWnkQ0RE/OqjTckcWPYcI62JWA2T9KpxRN37AdS7olTvM7BtQzo3q8PutJNcXDtCwaMCU/gQERG/+WN/MrGf3MntIT8C8JGzC38/MpjVVRpjP4/3s0eHK3RUAgofIiLiHzs/p+bCB6lnOcIJswoTcoaw1O3p3bFl91FqVlO/jkCl8CEiIr7lyoEv/g4bXiUU+MHdiBE5j5BsesY6DOCR+d+pX0cA04JTERHxnaN7yJ7dAza86rm+5iF+7L2IvdQHznwoqV9HYNPIh4iI+MaPS8lOHE6Y8zgOM4KxzofoWud+BrZtSKfLGrA77SSHM7IYMe+7fC9Tv47Ao/AhIiLlK+cUrJoASXMIA751X8IjOSP53azDqsTtdG5Wx7tQNMWRqX4dQUDTLiIiUn7SdsKcP0PSHABmOvtwe/Yz/G7WAc6MauRSv47gUKrwkZCQQNu2bYmMjKRu3br07duXX375Jd89gwcPxjCMfF/XXnttmRYtIiKVwNYP4c0u8Mc2iKjN4X4f8oLrDpx5Bt0LG9UY2LYh68d248MHr2X92G5abBqAShU+1q5dy/Dhw9m4cSOrV6/G6XTSo0cPMjIy8t3Xs2dPUlJSvF+ffvppmRYtIiIVWNYJWDzM0yY9J4Os2I4k9fqY7Iu7lXhUwx4dTvsmtTTiEaBKteZj5cqV+a7nzp1L3bp12bJlC507d/Y+brPZiImJKZsKRUSk8ji4Df59HxzeCYaFbU3/Sr9t7XHuTMZiJJMQ35L1Y7upC2mQu6A1Hw6HA4CaNWvme/yrr76ibt26NGvWjAcffJDU1NQi3yMrK4v09PR8XyIiUsmYJmyaDbNv8ASPyPocHrCIW7d1xGl6Pmpyt80CGtUIcucdPkzTZMyYMXTq1IkWLVp4H+/VqxcffPABa9as4cUXXyQpKYnrr7+erKysQt8nISGB6Oho71dsbOz5liQiIv6QeQw+uhc+fQxcWdD0Rhi2nl+qtCrymHsJboZpmua5byto+PDhLF++nPXr19OgQYMi70tJSaFRo0bMnz+f+Pj4As9nZWXlCybp6enExsbicDiIioo6n9JERMRXft8MC++DY3vBEgp/ngjth4NhkOLIpOOUNQW2za4f202jHgEoPT2d6OjoEn1+n1efj5EjR7Js2TLWrVtXbPAAsNvtNGrUiJ07dxb6vM1mw2aznU8ZIiLiQymOTJLTTp+3EmmD/073tEl3O6F6I7htLinVLif5t8PeM1kS4lsyPnE7LtPUtlnxKlX4ME2TkSNHsnjxYr766ivi4uLO+ZrDhw+zb98+7PbzOZ9QREQqggVJexmXuA23CbWNdJY1eJ/6h9Z7nryiH/R5hQXbHIxLXFPgTBYdcy9nK1X4GD58OPPmzWPp0qVERkZy8OBBAKKjowkPD+fEiRNMnDiR/v37Y7fb2b17N+PHj6d27dr069evXP4AIiJy4fKNapwVEFIcmd7gca3lR14OfY2YQ0cxrVVI7/p3frDHU/WQ23sPnFlcmrd7qUiuUoWPmTNnAtC1a9d8j8+dO5fBgwdjtVrZtm0b7733HseOHcNut9OtWzcWLFhAZGRkmRUtIiJlJ++oRmGnyCanZYDpZpR1MY+EJGI1TH511+fLK6aR8KkFt7kJAzh7AaHOZJGilHrapTjh4eF89tlnF1SQiIj4Tt5RDSg4YgHQpMpxPgibTHvLjwB85OzCJOcgMr+1eF9X2KeDzmSRouhsFxGRIJacllH8dtidq6n3wQ20t/xIhmljdPbDjHMN467rLivwOvCMnIDOZJHi6VRbEZEgFle7auGnyNYIgVVPwYbpngdjWpLR8w0Guuw8eXo0Y8765AKvS3y4PSez3VpcKsXSyIeISBAr7BTZV3pWx76o35ngcc1DcP/n1L24hbczaVGnz14ZW0PdS+WczrvJWHkpTZMSEREpGymOTHannaT5kS+o8fljkOWAKtFw62twWZ9zvk4jHVLuTcZERCSw2CMM7D89D5vf8jzQ4BoY8BZUL/44e22jlfOh8CEiEuwO7fC0SP/Dc+gbnR6FbhPAGurfuiRgKXyIiAS4sxuI5btOXgzL/wY5JyGiNsS/CZf82d8lS4BT+BARCWBnNxDr1/oiFn+3nyrmKZ4LnUt/69eeG+M6Q/xsiIzxvra4rqciF0LhQ0QkQBXWQGzRt/u5zNjDjLBXaWJJwWUanOz4BJF/fhIsVu9rz9X1VORCaKutiEiAKthAzORu62qWhD1DE0sKKWZN7sh+iu1NhuYLHkV1PU1xZPq0fglcGvkQEQlQeRuIRZHBlNDZ9LZuAuBzV2sezxlKuhFdoAV6cV1PNf0iZUEjHyIiASq3EdhVll0sDxtPb+smnITwD+c9PJDzGOlGdKEt0HNDS146p0XKkkY+REQCldvNwOwl3F5lEobbiTO6ESG3z+X+apdzQzGNwXJDy/jE7bhMU+e0SJlT+BARCSC5O1SaRGRS74tH4dfVGABX9COkzytQJRo7nDNIDGzbkM7N6qh7qZQLhQ8RkQCRu0PlGuNHXg59DYyjEFIFek2FqwaBYZz7TfJQ91IpLwofIiIBIMWRyYTE//GINZGR1sVYDZOd7ouofsf71Glylb/LE8lH4UNEJADs37uL90Of51rLTwAscHZlovNe3qYRdfxcm8jZFD5ERCq7Hato/elQrJYjnDCrMCFnCEvdnbRDRSoshQ8RkUqiQLtzZzas+TtsmI4VOBp1GQPSHmSXO0Y7VKRCU/gQEakEzm53/mrPGty84ynYv8VzwzVDqdHjOd7PcGuHilR4Ch8iIhXc2e3ObzS+ofOa2WCchCrRcOtrcFkfAOzR595GK+JvCh8iIhVcbrtzG9k8FfI+94R8DsDxOlcRede7UF0HvknlovbqIiIVXFztqlxi2c+SsGe8wWOm8xZO3LGUFKMOG3al6dA3qVQ08iEiUsHZkxezMvwZQlyZpJlRPJbzML363cW6Xcd07L1USgofIiIVVdYJWP43+H4+IUBWbCd2X/NPEhpeDEDHKWsKHHvfuVkdrfmQCk/hQ0SkIkr5HhbeB4d/BcMCXcdju24MbSxWADbsStOx91JpKXyIiFQkpglJc+CzCeDKgqiLoP8caNQh3225x97nDSBqKiaVhRaciohUFJnH4KN74NPHPMGjWU8Ytr5A8IAzx95bTx8Wp6ZiUplo5ENEpCLYlwQLh4BjL1hCofvf4dq/FnsSrY69l8pK4UNExJ/cbtjwKuaa5zDcTpzRjQi5/R24qGQn0erYe6mMNO0iIuIHKY5MNv3wC6fei4fPn8VwO/nEdS1Xpz7NggO1/V2eSLnSyIeIiI8tSNrLksULeDl0BlWMY5wyQ5noHMR8VzfA0JZZCXgKHyIiPpRy9AQHlz7L+6GLsRomO90XMSJnJL+YZ5qDacusBDqFDxERX0k/QMT8wYwK+QaAj5xdeNY5iEyq5Lstd8tsiiOT5LQM4mpXVRCRgKLwISLiCztWwZJhRJ88zAmzCuNz7meZuyMABp5NLW7zzJbZdTsOqXW6BCyFDxGR8uTMhjV/hw3TPdcxrVjbfDLLPzsOmN6wkXfLLKh1ugQ2hQ8RkfJydLend8f+LZ7ra4ZCj+e4KcTGVa0zC/TnyP1PtU6XQKfwISJSHn5YAssegSwHVImGW1+Dy/p4ny6uP4dap0ugU58PEZGylJMJnzwK/x7kCR6x7Twt0vMEj3NR63QJdBr5EBEpK4d2wL8HQ+oPgAGdHoVu48EaWuq3Uut0CWQKHyIiZWHrPFj+N8g5CVXrQL834ZIbLugt1TpdApXCh4jIhcg6Dssfg+/ne67jukD8bIis59+6RCowhQ8RkfOV8j0svA8O/wqGBbqNJ6XlX0lOPUWcO1OjFiJFUPgQESkt04SkOfDZBHBlkRURw/Gb3uCLk00YN22tGoOJnIPCh4hIaWQeg2Uj4KePAfjC1Zq/HRmG4/9OAtvI3R2btzEYoDbpInmUaqttQkICbdu2JTIykrp169K3b19++eWXfPeYpsnEiROpX78+4eHhdO3alR9++KFMixYR8Yt9SfDGdfDTx5iWUJ5z3sP9OY9xjEhM4Ky+YLhMk7nrd9NxyhrunP0NHaesYUHSXn9ULlKhlCp8rF27luHDh7Nx40ZWr16N0+mkR48eZGRkeO+ZNm0aL730EjNmzCApKYmYmBi6d+/O8ePHy7x4ERGfcLth/cswtyc49kKNi/m+50e85eyF52SWwlmAOet/K9AmPcWR6YuqRSqsUk27rFy5Mt/13LlzqVu3Llu2bKFz586YpsnLL7/MhAkTiI+PB+Ddd9+lXr16zJs3j6FDh5Zd5SIivnDiECweCru+8Fxf0Q/6vELdrDAsxpp8XUjPPiDu/k4XM+vr5HxvpzbpIhe45sPhcABQs2ZNAJKTkzl48CA9evTw3mOz2ejSpQsbNmwoNHxkZWWRlZXlvU5PT7+QkkREys5vayHxIThxEEKqcKzrP/gxpi9xWWHeLqTjE7fjMos+IG7O+mS1SRc5y3mHD9M0GTNmDJ06daJFixYAHDx4EIB69fLvb69Xrx579uwp9H0SEhKYNGnS+ZYhIlL2XE5YNw1z7TQMTHJqNuOLFlN4ePkp3OamfDtZCutCmndUo7CAolEPCXbnHT5GjBjB999/z/r16ws8Zxj550BN0yzwWK5x48YxZswY73V6ejqxsbHnW5aIyIVx7IfEB2HPfzCA+c6uTDowiFMHThW6k+VcXUjVJl2koPMKHyNHjmTZsmWsW7eOBg0aeB+PiYkBPCMgdrvd+3hqamqB0ZBcNpsNm812PmWIiJStHZ/B4mGQeYQTZhXG5zzAMneHQm8tzdoNtUkXya9Uu11M02TEiBEkJiayZs0a4uLi8j0fFxdHTEwMq1ev9j6WnZ3N2rVr6dCh8L/AIiL+kOLIZMOuNM/OE2e2p2HYvNs9waPmFdyUPbnI4AFauyFyIUo18jF8+HDmzZvH0qVLiYyM9K7xiI6OJjw8HMMwGD16NJMnT6Zp06Y0bdqUyZMnExERwZ133lkufwARkdJakLSXcYnbcJvQyPiDxLpvUcux3fNku2Ecv2Yc+/75n3yvOXsni9ZuiJw/wzTNs/viFH1zEes25s6dy+DBgwHP6MikSZN48803OXr0KO3ateO1117zLko9l/T0dKKjo3E4HERFRZW0NBGREklxZNJximeLbG/LRqaEzibKyMQZFs2v7acQfVU/7NHhLEjaW+xOFgUPkfxK8/ldqvDhCwofIlKeNuxK477ZX/N0yP9xd4ind8dmdzNG5Yxgv1k7306WFEemwoZICZXm81tnu4hIULnEcoClYU/T3LIPt2kw09WHfzkH4Dz967A0O1lE5PwofIhIwEtxZJJ86ASXp35C3S/HU9dykjQzikdzHuY/7la4z7pfXUhFypfCh4gEtAVJe/lH4iYmhcylg/V0X6K4Lri6T+fhzGo8Fmah3+sb1IVUxIcUPkQk4KQ4MklOy6BqmJX3Fn/MktDpNLGk4DINXnbdxp23/At7jWrkdh9SF1IR31L4EJGAcmYbrcm91tUkhn6AzcjhgFmTUdkjSDKb0+FIFvYa1byvURdSEd9S+BCRgJHiyGRc4jaqmSeYFjqbntYkAFa7ruLxnKEcI7LIKRUtLhXxHYUPEQkYyWkZXMlOptum08BII9u0kuC8k3fdPXFjaEpFpIJQ+BCRwOB20yL5bf4dlkCI4Wa3ux4jc0byI01Y/HB7Tma7NaUiUkEofIhIpZbiyOT3fXtomTSWqD1fggEfu9ozLud+Mo2qTI5vwZWxNfxdpojkofAhIpXWgqS9LF08n5dDX6OKcQynpQohN02jTZPbmH04UyMdIhWUwoeIVEopR4/zx9JneD90CRbDZIf7IkZlj+LtS273LB6trj4dIhWVwoeIVD6O/UR8OJhHQjYBMN/ZlYnOQZzCps6kIpWAwoeIVC47PoPFw4jOPMIJswrjcx5gmbsDoM6kIpWFxd8FiIiUiDMbPpsA826HzCNgv5K13Rax3OwIoG20IpWIRj5EpOI7kgwLh8CBbz3X7YZB979zU4iNq1rr2HuRykbhQ0Qqtu2J8PEoyEqHKtWh7+vQ/Cbv0+pMKlL5KHyISIWQexhcXO2qnjCRkwmfjYfNb3tuiL0W+s+B6rH+LVRELpjCh4j43ZnD4MBiwIzuVen983hI/QEw4Lox0HU8WPUrSyQQ6G+yiPhV7mFwbhPApL9lLV3XvgtGFq6IOvzc/p/UbHUjdmtIwdEREamUFD5ExK+S0zJwm1CVTJ4LnUu8db3n8ai2DDx0H6nLQ7F8uoZ+rS9i8Xf7vaMjCfEtGdi2oZ+rF5Hzoa22IuJXcbWr0sKym4/DJhBvXY/TtPDPnNv586FRpJrVAXCbsOjb/adHRzzX4xO3k+LI9F/hInLeFD5ExH9ME/vP/8fSKs/S2HKQA2ZN7sx5muwOj+Iyi//15DJNdqed9FGhIlKWNO0iIv6ReRSWjoCfP8EKnGp8I/vbJPDKRfUBmLM+2TvSURh1MxWpvDTyISK+t28TvNEZfv4ELKHQcwpV7llA28ubePt2JMS3xGoYgCdo9L/qonzX6mYqUnkZpmkW828L30tPTyc6OhqHw0FUVJS/yxGRsuR2w4ZX4IvnwHRBjTgY8DZcdFWht6c48ncvPftaRCqO0nx+a9pFRHzjxCFYPBR2feG5btEfbn4ZqhT9S+rs7qXqZioSGBQ+RKTcHd6+msjlfyUs8xCEhEOvqXDVvXB6GkVEgovWfIhI+XE5+eH9J6jx79sIyzzEDvdFrOzwAVw9iJT0U2zYlabtsiJBSCMfInJBiuw66thP1kdDuGL/RjBgvrMrE52DyFmdxRMhu5i64mc1DBMJUlpwKiLn7ewzWbwh4peVsOSvkHmE42Y4E3LuZ5m7g/d1hgF5f/NYDYP1Y7tpPYdIJaYFpyJS7vKfyeLpOvps4lZuSplBtW/fBCCnbitu2TeYZDPG+zrL6Xvzym0YpvAhEhy05kNEzkvumSy5Ghp/sCD0WW/woN1fCX3oc4bFd8/Xn+PJXs2xnLXOVA3DRIKLRj5E5LzE1a6KxfCMYtxk2UhC6GyijEzctupY+s2E5r0BGNi2IZ2b1cnXn6N6RCjjE7fjMk01DBMJQlrzISLnbeHGHeQsH8sdVk/vjkM1WlNn0P9B9dhzvlYNw0QCi9Z8iMg5FblLpaRSf2bAt/eB9UdMDDKueYQ6Nz4D1pL9WlHDMJHgpfAhEoSK3KVSEqYJWz+ATx+HnJNQtS5G/JtUa3J9+RYtIgFDC05Fgkxhu1TGJ24vWbOvrOOQ+BAsHe4JHo27wrD1oOAhIqWgkQ+RIHP2LhUo4VbXlP/Bv++DI7vAsEK38dBpDFgK/hvmgqd0RCSgKXyIBJm8u1RyFbvV1TRh02xYNQFc2RDVAPrPgUbtC739gqZ0RCQoaNpFJMjYo8NJiG+Zr/dGkVtdM4/CgrthxeOe4HFpbxj2dZHB44KmdEQkaGjkQyQIFdZ7o4B9m2Dh/eDYC9Yw6P4ctBta7Em05z2lIyJBReFDJEgVudXV7Yb/vAxr/gGmi8zIhpzoM5s6za4953uWekpHRIKSpl1E5IwTqfBBf/hiEpgulrk60PbQM7Sbe5gFSXsBz9TKhl1phU6llGpKR0SClkY+RMTjt7WQ+CCc+AMzJJxxmXcz39UVMOD02o1jmTlMXfFzsYtJSzSlIyJBrdQjH+vWraNPnz7Ur18fwzBYsmRJvucHDx6MYRj5vq699tzDtSLiJy6nZ4rlvVvhxB9Q5zK29kpkvqsbcGZ9h8s0mXI6eEDxi0nt0eG0b1JLwUNEClXq8JGRkcGVV17JjBkzirynZ8+epKSkeL8+/fTTCypSRMqJYz+82wfWvQCY/HHJ7aQMXE7MJa0LnDxrwbPrNq/cxaQiIqVR6mmXXr160atXr2LvsdlsxMTEnHdRIuIDv6yEJX+FzCPkhFTlb5n3sWx7Byw/bCQhviUJ8S3znTz7RM9LmbryZy0mFZELVi5rPr766ivq1q1L9erV6dKlC88//zx169Ytj28lIqXlzIbPJ8LG1wDIrtuKG/cNJtn0/IMhdzpl/dhurB/bLd/ajeoRofkCiRaTisj5KPPw0atXL2677TYaNWpEcnIyTz/9NNdffz1btmzBZrMVuD8rK4usrCzvdXp6elmXJCK5jvwGC4fAge881+3+ypYmI0l+e2u+23KnU85et6HFpCJSFso8fAwcOND731u0aEGbNm1o1KgRy5cvJz4+vsD9CQkJTJo0qazLEJGzbV8EH4+GrHSoUh36zoTmvbnYkVmq3hxF9gcRESmhcu/zYbfbadSoETt37iz0+XHjxuFwOLxf+/btK++SRIJLTiZ8PMoz4pGVDrHXwl//A817A+rNISK+V+59Pg4fPsy+ffuw2+2FPm+z2QqdjhGRMpD6Myy8D1J/BAy47m/QdRxY8//V13SKiPhSqcPHiRMn+PXXX73XycnJbN26lZo1a1KzZk0mTpxI//79sdvt7N69m/Hjx1O7dm369etXpoWLSDFME757H1Y8ATknoWpdiJ8FTboV+RJNp4iIr5Q6fGzevJlu3c78AhszZgwAgwYNYubMmWzbto333nuPY8eOYbfb6datGwsWLCAyMrLsqhaRomUdh0/GwLaPPNeNu0G/NyGynn/rEhE5zTDNs9sG+Vd6ejrR0dE4HA6ioqL8XY5I5ZLyP/j3fXBkFxhWuH4CdHwULDrGSUTKV2k+v3W2i0ggME3YNAtWPQWubIhqAAPegoY62kBEKh6FD5HKLvMoLB0BP3/iub60N9z6GkTU9G9dIiJFUPgQqcz2bfJsoXXsA2sYdH8O2g0Fwzj3a0VE/EThQ6QycrthwyvwxXNguqBGHNw2F+q3LvT2FEcmyWkZxNWuqh0tIuJ3Ch8iFUiJQsKJQ7B4KOz6wnPdoj/c/DJUKXyB14KkvYxL3IbbBIsBCfEtGdi2Yfn8AURESkDhQ6SCKFFI+G0troUPYD2ZihkSjtFrKlx1b4FpltwQUzXM6n1POHNoXOdmdTQCIiJ+o/AhUgGkODKLDwkuJ6ydirnuBayY7HBfxMiToxji7kbn9FP5RkvyhhgDOHsvfe6hcQofIuIvCh8ifpJ3iiU5LSPfwW6QJyRwBBY9AHs3YAAfOrsxyXkvp7AxdtE2jNOHwlkMeLJnc6au/Nn7XoU18Snu0DgREV9Q+BDxg7OnWJ7s2bzQk2UvTf8PLBwFmUdwhlbj0YzBfOzu4L3HxNPiAzyvnbriZ9yFfL/c99ahcSJSESh8iPhYYVMs01b+wpO9mjNtxS+4TJMqhoulzVdRc+n/AZBdtxXHbnqT5W8kF/vebjzLP8yzQkziw+05me3WoXEiUiGo57KIjxU1xdLqouqsH9uNxL/Y+b7hP7k02RM83nb2pNW+v/FlajUS4ltiPb241IJnTUdeVsNgbK/m3ntyRzqujK1B+ya1FDxEpELQyIeIj8XVrlroFMvFtSOw7/sU+4rRkJXOUbMaj+cM5XP31YBnAer6sd1YP7Ybu9NOcnHtCNbtOMT4xO24TNMbNAa2bcgtV9b33qPAISIVjcKHiI/Zo8NJiG+ZLzRMvfUS7OvGwpZ3AEivczW99w0mhVre1+UuQM07gjGwbUM6N6tTIGjYo8MVOkSkwlL4EPGDvKHhEmMfdVYOgtQfAQOuG0PGVY/yx7R1+barFLVLRUFDRCobrfkQ8RN7VBXaOz6lzryenuBRtS7csxhueAZ7jch86zu0S0VEAolGPkT8Ies4fDIGtn3kuW7cDeJnQbW63luKmlIREansFD5EfCDfmS0nd8C/74Mju8CwwvUToOOjYCk4EKkpFREJRAofIuXsTEMxk8HWVTxtm4fVnQNRDWDAW9DwWn+XKCLiUwofIuUot6FYNfMEL4TO4kbrZnDDqcY3UmXAGxBR098lioj4nBacipSj5LQM/sQOPrWN50brZrLMECbm3Mt3HV5X8BCRoKWRD5Hy4nbT4re3+SgsgRDDzW53PUbkjOQnmjC0TlV/Vyci4jcKHyLl4UQqLB5K1K41YMAyVwfG5wwh06iqLbMiEvQUPkTK2m9fQeJDcOIPCAmH3tNo23gAsw9nasusiAgKHyJlx+WErxLg6xcBk5PRTcm4ZTZ1mrTGDtirF+xOKiISjLTgVKQsOPbDuzfD1/8ETD50deOqP8bTbs4BFiTt9Xd1IiIVikY+RC7ULythyTDIPIo7rBqjMwazzNXB85zpOY22c7M6mm4RETlNIx8i58uZDSvHwYcDIfMo2P/E1l5LzwSP03JPoxUREQ+NfIicjyO/wcIhcOA7z/W1D8OfJ2LPcGMx/sBdgtNoRUSClUY+REpr+yJ4o7MneITXgDvmQ88ECLFhjw7XabQiIuegkQ8JWvkOeysiHOS7JwJYORa2vON5MvZaz9ks0Q3yvUan0YqIFE/hQ4LSmcPewGJAQnxLBrZtWOQ9zSy/81GtWVQ//itgwHVjoOt4sBb+V0in0YqIFE3TLhJ0cg97y12X4T69IyXFkVnIPSa3Wb9iaehTVD/+K66IOnDPYrjhmSKDh4iIFE+/PSVo5E6hHMnIzrcgFM7sSMkdrUhOyyDczOQfoW/Tz/ofANa5WhLRew4X1W5E8q60YqdrRESkaAofEhTyTqEYeL7y5o/cHSm5AaV2+k8sDxvPxZY/cJoWXnTezmx3Hx4/EsrtH6wpdrpGRESKp/AhAe/saRYTT/iwGJ4pl9wdKet2HGJc4vfcY1nF+JAPsFmc7DdrMzJ7BP/jUp7oeSlTV/xcYLpGDcREREpH4UMCXnJaRoFpFhOY/pfW1Kpm8/bg6D1lGTNDZnGjdTMAq11tsA+aw+NGFBfXjij0fVymyZbdR6lZrfhdMyIicobChwS8uNpVvaMcuayGwdUX1/CGhW3//YxPwsZxkXGYLDOEyc67eNfVgw+NKNo3qeV93dnvYwCPzP9O0zAiIqWg3S4S8Ipt/OV2w9cv0mLVHVxkHCbZXY/47Em867oRq2HJ15n07PfJ/ctT3K4ZEREpSCMfEhQKbfx1IhUWD4VdazCAPfV7c2vyANLNKkV2Js37Poczshgx77t8z5+9a0ZERApS+JCgka/x129fQeJDcOIPCAmH3i/QqPXdfJZ+6pydSXPfJ8WRWeh0js5xEREpnqZdJLi4nPDFc/BeX0/wqHMZPPQlXHUPGAb26HDaN6lVopELneMiInJ+NPIhwcOxHxbdD3v/67m+ahD0nAJh5z9SoXNcRERKT+FDKrSSHP5WIr+sgCV/hcyjEBYJfV6GlgPKpEad4yIiUjoKH1JhleTwt3NyZsPnz8LG1z3X9j/BgLehVpMyr1dEREpGaz6kQirJ4W/ndOQ3eKv7meBx7cNw/yoFDxERPyt1+Fi3bh19+vShfv36GIbBkiVL8j1vmiYTJ06kfv36hIeH07VrV3744YeyqleCRFHdRHennSzR649u+hDnzOsgZSuE14A75kPPBAixlX2xIiJSKqUOHxkZGVx55ZXMmDGj0OenTZvGSy+9xIwZM0hKSiImJobu3btz/PjxCy5WgkduV9K8SrSNNfsku94aQo1PhxGSc4Ik96Usa/8RXNqr/IoVEZFSKfWaj169etGrV+G/yE3T5OWXX2bChAnEx8cD8O6771KvXj3mzZvH0KFDL6xaCRq521jHJ27HZZrFbmPNXZTa1Pid6ssfosnhX3CbBjNct/KKsz+sSKPtlZlaFCoiUkGU6YLT5ORkDh48SI8ePbyP2Ww2unTpwoYNGwoNH1lZWWRlZXmv09PTy7IkqcRKso3Vsyj1ewZYvmJSyLuEGtkcMqMZlTOcDe4WnpvUdVREpEIp0/Bx8OBBAOrVq5fv8Xr16rFnz55CX5OQkMCkSZPKsgwJIMVtY01xZPJ84je8FPI2fa0bAFjnasljzodJNaO996nrqIhIxVIuu10MI/9kvWmaBR7LNW7cOBwOh/dr37595VGSBKDUX75haegE+lo34DQtTMsZyKCcJ+nbqbW6joqIVGBlOvIRExMDeEZA7Ha79/HU1NQCoyG5bDYbNpt2IEgpmCZ88yatVj+NYcnmd7M2j2SP4FuzGVbD4L5OF3Nfp4vVdVREpIIq05GPuLg4YmJiWL16tfex7Oxs1q5dS4cOHcryW0mwOnkEFtwNK5/EcGXze71u3JKd4A0euaMcpTmjRUREfKvUIx8nTpzg119/9V4nJyezdetWatasScOGDRk9ejSTJ0+madOmNG3alMmTJxMREcGdd95ZpoVLENr7DSwcAum/gzUMevyDBtc8xPISnEQrIiIVR6nDx+bNm+nWrZv3esyYMQAMGjSId955hyeeeILMzEwefvhhjh49Srt27Vi1ahWRkZFlV7UEF7cb/vMvWPM8mC6o2RgGzIX6fwJ0toqISGVjmKZpnvs230lPTyc6OhqHw0FUVJS/yxF/O5EKi4fCrjWe65a3wc3/ApvCrIhIRVKaz28dLCd+c84Ta3d9CYkPQUYqhIRD7xeg9d1QxM4pERGpHBQ+xC+KPbHW5YSvEuDrFwET6lwGt70DdZv7s2QRESkjCh/ic0WdWNu5WR3sHIaF98O+jZ4nrx4MNyZAmJqEiYgECoUP8bmiTqx1fLcM+zdPQuZRCIuEW16BFv0LvP6c0zUiIlKhKXyIz+WeWJsbQMLIYVzIhzT/aqXngfqtYcDbnl0tZyl2ukZERCqFcmmvLlKYFEcmG3alAZ7QYDUMGhkHSQybyH0hp4PHtcNhyKpCg0dR0zUpjkxf/RFERKQMaORDfKKwEYvN/RxErn6GkJwTEF4D+r5BSkwXkvekFzqlUtR0jU6sFRGpXBQ+pNydPWIRZmbBskeoYf3S80DD9tB/Dgt2uBn3zpoip1TOnq4BnVgrIlIZadpFyl3eEYumxu8sDXuagdYvMTGg8+Mw6BNSqHXOKRV7dLh3ugZ0Yq2ISGWlkQ8pd54RC5MBlq+YFPIu4UY2qWZ1rANmU6tlDwCS046VaEplYNuGdG5WR2e5iIhUYgofUu7sthy+jHufRgdWALDO3YojPV6hb8urvPeUZkpFZ7mIiFRumnaR8nXgO5jVhUYHVmAaVva0fpymY1bSt9NV+W7TlIqISPDQyIeUD9OEb96AVU+DOweiYzH6v0Wjhu2KfImmVEREgoPCh5S9k0dg6XD45VPPdfOb4ZbpEFHznC/VlIqISOBT+JCytXej52yW9N/BGgY9nodrHtRJtCIi4qXwIWXD7Yb1L8GXk8F0eTqU3vYO2K/0d2UiIlLBKHzIhTuRCokPwW+nm4a1vB1ufglskf6tS0REKiSFD8mnJCfG5rsnbaMneGSkQmgE9H4B/nSXpllERKRICh/iVZITY3PvMUwXY0IW8nDIMgxMqHs5DJgLdZv7qXoREaks1OdDgJKdGJt7Tz3zMPPDnmN4yFIMTDJa3gMPrlHwEBGRElH4EKD4E2Pz3nO9sYVPbeNoa9lBuhnO8OxH+P5Pk0g5CRt2pel4exEROSdNuwhQgvbmziyu3D6FOWGzAfifuzEjc0aynxha7T/GXXM2FjtdIyIikksjHwKco7354V3wVneqfucJHm85ezMgeyL7ieGJnpcydcXPxU7XiIiI5KWRD/EqrL350W/mEfn544TknIDwGtD3DXrHdOHy0/cUN12jTqUiIlIYhQ/Jx9vePPsku94aQpN9iwDY5G7OwfbTueXSa7Cfvi9XSU+jFRERAU27SGFSfyLnza402bcIt2nwirMfd2RP4NEVBReU6jRaEREpLY18yBmmCd++ByueJNSZSapZnVE5w/mv+wrv84VNpxQ2XVOSZmUiIhKcFD6CWL6AYMuBT0bDds80S1ajrty0YyCHzGjv/cVNp+Q9jbYkzcpERCR4KXwEqbwBoZXlNz6o/iaRJ/eBYYUbnsbWYRSPbfmd8YnbcZlmiadTimpW1rlZHY2AiIgIoPARlM4EBJP7rCsZFzKPsJMunJEXEXL7OxB7DVD4dMq5aPeLiIici8JHEMmdZjmSkU2keYJ/hr5Jd+sWAFa62lKz1yyuiW2c7zV5p1NK4pzNykREJOgpfASJvNMsbY2f+dT2GhcZh8kyQ3jeeRcfuG9k/UX2C/4+ubtfSjtdIyIiwUPhIwjkTrOYppuHrcsYE7KQEMNNsjuGETmP8DNxZRoQzme6RkREgofCRxBITsugpungpdDX6WzdBsBiV0eq3PoKT9WoWS4BobTTNSIiEjwUPiqh0vbQuDRjMytsY6ljODhp2njWOYhEd1fWN22ggCAiIj6n8FHJFNVD4+xAkuLIJDnVQaudr1Nr06tgmPzijmV4zkiSidU6DBER8RuFj0qkqB4axzJzvCfLWgzo1/oiNn73PS+HTqeaZYfn5qvvI6rDMzx3zNQ6DBER8SuFj0qkqB4aU1b8jJknkDi2LmN52BtUNzJIN8MZ73yICZ3HedZh1PJ93SIiInkpfFQihfXQsHDmOowcxoZ8yJCQlQBsdTdmZM5I9pn1uEtNvkREpILQqbaVSGEnyD7ZqzkWAxoZB1kU9qw3eMxy3sRt2RPZZ9ZTky8REalQNPJRyRTWQ6PV0dW0+G4ikUYmR8xqJDZ8iqm/NsKFmnyJiEjFo/BRCXl7aGSfhKUjaL/1/8CA9LptybnlTR5o0ISbHJlq8iUiIhWSwkdllfoT/HswHPoZMKDLE0R1foIoq+d/UjX5EhGRikrho7IxTfj2PVjxJDgzoVo9iJ8Njbv4uzIREZESUfioTE6lwyejYfsiz3WT66HfLKhWx69liYiIlEaZ73aZOHEihmHk+4qJiSnrbxNUUhyZ/G/TlzhnXucJHoYV/jwR7lqk4CEiIpVOuYx8XHHFFXz++efea6vVWh7fJigs2LSHX5a9wFjrPEIMFxnhdqre+R7EXuPv0kRERM5LuYSPkJAQjXacp7xntBiZR6j58X08E7IFgJWutow79hCfRrXE7uc6RUREzle5hI+dO3dSv359bDYb7dq1Y/LkyTRu3LjQe7OyssjKyvJep6enl0dJlULeQ+OusfzMnKpv0N2aSpYZwvPOu3jP1QMw2K1upSIiUomV+ZqPdu3a8d577/HZZ58xe/ZsDh48SIcOHTh8+HCh9yckJBAdHe39io2NLeuSKoXcQ+Mw3Qy3LmFe6D+Iykkl2R1DfPbfec91I2CoW6mIiFR6hmma5rlvO38ZGRk0adKEJ554gjFjxhR4vrCRj9jYWBwOB1FRUeVZWoWyYVcao2Z/xkuhr3OddTsAia5O/NZ2EjP/m4rLPNOtdGDbhn6uVkREJL/09HSio6NL9Pld7lttq1atSsuWLdm5c2ehz9tsNmw2W3mXUeFdmrGZFbax1DbSOWnaeMY5mMXuLqzvcgV3dblC3UpFRCRglHv4yMrK4qeffuK6664r729VObmc8OXz1Fr/LzBMfnbHMjznEXbTIN+ZLAodIiISKMo8fDz22GP06dOHhg0bkpqayj/+8Q/S09MZNGhQWX+rSu+PfTsJX/YQUYe+9TzQZgjR7Z/mH8dMjXKIiEjAKvPw8fvvv3PHHXeQlpZGnTp1uPbaa9m4cSONGjUq629VoeXdMltYiPj643douXk8UUYGx81wvr/6OTre/CB2wF7L9/WKiIj4SpmHj/nz55f1W1Y45woWebfMWgxIiG95ZpGoM4uMT8Zz3dY5YMBWd2NG5ozkwH9jWN8lU6MdIiIS8HS2SykVGyw4s2XWfXoPkduE8Ynb6dysDtZjyUQsfYBqR34AYJbzJl5wDiSHEMBU/w4REQkKCh8lkDvSUTXMWmSwyA0NyWkZ3udzuUyTb5bO4oZdCVQzMjliVuOxnL+yxt3ae4/6d4iISLBQ+DiHvCMdBnB2UxSXmX/EIq52VSwG3gBShSwmhbxL39++AgO+cTdnVPZw/qCW977c/h0a9RARkWCg8FGMs6dQCuvGdvaIhT06nIT4loxP3E5j9vFa6Cs0s+zHbRpMd/XjVWc/XHgO2pv+l9bUqmbTzhYREQkqCh/FKGwKBTjniMXANrHcmLWKyC+fweo6hatqPe45+gAb3Fd477EaBldfXEOhQ0REgo7CRzHOnkIBT2hIfLg9J7PdhY9YnEqHT0ZTffsiz3WTG7D2e5Nbf8rkm8Tt+dqkK3iIiEgwUvgoRt4plLyh4crYGoW/YP+3sHAIHE0GSwhc/zR0eAQsFga2hc7N6qhNuoiIBD2Fj3MY2LbhuUODacLGmbD6GXDnQHRDGPA2xLbNd5s9OlyhQ0REgp7CRwkUGxpOHoElD8OOFZ7ry/rALdMhvIjRERERkSCn8HEh9myARQ9A+n6whsGNk6HtA2AY/q5MRESkwlL4OB9uF3z9Enw1GUw31LoEBswFeyt/VyYiIlLhKXyc5VzntnD8D0h8EJLXeq5b/QVuehFs1XxbqIiISCWl8JHHuc5t4dcvYPFQyDgEoRHQ+5/Q+i6gBKFFREREgCALH8UFhOIOhLNXC4Evn4f1//I8WfcKuG0u1LkUKEFoEREREa+gCR/nCghFHQh3YPdO7Jsfg33feB5sM8SzsDTUE16KDS0aARERESnA4u8CfKGogJDiyPTek9vNNK+e1s20/rSPJ3jYouC2d+Dmf3mDBxQdWnannSynP42IiEjlFhThoyQBIbebqdUwCCOHSSHv8kboS1iyjkH9q2DoOriiX4H3Liy0nH3YnIiIiJwRFOGjpAFhYNuG/HdoI76tP41BIZ95Hmw/AoZ8BjXjCn3vvKEl9311bouIiEjRgmLNR1FntNijw/MvQt3zCXU/GQ3ZJyC8JvR7A5rdeM73L1ELdhEREQGCJHxA4QEhdxGqzTzFpND3uN36lefmRh2h/xyIql/i99e5LSIiIiUTNOED8geE3EWol7CP18JepallP27TIKP9GCJ7TACL1c/VioiIBKagWPNRmORDJ7jdsoZlYU/R1LKfP8zq3JUznu1Nhyt4iIiIlKOgGvnwOpVO601/o0PoUgC+cl3J33KGccyorl0qIiIi5Sz4wsf+b2HhfYQf3Y3bCGFazu286eyNxbBql4qIiIgPBE/4ME3Y+DqsfhbcORDdEMuAtxkU1YIu2qUiIiLiM8ETPlK2wmcTABMu6wO3TIfwGthBoUNERMSHgid81G8NXcdCRC1o+wAYxrlfIyIiImUueMIHeMKHiIiI+FXQbrUVERER/1D4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERn6pwp9qapglAenq6nysRERGRksr93M79HC9OhQsfx48fByA2NtbPlYiIiEhpHT9+nOjo6GLvMcySRBQfcrvdHDhwgMjISAzDKNP3Tk9PJzY2ln379hEVFVWm7y1n6OfsG/o5+45+1r6hn7NvlNfP2TRNjh8/Tv369bFYil/VUeFGPiwWCw0aNCjX7xEVFaX/Y/uAfs6+oZ+z7+hn7Rv6OftGefyczzXikUsLTkVERMSnFD5ERETEp4IqfNhsNp599llsNpu/Swlo+jn7hn7OvqOftW/o5+wbFeHnXOEWnIqIiEhgC6qRDxEREfE/hQ8RERHxKYUPERER8SmFDxEREfGpoAkfr7/+OnFxcVSpUoWrr76ar7/+2t8lBZyEhATatm1LZGQkdevWpW/fvvzyyy/+LivgJSQkYBgGo0eP9ncpAWf//v3cfffd1KpVi4iICP70pz+xZcsWf5cVUJxOJ0899RRxcXGEh4fTuHFj/v73v+N2u/1dWqW3bt06+vTpQ/369TEMgyVLluR73jRNJk6cSP369QkPD6dr16788MMPPqktKMLHggULGD16NBMmTOC7777juuuuo1evXuzdu9ffpQWUtWvXMnz4cDZu3Mjq1atxOp306NGDjIwMf5cWsJKSkpg1axatWrXydykB5+jRo3Ts2JHQ0FBWrFjBjz/+yIsvvkj16tX9XVpAmTp1Km+88QYzZszgp59+Ytq0abzwwgtMnz7d36VVehkZGVx55ZXMmDGj0OenTZvGSy+9xIwZM0hKSiImJobu3bt7z1grV2YQuOaaa8xhw4ble6x58+bm2LFj/VRRcEhNTTUBc+3atf4uJSAdP37cbNq0qbl69WqzS5cu5qhRo/xdUkB58sknzU6dOvm7jIB30003mUOGDMn3WHx8vHn33Xf7qaLABJiLFy/2XrvdbjMmJsacMmWK97FTp06Z0dHR5htvvFHu9QT8yEd2djZbtmyhR48e+R7v0aMHGzZs8FNVwcHhcABQs2ZNP1cSmIYPH85NN93En//8Z3+XEpCWLVtGmzZtuO2226hbty6tW7dm9uzZ/i4r4HTq1IkvvviCHTt2APC///2P9evX07t3bz9XFtiSk5M5ePBgvs9Gm81Gly5dfPLZWOEOlitraWlpuFwu6tWrl+/xevXqcfDgQT9VFfhM02TMmDF06tSJFi1a+LucgDN//ny+/fZbkpKS/F1KwPrtt9+YOXMmY8aMYfz48WzatIlHHnkEm83Gvffe6+/yAsaTTz6Jw+GgefPmWK1WXC4Xzz//PHfccYe/SwtouZ9/hX027tmzp9y/f8CHj1yGYeS7Nk2zwGNSdkaMGMH333/P+vXr/V1KwNm3bx+jRo1i1apVVKlSxd/lBCy3202bNm2YPHkyAK1bt+aHH35g5syZCh9laMGCBbz//vvMmzePK664gq1btzJ69Gjq16/PoEGD/F1ewPPXZ2PAh4/atWtjtVoLjHKkpqYWSHxSNkaOHMmyZctYt24dDRo08Hc5AWfLli2kpqZy9dVXex9zuVysW7eOGTNmkJWVhdVq9WOFgcFut3P55Zfne+yyyy5j0aJFfqooMD3++OOMHTuWv/zlLwC0bNmSPXv2kJCQoPBRjmJiYgDPCIjdbvc+7qvPxoBf8xEWFsbVV1/N6tWr8z2+evVqOnTo4KeqApNpmowYMYLExETWrFlDXFycv0sKSDfccAPbtm1j69at3q82bdpw1113sXXrVgWPMtKxY8cCW8V37NhBo0aN/FRRYDp58iQWS/6PIqvVqq225SwuLo6YmJh8n43Z2dmsXbvWJ5+NAT/yATBmzBjuuece2rRpQ/v27Zk1axZ79+5l2LBh/i4toAwfPpx58+axdOlSIiMjvaNN0dHRhIeH+7m6wBEZGVlgHU3VqlWpVauW1teUoUcffZQOHTowefJkbr/9djZt2sSsWbOYNWuWv0sLKH369OH555+nYcOGXHHFFXz33Xe89NJLDBkyxN+lVXonTpzg119/9V4nJyezdetWatasScOGDRk9ejSTJ0+madOmNG3alMmTJxMREcGdd95Z/sWV+36aCuK1114zGzVqZIaFhZlXXXWVtn+WA6DQr7lz5/q7tICnrbbl4+OPPzZbtGhh2mw2s3nz5uasWbP8XVLASU9PN0eNGmU2bNjQrFKlitm4cWNzwoQJZlZWlr9Lq/S+/PLLQn8nDxo0yDRNz3bbZ5991oyJiTFtNpvZuXNnc9u2bT6pzTBN0yz/iCMiIiLiEfBrPkRERKRiUfgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ9S+BARERGfUvgQERERn1L4EBEREZ/6fwLm7YObr39TAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predictions = model.predict( X )\n",
    "plt.plot(X, y, 'o', markersize=3)\n",
    "plt.plot(X, predictions)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a91f431-4197-445c-9dab-dda99d5d6dd4",
   "metadata": {},
   "source": [
    "Un réseau de neurones repose sur un principe relativement simple : il apprend les paramètres de combinaisons linéaires, à la manière d'une régression, puis applique à ces combinaisons des fonctions d'activation non linéaires. C'est l'enchaînement de ces transformations qui lui permet de modéliser des relations beaucoup plus complexes que celles capturées par une régression linéaire classique."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be1bb510-f306-42a1-8fda-001540d9eb1b",
   "metadata": {},
   "source": [
    "<b> Un réseau de neurones n'est pas une rupture avec la régression ; c'est une généralisation de la logique de régression à des architectures multicouches et non linéaires. <b>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e146531e-daf1-402b-b79b-f2f37f98183c",
   "metadata": {},
   "source": [
    "## Régression linéaire multiple et réseau de neurones"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75fba4b5-86e4-43f7-8176-0eec127bbfe9",
   "metadata": {},
   "source": [
    "Pour une régression linéaire multiple, un seul neurone avec une entrée de dimension n est toujours suffisant. Ca reste une combinaison linéaire. Pas besoins d'empiler des neurones en fonction du nombre de paramètres de votre régression linéaire, du nombre de variables (features). Attention néanmoins, plus n augmente, plus la convergence de la solution sera difficile."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65e5bd95-0408-4465-a58c-f5af1170ebf0",
   "metadata": {},
   "source": [
    "Utiliser plusieurs neurones n’apporte en théorie rien si : toutes les couches sont linéaires ; la sortie est une combinaison linéaire des entrées. Empiler plusieurs neurones linéaires est équivalent à un seul neurone linéaire. Autrement dit, une succession de transformations linéaires n'est rien d'autre qu'une seule grande transformation linéaire. De plus, plusieurs neurones détruisent le sens direct des coefficients. En gros, vous pourrez toujours aussi bien prédire, mais vous ne pourrez plus récupérer les coefficients de votre régression facilement (ils seront plus difficiles à obtenir, il faudra des traitements statistiques et mathématiques), votre modèle deviendra alors en quelque sorte une boite noire..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2c780611-5e15-4558-863a-2f7ad4eb197a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Données (on construit des donnees suivant une régression du type y = 3x1 + 6x2 + 2)\n",
    "np.random.seed(0)\n",
    "x1 = np.linspace(0, 10, 100).reshape(-1, 1)\n",
    "x2 = np.random.rand(100,1) * 10\n",
    "x2 = x2.reshape(-1,1)\n",
    "X = np.hstack([x1,x2])\n",
    "y = 3 * x1 + 6 * x2 + 2 + np.random.normal(0, 0.5, size=(100, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "626c37d0-fda8-4d1c-9d5c-ddc93dfcb302",
   "metadata": {},
   "source": [
    "Le code est donc quasiment identique à celui de la régression linéaire simple, car un seul neurone sans activation suffit. Il faut juste changer la dimension de l'entrée, donc ici de notre neurone."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d3b59584-8532-41ec-997a-7c4a9f035d77",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Modèle Keras : 1 neurone\n",
    "model = Sequential([Dense(units=1, input_shape=(2,), activation='linear')]) #input_shape passe a 2\n",
    "model.compile(optimizer='sgd', loss='mse')\n",
    "model.fit(X, y, epochs=200, verbose=0)\n",
    "weights, bias = model.layers[0].get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "55193da2-8fe5-4159-ba15-1df55351dac4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pente (a) estimée (proche de 3) : 3.0141\n",
      "Pente (b) estimée (proche de 6) : 6.0573\n",
      "Intercept (c) estimé (proche de 2) : 2.0957\n"
     ]
    }
   ],
   "source": [
    "a = weights[0][0]  # pente 1\n",
    "b = weights[1][0]  # pente 2\n",
    "c = bias[0]        # intercept\n",
    "print(f\"Pente (a) estimée (proche de 3) : {a:.4f}\")\n",
    "print(f\"Pente (b) estimée (proche de 6) : {b:.4f}\")\n",
    "print(f\"Intercept (c) estimé (proche de 2) : {c:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16705735-ef0e-41ee-95f0-d8378cc8285b",
   "metadata": {},
   "source": [
    "## Polynôme et réseau de neurones"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0093e45-b7a5-4743-b415-db9495eab867",
   "metadata": {},
   "source": [
    "Pour un polynôme, la problématique devient plus subtile : un polynôme est non linéaire, mais est linéaire en ses coefficients si on connait le modèle du polynôme. Ainsi, si le polynôme est explicitement construit, si on connait sa forme, alors un seul neurone suffit. Si on sait qu'un polynôme pourrait résoudre le problème posé, mais que l'on ignore sa forme, alors dans ce cas le problème n'est plus une simple combinaison linéaire de coefficients. Lorsque l'on ignore la forme exacte de la relation entre les variables, il devient nécessaire de disposer d'un modèle capable d'apprendre automatiquement des transformations non linéaires pertinentes à partir des données. C'est précisément l'un des intérêts des réseaux de neurones multicouches."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32bbccec-0d2e-4313-8e80-eafc49decf54",
   "metadata": {},
   "source": [
    "Imaginons que l'on cherche à modéliser un problème à l'aide d'un polynôme de degrés 3. En théorie, on peut alors utiliser un seul neurone et on peut s'intéresser aux valeurs des paramètres..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9addbf52-c84f-4a47-bbf7-31a82bf9c524",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Données (polynôme degré 3 :  y = 1 + 2x - 0.5x^2 + 0.1x^3 + bruit)\n",
    "np.random.seed(0)\n",
    "\n",
    "X = np.linspace(-3, 3, 200).reshape(-1, 1)\n",
    "\n",
    "y = (\n",
    "    1\n",
    "    + 2 * X\n",
    "    - 0.5 * X**2\n",
    "    + 0.1 * X**3\n",
    "    + np.random.normal(0, 0.5, size=(200, 1))\n",
    ")\n",
    "\n",
    "# Construction des features polynomiales\n",
    "X_poly = np.hstack([\n",
    "    X,\n",
    "    X**2,\n",
    "    X**3\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e777c92c-f148-46ed-84b2-db2127471523",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "β0 (intercept) : -108987176.000\n",
      "β1 (x)         : 158831024.000\n",
      "β2 (x²)        : 20081772.000\n",
      "β3 (x³)        : 282371104.000\n"
     ]
    }
   ],
   "source": [
    "# Modèle : 1 neurone linéaire\n",
    "model = Sequential([Dense(1, input_shape=(3,), activation='linear')])\n",
    "model.compile(optimizer='sgd', loss='mse')\n",
    "# Entraînement\n",
    "model.fit(X_poly, y, epochs=100, verbose=0)\n",
    "# Extraction des coefficients\n",
    "weights, bias = model.layers[0].get_weights()\n",
    "\n",
    "beta_1, beta_2, beta_3 = weights.flatten()\n",
    "beta_0 = bias[0]\n",
    "\n",
    "# Affichage\n",
    "print(f\"β0 (intercept) : {beta_0:.3f}\")\n",
    "print(f\"β1 (x)         : {beta_1:.3f}\")\n",
    "print(f\"β2 (x²)        : {beta_2:.3f}\")\n",
    "print(f\"β3 (x³)        : {beta_3:.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bf5aceeb-63a1-4b49-a4ff-5f697623e880",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0119843449399149e+19\n"
     ]
    }
   ],
   "source": [
    "loss = model.evaluate(X_poly, y, verbose=0)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "938f0cd6-8866-4942-86a0-5757eb07d88d",
   "metadata": {},
   "source": [
    "Comme on le voit ici, ça ne marche pas toujours très bien. On voit que la descente de gradient peut être un algorithme capricieux. On va donc passer à plusieurs neurones et surtout à plusieurs couches avec des fonctions d'activation non linéaires :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e8fdf56e-6b11-4d2b-ba96-0fcafa48daf6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"sequential_2\"</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1mModel: \"sequential_2\"\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\"> Layer (type)                         </span>┃<span style=\"font-weight: bold\"> Output Shape                </span>┃<span style=\"font-weight: bold\">         Param # </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n",
       "│ Dense_n1 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">12</span>)                  │              <span style=\"color: #00af00; text-decoration-color: #00af00\">48</span> │\n",
       "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
       "│ Dense_n2 (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">12</span>)                  │             <span style=\"color: #00af00; text-decoration-color: #00af00\">156</span> │\n",
       "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
       "│ Output (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                       │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">1</span>)                   │              <span style=\"color: #00af00; text-decoration-color: #00af00\">13</span> │\n",
       "└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                        \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape               \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m        Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n",
       "│ Dense_n1 (\u001b[38;5;33mDense\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m12\u001b[0m)                  │              \u001b[38;5;34m48\u001b[0m │\n",
       "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
       "│ Dense_n2 (\u001b[38;5;33mDense\u001b[0m)                     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m12\u001b[0m)                  │             \u001b[38;5;34m156\u001b[0m │\n",
       "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
       "│ Output (\u001b[38;5;33mDense\u001b[0m)                       │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m1\u001b[0m)                   │              \u001b[38;5;34m13\u001b[0m │\n",
       "└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">217</span> (868.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m217\u001b[0m (868.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">217</span> (868.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m217\u001b[0m (868.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Une couche d'entrée et une couche de sortie de dimension 1 pour les prédictions entre les deux deux couches cachées de 12 neurones non linéaires\n",
    "model = Sequential()\n",
    "model.add(Input((3,), name=\"InputLayer\"))\n",
    "model.add(Dense(12, activation='relu', name='Dense_n1'))\n",
    "model.add(Dense(12, activation='relu', name='Dense_n2'))\n",
    "model.add(Dense(1, name='Output'))\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "                loss      = 'mse',\n",
    "                metrics   = ['mae', 'mse'] )\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7b2050bf-fc6d-43f5-b474-e7db255b1bfb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 5ms/step - loss: 25.2928 - mae: 4.1509 - mse: 25.2928\n",
      "Epoch 2/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 21.9829 - mae: 3.9203 - mse: 21.9829 \n",
      "Epoch 3/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 21.2173 - mae: 3.8073 - mse: 21.2173 \n",
      "Epoch 4/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 17.5155 - mae: 3.5191 - mse: 17.5155 \n",
      "Epoch 5/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 14.4881 - mae: 3.2424 - mse: 14.4881 \n",
      "Epoch 6/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 12.4418 - mae: 3.0053 - mse: 12.4418 \n",
      "Epoch 7/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 10.0541 - mae: 2.7138 - mse: 10.0541 \n",
      "Epoch 8/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 7.7314 - mae: 2.3690 - mse: 7.7314 \n",
      "Epoch 9/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 7.0516 - mae: 2.3062 - mse: 7.0516 \n",
      "Epoch 10/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 5.6099 - mae: 2.0678 - mse: 5.6099 \n",
      "Epoch 11/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 4.6293 - mae: 1.9075 - mse: 4.6293 \n",
      "Epoch 12/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 3.6078 - mae: 1.6967 - mse: 3.6078 \n",
      "Epoch 13/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 3.5473 - mae: 1.6910 - mse: 3.5473 \n",
      "Epoch 14/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 3.0583 - mae: 1.5588 - mse: 3.0583 \n",
      "Epoch 15/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.6033 - mae: 1.4258 - mse: 2.6033 \n",
      "Epoch 16/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.5768 - mae: 1.4067 - mse: 2.5768 \n",
      "Epoch 17/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.4710 - mae: 1.3619 - mse: 2.4710 \n",
      "Epoch 18/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.3982 - mae: 1.3427 - mse: 2.3982 \n",
      "Epoch 19/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.4098 - mae: 1.3487 - mse: 2.4098 \n",
      "Epoch 20/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.3917 - mae: 1.3450 - mse: 2.3917 \n",
      "Epoch 21/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.3410 - mae: 1.3270 - mse: 2.3410 \n",
      "Epoch 22/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.1362 - mae: 1.2775 - mse: 2.1362 \n",
      "Epoch 23/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.3664 - mae: 1.3282 - mse: 2.3664 \n",
      "Epoch 24/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.0407 - mae: 1.2207 - mse: 2.0407 \n",
      "Epoch 25/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.1272 - mae: 1.2660 - mse: 2.1272 \n",
      "Epoch 26/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.2095 - mae: 1.2968 - mse: 2.2095 \n",
      "Epoch 27/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.2059 - mae: 1.2811 - mse: 2.2059 \n",
      "Epoch 28/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.9728 - mae: 1.2259 - mse: 1.9728 \n",
      "Epoch 29/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.2194 - mae: 1.2858 - mse: 2.2194 \n",
      "Epoch 30/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.9897 - mae: 1.2342 - mse: 1.9897 \n",
      "Epoch 31/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.9852 - mae: 1.2161 - mse: 1.9852 \n",
      "Epoch 32/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.1159 - mae: 1.2689 - mse: 2.1159 \n",
      "Epoch 33/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.8691 - mae: 1.1689 - mse: 1.8691 \n",
      "Epoch 34/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 1.8858 - mae: 1.1728 - mse: 1.8858 \n",
      "Epoch 35/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 2.0523 - mae: 1.2462 - mse: 2.0523 \n",
      "Epoch 36/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.9878 - mae: 1.2328 - mse: 1.9878 \n",
      "Epoch 37/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.9476 - mae: 1.2144 - mse: 1.9476 \n",
      "Epoch 38/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.7155 - mae: 1.1068 - mse: 1.7155 \n",
      "Epoch 39/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.5897 - mae: 1.0748 - mse: 1.5897 \n",
      "Epoch 40/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.8115 - mae: 1.1636 - mse: 1.8115 \n",
      "Epoch 41/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.7784 - mae: 1.1502 - mse: 1.7784 \n",
      "Epoch 42/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 8ms/step - loss: 1.7353 - mae: 1.1366 - mse: 1.7353 \n",
      "Epoch 43/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 1.8231 - mae: 1.1787 - mse: 1.8231 \n",
      "Epoch 44/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 1.6079 - mae: 1.0843 - mse: 1.6079 \n",
      "Epoch 45/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.7060 - mae: 1.1307 - mse: 1.7060 \n",
      "Epoch 46/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.5163 - mae: 1.0600 - mse: 1.5163 \n",
      "Epoch 47/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.7869 - mae: 1.1664 - mse: 1.7869 \n",
      "Epoch 48/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.5355 - mae: 1.0727 - mse: 1.5355 \n",
      "Epoch 49/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.7572 - mae: 1.1553 - mse: 1.7572 \n",
      "Epoch 50/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.5269 - mae: 1.0578 - mse: 1.5269 \n",
      "Epoch 51/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.6542 - mae: 1.1233 - mse: 1.6542 \n",
      "Epoch 52/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.4206 - mae: 1.0210 - mse: 1.4206 \n",
      "Epoch 53/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.4186 - mae: 1.0113 - mse: 1.4186 \n",
      "Epoch 54/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3353 - mae: 0.9878 - mse: 1.3353 \n",
      "Epoch 55/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3443 - mae: 1.0000 - mse: 1.3443 \n",
      "Epoch 56/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3308 - mae: 0.9780 - mse: 1.3308 \n",
      "Epoch 57/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3435 - mae: 0.9881 - mse: 1.3435 \n",
      "Epoch 58/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3493 - mae: 0.9980 - mse: 1.3493 \n",
      "Epoch 59/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3008 - mae: 0.9641 - mse: 1.3008 \n",
      "Epoch 60/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.3148 - mae: 0.9708 - mse: 1.3148 \n",
      "Epoch 61/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.1672 - mae: 0.8999 - mse: 1.1672 \n",
      "Epoch 62/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.1868 - mae: 0.9143 - mse: 1.1868 \n",
      "Epoch 63/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.2511 - mae: 0.9447 - mse: 1.2511 \n",
      "Epoch 64/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 6ms/step - loss: 1.1430 - mae: 0.9140 - mse: 1.1430 \n",
      "Epoch 65/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.2161 - mae: 0.9333 - mse: 1.2161 \n",
      "Epoch 66/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 1.1499 - mae: 0.9064 - mse: 1.1499 \n",
      "Epoch 67/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.0350 - mae: 0.8534 - mse: 1.0350 \n",
      "Epoch 68/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.1660 - mae: 0.9172 - mse: 1.1660 \n",
      "Epoch 69/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.0426 - mae: 0.8679 - mse: 1.0426 \n",
      "Epoch 70/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.9786 - mae: 0.8186 - mse: 0.9786 \n",
      "Epoch 71/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 1.0107 - mae: 0.8408 - mse: 1.0107 \n",
      "Epoch 72/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.9536 - mae: 0.8109 - mse: 0.9536 \n",
      "Epoch 73/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 1.0047 - mae: 0.8535 - mse: 1.0047 \n",
      "Epoch 74/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 0.8960 - mae: 0.7950 - mse: 0.8960 \n",
      "Epoch 75/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.9065 - mae: 0.7969 - mse: 0.9065 \n",
      "Epoch 76/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 0.8279 - mae: 0.7514 - mse: 0.8279 \n",
      "Epoch 77/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.8723 - mae: 0.7807 - mse: 0.8723 \n",
      "Epoch 78/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.8547 - mae: 0.7795 - mse: 0.8547 \n",
      "Epoch 79/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.8253 - mae: 0.7702 - mse: 0.8253 \n",
      "Epoch 80/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.7657 - mae: 0.7323 - mse: 0.7657 \n",
      "Epoch 81/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.6820 - mae: 0.6798 - mse: 0.6820 \n",
      "Epoch 82/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 0.8273 - mae: 0.7605 - mse: 0.8273 \n",
      "Epoch 83/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 0.7188 - mae: 0.7004 - mse: 0.7188 \n",
      "Epoch 84/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.7165 - mae: 0.7017 - mse: 0.7165 \n",
      "Epoch 85/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.6611 - mae: 0.6719 - mse: 0.6611 \n",
      "Epoch 86/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5889 - mae: 0.6346 - mse: 0.5889 \n",
      "Epoch 87/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step - loss: 0.6923 - mae: 0.6765 - mse: 0.6923 \n",
      "Epoch 88/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5968 - mae: 0.6367 - mse: 0.5968 \n",
      "Epoch 89/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.6324 - mae: 0.6669 - mse: 0.6324 \n",
      "Epoch 90/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5982 - mae: 0.6438 - mse: 0.5982 \n",
      "Epoch 91/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5485 - mae: 0.6107 - mse: 0.5485 \n",
      "Epoch 92/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5719 - mae: 0.6120 - mse: 0.5719 \n",
      "Epoch 93/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5381 - mae: 0.6000 - mse: 0.5381 \n",
      "Epoch 94/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5391 - mae: 0.5986 - mse: 0.5391 \n",
      "Epoch 95/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5115 - mae: 0.5823 - mse: 0.5115 \n",
      "Epoch 96/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.5177 - mae: 0.5878 - mse: 0.5177 \n",
      "Epoch 97/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.4677 - mae: 0.5556 - mse: 0.4677 \n",
      "Epoch 98/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.4303 - mae: 0.5198 - mse: 0.4303 \n",
      "Epoch 99/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.3980 - mae: 0.5144 - mse: 0.3980 \n",
      "Epoch 100/100\n",
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 5ms/step - loss: 0.4205 - mae: 0.5139 - mse: 0.4205 \n"
     ]
    }
   ],
   "source": [
    "history = model.fit(X_poly, y, epochs = 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "8c6fd952-f540-4ef1-b813-fa72e76a653b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss      : 0.3860\n",
      "mae       : 0.4979\n",
      "mse       : 0.3860\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(X_poly, y,  verbose=0)\n",
    "\n",
    "print('loss      : {:5.4f}'.format(score[0]))\n",
    "print('mae       : {:5.4f}'.format(score[1]))\n",
    "print('mse       : {:5.4f}'.format(score[2]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8e3d49c3-7e01-42f8-8b28-d97b634ea5de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m1/1\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 76ms/step\n",
      "Prediction : -3.5653694\n",
      "Reality    : -4.0401358473688\n"
     ]
    }
   ],
   "source": [
    "X1 = np.array([[-1.6342]]).reshape(-1, 1)\n",
    "y1 = (1 + 2 * X1 - 0.5 * X1**2 + 0.1 * X1**3)\n",
    "X_poly1 = np.hstack([X1, X1**2, X1**3])\n",
    "predictions = model.predict( X_poly1 )\n",
    "\n",
    "print(\"Prediction : \" + str(predictions[0][0]))\n",
    "print(\"Reality    : \" + str(format(y1[0][0])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "cec79c86-efbd-46e2-b439-be20ebbe3c59",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m7/7\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step \n",
      "MAE : 0.4979417879170137\n",
      "R2 : 0.9820949951418474\n"
     ]
    }
   ],
   "source": [
    "vary = np.sum((y - np.mean(y))**2)\n",
    "y_pred = model.predict( X_poly )\n",
    "\n",
    "residuals = y - y_pred\n",
    "error_moy = np.sum(np.abs(residuals)) / len(y)\n",
    "rss = np.sum(residuals**2)\n",
    "\n",
    "print(\"MAE : \" + str(error_moy))\n",
    "print(\"R2 : \" + str(1 - (rss / vary)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76b7bed2-a7e7-439d-abac-1692b4241fa6",
   "metadata": {},
   "source": [
    "Les prédictions obtenues sont alors plutôt bonnes avec plusieurs neurones non-linéaires, mais nous ne récupérons pas les paramètres du polynôme de départ, qui sont pourtant faciles à obtenir avec une régression linéaire classique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2406579f-5ec1-41b8-812b-d71f432ca491",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Les paramètres du polynome : [ 0.08049601 -0.49523311  2.09081018  1.02101087]\n"
     ]
    }
   ],
   "source": [
    "z = np.polyfit(list(X.flatten()), list(y.flatten()), 3)\n",
    "print(\"Les paramètres du polynome : \" + str(z)) #Plus haut degres en premier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1309f578-be5f-4bdc-a28f-624aafb42212",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4155176840113402\n",
      "R2 : 0.988246588509612\n"
     ]
    }
   ],
   "source": [
    "y_pred = z[3] + z[2] * X + z[1] * X**2 + z[0] * X**3\n",
    "\n",
    "residuals = y - y_pred\n",
    "error_moy = np.sum(np.abs(residuals)) / len(y)\n",
    "rss = np.sum(residuals**2)\n",
    "\n",
    "print(error_moy)\n",
    "print(\"R2 : \" + str(1 - (rss / vary)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0380083-0549-4afe-8b53-8af85705c1d6",
   "metadata": {},
   "source": [
    "## Exemple pratique : Prédiction des prix de l'immobilier à Boston"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abd17642-5807-48d5-994b-0293c8665d7a",
   "metadata": {},
   "source": [
    "Pour appliquer nos premières connaissances en matière de construction de réseaux de neurones, on va chercher à construire un modèle de prédiction des prix de l'immobilier sur un jeu de données très connu, les prix de l'immobilier à <a href='https://sergelhomme.fr/data/IA.zip'>Boston</a>. <br>\n",
    "Le jeu de données initial peut être retrouvé ici : https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html <br>\n",
    "Un autre exemple connu de prix de l'immobilier peut etre obtenu ici : https://keras.io/api/datasets/california_housing/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4f1ac443-6f00-4b05-9ed4-d276516f6ce1",
   "metadata": {},
   "outputs": [],
   "source": [
    "adresse = \"boston.csv\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "fd98e83c-8bda-46f7-993d-490a8a8b9c86",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>crim</th>\n",
       "      <th>zn</th>\n",
       "      <th>indus</th>\n",
       "      <th>chas</th>\n",
       "      <th>nox</th>\n",
       "      <th>rm</th>\n",
       "      <th>age</th>\n",
       "      <th>dis</th>\n",
       "      <th>rad</th>\n",
       "      <th>tax</th>\n",
       "      <th>ptratio</th>\n",
       "      <th>b</th>\n",
       "      <th>lstat</th>\n",
       "      <th>medv</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1</td>\n",
       "      <td>296</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2</td>\n",
       "      <td>242</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "      <td>21.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2</td>\n",
       "      <td>242</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "      <td>34.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3</td>\n",
       "      <td>222</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "      <td>33.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3</td>\n",
       "      <td>222</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "      <td>36.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      crim    zn  indus  chas    nox     rm   age     dis  rad  tax  ptratio  \\\n",
       "0  0.00632  18.0   2.31     0  0.538  6.575  65.2  4.0900    1  296     15.3   \n",
       "1  0.02731   0.0   7.07     0  0.469  6.421  78.9  4.9671    2  242     17.8   \n",
       "2  0.02729   0.0   7.07     0  0.469  7.185  61.1  4.9671    2  242     17.8   \n",
       "3  0.03237   0.0   2.18     0  0.458  6.998  45.8  6.0622    3  222     18.7   \n",
       "4  0.06905   0.0   2.18     0  0.458  7.147  54.2  6.0622    3  222     18.7   \n",
       "\n",
       "        b  lstat  medv  \n",
       "0  396.90   4.98  24.0  \n",
       "1  396.90   9.14  21.6  \n",
       "2  392.83   4.03  34.7  \n",
       "3  394.63   2.94  33.4  \n",
       "4  396.90   5.33  36.2  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(adresse,sep =',')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d55eac88-c366-4803-a9ca-816c4172ff84",
   "metadata": {},
   "source": [
    "CRIM: This is the per capita crime rate by town <br>\n",
    "ZN: This is the proportion of residential land zoned for lots larger than 25,000 sq.ft <br>\n",
    "INDUS: This is the proportion of non-retail business acres per town <br>\n",
    "CHAS: This is the Charles River dummy variable (this is equal to 1 if tract bounds river; 0 otherwise) <br>\n",
    "NOX: This is the nitric oxides concentration (parts per 10 million) <br>\n",
    "RM: This is the average number of rooms per dwelling <br>\n",
    "AGE: This is the proportion of owner-occupied units built prior to 1940 <br>\n",
    "DIS: This is the weighted distances to five Boston employment centers <br>\n",
    "RAD: This is the index of accessibility to radial highways <br>\n",
    "TAX: This is the full-value property-tax rate per 10,000 dollars <br>\n",
    "PTRATIO: This is the pupil-teacher ratio by town <br>\n",
    "B: This is calculated as 1000(Bk — 0.63)^2, where Bk is the proportion of people of African American descent by town <br>\n",
    "LSTAT: This is the percentage lower status of the population <br>\n",
    "MEDV: This is the median value of owner-occupied homes in 1000 dollars <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eef02d99-8381-42cc-b857-8b34ec3bed13",
   "metadata": {},
   "source": [
    "Premièrement, on va appliquer une régression linéaire multiple classique sur ces données. On doit alors clairement distinguer les données d'entrainement et les données de test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fb8fe415-0502-4d67-8a27-82fbf7ccbc98",
   "metadata": {},
   "outputs": [],
   "source": [
    "#On decoupe les donnees en donnees d'entrainement et de test en mélangeant celles-ci\n",
    "data = df.sample(frac=1., axis=0)\n",
    "data_train = data.sample(frac=0.7, axis=0)\n",
    "data_test  = data.drop(data_train.index)\n",
    "#On separe x et y\n",
    "x_train = data_train.drop('medv',  axis=1)\n",
    "y_train = data_train['medv']\n",
    "x_test  = data_test.drop('medv',   axis=1)\n",
    "y_test  = data_test['medv']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcda9417-09d6-4910-b93a-fd063c0b1191",
   "metadata": {},
   "source": [
    "On va normaliser les données, car elles sont très hétérogènes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "622627cb-9737-4279-8caf-6cae99e9fa7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#On normalise X\n",
    "mean = x_train.mean()\n",
    "std  = x_train.std()\n",
    "x_train = (x_train - mean) / std\n",
    "x_test  = (x_test  - mean) / std\n",
    "#On passe en array\n",
    "x_train, y_train = np.array(x_train), np.array(y_train)\n",
    "x_test,  y_test  = np.array(x_test),  np.array(y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ed2a9e4d-eb80-4731-a259-edd93383dcaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Calcul de la variance a reconstituer\n",
    "vary = np.sum((y_test - np.mean(y_test))**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20d51a79-1679-490c-b79c-70cafdaff198",
   "metadata": {},
   "source": [
    "On peut appliquer à ce jeu de données, plus précisément sur les données d'entrainement dans un premier temps (x_train, y_train), une régression linéaire classique :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "c1effa50-9e4d-45d2-a647-da0dfe132827",
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import t, f\n",
    "from scipy.stats import norm\n",
    "\n",
    "def reg(X, y, names, verbose=True) :\n",
    "    # === 1. Ajouter une constante pour l’intercept ===\n",
    "    X_ = np.c_[np.ones(len(X)), X]  # colonne de 1 pour le biais\n",
    "    # === 2. Calcul des coefficients OLS : β̂ = (XᵀX)⁻¹ Xᵀy ===\n",
    "    XtX_inv = np.linalg.inv(X_.T @ X_)\n",
    "    beta_hat = XtX_inv @ X_.T @ y\n",
    "    # === 3. Calcul des résidus ===\n",
    "    y_pred = X_ @ beta_hat\n",
    "    residuals = y - y_pred\n",
    "    # === 4. Variance résiduelle : σ² = RSS / (n - p) ===\n",
    "    n, p = X_.shape  # p = nb de paramètres (incluant l’intercept)\n",
    "    RSS = np.sum(residuals**2)\n",
    "    sigma2 = RSS / (n - p)\n",
    "    # === 5. Variance des coefficients : Var(β̂) = σ² * (XᵀX)⁻¹ ===\n",
    "    var_beta = sigma2 * XtX_inv\n",
    "    se_beta = np.sqrt(np.diag(var_beta))  # erreurs standard\n",
    "    # === 6. t-statistics : t = β̂ / SE(β̂) ===\n",
    "    t_stats = beta_hat / se_beta\n",
    "    # === 7. p-values (bilatérales) ===\n",
    "    p_values = 2 * (1 - t.cdf(np.abs(t_stats), df=n - p))\n",
    "    # === 8. R2 ===\n",
    "    r2 = 1 - RSS / np.sum((y - np.mean(y))**2)\n",
    "    k = p - 1\n",
    "    F = (r2 / k) / ((1 - r2) / (n - k - 1))\n",
    "    p_value = 1 - f.cdf(F, k, n - k - 1)\n",
    "    # === Affichage ===\n",
    "    if verbose :\n",
    "        for i, (b, se, tval, pval) in enumerate(zip(beta_hat, se_beta, t_stats, p_values)):\n",
    "            if i == 0 : print(f\"cst: {b:.4f}, SE={se:.4f}, t={tval:.4f}, p={pval:.4f}\")\n",
    "            if i != 0 : print(f\"{names[i-1]}: {b:.4f}, SE={se:.4f}, t={tval:.4f}, p={pval:.4f}\")\n",
    "        print(f\"\\nR² = {1 - RSS / np.sum((y - np.mean(y))**2):.4f}\")\n",
    "    return beta_hat, se_beta, t_stats, p_values, r2, p_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b284a55b-ed50-407c-957b-dc1624dd564f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cst: 22.3819, SE=0.2577, t=86.8399, p=0.0000\n",
      "crim: -0.9784, SE=0.3427, t=-2.8546, p=0.0046\n",
      "zn: 0.8455, SE=0.3727, t=2.2683, p=0.0239\n",
      "indus: 0.2121, SE=0.5103, t=0.4156, p=0.6780\n",
      "chas: 0.8184, SE=0.2661, t=3.0757, p=0.0023\n",
      "nox: -2.1003, SE=0.5356, t=-3.9212, p=0.0001\n",
      "rm: 2.8640, SE=0.3526, t=8.1217, p=0.0000\n",
      "age: 0.5534, SE=0.4417, t=1.2529, p=0.2111\n",
      "dis: -2.7938, SE=0.4935, t=-5.6616, p=0.0000\n",
      "rad: 2.7411, SE=0.7272, t=3.7693, p=0.0002\n",
      "tax: -1.7197, SE=0.8028, t=-2.1421, p=0.0329\n",
      "ptratio: -2.1600, SE=0.3370, t=-6.4093, p=0.0000\n",
      "b: 1.0519, SE=0.3075, t=3.4214, p=0.0007\n",
      "lstat: -4.0640, SE=0.4258, t=-9.5444, p=0.0000\n",
      "\n",
      "R² = 0.7398\n"
     ]
    }
   ],
   "source": [
    "noms = df.columns\n",
    "res = reg(x_train, y_train, noms)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7784b3fd-3479-4ed8-af4c-65b6053aa917",
   "metadata": {},
   "source": [
    "Le R2 nous apprend que la régression est plutôt de bonne qualité. La plupart des variables semblent significatives. Le jeu de données est donc plutôt bon à donner à un réseau de neurones. A priori il y a de la connaissance experte derrière. On peut alors faire des prédictions sur le jeu de données test en récupérant les coefficients calculés précédemment sur les données d'entrainement :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "cc157e3b-ddff-4b82-825d-145e2b5c264b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pred :  8.196582272310376 Value :  5.0\n",
      "Pred :  31.389964640513373 Value :  31.1\n"
     ]
    }
   ],
   "source": [
    "X_ = np.c_[np.ones(len(x_test)), x_test]\n",
    "y_pred = X_ @ res[0]\n",
    "ypred = res[-1]\n",
    "print(\"Pred : \", y_pred[10], \"Value : \", y_test[10])\n",
    "print(\"Pred : \", y_pred[11], \"Value : \", y_test[11])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "8555a8fe-f755-4758-9274-60f4d43a0e06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE : 3.508086014387225\n",
      "R2 : 0.7258450492857227\n"
     ]
    }
   ],
   "source": [
    "residuals = y_test - y_pred\n",
    "error_moy = np.sum(np.abs(residuals)) / len(y_test)\n",
    "rss = np.sum(residuals**2)\n",
    "print(\"MAE : \" + str(error_moy))\n",
    "print(\"R2 : \" + str(1 - (rss / vary)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32c3b623-5a2f-44bc-8ac1-807a4f1fcf80",
   "metadata": {},
   "source": [
    "Les prédictions sont bonnes : le R2 est à la hauteur de celui du jeu d'entrainement. On précise ici le calcul de la MAE qui est davantage utilisé lorsque l'on utilise des réseaux de neurones. Maintenant, étudions ce que l'on peut obtenir en utilisant un réseau de neurones avec des activations non linéaires, puisque sinon on a vu que l'on retrouvera les résultats de la régression linéaire. Peut-on mieux faire en termes de prédictions ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "bc2e015e-1766-4b69-8763-067a15dd5c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Input((13,), name=\"InputLayer\"))\n",
    "model.add(Dense(32, activation='relu', name='Dense_n1'))\n",
    "model.add(Dense(64, activation='relu', name='Dense_n2'))\n",
    "model.add(Dense(32, activation='relu', name='Dense_n3'))\n",
    "model.add(Dense(1, name='Output'))\n",
    "  \n",
    "model.compile(optimizer = 'adam', loss = 'mse', metrics = ['mae', 'mse'] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "fd4f0079-4a20-4ec3-acb7-ed3c8e23c709",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<keras.src.callbacks.history.History at 0x1aa4c2b6810>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs = 200, verbose = False, validation_data = (x_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "65675169-5437-498b-bb86-cdf2294f459b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss      : 13.8986\n",
      "mae       : 2.7022\n",
      "mse       : 13.8986\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print('loss      : {:5.4f}'.format(score[0]))\n",
    "print('mae       : {:5.4f}'.format(score[1]))\n",
    "print('mse       : {:5.4f}'.format(score[2]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "54ddfc6e-63e9-4532-a4d8-6c3ce4546dc8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction : 18.34\n",
      "Value :  22.0\n",
      "Prediction : 11.83\n",
      "Value :  13.4\n"
     ]
    }
   ],
   "source": [
    "predictions = model( x_test )\n",
    "print(\"Prediction : {:.2f}\".format(predictions[0][0]))\n",
    "print(\"Value : \", y_test[0])\n",
    "print(\"Prediction : {:.2f}\".format(predictions[1][0]))\n",
    "print(\"Value : \", y_test[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "281660cd-e837-4c85-bbf8-c829ee39a6af",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAE : 2.70217988804767\n",
      "R2 : 0.8234363701493373\n"
     ]
    }
   ],
   "source": [
    "residuals = np.array(np.array(predictions).flatten()) - y_test\n",
    "error_moy = np.sum(np.abs(residuals))/len(y_test)\n",
    "rss = np.sum(residuals**2)\n",
    "print(\"MAE : \" + str(error_moy))\n",
    "print(\"R2 : \" + str(1 - (rss / vary)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bda24e5-8710-4397-9f59-090a4c60b940",
   "metadata": {},
   "source": [
    "Le calcul du R2 que j'ai rajouté, mais aussi surtout celui de la MAE nous montre que le réseau de neurones arrive dans cet exemple à mieux prédire les prix de l'immobilier qu'une régression linéaire multiple. La prise en compte d'un modèle non linéaire semble donc plus appropriée ici. Il faut reconnaitre ici que c'est bien souvent le cas et qu'il est difficile battre les réseaux de neurones en matière de prédictions dans de nombreux domaines. Néanmoins, en matière de compréhension, pour exploiter les connaissances du réseau et donc ce qui influence les prix de l'immobilier, il faudra s'armer d'outils difficiles à présenter maintenant avec notre niveau de compétence. On a donc affaire à une boite noire pour l'instant. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d589026-327f-44c4-b19f-d3965868dff9",
   "metadata": {},
   "source": [
    "## L'alternative Scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d53ccf23-83a9-4fb4-8614-943ce3997427",
   "metadata": {},
   "source": [
    "Pour ce type de problème, il n'est pas nécessaire d'utiliser des architectures profondes ou particulièrement complexes. L'utilisation de TensorFlow peut même sembler excessive, dans la mesure où des outils plus simples permettent déjà de construire des réseaux de neurones performants. La bibliothèque scikit-learn propose notamment la classe MLPRegressor, qui permet d'appliquer rapidement un réseau de neurones multicouche à un jeu de données tabulaire. Nous utilisons néanmoins TensorFlow ici car il offre un contrôle plus fin sur l'architecture du réseau et constitue aujourd'hui une référence pour l'apprentissage du deep learning. Ci-dessous l'utilisation de MLPRegressor :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "2e57bac5-91f9-4aad-b08d-46992c2fb727",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPRegressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "e47b40ca-3284-4a44-807e-8eaf5159b1dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6221644268051829"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr = MLPRegressor(random_state=1, max_iter=2000, tol=0.1) # Une couche de 100 neurones\n",
    "regr.fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "8df85724-d044-4ace-a329-c11ede4873b9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.69868613458768"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Deux couches de neurones sont en théorie plus adaptées à notre problème\n",
    "regr = MLPRegressor(random_state=1, hidden_layer_sizes=(50,50), max_iter=2000, tol=0.1) \n",
    "regr.fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d5f71e0-2093-451b-8941-d10113cc69e8",
   "metadata": {},
   "source": [
    "Les résultats sont moins bons qu'avec notre réseau \"construit à la main\". Néanmoins, la force de Scikit-learn est que cette bibliothèque permet d'utiliser d'autres algorithmes de prédictions issus du machine learning qui sont en théorie meilleurs pour ce type de prédictions :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "fdb435cc-56fa-4934-a429-2b348dca872a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7812232553842731"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "regr = RandomForestRegressor(max_depth=2, random_state=0)\n",
    "regr.fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "9d20e835-ac8b-47e8-820a-d34c7ffa0bbf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.868236377908265"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "regr = GradientBoostingRegressor(random_state=0)\n",
    "regr.fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "8968d623-9df6-428e-8dca-46915cee2059",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.868236377908265"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import HistGradientBoostingRegressor #Pour de grands jeu de données\n",
    "HistGradientBoostingRegressor().fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "2c1f4c6b-605e-4c2d-b8f5-660d530ba679",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.620450119313478"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVR\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "regr = make_pipeline(StandardScaler(), SVR(C=1.0, epsilon=0.2))\n",
    "regr.fit(x_train, y_train)\n",
    "regr.predict(x_test)\n",
    "regr.score(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5612ac02-0608-4b20-b7f1-ea5f0ac5d568",
   "metadata": {},
   "source": [
    "A ce jeu de prédictions, sur ce jeu de données, c'est l'algorithme GradientBoostingRegressor qui a gagné et pas le réseau de neurones !"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}