Lab 01 - Encoding and Classifying Text.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rbblUQgh5QCi"
   },
   "source": [
    "# Encoding and Classifying Text\n",
    "\n",
    "This lab is about how to get started with ML using sklearn.\n",
    "\n",
    "* Read about the [20 newsgroups](https://scikit-learn.org/stable/datasets/real_world.html#the-20-newsgroups-text-dataset) data set. The data set provides training and test splits.\n",
    "* Experiment with feature extraction and classification. For example:\n",
    "  * Change which newsgroups are included (be ware of using too much data)\n",
    "  * Change the feature extraction: word counts vs tf-idf, ngrams for words and characters\n",
    "  * Try using clean data vs unprocessed. You can change this in the data loader.\n",
    "* Try out some classifiers\n",
    "  * [kNN](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html#sklearn.neighbors.KNeighborsClassifier)\n",
    "  * [Decision trees](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier)\n",
    "  * [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html)\n",
    "* Plot your data and classification results using PCA and matplotlib.\n",
    "\n",
    "This is intended as an exercise, not as an examination, so try to ask as many questions as you can during the lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KukO-BJWd0wu"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d.axes3d import Axes3D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "55Mz_fD78rsy"
   },
   "source": [
    "## Load the data set\n",
    "\n",
    "Let's load the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fhRh9wYOdvss"
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_20newsgroups\n",
    "categories = ['alt.atheism', 'talk.religion.misc', 'comp.graphics', 'sci.space']\n",
    "\n",
    "newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)\n",
    "newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)\n",
    "\n",
    "print(newsgroups_train.data[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_Ivd8mX2pe69"
   },
   "source": [
    "## Vectorization\n",
    "\n",
    "A vectorizer transforms your text into vectors. See more in the [docs](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html). Look at the loaded newsgroup objects to find the text (```newsgroups_train.data```), labels (```newsgroups_train.target_names```), and even label encodings (```newsgroups_train.target```)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hPa2TyAjs91V"
   },
   "source": [
    "## Classification\n",
    "\n",
    "Import a classifier from sklearn the try to train it on your training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "425RrA7Dt1Cr"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B8G6c60L84N6"
   },
   "source": [
    "## Linear dimensionality reduction\n",
    "\n",
    "Principal component analysis (PCA) finds a linear subspace for our data, so that it can be plotted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qjMD9-eT9qia"
   },
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "pca = PCA(n_components=min(100, X.shape[1]//2))\n",
    "X_pca = pca.fit_transform(X)\n",
    "\n",
    "plt.figure(figsize=(6, 4))\n",
    "plt.plot(np.cumsum(pca.explained_variance_ratio_)*100)\n",
    "plt.ylabel(\"Cumulative explained variance [%]\")\n",
    "plt.xlabel(\"n components\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MaVO1b6l9BIQ"
   },
   "outputs": [],
   "source": [
    "def plot_2d(X, y, labels, ax=None):\n",
    "  if ax is None:\n",
    "    fig = plt.figure(figsize=(6, 6), dpi=100)\n",
    "    ax = fig.subplots(1, 1)\n",
    "  ax.scatter(X[:, 0], X[:, 1], c=y, s=15, cmap='tab10', alpha=.5)\n",
    "  for label in np.unique(y):\n",
    "    ax.text(np.mean(X[y==label, 0]),\n",
    "            np.mean(X[y==label, 1]),\n",
    "            labels[label],\n",
    "            fontsize=16, zorder=1)\n",
    "  if ax is None:\n",
    "    fig.tight_layout(pad=0)\n",
    "    fig.show()\n",
    "\n",
    "fig = plt.figure(figsize=(13, 6))\n",
    "ax = fig.subplots(1, 2)\n",
    "plot_2d(X, y, labels, ax[0])\n",
    "plot_2d(X_pca, y, labels, ax[1])\n",
    "fig.tight_layout(pad=0)\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9EcLl4vREozK"
   },
   "outputs": [],
   "source": [
    "def plot_3d(X, y, labels, angle=None, ax=None):\n",
    "  if ax is None:\n",
    "    fig = plt.figure(figsize=(8, 6), dpi=100)\n",
    "    ax = fig.add_subplot(1, 1, 1, projection='3d')\n",
    "  ax.scatter(X[:, 0], X[:, 1], X[:, 2], c=y,\n",
    "              alpha=.5, cmap='tab10')\n",
    "  for label in np.unique(y):\n",
    "    ax.text(np.mean(X[y==label, 0]),\n",
    "            np.mean(X[y==label, 1]),\n",
    "            np.mean(X[y==label, 2]),\n",
    "            labels[label],\n",
    "            fontsize=10,\n",
    "            horizontalalignment='center',\n",
    "            verticalalignment='center')\n",
    "  if angle is not None:\n",
    "    ax.view_init(20, angle % 360)\n",
    "  if ax is None:\n",
    "    fig.tight_layout(pad=0)\n",
    "    fig.show()\n",
    "\n",
    "fig = plt.figure(figsize=(8, 6), dpi=100)\n",
    "ax = fig.add_subplot(1, 1, 1, projection='3d')\n",
    "plot_3d(X_pca, y, labels, ax=ax)\n",
    "fig.show()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Lab 01 - Encoding and Classifying Text.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}