Optical Character Recognition using Support Vector Machines.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optical Character Recognition using Support Vector Machines\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OCR of Hand-written Digits\n",
    "\n",
    "In kNN, we directly used pixel intensity as the feature vector. This time we will use Histogram of Oriented Gradients (HOG) as feature vectors.\n",
    "\n",
    "Here, before finding the HOG, we deskew the image using its second order moments. So we first define a function deskew() which takes a digit image and deskew it. Below is the deskew() function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def deskew(img):\n",
    "    m = cv2.moments(img)\n",
    "    if abs(m['mu02']) < 1e-2:\n",
    "        return img.copy()\n",
    "    skew = m['mu11']/m['mu02']\n",
    "    M = np.float32([[1, skew, -0.5*SZ*skew], [0, 1, 0]])\n",
    "    img = cv2.warpAffine(img,M,(SZ, SZ),flags=affine_flags)\n",
    "    return img"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below image shows above deskew function applied to an image of zero. Left image is the original image and right\n",
    "image is the deskewed image.\n",
    "\n",
    "<img src=\"images/skew.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we have to find the HOG Descriptor of each cell. For that, we find Sobel derivatives of each cell in X and Y\n",
    "direction. Then find their magnitude and direction of gradient at each pixel. This gradient is quantized to 16 integer\n",
    "values. Divide this image to four sub-squares. For each sub-square, calculate the histogram of direction (16 bins)\n",
    "weighted with their magnitude. So each sub-square gives you a vector containing 16 values. Four such vectors (of\n",
    "four sub-squares) together gives us a feature vector containing 64 values. This is the feature vector we use to train our\n",
    "data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def hog(img):\n",
    "    gx = cv2.Sobel(img, cv2.CV_32F, 1, 0)\n",
    "    gy = cv2.Sobel(img, cv2.CV_32F, 0, 1)\n",
    "    mag, ang = cv2.cartToPolar(gx, gy)\n",
    "    # quantizing binvalues in (0...16)\n",
    "    bins = np.int32(bin_n*ang/(2*np.pi))\n",
    "    # Divide to 4 sub-squares\n",
    "    bin_cells = bins[:10,:10], bins[10:,:10], bins[:10,10:], bins[10:,10:]\n",
    "    mag_cells = mag[:10,:10], mag[10:,:10], mag[:10,10:], mag[10:,10:]\n",
    "    hists = [np.bincount(b.ravel(), m.ravel(), bin_n) for b, m in zip(bin_cells, mag_cells)]\n",
    "    hist = np.hstack(hists)\n",
    "    return hist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, as in the previous case, we start by splitting our big dataset into individual cells. For every digit, 250 cells are\n",
    "reserved for training data and remaining 250 data is reserved for testing. Full code is given below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import cv2\n",
    "import numpy as np\n",
    "SZ=20\n",
    "bin_n = 16 # Number of bins\n",
    "svm_params = dict( kernel_type = cv2.SVM_LINEAR, svm_type = cv2.SVM_C_SVC, C=2.67, gamma=5.383 )\n",
    "affine_flags = cv2.WARP_INVERSE_MAP|cv2.INTER_LINEAR\n",
    "\n",
    "def deskew(img):\n",
    "    m = cv2.moments(img)\n",
    "    if abs(m['mu02']) < 1e-2:\n",
    "        return img.copy()\n",
    "    skew = m['mu11']/m['mu02']\n",
    "    M = np.float32([[1, skew, -0.5*SZ*skew], [0, 1, 0]])\n",
    "    img = cv2.warpAffine(img,M,(SZ, SZ),flags=affine_flags)\n",
    "    return img\n",
    "\n",
    "def hog(img):\n",
    "    gx = cv2.Sobel(img, cv2.CV_32F, 1, 0)\n",
    "    gy = cv2.Sobel(img, cv2.CV_32F, 0, 1)\n",
    "    mag, ang = cv2.cartToPolar(gx, gy)\n",
    "    bins = np.int32(bin_n*ang/(2*np.pi)) # quantizing binvalues in (0...16)\n",
    "    bin_cells = bins[:10,:10], bins[10:,:10], bins[:10,10:], bins[10:,10:]\n",
    "    mag_cells = mag[:10,:10], mag[10:,:10], mag[:10,10:], mag[10:,10:]\n",
    "    hists = [np.bincount(b.ravel(), m.ravel(), bin_n) for b, m in zip(bin_cells, mag_cells)]\n",
    "    hist = np.hstack(hists) # hist is a 64 bit vector\n",
    "    return hist\n",
    "\n",
    "img = cv2.imread('digits.png',0)\n",
    "cells = [np.hsplit(row,100) for row in np.vsplit(img,50)]\n",
    "\n",
    "# First half is trainData, remaining is testData\n",
    "train_cells = [ i[:50] for i in cells ]\n",
    "test_cells = [ i[50:] for i in cells]\n",
    "\n",
    "###### Now training ########################\n",
    "deskewed = [map(deskew,row) for row in train_cells]\n",
    "hogdata = [map(hog,row) for row in deskewed]\n",
    "trainData = np.float32(hogdata).reshape(-1,64)\n",
    "responses = np.float32(np.repeat(np.arange(10),250)[:,np.newaxis])\n",
    "svm = cv2.SVM()\n",
    "svm.train(trainData,responses, params=svm_params)\n",
    "svm.save('svm_data.dat')\n",
    "\n",
    "###### Now testing ########################\n",
    "deskewed = [map(deskew,row) for row in test_cells]\n",
    "hogdata = [map(hog,row) for row in deskewed]\n",
    "testData = np.float32(hogdata).reshape(-1,bin_n*4)\n",
    "result = svm.predict_all(testData)\n",
    "\n",
    "####### Check Accuracy ########################\n",
    "mask = result==responses\n",
    "correct = np.count_nonzero(mask)\n",
    "print correct*100.0/result.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This particular technique gave me nearly 94% accuracy. You can try different values for various parameters of SVM\n",
    "to check if higher accuracy is possible. Or you can read technical papers on this area and try to implement them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}