Doc updates

.ipynb_checkpoints/object_detection_CAFO_screencap-checkpoint.ipynb

@@ -0,0 +1,318 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using GIS and a TensorFlow Convolutional Neural Network to Perform CAFO Site Detection in NAIP Imagery\n",
+    "\n",
+    "Please be sure to visit the official [repo for this project] (https://github.com/Qberto/ML_ObjectDetection_CAFO/) for further information."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true,
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import os\n",
+    "import six.moves.urllib as urllib\n",
+    "import sys\n",
+    "import tarfile\n",
+    "import tensorflow as tf\n",
+    "import zipfile\n",
+    "\n",
+    "from PIL import ImageGrab\n",
+    "import time\n",
+    "\n",
+    "from collections import defaultdict\n",
+    "from io import StringIO\n",
+    "from matplotlib import pyplot as plt\n",
+    "from PIL import Image\n",
+    "\n",
+    "import cv2"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Env setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# This is needed to display the images.\n",
+    "%matplotlib inline\n",
+    "\n",
+    "# This is needed since the notebook is stored in the object_detection folder.\n",
+    "sys.path.append(\"..\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Object detection imports\n",
+    "Here are the imports from the object detection module."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "from utils import label_map_util\n",
+    "\n",
+    "from utils import visualization_utils as vis_util"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Model preparation "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Variables\n",
+    "\n",
+    "Any model exported using the `export_inference_graph.py` tool can be loaded here simply by changing `PATH_TO_CKPT` to point to a new .pb file.  \n",
+    "\n",
+    "By default we use an \"SSD with Mobilenet\" model here. See the [detection model zoo](https://github.com/tensorflow/models/blob/master/object_detection/g3doc/detection_model_zoo.md) for a list of other models that can be run out-of-the-box with varying speeds and accuracies."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# What model to download.\n",
+    "MODEL_NAME = 'cafo_graph'\n",
+    "\n",
+    "# Path to frozen detection graph. This is the actual model that is used for the object detection.\n",
+    "PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'\n",
+    "\n",
+    "# List of the strings that is used to add correct label for each box.\n",
+    "PATH_TO_LABELS = os.path.join('training', 'object-detection.pbtxt')\n",
+    "\n",
+    "NUM_CLASSES = 1"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Download Model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Load a (frozen) Tensorflow model into memory."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "detection_graph = tf.Graph()\n",
+    "with detection_graph.as_default():\n",
+    "  od_graph_def = tf.GraphDef()\n",
+    "  with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:\n",
+    "    serialized_graph = fid.read()\n",
+    "    od_graph_def.ParseFromString(serialized_graph)\n",
+    "    tf.import_graph_def(od_graph_def, name='')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Loading label map\n",
+    "Label maps map indices to category names, so that when our convolution network predicts `5`, we know that this corresponds to `airplane`.  Here we use internal utility functions, but anything that returns a dictionary mapping integers to appropriate string labels would be fine"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "label_map = label_map_util.load_labelmap(PATH_TO_LABELS)\n",
+    "categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True)\n",
+    "category_index = label_map_util.create_category_index(categories)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Helper code"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "def load_image_into_numpy_array(image):\n",
+    "  (im_width, im_height) = image.size\n",
+    "  return np.array(image.getdata()).reshape(\n",
+    "      (im_height, im_width, 3)).astype(np.uint8)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Detection"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# For the sake of simplicity we will use only 2 images:\n",
+    "# image1.jpg\n",
+    "# image2.jpg\n",
+    "# If you want to test the code with your images, just add path to the images to the TEST_IMAGE_PATHS.\n",
+    "PATH_TO_TEST_IMAGES_DIR = 'test_images'\n",
+    "# PATH_TO_TEST_IMAGES_DIR = 'custom_images'\n",
+    "TEST_IMAGE_PATHS = [ os.path.join(PATH_TO_TEST_IMAGES_DIR, 'image{}.jpg'.format(i)) for i in range(7, 8) ]\n",
+    "\n",
+    "# Size, in inches, of the output images.\n",
+    "IMAGE_SIZE = (12, 8)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "def cafo_in_image(classes_arr, scores_arr, obj_thresh=5):\n",
+    "    stacked_arr = np.stack((classes_arr, scores_arr), axis=-1)\n",
+    "    cafo_found_flag = False\n",
+    "    for ix in range(obj_thresh):\n",
+    "        if 1.00000000e+00 in stacked_arr[ix]:\n",
+    "            cafo_found_flag = True\n",
+    "            \n",
+    "    return cafo_found_flag"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true,
+    "scrolled": false
+   },
+   "outputs": [],
+   "source": [
+    "with detection_graph.as_default():\n",
+    "    with tf.Session(graph=detection_graph) as sess:\n",
+    "        # Definite input and output Tensors for detection_graph\n",
+    "        image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')\n",
+    "        # Each box represents a part of the image where a particular object was detected.\n",
+    "        detection_boxes = detection_graph.get_tensor_by_name('detection_boxes:0')\n",
+    "        # Each score represent how level of confidence for each of the objects.\n",
+    "        # Score is shown on the result image, together with the class label.\n",
+    "        detection_scores = detection_graph.get_tensor_by_name('detection_scores:0')\n",
+    "        detection_classes = detection_graph.get_tensor_by_name('detection_classes:0')\n",
+    "        num_detections = detection_graph.get_tensor_by_name('num_detections:0')\n",
+    "        while True:\n",
+    "            \n",
+    "            # Expand dimensions since the model expects images to have shape: [1, None, None, 3]\n",
+    "            image_np = np.array(ImageGrab.grab(bbox=(0,40,800,640)))\n",
+    "#             image_np = np.array(ImageGrab.grab(bbox=(30,100,900,640)))\n",
+    "            image_np_expanded = np.expand_dims(image_np, axis=0)\n",
+    "            # Actual detection.\n",
+    "            (boxes, scores, classes, num) = sess.run(\n",
+    "              [detection_boxes, detection_scores, detection_classes, num_detections],\n",
+    "              feed_dict={image_tensor: image_np_expanded})\n",
+    "            # Visualization of the results of a detection.\n",
+    "            vis_util.visualize_boxes_and_labels_on_image_array(\n",
+    "              image_np,\n",
+    "              np.squeeze(boxes),\n",
+    "              np.squeeze(classes).astype(np.int32),\n",
+    "              np.squeeze(scores),\n",
+    "              category_index,\n",
+    "              use_normalized_coordinates=True,\n",
+    "              line_thickness=8, min_score_thresh=0.8)\n",
+    "            \n",
+    "            cv2.imshow('object detection', cv2.resize(image_np, (800,600)))\n",
+    "            if cv2.waitKey(25) & 0xFF == ord('q'):\n",
+    "                cv2.destroyAllWindows()\n",
+    "                break\n",
+    "                \n",
+    "            cafo_found = cafo_in_image(np.squeeze(classes).astype(np.int32), \n",
+    "                                           np.squeeze(scores), \n",
+    "                                           obj_thresh=2)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

README.md

@@ -109,6 +109,8 @@ Goal: We can finally train the model! Let's teach this puppy how to do a new tri
 
 Description: We will use a script from the TensorFlow models repo to execute the training epochs (feedforward + backpropagation) for the CNN. You can find the script in this repo at 4_train.py but I recommend that you clone the [TensorFlow models repo](https://github.com/tensorflow/models/tree/master/research/object_detection) and execute train.py from that directory, referencing your workspace and configuration file as needed. 
 
+Neural Network Pattern Detection (https://github.com/Qberto/ML_ObjectDetection_CAFO/blob/master/doc/img/neuralnetwork.jpg)
+
 To execute training (finally!) run the following command:
 
 '''