james@codepost.io_hw01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 1: Causality and Expressions\n",
    "\n",
    "Please complete this notebook by filling in the cells provided. Before you begin, execute the following cell to load the provided tests. Note that you need to run the following cell in order for the `ok.submit()` and `ok.grade()` statements to work correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=====================================================================\n",
      "Assignment: Homework 1: Causality and Expressions\n",
      "OK, version v1.12.5\n",
      "=====================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Don't change this cell; just run it. \n",
    "from client.api.notebook import Notebook\n",
    "ok = Notebook('hw01.ok')\n",
    "##_ = ok.auth(inline=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reading:\n",
    "- Textbook chapters [1](http://www.inferentialthinking.com/chapters/01/what-is-data-science.html), [2](http://www.inferentialthinking.com/chapters/02/causality-and-experiments.html), and [3](http://www.inferentialthinking.com/chapters/03/programming-in-python.html)\n",
    "\n",
    "For all problems for which you must provide explanations, you **must** provide your answer in the designated space. Moreover, throughout this homework and all future ones, please be sure to not re-assign variables throughout the notebook! For example, if you use `max_temperature` in your answer to one question, do not reassign it later on. \n",
    "\n",
    "Deadline:\n",
    "\n",
    "This assignment is due Friday, Oct 19 at 9:00am. You will receive an early submission bonus point if you turn in your final submission by Thursday, October 18 at 9:00am. Late work will not be accepted as per the [course policies](https://piazza.com/ucsb/fall2018/int5/home).\n",
    "\n",
    "Directly sharing answers is **not okay**, but discussing problems with the course mentors or with other students is encouraged within the limits of the Integrity of Scholarship Agreement (see below). **To maximize your learning and understanding of the material, we strongly recommend carefully reading the instructions and making an honest attempt at solving the problem yourself (i.e., on your own) before discussing it with anyone or asking for help.**\n",
    "\n",
    "You should start early so that you have time to get help if you're stuck. The schedule for the drop-in office hours will be posted on Piazza."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Submitting your work:\n",
    "\n",
    "Once you're finished, select \"Save and Checkpoint\" in the File menu and then execute the `ok.submit()` cell below. See additional instructions on how to submit your work at the end of this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "test_output = [ok.grade(q[:-3]) for q in os.listdir('tests') if q.startswith('q')]\n",
    "%run upload_tests.py james@codepost.io Hello test_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Integrity of Scholarship Agreement\n",
    "\n",
    "Before starting the homework, please carefully read and fill out this integrity of scholarship agreement form. You will not recieve scores in this class until you submit the form.\n",
    "https://goo.gl/forms/qrYnW4b3tUc0CRI53\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Interpreting a Scatter Plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following chart is called a scatter plot. It shows the number of freshman admits and applicants to UCSB by year, from 2008 to 2018. If you are interested in the raw data, the admissions numbers from 2015-2018 can be found [here](https://www.ucop.edu/institutional-research-academic-planning/content-analysis/ug-admissions/ug-data.html) and the data from before 2015 can be found [here](https://www.universityofcalifornia.edu/infocenter/admissions-residency-and-ethnicity).\n",
    "Interpret the scatter plot to answer the two questions below.\n",
    "<img src=\"admissions_stats.PNG\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.** About how many students applied to UCSB in the year 2011? Assign either 1, 2, 3, 4, or 5 to the name `admissions_q1` below.\n",
    "\n",
    "1. 22,000\n",
    "2. 30,000\n",
    "3. 50,000\n",
    "4. 72,000\n",
    "5. 92,000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "admissions_q1 = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 2.** What is the approximate admission rate to UCSB during the past admissions cycle (2018)? Admission rate is defined as the total number of admits divided by the total number of applicants, then multiplied by 100. Round your answer to the nearest whole number and assign either 1, 2, or 3 to the name `admissions_q2` below.\n",
    "1. 33%\n",
    "2. 0.33%\n",
    "3. 306%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "admissions_q2 = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 3.** How does the 2018 admission rate compare to the previous year? Assign either 1, 2, or 3, which correspond to the options below, to the name `admissions_q3` below.\n",
    "1. The admission rate in 2017 was lower than in 2018.\n",
    "2. The admission rates are about the same.\n",
    "3. The admission rate in 2017 was higher than in 2018.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "admissions_q3 = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that your answers are in the correct format. This test *does not* check that you answered correctly; only that you assigned a number successfully in each multiple-choice answer cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Names and Assignment Statements\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.** When you run the following cell, Python produces a cryptic error message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "## 4 = 2 + 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choose the best explanation of what's wrong with the code, and then assign 1, 2, 3, or 4 to `names_q1` below to indicate your answer.\n",
    "\n",
    "1. Python is smart and already knows `# 4 = 2 + 2`.\n",
    "\n",
    "2. `4` is a number, and it doesn't make sense to make a number be a name for something else. In Python, \"`x = 2 + 2`\" means \"assign `x` as the name for the value of `2 + 2`.\"\n",
    "\n",
    "3. It should be `2 + 2 = 4`.\n",
    "\n",
    "4. I don't get an error message. This is a trick question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "names_q1 = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 2.** When you run the following cell, Python will produce another cryptic error message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "two = 3\n",
    "## six = two plus two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choose the best explanation of what's wrong with the code and assign 1, 2, 3, or 4 to `names_q2` below to indicate your answer.\n",
    "\n",
    "1. The `plus` operation only applies to numbers, not the word \"two\".\n",
    "\n",
    "2. The name \"two\" cannot be assigned to the number 3.\n",
    "\n",
    "3. Two plus two is four, not six.\n",
    "\n",
    "4. The name `two` cannot be followed directly by another name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "names_q2 = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that your answers are in the correct format. This test *does not* check that you answered correctly; only that you assigned a number successfully in each multiple-choice answer cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 3_ > Suite 1 > Case 2\n",
      "\n",
      ">>> names_q2 == 4\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q3 --suite 1 --case 2\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 1\n",
      "[oooook.....] 50.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q3')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Job Opportunities & Education in Rural India\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A [study](http://www.nber.org/papers/w16021.pdf) at UCLA investigated factors that might result in greater attention to the health and education of girls in rural India. One such factor is information about job opportunities for women. The idea is that if people know that educated women can get good jobs, they might take more care of the health and education of girls in their families, as an investment in the girls’ future potential as earners.\n",
    "\n",
    "The study focused on 160 villages outside the capital of India, all with little access to information about call centers and similar organizations that offer job opportunities to women. In 80 of the villages chosen at random, recruiters visited the village, described the opportunities, recruited women who had some English language proficiency and experience with computers, and provided ongoing support free of charge for three years. In the other 80 villages, no recruiters visited and no other intervention was made.\n",
    "\n",
    "At the end of the study period, the researchers recorded data about the school attendance and health of the children in the villages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.** Which statement best describes the *treatment* and *control* groups for this study? Assign either 1, 2, or 3 to the name `jobs_q1` below.\n",
    "\n",
    "1. The treatment group was the 80 villages visited by recruiters, and the control group was the other 80 villages with no intervention.\n",
    "\n",
    "2. The treatment group was the 160 villages selected, and the control group was the rest of the villages outside the capital of India.\n",
    "\n",
    "3. There is no clear notion of *treatment* and *control* group in this study."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "jobs_q1 = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 2.** Was this an observational study or a randomized controlled experiment? Assign either 1, 2, or 3 to the name `jobs_q2` below.\n",
    "\n",
    "1. This was an observational study.\n",
    "\n",
    "2. This was a randomized controlled experiment.  \n",
    "\n",
    "3. This was a randomized observational study."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "jobs_q2 = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "job_opportunities_3"
   },
   "source": [
    "**Question 3.** The study reported, “Girls aged 5-15 in villages that received the recruiting services were 3 to 5 percentage points more likely to be in school and experienced an increase in Body Mass Index, reflecting greater nutrition and/or medical care. However, there was no net gain in height. For boys, there was no change in any of these measures.” Why do you think the author points out the lack of change in the boys?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The author points this out because the study focused on girls, not boys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that your answers are in the correct format. This test *does not* check that you answered correctly; only that you assigned a number successfully in each multiple-choice answer cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Differences between Universities\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.** Suppose you'd like to *quantify* how *dissimilar* two universities are, using three quantitative characteristics.  The US Department of Education data on [UCSB](https://collegescorecard.ed.gov/school/?110705-University-of-California-Santa-Barbara) and [Cal Poly SLO](https://collegescorecard.ed.gov/school/?110422-California-Polytechnic-State-University-San-Luis-Obispo) describes the following three traits (among many others):\n",
    "\n",
    "| Trait                                | UCSB   | SLO    |\n",
    "|--------------------------------------|--------|--------|\n",
    "| Average annual cost to attend ($)    | 14,900 | 18,845 |\n",
    "| Graduation rate (percentage)         | 82     | 78     |\n",
    "| Socioeconomic Diversity (percentage) | 39     | 19     |\n",
    "\n",
    "You decide to define the dissimilarity between two universities as the maximum of the absolute values of the 3 differences in their respective trait values.\n",
    "\n",
    "_Hint: break up the previous statement into the various components that you can then assemble into a single statement. E.g., first, you need the difference between each trait value; next, the differences need to be ..._\n",
    "\n",
    "Using this method, compute the dissimilarity between UCSB and Cal Poly SLO.  Name the result `dissimilarity`.  Use a single expression (a single line of code) to compute the answer.  Let Python perform all the arithmetic (like subtracting 78 from 82) rather than simplifying the expression yourself. The built-in `abs` function takes absolute values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3945"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dissimilarity = max(abs(18845-14900),(abs(82-78)),(abs(39-19)))\n",
    "dissimilarity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the cell below to test for formatting (in this case, that dissimilarity is a number)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q5_1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Extra Credit Question.** Is UCSB more similar to 1) UCSD, 2) SLO, or 3) Berkeley?  Assign either 1, 2, or 3 to the name `q5_2` below.\n",
    "We will give extra credit only to the correct answers that neatly show the work below. Make sure that you **don't overwrite/reassign the value of `dissimilarity`** (i.e., use a different name).\n",
    "\n",
    "| Trait                                | UCSB   | SLO    | UCSD   | Berkeley |\n",
    "|--------------------------------------|--------|--------|--------|----------|\n",
    "| Average annual cost to attend ($)    | 14,900 | 18,845 |        |          |\n",
    "| Graduation rate (percentage)         | 82     | 78     |        |          |\n",
    "| Socioeconomic Diversity (percentage) | 39     | 19     |        |          |\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "q5_2 = ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Nearsightedness Study\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Myopia, or nearsightedness, results from a number of genetic and environmental factors. In 1999, Quinn et al studied the relation between myopia and ambient lighting at night (for example, from nightlights or room lights) during childhood."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "nearsightedness_1"
   },
   "source": [
    "**Question 1.** The data were gathered by the following procedure, reported in the study. “Between January and June 1998, parents of children aged 2-16 years [...] that were seen as outpatients in a university pediatric ophthalmology clinic completed a questionnaire on the child’s light exposure both at present and before the age of 2 years.” Was this study observational, or was it a controlled experiment?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observational"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "nearsightedness_2"
   },
   "source": [
    "**Question 2.** The study found that of the children who slept with a room light on before the age of 2, 55% were myopic. Of the children who slept with a night light on before the age of 2, 34% were myopic. Of the children who slept in the dark before the age of 2, 10% were myopic. The study concluded that, \"The prevalence of myopia [...] during childhood was strongly associated with ambient light exposure during sleep at night in the first two years after birth.\"\n",
    "\n",
    "Do the data support this statement? You may interpret “strongly” in any reasonable qualitative way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yes, moderately strong"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "nearsightedness_3"
   },
   "source": [
    "**Question 3.** On May 13, 1999, CNN reported the results of this study under the headline, “Night light may lead to nearsightedness.” Does the conclusion of the study claim that night light causes nearsightedness?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No, correlation does not prove causation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "nearsightedness_4"
   },
   "source": [
    "**Question 4.** The final paragraph of the CNN report said that “several eye specialists” had pointed out that the study should have accounted for heredity.\n",
    "\n",
    "Myopia is passed down from parents to children. It's reasonable to suppose that myopic parents are more likely to leave lights on in their children's rooms than other parents. In what way do you think this might have affected the data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The study may not have accounted for which kids' parents were myopic, so the groups could have been more controlled for that confounding factor by putting them into a group of kids with myopic parents and kids without myopic parents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Studying the Survivors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Reverend Henry Whitehead was skeptical of John Snow’s conclusion about the Broad Street pump. After the Broad Street cholera epidemic ended, Whitehead set about trying to prove Snow wrong.  (The history of the event is detailed [here](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1034367/pdf/medhist00183-0026.pdf).)\n",
    "\n",
    "He realized that Snow had focused his analysis almost entirely on those who had died. Whitehead, therefore, investigated the drinking habits of people in the Broad Street area who had not died in the outbreak.\n",
    "\n",
    "What is the main reason it was important to study this group?\n",
    "\n",
    "1) If Whitehead had found that many people had drunk water from the Broad Street pump and not caught cholera, that would have been evidence against Snow's hypothesis.\n",
    "\n",
    "2) Survivors could provide additional information about what else could have caused the cholera, potentially unearthing another cause.\n",
    "\n",
    "3) Through considering the survivors, Whitehead could have identified a cure for cholera."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assign survivor_answer to 1, 2, or 3\n",
    "survivor_answer = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 7_1 > Suite 1 > Case 1\n",
      "\n",
      ">>> type(survivor_answer) == int\n",
      "True\n",
      ">>> survivor_answer == 1\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q7_1 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "_ = ok.grade('q7_1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** Whitehead ended up finding further proof that the Broad Street pump played the central role in spreading the disease to the people who lived near it. Eventually, he became one of Snow’s greatest defenders."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Submission\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Once you're finished, select \"Save and Checkpoint\" in the File menu and then execute the `submit` cell below. The result will contain a link that you can use to check that your assignment has been submitted successfully. If you submit more than once before the deadline, we will only grade your final submission. If you mistakenly submit the wrong one, you can head to okpy.org and flag the correct version. To do so, go to the website, click on this assignment, and find the version you would like to be graded. There should be an option to flag that submission for grading!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 2_3 > Suite 1 > Case 1\n",
      "\n",
      ">>> admissions_q3 == 3\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q2_3 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 2\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 3_2 > Suite 1 > Case 1\n",
      "\n",
      ">>> names_q2 == 4\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q3_2 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 3_ > Suite 1 > Case 2\n",
      "\n",
      ">>> names_q2 == 4\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q3 --suite 1 --case 2\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 1\n",
      "[oooook.....] 50.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 7_1 > Suite 1 > Case 1\n",
      "\n",
      ">>> type(survivor_answer) == int\n",
      "True\n",
      ">>> survivor_answer == 1\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q7_1 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 2_1 > Suite 1 > Case 1\n",
      "\n",
      ">>> admissions_q1 == 3\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q2_1 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Question 5_2 > Suite 1 > Case 1\n",
      "\n",
      ">>> # EXTRA CREDIT QUESTION\n",
      ">>> q5_2 == 1\n",
      "False\n",
      "\n",
      "# Error: expected\n",
      "#     True\n",
      "# but got\n",
      "#     False\n",
      "\n",
      "Run only this test case with \"python3 ok -q q5_2 --suite 1 --case 1\"\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 0\n",
      "    Failed: 1\n",
      "[k..........] 0.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n",
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "Running tests\n",
      "\n",
      "---------------------------------------------------------------------\n",
      "Test summary\n",
      "    Passed: 1\n",
      "    Failed: 0\n",
      "[ooooooooook] 100.0% passed\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "test_output = [ok.grade(q[:-3]) for q in os.listdir('tests') if q.startswith('q')]\n",
    "%run upload_tests.py james@codepost.io Hello test_output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9.0"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Each question is worth 1 point\n",
    "q2_1 = 1 - 1\n",
    "q2_2 = 1 - 0\n",
    "q2_3 = 1 - 1\n",
    "q3_1 = 1 - 0\n",
    "q3_2 = 1 - 1\n",
    "q4_1 = 1 - 0\n",
    "q4_2 = 1 - 0\n",
    "q4_3 = 1 - 0.5 # incomplete explanation\n",
    "q5_1 = 1 - 0\n",
    "q5_extra = 0 + 0 \n",
    "#conceptual\n",
    "q6_1 = 1 - 0\n",
    "q6_2 = 1 - 0\n",
    "q6_3 = 1 - 0\n",
    "q6_4 = 1 - 0.5 # didn't address how this might have affected the data\n",
    "# \n",
    "q7_1 = 1 - 1\n",
    "total =  q2_1 + q2_2 + q2_3 + q3_1 + q3_2 + q4_1 + q4_2 + q4_3 + q5_1 + q5_extra + q6_1 + q6_2 + q6_3 + q6_4 + q7_1 \n",
    "total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}