{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercise 5.1: Mastering .loc for Pandas data frames\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pandas can be a bit frustrating during your first experiences with it. In this and the next few exercises, we will do our first practice with it. Stick with it! The more and more you use it, the more distant the memory of life without it will become.\n", "\n", "We will work with a data set from [Kleinteich and Gorb, *Sci. Rep.*, **4**, 5355, 2014](https://doi.org/10.1038/srep05225), and was [featured in the New York Times](http://www.nytimes.com/2014/08/25/science/a-frog-thats-a-living-breathing-pac-man.html). They measured several properties about the tongue strikes of horned frogs. Let's take a look at the data set, which is in the file `~/git/data/frog_tongue_adhesion.csv`." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# These data are from the paper,\n", "# Kleinteich and Gorb, Sci. Rep., 4, 5225, 2014.\n", "# It was featured in the New York Times.\n", "# http://www.nytimes.com/2014/08/25/science/a-frog-thats-a-living-breathing-pac-man.html\n", "#\n", "# The authors included the data in their supplemental information.\n", "#\n", "# Importantly, the ID refers to the identifites of the frogs they tested.\n", "# I: adult, 63 mm snout-vent-length (SVL) and 63.1 g body weight,\n", "# Ceratophrys cranwelli crossed with Ceratophrys cornuta\n", "# II: adult, 70 mm SVL and 72.7 g body weight,\n", "# Ceratophrys cranwelli crossed with Ceratophrys cornuta\n", "# III: juvenile, 28 mm SVL and 12.7 g body weight, Ceratophrys cranwelli\n", "# IV: juvenile, 31 mm SVL and 12.7 g body weight, Ceratophrys cranwelli\n", "date,ID,trial number,impact force (mN),impact time (ms),impact force / body weight,adhesive force (mN),time frog pulls on target (ms),adhesive force / body weight,adhesive impulse (N-s),total contact area (mm2),contact area without mucus (mm2),contact area with mucus / contact area without mucus,contact pressure (Pa),adhesive strength (Pa)\n", "2013_02_26,I,3,1205,46,1.95,-785,884,1.27,-0.290,387,70,0.82,3117,-2030\n", "2013_02_26,I,4,2527,44,4.08,-983,248,1.59,-0.181,101,94,0.07,24923,-9695\n", "2013_03_01,I,1,1745,34,2.82,-850,211,1.37,-0.157,83,79,0.05,21020,-10239\n", "2013_03_01,I,2,1556,41,2.51,-455,1025,0.74,-0.170,330,158,0.52,4718,-1381\n", "2013_03_01,I,3,493,36,0.80,-974,499,1.57,-0.423,245,216,0.12,2012,-3975\n" ] } ], "source": [ "!head -20 data/frog_tongue_adhesion.csv" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first lines all begin with `#` signs, signifying that they are comments and not data. They do give important information, though, such as the meaning of the ID data. The ID refers to which specific frog was tested.\n", "\n", "Immediately after the comments, we have a row of comma-separated headers. This row sets the number of columns in this data set and labels the meaning of the columns. So, we see that the first column is the date of the experiment, the second column is the ID of the frog, the third is the trial number, and so on.\n", "\n", "After this row, each row represents a single experiment where the frog struck the target. So, these data are already in tidy format. \n", "\n", "**a)** Load in the data set into a data frame. Be sure to use the appropriate value for the `comment` keyword argument of `pd.read_csv()`.\n", "\n", "**b)** Extract the impact time of all impacts that had an adhesive strength of magnitude greater than 2000 Pa. *Note*: The data in the `'adhesive strength (Pa)'` column is all negative. This is because the adhesive force is defined to be negative in the measurement. Without changing the data in the data frame, how can you check that the magnitude (the absolute value) is greater than 2000?\n", "\n", "**c)** Extract the impact force and adhesive force for all of Frog II's strikes.\n", "\n", "**d)** Extract the adhesive force and the time the frog pulls on the target for juvenile frogs (Frogs III and IV). *Hint*: We saw the `&` operator for Boolean indexing across more than one column. The `|` operator signifies OR, and works analogously. For technical reasons that we can discuss if you like, the Python operators `and` and `or` will **not** work for Boolean indexing of data frames. You could also approach this using the `isin()` method of a Pandas `Series`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Solution\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To read in the data frame, we use the `comment='#'` kwarg." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
dateIDtrial numberimpact force (mN)impact time (ms)impact force / body weightadhesive force (mN)time frog pulls on target (ms)adhesive force / body weightadhesive impulse (N-s)total contact area (mm2)contact area without mucus (mm2)contact area with mucus / contact area without mucuscontact pressure (Pa)adhesive strength (Pa)
02013_02_26I31205461.95-7858841.27-0.290387700.823117-2030
12013_02_26I42527444.08-9832481.59-0.181101940.0724923-9695
22013_03_01I11745342.82-8502111.37-0.15783790.0521020-10239
32013_03_01I21556412.51-45510250.74-0.1703301580.524718-1381
42013_03_01I3493360.80-9744991.57-0.4232452160.122012-3975
\n", "
" ], "text/plain": [ " date ID trial number impact force (mN) impact time (ms) \\\n", "0 2013_02_26 I 3 1205 46 \n", "1 2013_02_26 I 4 2527 44 \n", "2 2013_03_01 I 1 1745 34 \n", "3 2013_03_01 I 2 1556 41 \n", "4 2013_03_01 I 3 493 36 \n", "\n", " impact force / body weight adhesive force (mN) \\\n", "0 1.95 -785 \n", "1 4.08 -983 \n", "2 2.82 -850 \n", "3 2.51 -455 \n", "4 0.80 -974 \n", "\n", " time frog pulls on target (ms) adhesive force / body weight \\\n", "0 884 1.27 \n", "1 248 1.59 \n", "2 211 1.37 \n", "3 1025 0.74 \n", "4 499 1.57 \n", "\n", " adhesive impulse (N-s) total contact area (mm2) \\\n", "0 -0.290 387 \n", "1 -0.181 101 \n", "2 -0.157 83 \n", "3 -0.170 330 \n", "4 -0.423 245 \n", "\n", " contact area without mucus (mm2) \\\n", "0 70 \n", "1 94 \n", "2 79 \n", "3 158 \n", "4 216 \n", "\n", " contact area with mucus / contact area without mucus \\\n", "0 0.82 \n", "1 0.07 \n", "2 0.05 \n", "3 0.52 \n", "4 0.12 \n", "\n", " contact pressure (Pa) adhesive strength (Pa) \n", "0 3117 -2030 \n", "1 24923 -9695 \n", "2 21020 -10239 \n", "3 4718 -1381 \n", "4 2012 -3975 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('data/frog_tongue_adhesion.csv', comment='#')\n", "\n", "# Take a look\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**b)** To extract the entries with strong adhesive strength, we need to use the `np.abs()` function to esure that the absolute value of the adhesive strength is above 2000." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 46\n", "1 44\n", "2 34\n", "4 36\n", "7 46\n", "8 50\n", "11 48\n", "13 31\n", "14 38\n", "17 60\n", "19 40\n", "23 59\n", "24 33\n", "25 43\n", "27 31\n", "29 42\n", "31 57\n", "33 21\n", "35 29\n", "37 31\n", "38 15\n", "39 42\n", "42 105\n", "44 29\n", "45 16\n", "47 31\n", "49 32\n", "50 30\n", "51 16\n", "52 27\n", "53 30\n", "54 35\n", "55 39\n", "57 34\n", "59 34\n", "60 26\n", "61 20\n", "62 55\n", "65 33\n", "66 74\n", "67 26\n", "68 27\n", "69 33\n", "71 6\n", "73 31\n", "74 34\n", "75 38\n", "78 33\n", "Name: impact time (ms), dtype: int64" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.loc[np.abs(df['adhesive strength (Pa)']) > 2000, 'impact time (ms)']" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
impact force (mN)adhesive force (mN)
201612-655
21605-292
22327-246
23946-245
24541-553
251539-664
26529-261
27628-691
281453-92
29297-566
30703-223
31269-512
32751-227
33245-573
341182-522
35515-599
36435-364
37383-469
38457-844
39730-648
\n", "
" ], "text/plain": [ " impact force (mN) adhesive force (mN)\n", "20 1612 -655\n", "21 605 -292\n", "22 327 -246\n", "23 946 -245\n", "24 541 -553\n", "25 1539 -664\n", "26 529 -261\n", "27 628 -691\n", "28 1453 -92\n", "29 297 -566\n", "30 703 -223\n", "31 269 -512\n", "32 751 -227\n", "33 245 -573\n", "34 1182 -522\n", "35 515 -599\n", "36 435 -364\n", "37 383 -469\n", "38 457 -844\n", "39 730 -648" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# c) Impact force and adhesive force for Frog II\n", "df.loc[df['ID']=='II', ['impact force (mN)', 'adhesive force (mN)']]" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
adhesive force (mN)time frog pulls on target (ms)
40-94683
41-163245
42-172619
43-2251823
44-301918
45-931351
46-1311790
47-2891006
48-104883
49-2291218
50-259910
51-231550
52-2672081
53-178376
54-123289
55-151607
56-1272932
57-372680
58-236685
59-3901308
60-456462
61-193250
62-236743
63-225844
64-217728
65-161472
66-139959
67-264844
68-3421515
69-231279
70-2091427
71-2922874
72-3394251
73-371626
74-3311254
75-302986
76-2161627
77-1632021
78-3671366
79-2181269
\n", "
" ], "text/plain": [ " adhesive force (mN) time frog pulls on target (ms)\n", "40 -94 683\n", "41 -163 245\n", "42 -172 619\n", "43 -225 1823\n", "44 -301 918\n", "45 -93 1351\n", "46 -131 1790\n", "47 -289 1006\n", "48 -104 883\n", "49 -229 1218\n", "50 -259 910\n", "51 -231 550\n", "52 -267 2081\n", "53 -178 376\n", "54 -123 289\n", "55 -151 607\n", "56 -127 2932\n", "57 -372 680\n", "58 -236 685\n", "59 -390 1308\n", "60 -456 462\n", "61 -193 250\n", "62 -236 743\n", "63 -225 844\n", "64 -217 728\n", "65 -161 472\n", "66 -139 959\n", "67 -264 844\n", "68 -342 1515\n", "69 -231 279\n", "70 -209 1427\n", "71 -292 2874\n", "72 -339 4251\n", "73 -371 626\n", "74 -331 1254\n", "75 -302 986\n", "76 -216 1627\n", "77 -163 2021\n", "78 -367 1366\n", "79 -218 1269" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# d) Adhesive force and time frog pulls for frogs III and IV\n", "df.loc[\n", " df[\"ID\"].isin([\"III\", \"IV\"]),\n", " [\"adhesive force (mN)\", \"time frog pulls on target (ms)\"],\n", "]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing environment" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPython 3.7.7\n", "IPython 7.16.1\n", "\n", "numpy 1.18.5\n", "pandas 0.24.2\n", "jupyterlab 2.1.5\n" ] } ], "source": [ "%load_ext watermark\n", "%watermark -v -p numpy,pandas,jupyterlab" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 4 }