cord19_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  COVID-19 Open Research Dataset (CORD-19)\n",
    "\n",
    "- https://pages.semanticscholar.org/coronavirus-research\n",
    "- https://www.kaggle.com/acmiyaguchi/cord-19-citation-network-with-deduping/output\n",
    "- https://lg-covid-19-hotp.cs.duke.edu/\n",
    "\n",
    "### Stats\n",
    "\n",
    "Papers in CORD-19:\n",
    "- has valid DOI\n",
    "External papers cited by CORD-19 papers:\n",
    "- total\n",
    "- with DOI\n",
    "- scraped\n",
    "\n",
    "\n",
    "### Schema\n",
    "\n",
    "https://ai2-semanticscholar-cord-19.s3-us-west-2.amazonaws.com/2020-03-13/json_schema.txt\n",
    "```\n",
    "# JSON schema of full text documents\n",
    "{\n",
    "    \"paper_id\": <str>,                      # 40-character sha1 of the PDF\n",
    "    \"metadata\": {\n",
    "        \"title\": <str>,\n",
    "        \"authors\": [                        # list of author dicts, in order\n",
    "            {\n",
    "                \"first\": <str>,\n",
    "                \"middle\": <list of str>,\n",
    "                \"last\": <str>,\n",
    "                \"suffix\": <str>,\n",
    "                \"affiliation\": <dict>,\n",
    "                \"email\": <str>\n",
    "            },\n",
    "            ...\n",
    "        ],\n",
    "        \"abstract\": [                       # list of paragraphs in the abstract\n",
    "            {\n",
    "                \"text\": <str>,\n",
    "                \"cite_spans\": [             # list of character indices of inline citations\n",
    "                                            # e.g. citation \"[7]\" occurs at positions 151-154 in \"text\"\n",
    "                                            #      linked to bibliography entry BIBREF3\n",
    "                    {\n",
    "                        \"start\": 151,\n",
    "                        \"end\": 154,\n",
    "                        \"text\": \"[7]\",\n",
    "                        \"ref_id\": \"BIBREF3\"\n",
    "                    },\n",
    "                    ...\n",
    "                ],\n",
    "                \"ref_spans\": <list of dicts similar to cite_spans>,     # e.g. inline reference to \"Table 1\"\n",
    "                \"section\": \"Abstract\"\n",
    "            },\n",
    "            ...\n",
    "        ],\n",
    "        \"body_text\": [                      # list of paragraphs in full body\n",
    "                                            # paragraph dicts look the same as above\n",
    "            {\n",
    "                \"text\": <str>,\n",
    "                \"cite_spans\": [],\n",
    "                \"ref_spans\": [],\n",
    "                \"eq_spans\": [],\n",
    "                \"section\": \"Introduction\"\n",
    "            },\n",
    "            ...\n",
    "            {\n",
    "                ...,\n",
    "                \"section\": \"Conclusion\"\n",
    "            }\n",
    "        ],\n",
    "        \"bib_entries\": {\n",
    "            \"BIBREF0\": {\n",
    "                \"ref_id\": <str>,\n",
    "                \"title\": <str>,\n",
    "                \"authors\": <list of dict>       # same structure as earlier,\n",
    "                                                # but without `affiliation` or `email`\n",
    "                \"year\": <int>,\n",
    "                \"venue\": <str>,\n",
    "                \"volume\": <str>,\n",
    "                \"issn\": <str>,\n",
    "                \"pages\": <str>,\n",
    "                \"other_ids\": {\n",
    "                    \"DOI\": [\n",
    "                        <str>\n",
    "                    ]\n",
    "                }\n",
    "            },\n",
    "            \"BIBREF1\": {},\n",
    "            ...\n",
    "            \"BIBREF25\": {}\n",
    "        },\n",
    "        \"ref_entries\":\n",
    "            \"FIGREF0\": {\n",
    "                \"text\": <str>,                  # figure caption text\n",
    "                \"type\": \"figure\"\n",
    "            },\n",
    "            ...\n",
    "            \"TABREF13\": {\n",
    "                \"text\": <str>,                  # table caption text\n",
    "                \"type\": \"table\"\n",
    "            }\n",
    "        },\n",
    "        \"back_matter\": <list of dict>           # same structure as body_text\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "### Target output\n",
    "\n",
    "- With text `text,text_b,label`\n",
    "- With IDs `doc_id,doc_id_b,label`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "import math\n",
    "import random\n",
    "import pickle\n",
    "import json\n",
    "import re\n",
    "import numpy as np\n",
    "from tqdm import tqdm_notebook as tqdm\n",
    "from collections import defaultdict\n",
    "import requests\n",
    "import time\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "from fuzzywuzzy import fuzz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/mnt/hdd/experiments/mostendorff/acl-anthology/environments\n",
      "Environment detected: gpu_server (in default.yml)\n"
     ]
    }
   ],
   "source": [
    "from experiments.environment import get_env\n",
    "\n",
    "env = get_env()\n",
    "\n",
    "n_splits = 4\n",
    "\n",
    "scraper_dir = './output/cord19/'\n",
    "cord19_dir = os.path.join(env['datasets_dir'], 'cord-19')\n",
    "dummy_id = '21a4369f83891bf6975dd916c0aa495d5df8709e'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "meta_df = pd.read_csv(os.path.join(cord19_dir, 'metadata.csv'), index_col=0, dtype={'doi': str})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sha</th>\n",
       "      <th>source_x</th>\n",
       "      <th>title</th>\n",
       "      <th>doi</th>\n",
       "      <th>pmcid</th>\n",
       "      <th>pubmed_id</th>\n",
       "      <th>license</th>\n",
       "      <th>abstract</th>\n",
       "      <th>publish_time</th>\n",
       "      <th>authors</th>\n",
       "      <th>journal</th>\n",
       "      <th>Microsoft Academic Paper ID</th>\n",
       "      <th>WHO #Covidence</th>\n",
       "      <th>has_full_text</th>\n",
       "      <th>full_text_file</th>\n",
       "      <th>url</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cord_uid</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4360s2yu</th>\n",
       "      <td>289deae0b2050aa259a05ba84565a4df82fa099a</td>\n",
       "      <td>Elsevier</td>\n",
       "      <td>Personal Protective Equipment: Protecting Heal...</td>\n",
       "      <td>10.1016/j.clinthera.2015.07.007</td>\n",
       "      <td>PMC4661082</td>\n",
       "      <td>26452427.0</td>\n",
       "      <td>els-covid</td>\n",
       "      <td>Abstract Purpose The recent Ebola epidemic tha...</td>\n",
       "      <td>2015-11-01</td>\n",
       "      <td>Fischer, William A.; Weber, David J.; Wohl, Da...</td>\n",
       "      <td>Clinical Therapeutics</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>True</td>\n",
       "      <td>custom_license</td>\n",
       "      <td>https://doi.org/10.1016/j.clinthera.2015.07.007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66jumbir</th>\n",
       "      <td>21a4369f83891bf6975dd916c0aa495d5df8709e</td>\n",
       "      <td>Elsevier</td>\n",
       "      <td>Viruses and asthma</td>\n",
       "      <td>10.1016/j.bbagen.2011.01.012</td>\n",
       "      <td>PMC3130828</td>\n",
       "      <td>21291960.0</td>\n",
       "      <td>els-covid</td>\n",
       "      <td>Abstract Background Viral respiratory infectio...</td>\n",
       "      <td>2011-11-30</td>\n",
       "      <td>Dulek, Daniel E.; Peebles, R. Stokes</td>\n",
       "      <td>Biochimica et Biophysica Acta (BBA) - General ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>True</td>\n",
       "      <td>custom_license</td>\n",
       "      <td>https://doi.org/10.1016/j.bbagen.2011.01.012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3wk36h9p</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Elsevier</td>\n",
       "      <td>Why the WHO won't use the p-word</td>\n",
       "      <td>10.1016/s0262-4079(20)30474-7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>els-covid</td>\n",
       "      <td>There are no criteria for a pandemic, but covi...</td>\n",
       "      <td>2020-03-07</td>\n",
       "      <td>MacKenzie, Debora</td>\n",
       "      <td>New Scientist</td>\n",
       "      <td>NaN</td>\n",
       "      <td>#5716</td>\n",
       "      <td>False</td>\n",
       "      <td>custom_license</td>\n",
       "      <td>https://doi.org/10.1016/s0262-4079(20)30474-7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0ujw0gak</th>\n",
       "      <td>NaN</td>\n",
       "      <td>WHO</td>\n",
       "      <td>Communication, transparency key as Canada face...</td>\n",
       "      <td>10.1503/cmaj.1095846</td>\n",
       "      <td>PMC7030882</td>\n",
       "      <td>32071113.0</td>\n",
       "      <td>unk</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-02-17</td>\n",
       "      <td>Glauser, Wendy</td>\n",
       "      <td>Canadian Medical Association Journal</td>\n",
       "      <td>1.953688e+09</td>\n",
       "      <td>#4117</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>https://doi.org/10.1503/cmaj.1095846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28vx9w58</th>\n",
       "      <td>3369a14e1d116943f48b3a33597796c9802de279; f523...</td>\n",
       "      <td>PMC</td>\n",
       "      <td>Searching for animal models and potential targ...</td>\n",
       "      <td>10.1016/j.onehlt.2017.03.001</td>\n",
       "      <td>PMC5454147</td>\n",
       "      <td>28616501.0</td>\n",
       "      <td>cc-by-nc-nd</td>\n",
       "      <td>Emerging and re-emerging pathogens represent a...</td>\n",
       "      <td>2017-03-03</td>\n",
       "      <td>Vergara-Alert, Júlia; Vidal, Enric; Bensaid, A...</td>\n",
       "      <td>One Health</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>True</td>\n",
       "      <td>noncomm_use_subset</td>\n",
       "      <td>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                        sha  source_x  \\\n",
       "cord_uid                                                                \n",
       "4360s2yu           289deae0b2050aa259a05ba84565a4df82fa099a  Elsevier   \n",
       "66jumbir           21a4369f83891bf6975dd916c0aa495d5df8709e  Elsevier   \n",
       "3wk36h9p                                                NaN  Elsevier   \n",
       "0ujw0gak                                                NaN       WHO   \n",
       "28vx9w58  3369a14e1d116943f48b3a33597796c9802de279; f523...       PMC   \n",
       "\n",
       "                                                      title  \\\n",
       "cord_uid                                                      \n",
       "4360s2yu  Personal Protective Equipment: Protecting Heal...   \n",
       "66jumbir                                 Viruses and asthma   \n",
       "3wk36h9p                   Why the WHO won't use the p-word   \n",
       "0ujw0gak  Communication, transparency key as Canada face...   \n",
       "28vx9w58  Searching for animal models and potential targ...   \n",
       "\n",
       "                                      doi       pmcid   pubmed_id  \\\n",
       "cord_uid                                                            \n",
       "4360s2yu  10.1016/j.clinthera.2015.07.007  PMC4661082  26452427.0   \n",
       "66jumbir     10.1016/j.bbagen.2011.01.012  PMC3130828  21291960.0   \n",
       "3wk36h9p    10.1016/s0262-4079(20)30474-7         NaN         NaN   \n",
       "0ujw0gak             10.1503/cmaj.1095846  PMC7030882  32071113.0   \n",
       "28vx9w58     10.1016/j.onehlt.2017.03.001  PMC5454147  28616501.0   \n",
       "\n",
       "              license                                           abstract  \\\n",
       "cord_uid                                                                   \n",
       "4360s2yu    els-covid  Abstract Purpose The recent Ebola epidemic tha...   \n",
       "66jumbir    els-covid  Abstract Background Viral respiratory infectio...   \n",
       "3wk36h9p    els-covid  There are no criteria for a pandemic, but covi...   \n",
       "0ujw0gak          unk                                                NaN   \n",
       "28vx9w58  cc-by-nc-nd  Emerging and re-emerging pathogens represent a...   \n",
       "\n",
       "         publish_time                                            authors  \\\n",
       "cord_uid                                                                   \n",
       "4360s2yu   2015-11-01  Fischer, William A.; Weber, David J.; Wohl, Da...   \n",
       "66jumbir   2011-11-30               Dulek, Daniel E.; Peebles, R. Stokes   \n",
       "3wk36h9p   2020-03-07                                  MacKenzie, Debora   \n",
       "0ujw0gak   2020-02-17                                     Glauser, Wendy   \n",
       "28vx9w58   2017-03-03  Vergara-Alert, Júlia; Vidal, Enric; Bensaid, A...   \n",
       "\n",
       "                                                    journal  \\\n",
       "cord_uid                                                      \n",
       "4360s2yu                              Clinical Therapeutics   \n",
       "66jumbir  Biochimica et Biophysica Acta (BBA) - General ...   \n",
       "3wk36h9p                                      New Scientist   \n",
       "0ujw0gak               Canadian Medical Association Journal   \n",
       "28vx9w58                                         One Health   \n",
       "\n",
       "          Microsoft Academic Paper ID WHO #Covidence  has_full_text  \\\n",
       "cord_uid                                                              \n",
       "4360s2yu                          NaN            NaN           True   \n",
       "66jumbir                          NaN            NaN           True   \n",
       "3wk36h9p                          NaN          #5716          False   \n",
       "0ujw0gak                 1.953688e+09          #4117          False   \n",
       "28vx9w58                          NaN            NaN           True   \n",
       "\n",
       "              full_text_file  \\\n",
       "cord_uid                       \n",
       "4360s2yu      custom_license   \n",
       "66jumbir      custom_license   \n",
       "3wk36h9p      custom_license   \n",
       "0ujw0gak                 NaN   \n",
       "28vx9w58  noncomm_use_subset   \n",
       "\n",
       "                                                        url  \n",
       "cord_uid                                                     \n",
       "4360s2yu    https://doi.org/10.1016/j.clinthera.2015.07.007  \n",
       "66jumbir       https://doi.org/10.1016/j.bbagen.2011.01.012  \n",
       "3wk36h9p      https://doi.org/10.1016/s0262-4079(20)30474-7  \n",
       "0ujw0gak               https://doi.org/10.1503/cmaj.1095846  \n",
       "28vx9w58  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "meta_df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9271638921658584"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(meta_df['doi'].unique()) / len(meta_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "31745"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id2meta = {row['sha']: row for idx, row in meta_df.iterrows() if row['sha']}\n",
    "len(id2meta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'10.1016/j.bbagen.2011.01.012'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id2meta[dummy_id]['doi']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load paper data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "subsets = ['biorxiv_medrxiv', 'comm_use_subset', 'custom_license', 'noncomm_use_subset']\n",
    "id2paper = {}\n",
    "\n",
    "has_doi = 0\n",
    "bib_count = 0  \n",
    "cits = [] # from_doi, to_doi, <section title>\n",
    "    \n",
    "for ss in subsets:\n",
    "    ss_dir = os.path.join(cord19_dir, ss)\n",
    "    \n",
    "    # iterate over files\n",
    "    for fn in os.listdir(ss_dir):\n",
    "        if not fn.endswith('.json'):\n",
    "            continue\n",
    "            \n",
    "        fp = os.path.join(ss_dir, fn)\n",
    "        with open(fp, 'r') as f:\n",
    "            paper = json.load(f)            \n",
    "            \n",
    "            if paper['paper_id'] not in id2meta:\n",
    "                continue\n",
    "            \n",
    "            meta = id2meta[paper['paper_id']]\n",
    "            \n",
    "            paper['_meta'] = dict(meta)\n",
    "            \n",
    "            id2paper[paper['paper_id']] = paper\n",
    "            \n",
    "            # has valid DOI\n",
    "            if isinstance(meta['doi'], str) and len(meta['doi']) > 10:\n",
    "                # iterate over body text            \n",
    "                for paragraph in paper['body_text']:\n",
    "                    # iterate over each citation marker\n",
    "                    for cit in paragraph['cite_spans']:\n",
    "                        # find corresponding bib entry\n",
    "                        if cit['ref_id'] in paper['bib_entries']:\n",
    "                            bib = paper['bib_entries'][cit['ref_id']]\n",
    "                            bib_count += 1\n",
    "\n",
    "                            # only use bib entries with DOI\n",
    "                            if 'DOI' in bib['other_ids']:\n",
    "                                has_doi += 1\n",
    "\n",
    "                                for out_doi in bib['other_ids']['DOI']:\n",
    "                                    cits.append((\n",
    "                                        meta['doi'],\n",
    "                                        out_doi,\n",
    "                                        paragraph['section']\n",
    "                                    ))\n",
    "        #break\n",
    "    #break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Viruses and asthma ☆'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id2paper[dummy_id]['metadata']['title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 65,309 from disk\n"
     ]
    }
   ],
   "source": [
    "# Load paper data from disk (scraped)\n",
    "if os.path.exists(os.path.join(scraper_dir, 'doi2paper.json')):\n",
    "    with open(os.path.join(scraper_dir, 'doi2paper.json'), 'r') as f:\n",
    "        doi2s2paper = json.load(f)\n",
    "\n",
    "    print(f'Loaded {len(doi2s2paper):,} from disk')\n",
    "else:\n",
    "    doi2s2paper = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.1142/S1793048008000708\n"
     ]
    }
   ],
   "source": [
    "for k in doi2s2paper:\n",
    "    print(k)\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 29891 from CORD-19\n"
     ]
    }
   ],
   "source": [
    "# Load paper data from CORD-19\n",
    "doi2paper = {id2meta[pid]['doi']: paper for pid, paper in id2paper.items() if pid in id2meta}\n",
    "\n",
    "print(f'Loaded {len(doi2paper)} from CORD-19')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merge CORD-19 + S2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Paper count: 30197\n",
      "DOI exists: 0.166464416275948 (total: 1971154; doi: 328127)\n",
      "Citation pairs: 328127\n"
     ]
    }
   ],
   "source": [
    "print(f'Paper count: {len(id2paper)}')\n",
    "print(f'DOI exists: {has_doi/bib_count} (total: {bib_count}; doi: {has_doi})')\n",
    "print(f'Citation pairs: {len(cits)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#cits_with_doi = [c for c in cits if c[0] in doi2paper and c[1] in doi2paper]\n",
    "cits_with_doi = [c for c in cits if (c[0] in doi2paper or c[0] in doi2s2paper) and (c[1] in doi2paper or c[1] in doi2s2paper)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Citations with DOI: 170454 (0.5194756908148369)\n"
     ]
    }
   ],
   "source": [
    "# CORD-19 only: Citations with DOI: 30655 (0.09342419246206499)\n",
    "# + S2: Citations with DOI: 170454 (0.5194756908148369)\n",
    "\n",
    "print(f'Citations with DOI: {len(cits_with_doi)} ({len(cits_with_doi)/len(cits)})')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Missing paper data, but DOI: 297472\n"
     ]
    }
   ],
   "source": [
    "missing_papers = [c[0] for c in cits if c[0] not in doi2paper]\n",
    "missing_papers += [c[1] for c in cits if c[1] not in doi2paper]\n",
    "\n",
    "print(f'Missing paper data, but DOI: {len(missing_papers)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique DOIs of missing papers: 140682\n"
     ]
    }
   ],
   "source": [
    "unique_missing_papers = set(missing_papers)\n",
    "\n",
    "print(f'Unique DOIs of missing papers: {len(unique_missing_papers)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "108001"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unique_cits = {(c[0], c[1]) for c in cits_with_doi}\n",
    "len(unique_cits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# section titles\n",
    "sect_titles_count = defaultdict(int)\n",
    "\n",
    "for from_doi, to_doi, sect_title in cits_with_doi:\n",
    "    for t in normalize_section(sect_title).split(' and '):\n",
    "        sect_titles_count[t] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('discussion', 28879),\n",
       " ('introduction', 27012),\n",
       " ('', 9706),\n",
       " ('conclusion', 1971),\n",
       " ('results', 1941),\n",
       " ('background', 1192),\n",
       " ('methods', 1192),\n",
       " ('materials', 894),\n",
       " ('virus', 452),\n",
       " ('future work', 332),\n",
       " ('autophagy', 291),\n",
       " ('epidemiology', 250),\n",
       " ('cells', 243),\n",
       " ('treatment', 216),\n",
       " ('pathogenesis', 210),\n",
       " ('the study', 186),\n",
       " ('figure 2.', 172),\n",
       " ('influenza virus', 172),\n",
       " ('statistical analysis', 172),\n",
       " ('author summary', 171),\n",
       " ('phylogenetic analysis', 148),\n",
       " ('government-published information (n = 33; 15%)', 141),\n",
       " ('perspectives', 141),\n",
       " ('rna virus', 132),\n",
       " ('innate immunity', 128),\n",
       " ('influenza', 127),\n",
       " ('structure', 125),\n",
       " ('apoptosis', 120),\n",
       " ('vaccines', 117),\n",
       " ('evolution', 111),\n",
       " ('trpml1-3 channels', 110),\n",
       " ('expression', 108),\n",
       " ('prevention', 108),\n",
       " ('macrophages', 107),\n",
       " ('classification', 106),\n",
       " ('plasmids', 105),\n",
       " ('quantification', 102),\n",
       " ('summary', 97),\n",
       " ('replication', 97),\n",
       " ('mda5', 96),\n",
       " ('clinical features', 91),\n",
       " ('disease', 91),\n",
       " ('sequencing', 91),\n",
       " ('antibodies', 90),\n",
       " ('viral infections', 89),\n",
       " ('effects of ar', 88),\n",
       " ('inflammation', 88),\n",
       " ('cell culture', 88),\n",
       " ('disease pathogenesis', 87),\n",
       " ('analysis', 87),\n",
       " ('disease important finding reference', 86),\n",
       " ('cell lines', 86),\n",
       " ('atg8/lc3 detection', 85),\n",
       " ('host immune suppression', 85),\n",
       " ('diagnosis', 85),\n",
       " ('identification', 84),\n",
       " ('limitations', 84),\n",
       " ('hcv', 83),\n",
       " ('chemistry', 82),\n",
       " ('transmission', 82),\n",
       " ('seasonal variations of rsv infections versus other viral infections', 81),\n",
       " ('demographic, clinical characteristics', 81),\n",
       " ('structural studies', 81),\n",
       " ('identified erentds', 80),\n",
       " ('dalis (dendritic cell aggresome-like induced structures):', 80),\n",
       " ('risk factors', 80),\n",
       " ('rna degradation', 79),\n",
       " ('pa-x', 79),\n",
       " ('data', 79),\n",
       " ('influenza a virus', 78),\n",
       " ('bacteria', 78),\n",
       " ('coronavirus', 78),\n",
       " ('trims', 77),\n",
       " ('oligosaccharide-binding specificity of hrl40', 77),\n",
       " ('importance', 77),\n",
       " ('retrovirus', 75),\n",
       " ('herpesvirus-triggered rna degradation: two proteins, one mechanism', 75),\n",
       " ('assembly', 74),\n",
       " ('dengue virus', 74),\n",
       " ('sample collection', 74),\n",
       " ('mortality in hematology patients', 73),\n",
       " ('evidence of crv-infection-related morbidity', 73),\n",
       " ('viral metagenomics', 73),\n",
       " ('implementation of core capacities', 72),\n",
       " ('viral replication', 72),\n",
       " ('interaction of catechins', 71),\n",
       " ('other virus', 71),\n",
       " ('data analysis', 71),\n",
       " ('inducers', 70),\n",
       " ('autophagy inhibitors', 70),\n",
       " ('adaptive immunity', 70),\n",
       " ('control', 70),\n",
       " ('antiviral activity', 67),\n",
       " ('herpes simplex virus', 66),\n",
       " ('trim5α', 66),\n",
       " ('diagnostics', 66),\n",
       " ('plos one', 65),\n",
       " ('mice', 65),\n",
       " ('their biological activities', 64),\n",
       " ('biology', 64),\n",
       " ('viral entry', 64),\n",
       " ('solid-phase synthesis of cyclohexapeptides using solution-phase cyclization',\n",
       "  63),\n",
       " ('ebola virus', 63),\n",
       " ('respiratory syncytial virus', 63),\n",
       " ('upr', 63),\n",
       " ('study design', 63),\n",
       " ('virology', 62),\n",
       " ('cancer-related proteins', 61),\n",
       " ('dengue virus (denv)', 61),\n",
       " ('its immunogenicity', 61),\n",
       " ('cancer', 61),\n",
       " ('phylogenetic analyses', 61),\n",
       " ('asthma', 61),\n",
       " ('literature review', 60),\n",
       " ('infections', 60),\n",
       " ('hbx', 59),\n",
       " ('evs', 59),\n",
       " ('flavonoids against non-picornavirus', 58),\n",
       " ('dovepress', 58),\n",
       " ('relevant domains of cov e proteins', 58),\n",
       " ('sequence analysis', 58),\n",
       " ('(which was not peer-reviewed)', 58),\n",
       " ('commercial/industry (n = 131; 62%)', 57),\n",
       " ('bacterial treatment: quorum sensing', 57),\n",
       " ('gp', 57),\n",
       " ('therapeutics', 57),\n",
       " ('for antibody production', 56),\n",
       " ('the yeast saccharomyces cerevisiae: a powerful system for identifying cellular targets of viral proteins, for reconstituting viral machineries, for drug screening',\n",
       "  56),\n",
       " ('ros in respiratory virus pathology', 56),\n",
       " ('hepatitis c virus (hcv)', 56),\n",
       " ('viral infection', 56),\n",
       " ('transcription', 56),\n",
       " ('neutrophils', 55),\n",
       " ('vaccination', 55),\n",
       " ('glutamic acid', 54),\n",
       " ('denv', 54),\n",
       " ('positive-sense single strand rna ((+)ssrna) virus', 54),\n",
       " ('generation of reporter-expressing, replicating-competent iav containing a recombinant ns segment harboring the foreign gene',\n",
       "  54),\n",
       " ('bmj global health', 53),\n",
       " ('high-risk human papillomavirus (hpv)', 53),\n",
       " ('data collection', 53),\n",
       " ('design', 53),\n",
       " ('interferon signaling', 52),\n",
       " ('telomere dynamics', 52),\n",
       " ('management', 52),\n",
       " ('lifeguard family: an evolutionary perspective', 52),\n",
       " ('gaaps within the tmbim', 52),\n",
       " ('outlook', 52),\n",
       " ('animal models', 52),\n",
       " ('mers-cov', 52),\n",
       " ('host shutoff as a mechanism of immune evasion', 51),\n",
       " ('respiratory virus', 51),\n",
       " ('dna virus', 51),\n",
       " ('extracellular matrix', 51),\n",
       " ('cellular', 51),\n",
       " ('box 2. lessons in zoonosis from the hiv pandemic', 50),\n",
       " ('general features of (+)ssrna virus nucleocapsid assembly', 50),\n",
       " ('clinical manifestations', 50),\n",
       " ('nipah virus', 50),\n",
       " ('figure 1.', 50),\n",
       " ('losses.', 49),\n",
       " ('loss events. (i) reconstruction of gene gains', 49),\n",
       " ('reconstruction of gene gain', 49),\n",
       " ('evolution of the 5 major branches of rna virus', 49),\n",
       " ('the role of interferons in coronavirus pathogenesis', 49),\n",
       " ('•', 49),\n",
       " ('blood', 49),\n",
       " ('diseases', 49),\n",
       " ('challenges', 49),\n",
       " ('snps in host genes affecting iav disease', 48),\n",
       " ('enterocytes', 48),\n",
       " ('effects on gut microbiota', 48),\n",
       " ('glial scar', 48),\n",
       " ('ceacam1: implication in non-alcoholic fatty liver disease', 48),\n",
       " ('zika virus', 48),\n",
       " ('its role in limiting cross-species transmission', 48),\n",
       " ('tetherin counteraction by primate lentivirus', 48),\n",
       " ('viral', 48),\n",
       " ('detection', 48),\n",
       " ('bm2', 47),\n",
       " ('polymeric nanoparticles', 47),\n",
       " ('history', 47),\n",
       " ('bacterial infections', 47),\n",
       " ('description', 47),\n",
       " ('www.frontiersin.org', 47),\n",
       " ('m iddle east respiratory syndrome coronavirus (mers-', 46),\n",
       " ('specific', 46),\n",
       " ('heat-clearing drugs', 46),\n",
       " ('pure compounds isolated from tcm with anti-hsv-1 activities', 46),\n",
       " ('respiratory syncytial virus (rsv)', 46),\n",
       " ('salmonellosis', 46),\n",
       " ('immunity', 46),\n",
       " ('viral countermeasures', 46),\n",
       " ('oxygen perturbation of proper lung development', 46),\n",
       " ('ethics statement', 46),\n",
       " ('characteristics of the pericollapse stage of osteonecrosis of the femoral head',\n",
       "  45),\n",
       " ('glycan histo-blood groups', 45),\n",
       " ('ns1', 45),\n",
       " ('sars-cov viroporins in replication', 45),\n",
       " ('diffusion', 45),\n",
       " ('human', 45),\n",
       " ('nanoparticles', 45),\n",
       " ('clinical trials', 45),\n",
       " ('purification', 45),\n",
       " ('animals', 45),\n",
       " ('vaccine development', 44),\n",
       " ('the hiv life cycle', 44),\n",
       " ('oxidative stress', 44),\n",
       " ('their impact on human health', 44),\n",
       " ('type i ifn', 44),\n",
       " ('figure 1 | (a)', 44),\n",
       " ('ion channels formed by viroporins', 44),\n",
       " ('infectious diseases', 44),\n",
       " ('bioinformatics', 44),\n",
       " ('other proposals', 43),\n",
       " ('specific rna features recognized by rig-i', 43),\n",
       " ('autoinflammatory diseases', 43),\n",
       " ('implications for autoimmunity', 43),\n",
       " ('small rna', 43),\n",
       " ('ire1', 43),\n",
       " ('microglia', 43),\n",
       " ('flow cytometry', 43),\n",
       " ('reagents', 43),\n",
       " ('other flavivirus elicit cross-reactive t-cell responses', 42),\n",
       " ('morphology', 42),\n",
       " ('adjuvants shaping dc functions', 42),\n",
       " ('zika virus (zikv)', 42),\n",
       " ('antigen presentation', 42),\n",
       " ('viral pathogenesis', 42),\n",
       " ('pteropodidae', 42),\n",
       " ('ribavirin', 42),\n",
       " ('clinical presentation', 42),\n",
       " ('discussions', 42),\n",
       " ('infection', 42),\n",
       " ('conjunctival swabs', 41),\n",
       " ('direct naats for oral, dermal,', 41),\n",
       " ('other ligands reported in the literature as g4 fluorescence turn-on ligands',\n",
       "  41),\n",
       " ('gc', 41),\n",
       " ('widespread rna degradation by pa-x', 41),\n",
       " ('genome organization', 41),\n",
       " ('receptor-mediated endocytosis', 40),\n",
       " ('contents lists available at sciencedirect', 40),\n",
       " ('human coronavirus in the cns: possible associated neurological pathologies',\n",
       "  40),\n",
       " ('host factors in mers-cov transmission', 40),\n",
       " ('antivirals against arenavirus infections', 40),\n",
       " ('development of vaccines for', 40),\n",
       " ('mechanisms involved in immune evasion innate immune response', 40),\n",
       " ('immunology of prrsv', 40),\n",
       " ('common characteristics of fidelity mutants', 40),\n",
       " ('inflammatory responses during virus infection', 40),\n",
       " ('virus-induced serine monophosphorylation of stats', 40),\n",
       " ('gene functions', 40),\n",
       " ('genome structure', 40),\n",
       " ('mouse models', 40),\n",
       " ('type i', 40),\n",
       " ('virus.', 40),\n",
       " ('strengths', 40),\n",
       " ('comment', 39),\n",
       " ('atg1/ulk1', 39),\n",
       " ('tor/mtor, ampk', 39),\n",
       " ('interference with rlr-dependent signalling', 39),\n",
       " ('feline bocaparvovirus', 39),\n",
       " ('middle east respiratory syndrome coronavirus (mers-cov)', 39),\n",
       " ('the isg response', 39),\n",
       " ('asia', 39),\n",
       " ('molecular mechanisms in apoptosis', 39),\n",
       " ('adolescents', 39),\n",
       " ('studies conducted in children', 39),\n",
       " ('exosomes', 39),\n",
       " ('advantages', 39),\n",
       " ('isolation', 39),\n",
       " ('astrocytes', 39),\n",
       " ('rna extraction', 39),\n",
       " ('in vivo', 39),\n",
       " ('dendritic cells', 39),\n",
       " ('improvements in traditional culturing techniques', 38),\n",
       " ('the na/k-atpase oxidant amplification loop', 38),\n",
       " ('non-specific inhibition of epac1 by ce3f4r', 38),\n",
       " ('protease engineering', 38),\n",
       " ('middle east respiratory syndrome', 38),\n",
       " ('ligands functions', 38),\n",
       " ('dioscin', 38),\n",
       " ('quorum sensing interfering agents: an alternative to antibiotics for controlling bacterial infections',\n",
       "  38),\n",
       " ('protect globally-think locally! trm for influenza infection', 38),\n",
       " ('viral co-infections', 38),\n",
       " ('specific ibs', 38),\n",
       " ('rsv takes advantage of cytoplasmic sgs', 38),\n",
       " ('results: patients characteristics', 37),\n",
       " ('role of asymptomatic individuals in mers-cov infection', 37),\n",
       " ('neurotropic properties: possible associated neuropathologies', 37),\n",
       " ('respiratory virus with neuroinvasive', 37),\n",
       " ('covalently linked or fusion rnases hetero-oligomers', 37),\n",
       " ('etiology', 37),\n",
       " ('gene name abbreviation enzyme principal functions', 37),\n",
       " ('mda5 functions', 37),\n",
       " ('geographical distribution of bat covs', 37),\n",
       " ('bat-human interactions: interspecies jumping of bat covs', 37),\n",
       " ('bat-animal', 37),\n",
       " ('pathogenesis of influenza d virus', 37),\n",
       " ('origin, epidemiology', 37),\n",
       " ('vrna packaging', 37),\n",
       " ('figure 5)', 37),\n",
       " ('subversion by virus-encoded homologs of cell cycle regulators (table 5',\n",
       "  37),\n",
       " ('biological activity of thiopyrano[2,3-d]thiazole derivatives', 37),\n",
       " ('canonical translation of eukaryotic mrnas', 37),\n",
       " ('post-translational modification processes', 37),\n",
       " ('motif involved in immune response', 37),\n",
       " ('disclaimer', 37),\n",
       " ('hcv p7 topology', 37),\n",
       " ('ace2', 37),\n",
       " ('functions of idps/idprs', 36),\n",
       " ('trpml1', 36),\n",
       " ('alternatives to in vitro plant culture', 36),\n",
       " ('sulfated polysaccharides', 36),\n",
       " ('pteropid bats', 36),\n",
       " ('stability', 36),\n",
       " ('circular rna (circrna)', 36),\n",
       " ('under-appreciated components of bacilli secretome', 36),\n",
       " ('modifications at position 34', 36),\n",
       " ('multiplex detection', 36),\n",
       " ('a non-canonical unfolded-protein response (upr)', 36),\n",
       " ('rsv induces er stress', 36),\n",
       " ('flaviviridae', 36),\n",
       " ('mns', 36),\n",
       " ('zfns, talens,', 36),\n",
       " ('neuroprotection', 35),\n",
       " ('the role of siglec-1 in dc infection', 35),\n",
       " ('sputum', 35),\n",
       " ('direct naats for saliva', 35),\n",
       " ('effusion', 35),\n",
       " ('acid activation mechanism of am2', 35),\n",
       " ('neutrophil extracellular trap formation (netosis)', 35),\n",
       " ('host factors in mers-cov pathogenesis', 35),\n",
       " ('host-virus interaction', 35),\n",
       " ('occurrence', 35),\n",
       " ('sepsis', 35),\n",
       " ('immune response', 35),\n",
       " ('25-hc inhibits a variety of virus by modulating sterol synthesis', 35),\n",
       " ('its influence on protein localization, trafficking,', 35),\n",
       " ('s-palmitoylation of proteins', 35),\n",
       " ('phylogeny', 35),\n",
       " ('dc-sign', 35),\n",
       " ('influenza reverse genetics', 35),\n",
       " ('lactoferrin', 35),\n",
       " ('complementary findings', 35),\n",
       " ('hepatitis b virus (hbv)', 35),\n",
       " ('the ddr', 35),\n",
       " ('rsv', 35),\n",
       " ('data processing', 35),\n",
       " ('material', 35),\n",
       " ('severe acute respiratory syndrome-associated coronavirus', 34),\n",
       " ('lc3-positive compartments', 34),\n",
       " ('involvement of autophagosomes', 34),\n",
       " ('cell death in spinal cord injury', 34),\n",
       " ('human rsv mouse model', 34),\n",
       " ('accessory proteins', 34),\n",
       " ('cap-independent translation elements', 34),\n",
       " ('genotyping', 34),\n",
       " ('budding', 34),\n",
       " ('datasets', 34),\n",
       " ('model', 34),\n",
       " ('spike protein', 33),\n",
       " ('clinical setting', 33),\n",
       " ('mda5 by post-translational modifications', 33),\n",
       " ('activation of rig-i', 33),\n",
       " ('iii interferon responses', 33),\n",
       " ('herpesvirus', 33),\n",
       " ('recent advances in the development of vaccines based on the mers-cov s-protein rbd',\n",
       "  33),\n",
       " ('extraction', 33),\n",
       " ('metagenomics', 33),\n",
       " ('cd8 t cell responses to acute respiratory virus infections', 33),\n",
       " ('magnetic fields', 33),\n",
       " ('electric', 33),\n",
       " ('trim25 regulates intracellular signaling', 33),\n",
       " ('top-appreciated components of bacilli secretome', 33),\n",
       " ('er invaginations', 33),\n",
       " ('anti-inflammatory activity', 33),\n",
       " ('measures', 33),\n",
       " ('sequence', 33),\n",
       " ('the coronavirus envelope protein regulates er stress', 33),\n",
       " ('pcr', 33),\n",
       " ('substrate recognition for erad', 33),\n",
       " ('viral evasion', 33),\n",
       " ('degradation of viral rna by rnasel', 33),\n",
       " ('processing', 33),\n",
       " ('review of the literature', 32),\n",
       " ('bat-guano crazy about sampling', 32),\n",
       " ('step 3: lamp modes of output', 32),\n",
       " ('rig-i/mda5-like proteins', 32),\n",
       " ('speciation of rig-i/mda5', 32),\n",
       " ('interferon-stimulated genes (isgs)', 32),\n",
       " ('the physiological role of tpcs: local vs. global ca 2+ signals', 32),\n",
       " ('zika virus background', 32),\n",
       " ('•−', 32),\n",
       " ('utilization of 5-arylidene-4-thiazolidinethiones in the hetero-diels-alder reactions',\n",
       "  32),\n",
       " ('basics of t cell responses during infection', 32),\n",
       " ('malaria', 32),\n",
       " ('mechanisms of cytokine regulation by er stress', 32),\n",
       " ('antifibrotic effects', 32),\n",
       " ('many different infections have been observed at the outset of me/cfs like disease',\n",
       "  32),\n",
       " ('geographic distribution', 32),\n",
       " ('is emergence predictable?', 32),\n",
       " ('disadvantages', 32),\n",
       " ('er-associated protein degradation', 32),\n",
       " ('er exvaginations: single-and double-membrane vesicles (dmvs)', 32),\n",
       " ('miscellaneous', 32),\n",
       " ('dna vaccines', 32),\n",
       " ('mycobacterial ifitms provide antiviral activity when expressed in human cells',\n",
       "  32),\n",
       " ('fungal infections', 32),\n",
       " ('life cycle of hepatotropic virus', 31),\n",
       " ('detection of fcov mutations', 31),\n",
       " ('the role of interferons in flavivirus pathogenesis', 31),\n",
       " ('host-derived fpr ligands', 31),\n",
       " ('pdcov infection induces cellular ultrastructural changes', 31),\n",
       " ('retroviridae', 31),\n",
       " ('opportunities', 31),\n",
       " ('replicating competent, reporter gene-expressing rzikvs', 31),\n",
       " ('polyphenols', 31),\n",
       " ('genomics', 31),\n",
       " ('models', 31),\n",
       " ('vp40', 31),\n",
       " ('the autophagy-related proteins', 31),\n",
       " ('role of ibalt in pulmonary immune responses to infection', 31),\n",
       " ('without mrna cleavage', 31),\n",
       " ('translational inhibition with', 31),\n",
       " ('gene expression', 31),\n",
       " ('diadenosine tetraphosphate', 31),\n",
       " ('influence of cd8 + tregs on monocytes/ macrophages', 31),\n",
       " ('uncoating', 31),\n",
       " ('serology', 31),\n",
       " ('hypertension', 31),\n",
       " ('statistical analyses', 31),\n",
       " ('data sources', 31),\n",
       " ('treatment-chikungunya arthritis/pain', 30),\n",
       " ('infections in transplant patients', 30),\n",
       " ('systemic infection', 30),\n",
       " ('astroglia protect the cns against invasive', 30),\n",
       " ('regulatory t cells in neuroinflammation', 30),\n",
       " ('hiv-1', 30),\n",
       " ('sirnas against respiratory virus', 30),\n",
       " ('mrna expression of therapeutic proteins in vivo', 30),\n",
       " ('bacterial infections in aecopd', 30),\n",
       " ('the secretory pathway in zika virus infection: er modifications', 30),\n",
       " ('other peptide agonists for fprs', 30),\n",
       " ('virulence factors description references', 30),\n",
       " ('np', 30),\n",
       " ('other agents', 30),\n",
       " ('orthopoxvirus (opv), herpesvirus', 30),\n",
       " ('beyond', 30),\n",
       " ('pattern recognition receptors', 30),\n",
       " ('lncrna', 30),\n",
       " ('modifications at position 37', 30),\n",
       " ('vaccines against cancer', 30),\n",
       " ('toxicity', 30),\n",
       " ('multiple modes of action for ribavirin', 30),\n",
       " ('anticancer activity', 30),\n",
       " ('biosensors', 30),\n",
       " ('others', 30),\n",
       " ('study population', 30),\n",
       " ('experimental design', 30),\n",
       " ('western blot analysis', 30),\n",
       " ('establishment of the infected cell state', 29),\n",
       " ('tpc2', 29),\n",
       " ('11-16', 29),\n",
       " ('carbodiimide chemistry', 29),\n",
       " ('ge-gen-tang (ggt; kakkon-to in japan; galgeun-tang in korea; gegen decoction in china)',\n",
       "  29),\n",
       " ('teratogenic potential', 29),\n",
       " ('sbv display placental crossing abilities', 29),\n",
       " ('btv', 29),\n",
       " ('potential antibody dependent enhancement (ade) of mers-cov infection', 29),\n",
       " ('rate of proton conductance in am2', 29),\n",
       " ('proviral role of autophagy', 29),\n",
       " ('mda5 functions by their post-translational modifications and/or by other viral or cellular proteins',\n",
       "  29),\n",
       " ('modulations of rig-i', 29),\n",
       " ('t-cell response after yfv vaccination', 29),\n",
       " ('b lymphocytes', 29),\n",
       " ('t', 29),\n",
       " ('aptamers as therapeutics', 29),\n",
       " ('serum', 29),\n",
       " ('artesunate', 29),\n",
       " ('pedv', 29),\n",
       " ('aptamer selection', 29),\n",
       " ('microbeads for molecular recognition analysis', 29),\n",
       " ('uses of epcr', 29),\n",
       " ('the biogenesis of autophagic vacuoles', 29),\n",
       " ('molecular organization of porcine circovirus', 29),\n",
       " ('human respiratory syncytial virus (hrsv)', 29),\n",
       " ('tbk1', 29),\n",
       " ('viral impact on the nucleolus', 29),\n",
       " ('t-cell exhaustion during chronic/latent viral infections', 29),\n",
       " ('replicative senescence in chronic viral infection', 29),\n",
       " ('hbv', 29),\n",
       " ('envelope (e) protein', 29),\n",
       " ('scorpion venoms', 29),\n",
       " ('japanese encephalitis virus (jev)', 29),\n",
       " ('viral replication in the mammalian host', 29),\n",
       " ('methods of studying the hcv membranous web', 29),\n",
       " ('editorial', 29),\n",
       " ('the ridd pathway', 29),\n",
       " ('human immunodeficiency virus', 29),\n",
       " ('findings', 29),\n",
       " ('symptoms', 29),\n",
       " ('table 6', 28),\n",
       " ('mefloquine', 28),\n",
       " ('viral dissemination routes', 28),\n",
       " ('siglec-1 expression on different anatomical compartments', 28),\n",
       " ('camp-dependent regulation of epac function', 28),\n",
       " ('formation of subepithelial infiltrates', 28),\n",
       " ('autophagy in tumor suppression', 28),\n",
       " ('its connection with cellular lipids', 28),\n",
       " ('biology of zikv', 28),\n",
       " ('biological processes', 28),\n",
       " ('common drivers', 28),\n",
       " ('histological lesions', 28),\n",
       " ('gross', 28),\n",
       " ('non-structural proteins', 28),\n",
       " ('zikv replicons', 28),\n",
       " ('isgylated cellular proteins', 28),\n",
       " ('immune signaling are delicately balanced to protect fetal development', 28),\n",
       " ('hormonal regulation of pregnancy', 28),\n",
       " ('ceacam1 in metabolism: a role in promoting insulin clearance', 28),\n",
       " ('productive viral infection', 28),\n",
       " ('flavonoids', 28),\n",
       " ('antimicrobial phytochemicals in veterinary diseases', 28),\n",
       " ('t cell-mediated protection in human studies', 28),\n",
       " ('host factors', 28),\n",
       " ('sequence data', 28),\n",
       " ('flow visualization.', 28),\n",
       " ('regulation of the jak/stat pathway', 28),\n",
       " ('modification', 28),\n",
       " ('hiv', 28),\n",
       " ('arterivirus', 28),\n",
       " ('reporter (b-d) influenza a virus as described in', 28),\n",
       " ('figure 4. pa reporter influenza a virus: schematic representation of the pa segment from wt (a)',\n",
       "  28),\n",
       " ('initiation of viral rna synthesis', 28),\n",
       " ('the cytoskeleton', 28),\n",
       " ('rnase-l', 28),\n",
       " ('pi4p', 28),\n",
       " ('virus production', 28),\n",
       " ('plasmid construction', 28),\n",
       " ('origin', 28),\n",
       " ('patients', 28),\n",
       " ('virus strains', 28),\n",
       " ('eukaryotic virus', 28),\n",
       " ('challenges of erentds', 27),\n",
       " ('degradation pathway', 27),\n",
       " ('o. vacuole import', 27),\n",
       " ('ddx1', 27),\n",
       " ('metagenomic approach', 27),\n",
       " ('transgene containment technologies', 27),\n",
       " ('damps', 27),\n",
       " ('direct naats for plasma', 27),\n",
       " ('amphibian secretory rnases', 27),\n",
       " ('mammalian', 27),\n",
       " ('emerging high-risk virus', 27),\n",
       " ('cd8 t cell response to viral infections of the brain', 27),\n",
       " ('reactive species (rs)', 27),\n",
       " ('the relationship between g6pd', 27),\n",
       " ('antibodies or recombinant proteins targeting gp120', 27),\n",
       " ('viral small rna profiles', 27),\n",
       " ('t-cell response against ebov infection', 27),\n",
       " ('| d iscuss i on', 27),\n",
       " ('promote tumor vascularization', 27),\n",
       " ('2 support cancer cell migration', 27),\n",
       " ('two-pore channels 1', 27),\n",
       " ('human coronavirus origins', 27),\n",
       " ('vaccination during pregnancy', 27),\n",
       " ('humoral immune responses following infection', 27),\n",
       " ('translation of equine torovirus', 27),\n",
       " ('rainfall', 27),\n",
       " ('wide-field microscopy:', 27),\n",
       " ('antiviral defense', 27),\n",
       " ('palmitoylation of host proteins involved in antibacterial', 27),\n",
       " ('trim21', 27),\n",
       " ('replaced by triazole', 27),\n",
       " ('synthesis of dna aptamers', 27),\n",
       " ('h19', 27),\n",
       " ('genetics', 27),\n",
       " ('lung viral infections', 27),\n",
       " ('vaccines against infectious diseases', 27),\n",
       " ('influenza a (h1n1) viral-induced ards', 27),\n",
       " ('orf3-encoded protein vp13', 27),\n",
       " ('further notes in support of our hypothesis concerning zikv-mediated microcephaly',\n",
       "  27),\n",
       " ('generation of reporter-expressing iav containing a modified na segment',\n",
       "  27),\n",
       " ('thus inhibit hepatocyte interferon signaling via usp18 induction. these findings represent a new paradigm for how inflammation alters hepatic innate immune responses, with the induction of usp18 representing a potential target for intervention in various inflammatory states.',\n",
       "  27),\n",
       " ('spread. blunted immunity as a result of inappropriate innate inflammatory responses is a common characteristic of chronic viral infections. previous studies have shown that expression of certain interferon-stimulated genes is upregulated in chronic hcv infection, leading to impaired hepatocyte responses. in this study, we show that multiple inflammatory stimuli can modulate interferon stimulated gene expression',\n",
       "  27),\n",
       " ('inflammation may prevent the control of viral infection when it impairs the innate immune response, enhancing viral replication',\n",
       "  27),\n",
       " ('egress', 27),\n",
       " ('antimicrobial resistance', 27),\n",
       " ('function', 27),\n",
       " ('coronaviridae', 27),\n",
       " ('protein expression', 27),\n",
       " ('virus isolation', 27),\n",
       " ('viral manipulation of the immune response by erad', 27),\n",
       " ('signaling', 27),\n",
       " ('cells, virus,', 27),\n",
       " ('implications', 27),\n",
       " ('host-pathogen interactions', 26),\n",
       " ('price factors affecting international tourism demand', 26),\n",
       " ('pexophagy', 26),\n",
       " ('d. animal mitophagy', 26),\n",
       " ('synthesis of ghi studies', 26),\n",
       " ('antiviral agents used in nanotechnology', 26),\n",
       " ('innate immunity in iav infections', 26),\n",
       " ('tissue', 26),\n",
       " ('sequelae of subclinical in utero zika virus infection in offspring', 26),\n",
       " ('innate immune evasion', 26),\n",
       " ('phlorotannins', 26),\n",
       " ('autoimmunity', 26),\n",
       " ('unconventional prospects in porcine ifn biology', 26),\n",
       " ('conventional', 26),\n",
       " ('therapeutic potential', 26),\n",
       " ('other t-cell subsets in vhf', 26),\n",
       " ('regulatory', 26),\n",
       " ('macrocyclic inhibitors targeting norovirus 3cl pro', 26),\n",
       " ('rna g-quadruplexes', 26),\n",
       " ('membrane binding motifs', 26),\n",
       " ('vp40 oligomerisation', 26),\n",
       " ('fidelity mutant (in)stability', 26),\n",
       " ('current hypotheses for the maintenance of ebov', 26),\n",
       " ('protects against immune-mediated hepatitis in mice', 26),\n",
       " ('ceacam1 is required for treg induction in the liver', 26),\n",
       " ('human rhinovirus', 26),\n",
       " ('antioxidant therapy of respiratory virus', 26),\n",
       " ('| introduc ti on', 26),\n",
       " ('ddx5 facilitates human immunodeficiency virus 1 mrna export', 26),\n",
       " ('designer nucleases', 26),\n",
       " ('rhinovirus', 26),\n",
       " ('regulation by wd40-repeat proteins', 26),\n",
       " ('outbreak investigations', 26),\n",
       " ('methylphosphonate or phosphorothioate', 26),\n",
       " ('cyclic peptides', 26),\n",
       " ('safety', 26),\n",
       " ('bioinformatics analysis', 26),\n",
       " ('retroviral integration', 26),\n",
       " ('three classes of viral fusogens', 26),\n",
       " ('regulatory role for cd8 + t-cells in eae', 26),\n",
       " ('multiple sclerosis', 26),\n",
       " ('inclusion of symptomatic (urti) controls inclusion of outpatient (op) cases',\n",
       "  26),\n",
       " ('the dpp family in inflammatory disease', 26),\n",
       " ('chronic disease', 26),\n",
       " ('intracellular ang-(1-7)-mas receptor system', 26),\n",
       " ('hrv', 26),\n",
       " ('sars-cov', 26),\n",
       " ('human metapneumovirus', 26),\n",
       " ('class iii ptdins3k', 25),\n",
       " ('e. chaffeensis inclusion membranes are enriched with ptdins3p', 25),\n",
       " ('shotgun proteomics', 25),\n",
       " ('in vitro studies', 25),\n",
       " ('regulation of adaptive immune responses within the cns', 25),\n",
       " ('selective types of autophagy', 25),\n",
       " ('related lc3 binding protein turnover assays', 25),\n",
       " ('sqstm1', 25),\n",
       " ('. transmission electron microscopy', 25),\n",
       " ('jcpyv', 25),\n",
       " ('rnase a', 25),\n",
       " ('apobec3g', 25),\n",
       " ('p53 stabilization in cancer', 25),\n",
       " ('nucleolar stress remodelling', 25),\n",
       " ('nanodiamonds in infection diagnosis', 25),\n",
       " ('household id shared infection(s) cases in cluster (position in household a , age in years) interval between onset of cases (days)',\n",
       "  25),\n",
       " ('rna modifications affecting innate immune receptors other than tlrs', 25),\n",
       " ('lysosomes contribute to cancer hallmarks', 25),\n",
       " ('cell culture model systems', 25),\n",
       " ('bats', 25),\n",
       " ('ifitms restrict a broad spectrum of virus in cultured cells', 25),\n",
       " ('hiv-1 creates an unfavorable environment for cap-dependent initiation', 25),\n",
       " ('selective autophagy', 25),\n",
       " ('animal studies', 25),\n",
       " ('lipid metabolism modulators', 25),\n",
       " ('life cycle', 25),\n",
       " ('veterinary health impact', 25),\n",
       " ('risk of zoonotic bacterial infection', 25),\n",
       " ('enzymes.', 25),\n",
       " ('treatment of cells with chemicals', 25),\n",
       " ('structures', 25),\n",
       " ('cell type main reported sp-d-mediated effects reference', 25),\n",
       " ('tyrosine phosphorylation-independent non-canonical stat activation', 25),\n",
       " ('non-human primate models', 25),\n",
       " ('infectious bronchitis corona virus-avian macrophage interaction', 25),\n",
       " ('ifns in acute respiratory viral infection', 25),\n",
       " ('nuclear trafficking', 25),\n",
       " ('figure 2. pb2 reporter influenza a virus:', 25),\n",
       " ('hcv genome replication', 25),\n",
       " ('the membranous web', 25),\n",
       " ('perspective', 25),\n",
       " ('nucleic acid extraction', 25),\n",
       " ('samples', 25),\n",
       " ('factors associated with clinical signs of ili', 25),\n",
       " ('rna isolation', 25),\n",
       " ('frontiers in microbiology | virology', 25),\n",
       " ('evidence of upr in hepatitis c', 25),\n",
       " ('induction', 25),\n",
       " ('angiotensin-(1-12)', 25),\n",
       " ('testing techniques evolving while the mechanism of microcephaly uncertain',\n",
       "  24),\n",
       " ('sgta', 24),\n",
       " ('zoonotic potential of astrovirus', 24),\n",
       " ('genetic variability, interspecies transmission', 24),\n",
       " ('histopathologic features', 24),\n",
       " ('protease inhibitors as anti-hepatitis c viral agents in clinical trials',\n",
       "  24),\n",
       " ('nucleoside, non-nucleoside', 24),\n",
       " ('broad-spectrum molecules', 24),\n",
       " ('a. atg9', 24),\n",
       " ('44.', 24),\n",
       " ('coronavirus model of demyelination', 24),\n",
       " ('the spreading of global cpv strains', 24),\n",
       " ('human coronavirus: eclosions of recognized respiratory pathogens', 24),\n",
       " ('ho-1: implications for obesity', 24),\n",
       " ('molecular mechanisms for arenavirus antagonism of the ifn response', 24),\n",
       " ('direct naats for dried blood', 24),\n",
       " ('zika vaccine candidates in development', 24),\n",
       " ('factors influencing the differentiation of cd8 btrm', 24),\n",
       " ('biochemical support for exon-driven error correction', 24),\n",
       " ('myagropsis myagroides', 24),\n",
       " ('viral suppressors of rnai in mammalian virus', 24),\n",
       " ('(f ) genetic analyses', 24),\n",
       " ('immune-surveillance', 24),\n",
       " ('viral proteins interfere with innate immune responses', 24),\n",
       " ('expression of pd-l1 within the brain', 24),\n",
       " ('stimulation of viral rna accumulation by host mirnas', 24),\n",
       " ('the er', 24),\n",
       " ('packaging signals', 24),\n",
       " ('the mapks', 24),\n",
       " ('mers-cov s-protein rbd-targeting mabs', 24),\n",
       " ('flaviviridae virus', 24),\n",
       " ('electrochemical (ec) detection of avian influenza virus (aiv) using nanobio hybrid materials',\n",
       "  24),\n",
       " ('other small-molecule inhibitors with defined or undefined mechanisms of action',\n",
       "  24),\n",
       " ('mers-cov inhibitors targeting papain-like protease', 24),\n",
       " ('mers-cov entry inhibitors', 24),\n",
       " ('addressing barriers to gene transfer', 24),\n",
       " ('colonization', 24),\n",
       " ('proteins related to neurodegenerative diseases', 24),\n",
       " ('interaction between catechins', 24),\n",
       " ('ferruginol', 24),\n",
       " ('other vaccines', 24),\n",
       " ('effect on host immunity', 24),\n",
       " ('iav assembly', 24),\n",
       " ('viral dynamics', 24),\n",
       " ('campylobacteriosis', 24),\n",
       " ('structural', 24),\n",
       " ('trim-rancyp restriction assay', 24),\n",
       " ('ifns deplete polyamines as a strategy to check viral infection', 24),\n",
       " ('the click chemistry-based method of analysis of protein fatty acylation',\n",
       "  24),\n",
       " ('porcine models', 24),\n",
       " ('tlr signaling', 24),\n",
       " ('6k is a viroporin', 24),\n",
       " ('rooting trees of life (tols): outgroup vs. generality criterion', 24),\n",
       " ('cross-sectional studies', 24),\n",
       " ('systemic lipopolysaccharide (lps)-related inflammation', 24),\n",
       " ('gut', 24),\n",
       " ('induction of type i ifns', 24),\n",
       " ('unresolved issues in ards', 24),\n",
       " ('histopathologic findings', 24),\n",
       " ('nucleic acid-based rabies vaccines', 24),\n",
       " ('mode of action', 24),\n",
       " ('the lgi network of the e. coli fimh-ld adhesin. the fimh', 24),\n",
       " ('congenital epigenetic diseases', 24),\n",
       " ('glycoprotein maturation, viral assembly,', 24),\n",
       " ('rna processing', 24),\n",
       " ('viral replication-the molecular processes', 24),\n",
       " ('entry', 24),\n",
       " ('humanized glycosylation in plants for \"glycan-better\" products', 24),\n",
       " ('transmembrane domain', 24),\n",
       " ('metabolism', 24),\n",
       " ('dengue', 24),\n",
       " ('arginase activity can promote viral persistence and/or exacerbated immunopathology arginase activity can inhibit viral clearance',\n",
       "  24),\n",
       " ('the upr', 24),\n",
       " ('refinement', 24),\n",
       " ('post-translational modifications', 24),\n",
       " ('study area', 24),\n",
       " ('molecular dynamics simulations', 24),\n",
       " ('covid-19', 24),\n",
       " ('antibody technologies', 23),\n",
       " ('pipeline of mers-cov vaccine', 23),\n",
       " ('associations of common infectious agents with ms', 23),\n",
       " ('the composition of replication compartments', 23),\n",
       " ('function of amyloidogenic peptides in microbial organisms', 23),\n",
       " ('characteristics of the outbreaks', 23),\n",
       " ('immune activation triggers siglec-1 induction on dcs', 23),\n",
       " ('viral sensing', 23),\n",
       " ('fcov ii s protein', 23),\n",
       " ('structural differences between fcov i', 23),\n",
       " ('detection methods of o-acetylated neu5ac', 23),\n",
       " ('zikv replication inhibitors', 23),\n",
       " ('the role of interferons in alphavirus pathogenesis', 23),\n",
       " ('insulating foams', 23),\n",
       " ('fireproof', 23),\n",
       " ('the rdna chromatin compaction', 23),\n",
       " ('torc1, condensin, epigenetics', 23),\n",
       " ('redox activity of biochar-based feed additives', 23),\n",
       " ('alkaline proteases', 23),\n",
       " ('lipid binding', 23),\n",
       " ('neuroinflammation', 23),\n",
       " ('role of cspgs on endogenous cell response', 23),\n",
       " ('targets of zikv', 23),\n",
       " ('cellular entry', 23),\n",
       " ('ow arenavirus', 23),\n",
       " ('entry mechanisms by nw', 23),\n",
       " ('restricts viral fusion in a549 cells. (a)', 23),\n",
       " ('ifitm3 proximity to viral fusion proteins allows virus restriction. (a-d) ifitm3 incorporates into virions',\n",
       "  23),\n",
       " ('adenovirus', 23),\n",
       " ('autophagy as a therapeutic target for curing flavivirus infection', 23),\n",
       " ('models of human rsv-induced asthma development in immature mice', 23),\n",
       " ('mers-cov inhibitors targeting 3c-like protease', 23),\n",
       " ('tw strains on sgmrna synthesis', 23),\n",
       " ('evaluation of the effect of variations in trs between cv777', 23),\n",
       " ('isg15 definition', 23),\n",
       " ('cd4 t cells control homeostasis of brain-resident cd8 t cells during polyomavirus infection',\n",
       "  23),\n",
       " ('environmental sample collection', 23),\n",
       " ('influenza virus a (iav) infection', 23),\n",
       " ('antiretroviral therapy', 23),\n",
       " ('mechanism of cpp mediated uptake', 23),\n",
       " ('lung epithelium tolerance mechanisms', 23),\n",
       " ('oxidative damage', 23),\n",
       " ('transmission: known unknowns', 23),\n",
       " ('t-cell exhaustion during chronic/productive viral infections', 23),\n",
       " ('schistosomiasis', 23),\n",
       " ('propensity to form α-helix in incomplete polypeptide segments that adopt native β-strands',\n",
       "  23),\n",
       " ('how are edible vaccines developed?', 23),\n",
       " ('immunosuppression', 23),\n",
       " ('microbiome', 23),\n",
       " ('astrovirus detected in livestock', 23),\n",
       " ('signaling pathways', 23),\n",
       " ('ifn-λ expression', 23),\n",
       " ('production', 23),\n",
       " ('emerging virus', 23),\n",
       " ('receptors', 23),\n",
       " ('ibalt in chronic pulmonary diseases', 23),\n",
       " ('stat3', 23),\n",
       " ('education to action: improving public perception of bats', 23),\n",
       " ('rotavirus', 23),\n",
       " ('alpha herpesvirus strains expressing fluorescent proteins', 23),\n",
       " ('definitions', 23),\n",
       " ('different pathogen sensing by dc subsets in humans', 23),\n",
       " ('specialized dc subsets induce different classes of t-cell responses in mice',\n",
       "  23),\n",
       " ('environmental factors', 23),\n",
       " ('lipid metabolism', 23),\n",
       " ('relevant domains of rsv sh protein', 23),\n",
       " ('general features of the envelope (e) protein in covs', 23),\n",
       " ('respiratory syndrome virus', 23),\n",
       " ('outcomes', 23),\n",
       " ('bioinformatic analysis', 23),\n",
       " ('no production can be beneficial during virus infection', 23),\n",
       " ('asthma exacerbations: clinical findings', 23),\n",
       " ('structure determination', 23),\n",
       " ('n vpu', 23),\n",
       " ('hiv-1 m', 23),\n",
       " ('statistical methods', 23),\n",
       " ('the hepatitis c virus p7 protein (hcv-p7)', 22),\n",
       " ('etiology, diffusion', 22),\n",
       " ('crisis communication on social media', 22),\n",
       " ('analyses in vivo', 22),\n",
       " ('hybridization-based enrichment', 22),\n",
       " ('dna replication', 22),\n",
       " ('hydroxychloroquine', 22),\n",
       " ('chloroquine', 22),\n",
       " ('step 1: template preparation', 22),\n",
       " ('s. pneumoniae at the mucosal barrier', 22),\n",
       " ('targeting iav', 22),\n",
       " ('hypotheses', 22),\n",
       " ('conceptual model', 22),\n",
       " ('clinical manifestation', 22),\n",
       " ('non-specific effects of csf1r inhibition: a confounding factor?', 22),\n",
       " ('viral risks for humans who share ecosystems with nhp', 22),\n",
       " ('the size of the nucleolus', 22),\n",
       " ('torc1, stress', 22),\n",
       " ('advancement in mers-cov vaccine development', 22),\n",
       " ('(c) descriptive analyses', 22),\n",
       " ('after the intervention', 22),\n",
       " ('(b) questions during', 22),\n",
       " ('therapeutic small molecules', 22),\n",
       " ('declining lung function', 22),\n",
       " ('cystic fibrosis disease pathophysiology: mucus dehydration', 22),\n",
       " ('other rna helicases potentially involved in lasv replication', 22),\n",
       " ('ionic imbalance, excitotoxicity', 22),\n",
       " ('host cell restriction factors that limit zikv replication', 22),\n",
       " ('viral genome replication, gene expression,', 22),\n",
       " ('targ1-type proteins are scarce, representing just one percent of sequenced macrodomains.',\n",
       "  22),\n",
       " ('dna g-quadruplexes', 22),\n",
       " ('the differentiation of virus-specific tfh cells during chronic viral infection',\n",
       "  22),\n",
       " ('lysosomes: their emerging role in ca 2+ signaling', 22),\n",
       " ('the function of endosomes', 22),\n",
       " ('cap-and helicase-dependent scanning model', 22),\n",
       " ('remaining questions', 22),\n",
       " ('parainfluenza virus', 22),\n",
       " ('the role of lncrnas in gammaherpesvirus', 22),\n",
       " ('human influenza virus (ivs) are members of the family', 22),\n",
       " ('route of vaccination', 22),\n",
       " ('spirit calming drugs', 22),\n",
       " ('ginsenoside rh2', 22),\n",
       " ('extracellular barriers', 22),\n",
       " ('hct', 22),\n",
       " ('community acquired respiratory virus', 22),\n",
       " ('in liver regeneration', 22),\n",
       " ('ceacam1 in hepatocyte adhesion, differentiation,', 22),\n",
       " ('west nile virus (wnv)', 22),\n",
       " ('ethnicity', 22),\n",
       " ('universal coverage: epitopes, hla,', 22),\n",
       " ('polymicrobial lung infections', 22),\n",
       " ('a b s t r a c t', 22),\n",
       " ('pupillary light reflex', 22),\n",
       " ('csf in me/cfs', 22),\n",
       " ('cytokine patterns in blood', 22),\n",
       " ('association with all-cause mortality', 22),\n",
       " ('virus in tombusviridae', 22),\n",
       " ('hiv-1 genome', 22),\n",
       " (\"tetherin's role in linking innate\", 22),\n",
       " ('ap33', 22),\n",
       " ('validation of high-throughput analysis using hcv1', 22),\n",
       " ('lassa virus', 22),\n",
       " ('mva as a vaccine vector', 22),\n",
       " ('hsv-1 replication was inhibited by the rnase activity of ire1α', 22),\n",
       " ('bacillus', 22),\n",
       " ('beneficial effects of lactobacillus', 22),\n",
       " ('non-structural protein 1 (ns1)', 22),\n",
       " ('non-human mastv', 22),\n",
       " ('virus-encoded lncrna', 22),\n",
       " ('emotional components', 22),\n",
       " ('outcome', 22),\n",
       " ('coronavirus infections', 22),\n",
       " ('virus egress from the cell', 22),\n",
       " ('its consequences', 22),\n",
       " ('genetic determinants of hemagglutinin ph of activation', 22),\n",
       " ('b. functional, structural', 22),\n",
       " ('ifn transcription', 22),\n",
       " ('activation of transcription factors', 22),\n",
       " ('c', 22),\n",
       " ('roles of erad in inhibition of virus replication', 22),\n",
       " ('validation', 22),\n",
       " ('viral etiologies of study population', 22),\n",
       " ('demographic, clinical features', 22),\n",
       " ('luciferase reporter genes', 22),\n",
       " ('comparison of fluorescent', 22),\n",
       " ('toward a model of ibalt development', 22),\n",
       " ('reactive oxygen species (ros) during rsv infection: too much of a good thing',\n",
       "  22),\n",
       " ('kidney protection', 22),\n",
       " ('the role of dpp-4 inhibitors in diabetes', 22),\n",
       " ('astrocyte as a possible therapeutic target', 22),\n",
       " ('cytoplasmic domain', 22),\n",
       " ('the orchestration of herpesvirus membrane fusion', 22),\n",
       " ('mers', 22),\n",
       " ('dna extraction', 22),\n",
       " ('jc virus agnoprotein', 22),\n",
       " ('endocytic pathway', 22),\n",
       " ('hcv p7 channel as antiviral drug target', 22),\n",
       " ('3', 22),\n",
       " ('mycobacterial ifitms partially localize to cathepsin b-positive cellular compartments',\n",
       "  22),\n",
       " ('retroviral', 22),\n",
       " ('er stress', 22),\n",
       " ('glial reactions during cuprizone-induced de-and remyelination the fate of oligodendrocytes',\n",
       "  22),\n",
       " ('tuberculosis', 22),\n",
       " ('development', 22),\n",
       " ('sensitivity analysis', 22),\n",
       " ('protein-protein interactions', 22),\n",
       " ('mechanisms of immune evasion used by intracellular pathogens against macrophage responses',\n",
       "  21),\n",
       " ('| depleting microglia in diseases: the friend in need may not always be desirable indeed',\n",
       "  21),\n",
       " ('formulation of mathematical model', 21),\n",
       " ('plasmid-encoded cytokines', 21),\n",
       " ('hazards related to the recombinant mv vectors', 21),\n",
       " ('innate immune responses', 21),\n",
       " ('structure of pp1 holoenzymes', 21),\n",
       " ('application of the one health initiative to erentds', 21),\n",
       " ('immunogenicity of glial allografts', 21),\n",
       " ('probable mechanisms of antiviral action elucidated', 21),\n",
       " ('viral pneumonia', 21),\n",
       " ('border screening', 21),\n",
       " ('pathophysiologic rationale for udca therapy', 21),\n",
       " ('genetic factors/susceptibility', 21),\n",
       " ('close contact behavior', 21),\n",
       " ('modulation of antiviral responses', 21),\n",
       " ('multivalent', 21),\n",
       " ('somamers', 21),\n",
       " ('modifications on the bases', 21),\n",
       " ('madin-darby canine kidney (mdck) cells', 21),\n",
       " ('mers-cov virology', 21),\n",
       " ('bs-rnase', 21),\n",
       " ('rational design of lav candidates using reverse genetics technology', 21),\n",
       " ('feline kobuvirus', 21),\n",
       " ('lectins', 21),\n",
       " ('the generation of homologous intertypic recombinant enterovirus, a multi-step process',\n",
       "  21),\n",
       " ('corneal keratocyte responses', 21),\n",
       " ('the human holobiont', 21),\n",
       " ('n7-methyltransferase activities', 21),\n",
       " ('coronavirus nsp14 harbors exoribonuclease', 21),\n",
       " ('aslv subgroup a to e envelope glycoproteins', 21),\n",
       " ('favipiravir', 21),\n",
       " ('the sirna pathway in insects', 21),\n",
       " ('australia', 21),\n",
       " ('(d) quantifying transmissibility', 21),\n",
       " ('orthomyxoviridae', 21),\n",
       " ('env antagonists in the membrane of target cells: the other half of the fusion story',\n",
       "  21),\n",
       " ('er-associated degradation: traps at its place of birth', 21),\n",
       " ('isoforms, biological expression', 21),\n",
       " ('gene alleles', 21),\n",
       " ('pathways of type i ifn activation in different cell types', 21),\n",
       " ('its main components on nervous system diseases', 21),\n",
       " ('role of dead-box rna helicase 3 (ddx3) in viral replication', 21),\n",
       " ('structure-guided optimization of dipeptidyl inhibitor lead series', 21),\n",
       " ('importance of rna modifications for immune regulation in vivo', 21),\n",
       " ('trafficking', 21),\n",
       " ...]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import operator\n",
    "top_sect_titles = sorted(sect_titles_count.items(), key=operator.itemgetter(1))\n",
    "top_sect_titles.reverse()\n",
    "top_sect_titles\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# normalize section title\n",
    "def normalize_section(title):\n",
    "    return title.strip().lower()\\\n",
    "        .replace('conclusions', 'conclusion')\\\n",
    "        .replace('concluding remarks', 'conclusion')\\\n",
    "        .replace('future perspectives', 'future work')\\\n",
    "        .replace('future directions', 'future work')\\\n",
    "        .replace('viruses.', 'virus')\\\n",
    "        .replace('viruses', 'virus')            \n",
    "        #.replace('conclusion and future perspectives', 'conclusion')\\\n",
    "        #.replace('materials and methods', 'methods')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# resolve 'and' titles\n",
    "def resolve_and_sect_titles(cits):\n",
    "    for from_doi, to_doi, sect_title in cits:\n",
    "        for t in normalize_section(sect_title).split(' and '):\n",
    "            yield (from_doi, to_doi, t)\n",
    "\n",
    "normalized_cits_with_doi = resolve_and_sect_titles(cits_with_doi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('10.1101/001727', '10.1038/nature11711', 'introduction'),\n",
       " ('10.1101/003889',\n",
       "  '10.1111/irv.12226',\n",
       "  'validity of networked metapopulation'),\n",
       " ('10.1101/006866', '10.4049/jimmunol.1000445', 'introduction')]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(resolve_and_sect_titles(cits_with_doi[:3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>from_doi</th>\n",
       "      <th>to_doi</th>\n",
       "      <th>citing_section</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1101/001727</td>\n",
       "      <td>10.1038/nature11711</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.1101/003889</td>\n",
       "      <td>10.1111/irv.12226</td>\n",
       "      <td>validity of networked metapopulation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.1101/006866</td>\n",
       "      <td>10.4049/jimmunol.1000445</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1101/006866</td>\n",
       "      <td>10.1073/pnas.1107498108</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.1101/006866</td>\n",
       "      <td>10.4049/jimmunol.1102097</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199767</th>\n",
       "      <td>10.1093/jtm/taaa021</td>\n",
       "      <td>10.2139/ssrn.3525558</td>\n",
       "      <td>results</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199768</th>\n",
       "      <td>10.3348/kjr.2020.0096</td>\n",
       "      <td>10.3348/kjr.2020.0078</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199769</th>\n",
       "      <td>10.2478/jccm-2020-0013</td>\n",
       "      <td>10.1016/s0140-6736(20)30183-5</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199770</th>\n",
       "      <td>10.2478/jccm-2020-0013</td>\n",
       "      <td>10.1016/s0140-6736(20)30183-5</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199771</th>\n",
       "      <td>10.2478/jccm-2020-0013</td>\n",
       "      <td>10.1016/s0140-6736(20)30183-5</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>199772 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      from_doi                         to_doi  \\\n",
       "0               10.1101/001727            10.1038/nature11711   \n",
       "1               10.1101/003889              10.1111/irv.12226   \n",
       "2               10.1101/006866       10.4049/jimmunol.1000445   \n",
       "3               10.1101/006866        10.1073/pnas.1107498108   \n",
       "4               10.1101/006866       10.4049/jimmunol.1102097   \n",
       "...                        ...                            ...   \n",
       "199767     10.1093/jtm/taaa021           10.2139/ssrn.3525558   \n",
       "199768   10.3348/kjr.2020.0096          10.3348/kjr.2020.0078   \n",
       "199769  10.2478/jccm-2020-0013  10.1016/s0140-6736(20)30183-5   \n",
       "199770  10.2478/jccm-2020-0013  10.1016/s0140-6736(20)30183-5   \n",
       "199771  10.2478/jccm-2020-0013  10.1016/s0140-6736(20)30183-5   \n",
       "\n",
       "                              citing_section  \n",
       "0                               introduction  \n",
       "1       validity of networked metapopulation  \n",
       "2                               introduction  \n",
       "3                               introduction  \n",
       "4                               introduction  \n",
       "...                                      ...  \n",
       "199767                               results  \n",
       "199768                                        \n",
       "199769                                        \n",
       "199770                                        \n",
       "199771                                        \n",
       "\n",
       "[199772 rows x 3 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cits_df = pd.DataFrame(normalized_cits_with_doi, columns=['from_doi', 'to_doi', 'citing_section'])\n",
    "cits_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After normalization: 199,772 (before: 170,454)\n"
     ]
    }
   ],
   "source": [
    "print(f'After normalization: {len(cits_df):,} (before: {len(cits_with_doi):,})')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "cits_df['citing_section'] = [normalize_section(t) for t in cits_df['citing_section'].values]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "discussion      28879\n",
       "introduction    27012\n",
       "                 9706\n",
       "conclusion       1971\n",
       "results          1941\n",
       "methods          1192\n",
       "background       1192\n",
       "materials         894\n",
       "virus             452\n",
       "future work       332\n",
       "Name: citing_section, dtype: int64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_sections = 10\n",
    "\n",
    "cits_df['citing_section'].value_counts()[:top_sections]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['discussion',\n",
       " 'introduction',\n",
       " 'conclusion',\n",
       " 'results',\n",
       " 'methods',\n",
       " 'background',\n",
       " 'materials',\n",
       " 'virus',\n",
       " 'future work']"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels = list(filter(lambda t: t, cits_df['citing_section'].value_counts()[:top_sections].keys()))\n",
    "labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def to_label(t, labels):\n",
    "    t = normalize_section(t)\n",
    "    \n",
    "    if t in labels:\n",
    "        return t\n",
    "    else:\n",
    "        return 'other'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "label_col = 'label'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "122598"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cits_df[label_col] = [to_label(t, labels) for t in cits_df['citing_section']]\n",
    "cits_df.drop_duplicates(['from_doi', 'to_doi', 'label'], keep='first', inplace=True)\n",
    "len(cits_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>from_doi</th>\n",
       "      <th>to_doi</th>\n",
       "      <th>label_count</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1002/1873-3468.13478</td>\n",
       "      <td>10.1128/JVI.01240-17</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.1002/ame2.12017</td>\n",
       "      <td>10.3201/eid2206.152113</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.1002/bdm.2056</td>\n",
       "      <td>10.3102/10769986031004437</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1002/cbf.3182</td>\n",
       "      <td>10.1182/blood-2011-01-329060</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.1002/cbf.3182</td>\n",
       "      <td>10.1186/s12977-015-0178-0</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107996</th>\n",
       "      <td>10.9745/ghsp-d-19-00188</td>\n",
       "      <td>10.1371/journal.pntd.0006007</td>\n",
       "      <td>1</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107997</th>\n",
       "      <td>10.9745/ghsp-d-19-00188</td>\n",
       "      <td>10.1371/journal.pone.0129054</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107998</th>\n",
       "      <td>10.9745/ghsp-d-19-00188</td>\n",
       "      <td>10.15406/ogij.2016.05.00180</td>\n",
       "      <td>1</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107999</th>\n",
       "      <td>10.9745/ghsp-d-19-00188</td>\n",
       "      <td>10.2471/BLT.17.201541</td>\n",
       "      <td>1</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>108000</th>\n",
       "      <td>10.9745/ghsp-d-19-00188</td>\n",
       "      <td>10.3390/ijerph15010096</td>\n",
       "      <td>1</td>\n",
       "      <td>introduction</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>108001 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                       from_doi                        to_doi  label_count  \\\n",
       "0       10.1002/1873-3468.13478          10.1128/JVI.01240-17            1   \n",
       "1            10.1002/ame2.12017        10.3201/eid2206.152113            1   \n",
       "2              10.1002/bdm.2056     10.3102/10769986031004437            1   \n",
       "3              10.1002/cbf.3182  10.1182/blood-2011-01-329060            1   \n",
       "4              10.1002/cbf.3182     10.1186/s12977-015-0178-0            1   \n",
       "...                         ...                           ...          ...   \n",
       "107996  10.9745/ghsp-d-19-00188  10.1371/journal.pntd.0006007            1   \n",
       "107997  10.9745/ghsp-d-19-00188  10.1371/journal.pone.0129054            1   \n",
       "107998  10.9745/ghsp-d-19-00188   10.15406/ogij.2016.05.00180            1   \n",
       "107999  10.9745/ghsp-d-19-00188         10.2471/BLT.17.201541            1   \n",
       "108000  10.9745/ghsp-d-19-00188        10.3390/ijerph15010096            1   \n",
       "\n",
       "               label  \n",
       "0              other  \n",
       "1              other  \n",
       "2              other  \n",
       "3              other  \n",
       "4              other  \n",
       "...              ...  \n",
       "107996  introduction  \n",
       "107997         other  \n",
       "107998         other  \n",
       "107999  introduction  \n",
       "108000  introduction  \n",
       "\n",
       "[108001 rows x 4 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_df = cits_df.groupby(['from_doi', 'to_doi']).label.agg([('label_count', 'count'), (label_col, ','.join)]).reset_index()\n",
    "tmp_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    95524\n",
       "2    10633\n",
       "3     1609\n",
       "4      196\n",
       "5       37\n",
       "6        2\n",
       "Name: label_count, dtype: int64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_df['label_count'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Next-generation sequencing is increasingly being used to study samples composed of mixtures of organisms, such as in clinical applications where the presence of a pathogen at very low abundance may be highly important. We present an analytical method (SIANN: Strain Identification by Alignment to Near Neighbors) specifically designed to rapidly detect a set of target organisms in mixed samples that achieves a high degree of species-and strain-specificity by aligning short sequence reads to the genomes of near neighbor organisms, as well as that of the target. Empirical benchmarking alongside the current state-of-the-art methods shows an extremely high Positive Predictive Value, even at very low abundances of the target organism in a mixed sample. SIANN is available as an Illumina BaseSpace app, as well as through Signature Science, LLC. SIANN results are presented in a streamlined report designed to be comprehensible to the non-specialist user, providing a powerful tool for rapid species detection in a mixed sample. By focusing on a set of (customizable) target organisms and their near neighbors, SIANN can operate quickly and with low computational requirements while delivering highly accurate results.'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for k, p in doi2paper.items():\n",
    "    break\n",
    "    \n",
    "p['abstract'][0]['text']\n",
    "#p['metadata']['title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"The spread of epidemics is inevitably entangled with human behavior, social contacts, and population flows among different geographical regions. The collection and analysis of datasets which trace the activities and interactions of individuals, social patterns, transportation infrastructures and travel fluxes, have unveiled the presence of connectivity patterns characterized by complex features encoded in large-scale heterogeneities and unbounded statistical fluctuations. These features dramatically affect the behavior of dynamical processes occurring on networks, and are responsible for the observed statistical properties of the processes' dynamics and evolution patterns. Here we will present a large-scale stochastic computational approach for the study of the global spread of emergent infectious diseases which explicitly incorporates real world transportation networks and census data.\""
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for k, p in doi2s2paper.items():\n",
    "    break\n",
    "    \n",
    "p['title']\n",
    "p['abstract']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generate output as TSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_text_from_doi(doi):\n",
    "    text = ''\n",
    "    sep = '\\n'\n",
    "    \n",
    "    if doi in doi2s2paper:\n",
    "        # from s2 scraper\n",
    "        #text += doi2s2paper[doi]['title']\n",
    "        \n",
    "        if doi2s2paper[doi]['abstract']:\n",
    "            #text += '\\n' + doi2s2paper[doi]['abstract']\n",
    "            text = doi2s2paper[doi]['title'] + sep + doi2s2paper[doi]['abstract']\n",
    "\n",
    "    elif doi in doi2paper:\n",
    "        #text += doi2paper[doi]['metadata']['title']\n",
    "        \n",
    "        if len(doi2paper[doi]['abstract']) > 0:\n",
    "            #text += doi2paper[doi]['metadata']['title'] + '\\n' + doi2paper[doi]['abstract'][0]['text'] \n",
    "            text = doi2paper[doi]['metadata']['title'] + sep + doi2paper[doi]['abstract'][0]['text'] \n",
    "    else:\n",
    "        raise ValueError('DOI not found')\n",
    "        \n",
    "    return text\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Positive samples\n",
    "pos_rows = []\n",
    "\n",
    "for idx, r in tmp_df.iterrows():\n",
    "    text = get_text_from_doi(r['from_doi'])\n",
    "    text_b = get_text_from_doi(r['to_doi'])\n",
    "    \n",
    "    # Filter out empty texts\n",
    "    if text != '' and text_b != '':\n",
    "        pos_rows.append((r['from_doi'], r['to_doi'], text, text_b, r[label_col]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Negative sampling\n",
    "\n",
    "Requirements:\n",
    "- no connected with citation\n",
    "- no co-citation\n",
    "- no shared author\n",
    "- no shared venue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total DOIs: 95,200\n"
     ]
    }
   ],
   "source": [
    "all_dois = list(map(str, set(list(doi2s2paper.keys()) + list(doi2paper.keys()))))\n",
    "\n",
    "print(f'Total DOIs: {len(all_dois):,}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total citation count: 108,001\n"
     ]
    }
   ],
   "source": [
    "def get_cit_pair(a, b):\n",
    "    # ensure citation pair is always in same order\n",
    "    if a > b:\n",
    "        return (a, b)\n",
    "    else:\n",
    "        return (b, a)\n",
    "\n",
    "\n",
    "cits_set = set([get_cit_pair(from_doi, to_doi) for from_doi, to_doi, label in cits_with_doi])\n",
    "\n",
    "print(f'Total citation count: {len(cits_set):,}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total co-citation count: 2,339,001\n"
     ]
    }
   ],
   "source": [
    "# co cits\n",
    "from_to_cits = defaultdict(set)\n",
    "\n",
    "for from_doi, to_doi, label in cits_with_doi:\n",
    "    from_to_cits[from_doi].add(to_doi)\n",
    "\n",
    "\n",
    "cocits_set = set()\n",
    "\n",
    "for from_cit, to_cits in from_to_cits.items():\n",
    "    for a in to_cits:\n",
    "        for b in to_cits:\n",
    "            cocits_set.add(get_cit_pair(a,b))\n",
    "            \n",
    "print(f'total co-citation count: {len(cocits_set):,}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# shared author\n",
    "def get_authors(doi):\n",
    "    if doi in doi2s2paper:\n",
    "        s2paper = doi2s2paper[doi]\n",
    "        last_names = [a['name'].split()[-1].lower() for a in s2paper['authors']]\n",
    "        return last_names\n",
    "    elif doi in doi2paper:\n",
    "        paper = doi2paper[doi]\n",
    "        last_names = [a['last'].lower() for a in paper['metadata']['authors']]\n",
    "        return last_names\n",
    "    else:\n",
    "        raise ValueError(f'DOI not found: {doi}')\n",
    "\n",
    "def have_no_shared_authors(a_doi, b_doi):\n",
    "    try:\n",
    "        a_authors = set(get_authors(a_doi))\n",
    "        b_authors = set(get_authors(b_doi))\n",
    "        \n",
    "        overlap = a_authors & b_authors\n",
    "        \n",
    "        if len(overlap) == 0:\n",
    "            return True\n",
    "        else:\n",
    "            return False\n",
    "        \n",
    "    except ValueError:\n",
    "        return False\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# has same venue\n",
    "def get_venue(doi):\n",
    "    if doi in doi2s2paper:\n",
    "        s2paper = doi2s2paper[doi]\n",
    "        return s2paper['venue'].lower().strip()\n",
    "    \n",
    "    elif doi in doi2paper:\n",
    "        paper = doi2paper[doi]\n",
    "        venue = paper['_meta']['journal']\n",
    "        \n",
    "        if isinstance(venue, float) and math.isnan(venue):\n",
    "            return ''\n",
    "        else:\n",
    "            return venue.lower().strip()\n",
    "    else:\n",
    "        raise ValueError(f'DOI not found: {doi}')\n",
    "\n",
    "def have_not_same_venue(a_doi, b_doi):\n",
    "    a_venue = get_venue(a_doi)\n",
    "    b_venue = get_venue(b_doi)\n",
    "    \n",
    "    if a_venue == \"\" or b_venue == \"\":\n",
    "        # cant answer if venue is not set\n",
    "        return False\n",
    "        \n",
    "    if fuzz.ratio(a_venue, b_venue) < 0.75:\n",
    "        # fuzzy string matching score must be low!\n",
    "        return True\n",
    "    else:\n",
    "        return False\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Negatives needed: 45,923 (ratio: 0.5)\n",
      "Found 45,923 negative rows (tried 16,176,619 random samples)\n"
     ]
    }
   ],
   "source": [
    "negative_label = 'none'\n",
    "#negative_needed = 10000 #105492  # len(df)\n",
    "negative_ratio = 0.5\n",
    "negative_needed = math.ceil(len(pos_rows) * 0.5)\n",
    "negative_rows = []\n",
    "negative_pairs = set()\n",
    "tries = 0\n",
    "\n",
    "# Negatives needed: 52,746 (ratio: 0.5)\n",
    "print(f'Negatives needed: {negative_needed:,} (ratio: {negative_ratio})')\n",
    "\n",
    "while len(negative_pairs) < negative_needed:\n",
    "    a = random.choice(all_dois)\n",
    "    b = random.choice(all_dois)\n",
    "    \n",
    "    if a == b:\n",
    "        tries += 1\n",
    "        continue\n",
    "        \n",
    "    pair = tuple((a,b))\n",
    "    \n",
    "    if pair in negative_pairs:\n",
    "        continue\n",
    "            \n",
    "    cit_pair = get_cit_pair(a,b)\n",
    "    if cit_pair in cits_set:\n",
    "        tries += 1\n",
    "        continue\n",
    "    \n",
    "    if cit_pair in cocits_set:\n",
    "        tries += 1\n",
    "        continue\n",
    "    \n",
    "    if not have_no_shared_authors(a, b):\n",
    "        tries += 1\n",
    "        continue\n",
    "    \n",
    "    if not have_not_same_venue(a, b):\n",
    "        tries += 1\n",
    "        continue\n",
    "        \n",
    "    text = get_text_from_doi(a)\n",
    "    text_b = get_text_from_doi(b)\n",
    "    \n",
    "    if text == '' or text_b == '':\n",
    "        continue\n",
    "\n",
    "    # None of the criteria above matches...\n",
    "    negative_pairs.add(pair)\n",
    "    negative_rows.append((\n",
    "        a,\n",
    "        b,\n",
    "        text,\n",
    "        text_b,\n",
    "        negative_label,\n",
    "    ))\n",
    "    \n",
    "# Found 45,923 negative rows (tried 16,069,718 random samples)\n",
    "print(f'Found {len(negative_rows):,} negative rows (tried {tries:,} random samples)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Merge pos + neg samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total df rows: 137769\n",
      "After drop_duplicates - df rows: 137769\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>from_doi</th>\n",
       "      <th>to_doi</th>\n",
       "      <th>text</th>\n",
       "      <th>text_b</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1002/1873-3468.13478</td>\n",
       "      <td>10.1128/JVI.01240-17</td>\n",
       "      <td>Mechanisms and biomedical implications of -1 p...</td>\n",
       "      <td>HIV-1 Exploits a Dynamic Multi-aminoacyl-tRNA ...</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.1002/ame2.12017</td>\n",
       "      <td>10.3201/eid2206.152113</td>\n",
       "      <td>The battle against SARS and MERS coronaviruses...</td>\n",
       "      <td>MERS-CoV Infection of Alpaca in a Region Where...</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.1002/bdm.2056</td>\n",
       "      <td>10.3102/10769986031004437</td>\n",
       "      <td>Suffering a Loss Is Good Fortune: Myth or Real...</td>\n",
       "      <td>Computational Tools for Probing Interactions i...</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1002/cbf.3182</td>\n",
       "      <td>10.1182/blood-2011-01-329060</td>\n",
       "      <td>Chloroquine could be used for the treatment of...</td>\n",
       "      <td>Hydroxychloroquine drastically reduces immune ...</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.1002/cbf.3182</td>\n",
       "      <td>10.1186/s12977-015-0178-0</td>\n",
       "      <td>Chloroquine could be used for the treatment of...</td>\n",
       "      <td>Chloroquine and beyond: exploring anti-rheumat...</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137764</th>\n",
       "      <td>10.1136/bmj.g6120</td>\n",
       "      <td>10.1007/s11262-010-0528-x</td>\n",
       "      <td>Spanish authorities investigate how nurse cont...</td>\n",
       "      <td>Molecular analysis of infectious bronchitis vi...</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137765</th>\n",
       "      <td>10.1001/jama.2014.14601</td>\n",
       "      <td>10.1016/j.virol.2013.12.002</td>\n",
       "      <td>Molecular findings among patients referred for...</td>\n",
       "      <td>Genome rearrangement of a mycovirus Rosellinia...</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137766</th>\n",
       "      <td>10.1016/S0140-6736(10)62356-2</td>\n",
       "      <td>10.1128/mbio.00077-11</td>\n",
       "      <td>Development assistance for health: trends and ...</td>\n",
       "      <td>An Insect Nidovirus Emerging from a Primary Tr...</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137767</th>\n",
       "      <td>10.1016/j.cell.2017.12.023</td>\n",
       "      <td>10.1261/rna.039438.113</td>\n",
       "      <td>Transmembrane Pickets Connect Cyto- and Perice...</td>\n",
       "      <td>Automated classification of RNA 3D motifs and ...</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137768</th>\n",
       "      <td>10.1136/bmj.g6802</td>\n",
       "      <td>10.1136/archdischild-2012-302760</td>\n",
       "      <td>Innovative research methods for studying treat...</td>\n",
       "      <td>Protracted bacterial bronchitis: reinventing a...</td>\n",
       "      <td>none</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>137769 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                             from_doi                            to_doi  \\\n",
       "0             10.1002/1873-3468.13478              10.1128/JVI.01240-17   \n",
       "1                  10.1002/ame2.12017            10.3201/eid2206.152113   \n",
       "2                    10.1002/bdm.2056         10.3102/10769986031004437   \n",
       "3                    10.1002/cbf.3182      10.1182/blood-2011-01-329060   \n",
       "4                    10.1002/cbf.3182         10.1186/s12977-015-0178-0   \n",
       "...                               ...                               ...   \n",
       "137764              10.1136/bmj.g6120         10.1007/s11262-010-0528-x   \n",
       "137765        10.1001/jama.2014.14601       10.1016/j.virol.2013.12.002   \n",
       "137766  10.1016/S0140-6736(10)62356-2             10.1128/mbio.00077-11   \n",
       "137767     10.1016/j.cell.2017.12.023            10.1261/rna.039438.113   \n",
       "137768              10.1136/bmj.g6802  10.1136/archdischild-2012-302760   \n",
       "\n",
       "                                                     text  \\\n",
       "0       Mechanisms and biomedical implications of -1 p...   \n",
       "1       The battle against SARS and MERS coronaviruses...   \n",
       "2       Suffering a Loss Is Good Fortune: Myth or Real...   \n",
       "3       Chloroquine could be used for the treatment of...   \n",
       "4       Chloroquine could be used for the treatment of...   \n",
       "...                                                   ...   \n",
       "137764  Spanish authorities investigate how nurse cont...   \n",
       "137765  Molecular findings among patients referred for...   \n",
       "137766  Development assistance for health: trends and ...   \n",
       "137767  Transmembrane Pickets Connect Cyto- and Perice...   \n",
       "137768  Innovative research methods for studying treat...   \n",
       "\n",
       "                                                   text_b  label  \n",
       "0       HIV-1 Exploits a Dynamic Multi-aminoacyl-tRNA ...  other  \n",
       "1       MERS-CoV Infection of Alpaca in a Region Where...  other  \n",
       "2       Computational Tools for Probing Interactions i...  other  \n",
       "3       Hydroxychloroquine drastically reduces immune ...  other  \n",
       "4       Chloroquine and beyond: exploring anti-rheumat...  other  \n",
       "...                                                   ...    ...  \n",
       "137764  Molecular analysis of infectious bronchitis vi...   none  \n",
       "137765  Genome rearrangement of a mycovirus Rosellinia...   none  \n",
       "137766  An Insect Nidovirus Emerging from a Primary Tr...   none  \n",
       "137767  Automated classification of RNA 3D motifs and ...   none  \n",
       "137768  Protracted bacterial bronchitis: reinventing a...   none  \n",
       "\n",
       "[137769 rows x 5 columns]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# construct\n",
    "df = pd.DataFrame(pos_rows + negative_rows, columns=['from_doi', 'to_doi', 'text', 'text_b', label_col])\n",
    "\n",
    "print(f'Total df rows: {len(df)}')\n",
    "\n",
    "df.drop_duplicates(['text', 'text_b'], keep='first', inplace=True)\n",
    "\n",
    "print(f'After drop_duplicates - df rows: {len(df)}')\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "other                                 52317\n",
       "none                                  45923\n",
       "introduction                          13447\n",
       "discussion                            13366\n",
       "introduction,other                     2434\n",
       "                                      ...  \n",
       "results,materials,methods,other           1\n",
       "materials,methods,other,discussion        1\n",
       "background,methods,discussion             1\n",
       "introduction,materials                    1\n",
       "results,conclusion                        1\n",
       "Name: label, Length: 161, dtype: int64"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[label_col].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "other                 525\n",
       "none                  405\n",
       "introduction           41\n",
       "discussion             27\n",
       "other,discussion        1\n",
       "introduction,other      1\n",
       "Name: label, dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total: 1,000; Train: 747; Test: 253\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mostendorff/miniconda2/envs/acl-anthology/lib/python3.7/site-packages/sklearn/model_selection/_split.py:657: Warning: The least populated class in y has only 1 members, which is too few. The minimum number of members in any class cannot be less than n_splits=4.\n",
      "  % (min_groups, self.n_splits)), Warning)\n"
     ]
    }
   ],
   "source": [
    "# Sample data (for debugging & development)\n",
    "sample_df = df.sample(n=1000, weights=df[label_col].value_counts()[df[label_col]].values.tolist())\n",
    "\n",
    "display(sample_df[label_col].value_counts())\n",
    "\n",
    "sample_kf = StratifiedKFold(n_splits=4, random_state=0, shuffle=True)\n",
    "for train_index, test_index in sample_kf.split(sample_df.index.tolist(), sample_df[label_col].values.tolist()):\n",
    "    k = 'sample_1k'\n",
    "    split_dir = os.path.join(cord19_dir, 'splits', str(k))\n",
    "    \n",
    "    if not os.path.exists(split_dir):\n",
    "        os.makedirs(split_dir)\n",
    "        \n",
    "    split_train_df = sample_df.iloc[train_index]\n",
    "    split_test_df = sample_df.iloc[test_index]\n",
    "\n",
    "    print(f'Total: {len(sample_df):,}; Train: {len(split_train_df):,}; Test: {len(split_test_df):,}')\n",
    "\n",
    "    split_train_df.to_csv(os.path.join(split_dir, 'train.tsv'), sep='\\t', index=False)\n",
    "    split_test_df.to_csv(os.path.join(split_dir, 'test.tsv'), sep='\\t', index=False)\n",
    "    \n",
    "    break # we only need one sample set!\n",
    "\n",
    "del sample_kf\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['discussion',\n",
       " 'introduction',\n",
       " 'conclusion',\n",
       " 'results',\n",
       " 'methods',\n",
       " 'background',\n",
       " 'materials',\n",
       " 'virus',\n",
       " 'future work']"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mostendorff/miniconda2/envs/acl-anthology/lib/python3.7/site-packages/sklearn/model_selection/_split.py:657: Warning: The least populated class in y has only 1 members, which is too few. The minimum number of members in any class cannot be less than n_splits=4.\n",
      "  % (min_groups, self.n_splits)), Warning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total: 137,769; Train: 103,293; Test: 34,476\n",
      "Total: 137,769; Train: 103,326; Test: 34,443\n",
      "Total: 137,769; Train: 103,323; Test: 34,446\n",
      "Total: 137,769; Train: 103,365; Test: 34,404\n"
     ]
    }
   ],
   "source": [
    "# Full training and test set\n",
    "kf = StratifiedKFold(n_splits=n_splits, random_state=0, shuffle=True)\n",
    "\n",
    "# Stratified K-Folds cross-validator\n",
    "for k, (train_index, test_index) in enumerate(kf.split(df.index.tolist(), df[label_col].values.tolist()), 1):\n",
    "    split_dir = os.path.join(cord19_dir, 'splits', str(k))\n",
    "    \n",
    "    if not os.path.exists(split_dir):\n",
    "        os.makedirs(split_dir)\n",
    "    \n",
    "    split_train_df = df.iloc[train_index]\n",
    "    split_test_df = df.iloc[test_index]\n",
    "    \n",
    "    print(f'Total: {len(df):,}; Train: {len(split_train_df):,}; Test: {len(split_test_df):,}')\n",
    "    \n",
    "    split_train_df.to_csv(os.path.join(split_dir, 'train.tsv'), sep='\\t', index=False)\n",
    "    split_test_df.to_csv(os.path.join(split_dir, 'test.tsv'), sep='\\t', index=False)\n",
    "    #break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scrape missing paper data from S2 API with DOI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join(scraper_dir, 'unique_missing_papers.json'), 'r') as f:\n",
    "    unique_missing_papers = set(json.load(f))  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "140682"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(unique_missing_papers)  # list of DOIs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join(scraper_dir, 'unique_missing_papers.json'), 'w') as f:\n",
    "    json.dump(list(unique_missing_papers), f)     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "errors = set()\n",
    "notfound = set()\n",
    "doi2paper = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join(scraper_dir, 'doi2paper.json'), 'r') as f:\n",
    "    doi2paper = json.load(f)\n",
    "with open(os.path.join(scraper_dir, 'errors.json'), 'r') as f:\n",
    "    errors = set(json.load(f))\n",
    "with open(os.path.join(scraper_dir, 'notfound.json'), 'r') as f:\n",
    "    notfound = set(json.load(f))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doi2paper: 23715\n",
      "errors: 98\n",
      "notfound: 1526\n"
     ]
    }
   ],
   "source": [
    "print(f'doi2paper: {len(doi2paper)}')  # 12451, 22938\n",
    "print(f'errors: {len(errors)}')\n",
    "print(f'notfound: {len(notfound)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3baeaa8bec584f2c9a843f57cb04b029",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=140682), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1667/0033-7587(2003)159[0484:RROCDA]2.0.CO;2\n",
      "Error status: 500 - 10.1002/(sici)1520-6300(1996)8:4<497::Aid-ajhb10>3.0.Co;2-h\n",
      "Error status: 500 - 10.1002/1097-0142(19920115)69:2<537::AID-CNCR2820690242>3.0.CO;2-3\n",
      "Error status: 500 - 10.1002/1522-2683(200203)23:5<677::AID-ELPS677>3.0.CO;2â•ﬁ8\n",
      "Error status: 500 - 10.1128/CMR.14.2.430&ndash;445.2001\n",
      "Error status: 500 - 10.1002/1521-4141(200209)32:9<2635::AID-IMMU2635>3.0.CO;2-N\n",
      "Error status: 500 - 10.1073/pnas.1104306108;10.1073/pnas.1104306108\n",
      "Error status: 500 - 10.1175/1520-0450(1984)023<1674:AUSOAT>2.0.CO;2\n",
      "Error status: 500 - 10.1002/(SICI)1097-4547(19980201)51:3<403::AID-JNR13>3.0.CO;2-7\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1603/0022-2585(2005)042[0473:DORBAB]2.0.CO;2\n",
      "Error status: 500 - 10.1002/(SICI)1521-1878(199911)21:11<932::AID-BIES5>3.0.CO;2-N\n",
      "Error status: 500 - 10.1002/(SICI)1097-0290(19990720)64:2<135::AID-BIT2>3.3.CO;2-H\n",
      "Error status: 500 - 10.1128/CMR.14.1.129&ndash;149.2001\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/(SICI)1096-9071(199706)52:2<121::AID-JMV1>3.0.CO;2-5\n",
      "Error status: 500 - 10.1128/JVI.79.20.12989&ndash;12998.2005\n",
      "Error status: 500 - 10.1038/nmeth.1923\n",
      "Error status: 500 - 10.1128/IAI.70.11.6365&ndash;6372.2002\n",
      "Error status: 500 - 10.1002/(SICI)1097-4598(199904)22:4<460::AID-MUS6>3.0.CO;2-L\n",
      "Error status: 500 - 10.1002/(SICI)1522-2683(19991201)20:18%3C3551::AID-ELPS3551%3E3.0.CO;2-2\n",
      "Error status: 500 - 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2\n",
      "Error status: 500 - 10.1093/molbev/msr121\n",
      "Error status: 500 - 10.1002/(SICI)1099-1573(199702)11:1<42::AID-PTR940>3.0.CO;2-5\n",
      "Error status: 500 - 10.1007/s00705&ndash;012&ndash;1591&ndash;5\n",
      "Error status: 500 - 10.1002/1521-4141(200207)32:7<2004::AID-IMMU2004>3.0.CO;2-5\n",
      "Error status: 500 - 10.1002/(SICI)1098-1004(200001)15:1<57::AID-HUMU12>3.0.CO;2-G\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(199912)59:4<552::AID-JMV21>3.0.CO;2-A\n",
      "Error status: 500 - 10.1002/(SICI)1097-0290(19970605)54:5<468::AID-BIT7>3.0.CO;2-C\n",
      "Error status: 500 - 10.1099/vir.0.038679&ndash;0\n",
      "Error status: 500 - 10.1385/1-59259-192-2:365\n",
      "Error status: 500 - 10.1637/0005-2086(2003)047[0149:MTOTCB]2.0.CO;2\n",
      "Error status: 500 - 10.1002/jcc.20084\n",
      "Error status: 500 - 10.1063/1.3297877\n",
      "Error status: 500 - 10.1002/(SICI)1096-987X(199802)19:3%3C319::AID-JCC6%3E3.0.CO;2-W\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/(SICI)1521-4095(199903)11:3<253::AID-ADMA253>3.0.CO;2-7\n",
      "Error status: 500 - 10.1093/bioinformatics/btp352\n",
      "Error status: 500 - 10.1002/(SICI)1097-0010(199601)70:1<55::AID-JSFA471>3.0.CO;2-X\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(199810)56:2<159::AID-JMV10>3.0.CO;2-B\n",
      "Error status: 500 - 10.1002/1531-8249(200002)47:2ï»¿<ï»¿276::AID-ANA28ï»¿>ï»¿3.3.CO;2-T\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/(SICI)1096-9071(199605)49:1<1::AID-JMV1>3.0.CO;2-A\n",
      "Error status: 500 - 10.3398/1527-0904(2006)66[390:BIBCCL]2.0.CO;2\n",
      "Error status: 500 - 10.1128/IAI.01539-07\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/1521-4141(200201)32:1<97::AID-IMMU97>3.0.CO;2-Y\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(200005)61:1%6052::AID-JMV8%623.0.CO;2-L\n",
      "Error status: 500 - 10.1128/JVI.75.8.4014&ndash;4018.2001\n",
      "Error status: 500 - 10.1202/0002-8894(2000)061<0056:IOBMEF>2.0.CO;2\n",
      "Error status: 500 - 10.1002/1521-3773(20001002)39:19<3430::AID-ANIE3430>3.0.CO;2-3\n",
      "Error status: 500 - 10.1016/0021-9681(87)90171-8\n",
      "Error status: 500 - 10.1099/vir.0.056341&ndash;0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1128/JB.183.22.6573&ndash;6578.2001\n",
      "Error status: 500 - 10.1128/JVI.79.15.9439&ndash;9448.2005\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(199811)56:3<186::AID-JMV2>3.0.CO;2-3\n",
      "Error status: 500 - 10.1099/vir.0.043935&ndash;0\n",
      "Error status: 500 - 10.1637/0005-2086(2002)046[0053:poahko]2.0.co;2\n",
      "Error status: 500 - 10.1002/(SICI)1097-0010(199909)79:12<1601::AID-JSFA407>3.0.CO;2-1\n",
      "Error status: 500 - 10.1290/1071-2690(2002)038%3c0123:MIECIP%3e2.0.CO;2\n",
      "Error status: 500 - 10.1002/1521-4141(200107)31:7<2104::AID-IMMU2104>3.0.CO;2-3\n",
      "Error status: 500 - 10.1002/(SICI)1098-1136(19990101)25:1<21::AID-GLIA3>3.0.CO;2-R\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/(ISSN)1521-414110.1002/1521-4141(2000)30:8<>1.0.CO;2-010.1002/1521-4141(2000)30:8<2372::AID-IMMU2372>3.0.CO;2-D\n",
      "Error status: 500 - 10.1002/(SICI)1099-1573(199905)13:3<222::AID-PTR447>3.0.CO;2-P\n",
      "Error status: 500 - 10.1086/519795\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1002/9781119062585&copy;2017\n",
      "Error status: 500 - 10.1002/(SICI)1521-4141(199804)28:04<1251::AID-IMMU1251>3.0.CO;2-O\n",
      "Error status: 500 - 10.1017/S1049023&times;13008601\n",
      "Error status: 500 - 10.1002/(SICI)1520-667X(1998)10:4%3c313:AID-MCS1%3e3.0.CO;2-J\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(199708)52:4<406::AID-JMV11>3.0.CO;2-E\n",
      "Error status: 500 - 10.1099/vir.0.040287&ndash;0\n",
      "Error status: 500 - 10.1645/0022-3395(2000)086[0627:POTGAI]2.0.CO;2\n",
      "Error status: 500 - 10.1002/1098-1136(200012)32:3%3C214::AID-GLIA20%3E3.0.CO;2-7\n",
      "Error status: 500 - 10.1637/0005-2086(2007)51[725:IOSDOV]2.0.CO;2\n",
      "Error status: 500 - 10.1638/1042-7260(2000)031[0353:ALSOEC]2.0.CO;2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error status: 500 - 10.1638/1042-7260(2000)031[0335:ASOCCD]2.0.CO;2\n",
      "Error status: 500 - 10.1002/(SICI)1097-4547(19960915)45:6%3C735::AID-JNR10%3E3.0.CO;2-V\n",
      "Error status: 500 - 10.1038/nature10404\n",
      "Error status: 500 - 10.1093/jpids/piw091\n",
      "Error status: 500 - 10.1002/rmv.483\n",
      "Error status: 500 - 10.1073/pnas.85.21.7972\n",
      "Error status: 500 - 10.3390/ijerph17020428\n",
      "Error status: 500 - 10.3201/eid2402.171216\n",
      "Error status: 500 - 10.1097/INF.0b013e31814536ba\n",
      "Error status: 500 - 10.1002/(SICI)1096-9071(199902)57:2<186::AID-JMV17>3.0.CO;2-Q\n",
      "Error status: 500 - 10.1890/0012-9658(2000)081[0654:LHAEPP]2.0.CO;2\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub message rate exceeded.\n",
      "The notebook server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--NotebookApp.iopub_msg_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)\n",
      "NotebookApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-16-1fb6d0cbad76>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     39\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     40\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 41\u001b[0;31m     \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msleep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m2.5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     42\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "check_points = []\n",
    "api_url = 'http://api.semanticscholar.org/v1/paper/'\n",
    "offset = 0 \n",
    "\n",
    "for i, doi in enumerate(tqdm(unique_missing_papers, total=len(unique_missing_papers))):\n",
    "    if i < offset:  # skip \n",
    "        continue\n",
    "        \n",
    "    if doi in doi2paper or doi in errors or doi in notfound:\n",
    "        continue\n",
    "        \n",
    "    res = requests.get(api_url + doi)    \n",
    "    \n",
    "    if res.status_code == 200:\n",
    "        try:\n",
    "            doi2paper[doi] = res.json()\n",
    "        except ValueError:\n",
    "            print(f'Error cannot parse JSON: {doi}')\n",
    "            errors.add(doi)\n",
    "    elif res.status_code == 429:\n",
    "        print(f'Stop! Rate limit reached at: {i}')\n",
    "        break\n",
    "    elif res.status_code == 403:\n",
    "        print(f'Stop! Forbidden / rate limit reached at: {i}')\n",
    "        break\n",
    "    elif res.status_code == 404:\n",
    "        notfound.add(doi)\n",
    "    else:\n",
    "        print(f'Error status: {res.status_code} - {doi}')\n",
    "        errors.add(doi)\n",
    "    \n",
    "    if (i % 10000) == 0 and i > 0:\n",
    "        with open(os.path.join(scraper_dir, 'doi2paper.json'), 'w') as f:\n",
    "            json.dump(doi2paper, f)\n",
    "        with open(os.path.join(scraper_dir, 'errors.json'), 'w') as f:\n",
    "            json.dump(list(errors), f)\n",
    "        with open(os.path.join(scraper_dir, 'notfound.json'), 'w') as f:\n",
    "            json.dump(list(notfound), f)\n",
    "            \n",
    "            \n",
    "    time.sleep(2.5)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "54680"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'abstract': \"Context: Qigong, Tai‐chi and dancing have all been proven effective for Parkinson's disease (PD); however, no study has yet assessed the efficacy of Turo, a hybrid qigong dancing program developed to relieve symptoms in PD patients. Objective: To determine whether Turo may provide benefit in addressing the symptoms of PD patients. Design: Randomized, assessor blind, waiting‐list control, partial crossover study. Setting: Kyung Hee University Korean Medicine Hospital, Seoul, Republic of Korea. Participants: A total of 32 PD patients (mean age 65.7 ± 6.8). Intervention: Participants were assigned to the Turo group or the waiting‐list control group. The Turo group participated in an 8‐week Turo training program (60‐minute sessions twice a week). The waiting‐list control group received no additional treatment during the same period; then underwent the same 8‐week Turo training. Outcome Measures: The primary outcome was a score on the Unified Parkinson's Disease Rating Scale (UPDRS), and the secondary outcomes included the perceived health status assessed using the Parkinson's disease Quality of Life questionnaire (PDQL), balance function as assessed by the Berg Balance Scale (BBS) and the results of the Beck Depression Inventory (BDI). Results: The Turo group showed statistically significant improvements in the UPDRS (P < 0.01) and PDQL (P < 0.05) as compared to the control group. The changes in BBS scores displayed a tendency toward improvement, but was not statistically significant (P = 0.051). Conclusion: These findings suggest that Turo PD training might improve the symptoms of PD patients.\",\n",
       " 'arxivId': None,\n",
       " 'authors': [{'authorId': '15346455',\n",
       "   'name': 'Hwa-jin Lee',\n",
       "   'url': 'https://www.semanticscholar.org/author/15346455'},\n",
       "  {'authorId': '4133307',\n",
       "   'name': 'Song-Yi Kim',\n",
       "   'url': 'https://www.semanticscholar.org/author/4133307'},\n",
       "  {'authorId': '143762929',\n",
       "   'name': 'Younbyoung Chae',\n",
       "   'url': 'https://www.semanticscholar.org/author/143762929'},\n",
       "  {'authorId': '1719843',\n",
       "   'name': 'Mi-Young Kim',\n",
       "   'url': 'https://www.semanticscholar.org/author/1719843'},\n",
       "  {'authorId': '5797062',\n",
       "   'name': 'Changshik Yin',\n",
       "   'url': 'https://www.semanticscholar.org/author/5797062'},\n",
       "  {'authorId': '6790214',\n",
       "   'name': 'Woo-sang Jung',\n",
       "   'url': 'https://www.semanticscholar.org/author/6790214'},\n",
       "  {'authorId': '145979133',\n",
       "   'name': 'Ki-Ho Cho',\n",
       "   'url': 'https://www.semanticscholar.org/author/145979133'},\n",
       "  {'authorId': '2231651',\n",
       "   'name': 'Seung-Nam Kim',\n",
       "   'url': 'https://www.semanticscholar.org/author/2231651'},\n",
       "  {'authorId': '8043377',\n",
       "   'name': 'Hi-Joon Park',\n",
       "   'url': 'https://www.semanticscholar.org/author/8043377'},\n",
       "  {'authorId': '1891421',\n",
       "   'name': 'Hyejung Lee',\n",
       "   'url': 'https://www.semanticscholar.org/author/1891421'}],\n",
       " 'citationVelocity': 0,\n",
       " 'citations': [{'arxivId': None,\n",
       "   'authors': [{'authorId': '1471185000',\n",
       "     'name': 'Xiaohu Jin',\n",
       "     'url': 'https://www.semanticscholar.org/author/1471185000'},\n",
       "    {'authorId': '48170377',\n",
       "     'name': 'Lin Wang',\n",
       "     'url': 'https://www.semanticscholar.org/author/48170377'},\n",
       "    {'authorId': '48641924',\n",
       "     'name': 'Shijie Liu',\n",
       "     'url': 'https://www.semanticscholar.org/author/48641924'},\n",
       "    {'authorId': '145857011',\n",
       "     'name': 'Lin Zhu',\n",
       "     'url': 'https://www.semanticscholar.org/author/145857011'},\n",
       "    {'authorId': '3946908',\n",
       "     'name': 'Paul D Loprinzi',\n",
       "     'url': 'https://www.semanticscholar.org/author/3946908'},\n",
       "    {'authorId': '144309185',\n",
       "     'name': 'Xin Tian Fan',\n",
       "     'url': 'https://www.semanticscholar.org/author/144309185'}],\n",
       "   'doi': '10.3390/ijerph17010031',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': True,\n",
       "   'paperId': '08ba16ea187fbf8afc83e33723e81c3b2afc9a39',\n",
       "   'title': 'The Impact of Mind-Body Exercises on Motor Function, Depressive Symptoms, and Quality of Life in Parkinson’s Disease: A Systematic Review and Meta-Analysis',\n",
       "   'url': 'https://www.semanticscholar.org/paper/08ba16ea187fbf8afc83e33723e81c3b2afc9a39',\n",
       "   'venue': 'International journal of environmental research and public health',\n",
       "   'year': 2019},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '17136793',\n",
       "     'name': 'Yeonju Woo',\n",
       "     'url': 'https://www.semanticscholar.org/author/17136793'},\n",
       "    {'authorId': '5386992',\n",
       "     'name': 'Min Kyung Hyun',\n",
       "     'url': 'https://www.semanticscholar.org/author/5386992'}],\n",
       "   'doi': '10.3389/fnagi.2019.00040',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4eb512e8b51929e92cae935e6355b1b571e9e7b0',\n",
       "   'title': 'Effectiveness of Integrative Therapy for Parkinson’s Disease Management',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4eb512e8b51929e92cae935e6355b1b571e9e7b0',\n",
       "   'venue': 'Front. Aging Neurosci.',\n",
       "   'year': 2019}],\n",
       " 'corpusId': 4799232,\n",
       " 'doi': '10.1016/j.explore.2017.11.002',\n",
       " 'fieldsOfStudy': ['Medicine'],\n",
       " 'influentialCitationCount': 1,\n",
       " 'is_open_access': False,\n",
       " 'is_publisher_licensed': True,\n",
       " 'paperId': 'ba02eb4199691ddf8974a5ff8c776ea1b37fb2a0',\n",
       " 'references': [{'arxivId': None,\n",
       "   'authors': [{'authorId': '46901141',\n",
       "     'name': 'Hwajin Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/46901141'},\n",
       "    {'authorId': '1891421',\n",
       "     'name': 'Hyejung Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/1891421'},\n",
       "    {'authorId': '145673414',\n",
       "     'name': 'Chang-Shik Yin',\n",
       "     'url': 'https://www.semanticscholar.org/author/145673414'},\n",
       "    {'authorId': '143762929',\n",
       "     'name': 'Younbyoung Chae',\n",
       "     'url': 'https://www.semanticscholar.org/author/143762929'},\n",
       "    {'authorId': '74883380',\n",
       "     'name': 'You-sang Baik',\n",
       "     'url': 'https://www.semanticscholar.org/author/74883380'},\n",
       "    {'authorId': '152692357',\n",
       "     'name': 'Yong-cheol Shin',\n",
       "     'url': 'https://www.semanticscholar.org/author/152692357'},\n",
       "    {'authorId': '80602646',\n",
       "     'name': 'Sang-Jea Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/80602646'},\n",
       "    {'authorId': '8043377',\n",
       "     'name': 'Hi-Joon Park',\n",
       "     'url': 'https://www.semanticscholar.org/author/8043377'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '7fa2b8a6885bece1d6621741e2bad370f4ec52c4',\n",
       "   'title': \"Development of disease-specific qigong program for Parkinson's disease\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/7fa2b8a6885bece1d6621741e2bad370f4ec52c4',\n",
       "   'venue': '',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '36670591',\n",
       "     'name': 'Ryan P. Duncan',\n",
       "     'url': 'https://www.semanticscholar.org/author/36670591'},\n",
       "    {'authorId': '5957474',\n",
       "     'name': 'Gammon M Earhart',\n",
       "     'url': 'https://www.semanticscholar.org/author/5957474'}],\n",
       "   'doi': '10.1177/1545968311421614',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '562c64c51b74da83b476270d16a8eea725a1c8d0',\n",
       "   'title': 'Randomized Controlled Trial of Community-Based Dancing to Modify Disease Progression in Parkinson Disease',\n",
       "   'url': 'https://www.semanticscholar.org/paper/562c64c51b74da83b476270d16a8eea725a1c8d0',\n",
       "   'venue': 'Neurorehabilitation and neural repair',\n",
       "   'year': 2012},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5559622',\n",
       "     'name': 'Colleen G. Canning',\n",
       "     'url': 'https://www.semanticscholar.org/author/5559622'},\n",
       "    {'authorId': '143882177',\n",
       "     'name': 'Jennifer A Alison',\n",
       "     'url': 'https://www.semanticscholar.org/author/143882177'},\n",
       "    {'authorId': '4383397',\n",
       "     'name': 'Naomi E Allen',\n",
       "     'url': 'https://www.semanticscholar.org/author/4383397'},\n",
       "    {'authorId': '4444373',\n",
       "     'name': 'Herbert Groeller',\n",
       "     'url': 'https://www.semanticscholar.org/author/4444373'}],\n",
       "   'doi': '10.1016/S0003-9993(97)90264-1',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '98d7bf36b05811f00ebfd7bca9a811c92bb66b01',\n",
       "   'title': \"Parkinson's disease: an investigation of exercise capacity, respiratory function, and gait.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/98d7bf36b05811f00ebfd7bca9a811c92bb66b01',\n",
       "   'venue': 'Archives of physical medicine and rehabilitation',\n",
       "   'year': 1997},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '48586580',\n",
       "     'name': 'Chenchen Wang',\n",
       "     'url': 'https://www.semanticscholar.org/author/48586580'},\n",
       "    {'authorId': '7880706',\n",
       "     'name': 'Jean Paul Collet',\n",
       "     'url': 'https://www.semanticscholar.org/author/7880706'},\n",
       "    {'authorId': '143917291',\n",
       "     'name': 'Joseph Lau',\n",
       "     'url': 'https://www.semanticscholar.org/author/143917291'}],\n",
       "   'doi': '10.1001/archinte.164.5.493',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f064084c7832a221afe14fc997e17580f67e46d8',\n",
       "   'title': 'The Effect of Tai Chi on Health Outcomes in Patients With Chronic Conditions: A Systematic Review',\n",
       "   'url': 'https://www.semanticscholar.org/paper/f064084c7832a221afe14fc997e17580f67e46d8',\n",
       "   'venue': '',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '40407893',\n",
       "     'name': 'Monique L. Giroux',\n",
       "     'url': 'https://www.semanticscholar.org/author/40407893'}],\n",
       "   'doi': '10.3949/ccjm.74.5.313',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'faa3b1f453cc7937aacb31b4d2f97d46c0afca99',\n",
       "   'title': 'Parkinson disease: managing a complex, progressive disease at all stages.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/faa3b1f453cc7937aacb31b4d2f97d46c0afca99',\n",
       "   'venue': 'Cleveland Clinic journal of medicine',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1938172',\n",
       "     'name': 'Yan Yang',\n",
       "     'url': 'https://www.semanticscholar.org/author/1938172'},\n",
       "    {'authorId': '13799517',\n",
       "     'name': 'Xiang-Yuan Li',\n",
       "     'url': 'https://www.semanticscholar.org/author/13799517'},\n",
       "    {'authorId': '144510234',\n",
       "     'name': 'Li Gong',\n",
       "     'url': 'https://www.semanticscholar.org/author/144510234'},\n",
       "    {'authorId': '4340980',\n",
       "     'name': 'Yun-liang Zhu',\n",
       "     'url': 'https://www.semanticscholar.org/author/4340980'},\n",
       "    {'authorId': '35241465',\n",
       "     'name': 'Yan-lei Hao',\n",
       "     'url': 'https://www.semanticscholar.org/author/35241465'}],\n",
       "   'doi': '10.1371/journal.pone.0102942',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '84816ad743c9b3b01a5aa95b72a93ccaf9717b1f',\n",
       "   'title': \"Tai Chi for Improvement of Motor Function, Balance and Gait in Parkinson's Disease: A Systematic Review and Meta-Analysis\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/84816ad743c9b3b01a5aa95b72a93ccaf9717b1f',\n",
       "   'venue': 'PloS one',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '153721023',\n",
       "     'name': 'P Hobson',\n",
       "     'url': 'https://www.semanticscholar.org/author/153721023'},\n",
       "    {'authorId': '48954168',\n",
       "     'name': 'Arthur Cort Holden',\n",
       "     'url': 'https://www.semanticscholar.org/author/48954168'},\n",
       "    {'authorId': '33661982',\n",
       "     'name': 'Jill R. Meara',\n",
       "     'url': 'https://www.semanticscholar.org/author/33661982'}],\n",
       "   'doi': '10.1093/ageing/28.4.341',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '63a658db95ae998a682744ee9fde6e65c7772e8a',\n",
       "   'title': \"Measuring the impact of Parkinson's disease with the Parkinson's Disease Quality of Life questionnaire.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/63a658db95ae998a682744ee9fde6e65c7772e8a',\n",
       "   'venue': 'Age and ageing',\n",
       "   'year': 1999},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '46901141',\n",
       "     'name': 'Hwajin Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/46901141'},\n",
       "    {'authorId': '143762929',\n",
       "     'name': 'Younbyoung Chae',\n",
       "     'url': 'https://www.semanticscholar.org/author/143762929'},\n",
       "    {'authorId': '8043377',\n",
       "     'name': 'Hi-Joon Park',\n",
       "     'url': 'https://www.semanticscholar.org/author/8043377'},\n",
       "    {'authorId': '3812929',\n",
       "     'name': 'Dae-Hyun Hahm',\n",
       "     'url': 'https://www.semanticscholar.org/author/3812929'},\n",
       "    {'authorId': '47648783',\n",
       "     'name': 'Kyungeh An',\n",
       "     'url': 'https://www.semanticscholar.org/author/47648783'},\n",
       "    {'authorId': '1891421',\n",
       "     'name': 'Hyejung Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/1891421'}],\n",
       "   'doi': '10.1093/ecam/nem120',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '2358b445c151e169cc464539008ee8a5f3e0767e',\n",
       "   'title': 'Turo (Qi Dance) Training Attenuates Psychological Symptoms and Sympathetic Activation Induced by Mental Stress in Healthy Women',\n",
       "   'url': 'https://www.semanticscholar.org/paper/2358b445c151e169cc464539008ee8a5f3e0767e',\n",
       "   'venue': 'Evidence-based complementary and alternative medicine : eCAM',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3188076',\n",
       "     'name': 'Arianne P Verhagen',\n",
       "     'url': 'https://www.semanticscholar.org/author/3188076'},\n",
       "    {'authorId': '40063701',\n",
       "     'name': 'M D Immink',\n",
       "     'url': 'https://www.semanticscholar.org/author/40063701'},\n",
       "    {'authorId': '51924087',\n",
       "     'name': 'Annemieke van der Meulen',\n",
       "     'url': 'https://www.semanticscholar.org/author/51924087'},\n",
       "    {'authorId': '1387045265',\n",
       "     'name': 'Sita M. A. Bierma-Zeinstra',\n",
       "     'url': 'https://www.semanticscholar.org/author/1387045265'}],\n",
       "   'doi': '10.1093/fampra/cmh122',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '99c67ca2339cbaee25221f5edd4d800242ffed04',\n",
       "   'title': 'The efficacy of Tai Chi Chuan in older adults: a systematic review.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/99c67ca2339cbaee25221f5edd4d800242ffed04',\n",
       "   'venue': 'Family practice',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '34344513',\n",
       "     'name': 'Myeong Soo Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/34344513'},\n",
       "    {'authorId': '153626825',\n",
       "     'name': 'Ezard Ernst',\n",
       "     'url': 'https://www.semanticscholar.org/author/153626825'}],\n",
       "   'doi': '10.1002/mds.22275',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9832399e4fedf6f316ff37337a977c285bb6645b',\n",
       "   'title': 'Qigong for movement disorders: A systematic review.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9832399e4fedf6f316ff37337a977c285bb6645b',\n",
       "   'venue': 'Movement disorders : official journal of the Movement Disorder Society',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '32661709',\n",
       "     'name': 'P Ferry',\n",
       "     'url': 'https://www.semanticscholar.org/author/32661709'},\n",
       "    {'authorId': '40161530',\n",
       "     'name': 'Mw Johnson',\n",
       "     'url': 'https://www.semanticscholar.org/author/40161530'},\n",
       "    {'authorId': '144362430',\n",
       "     'name': 'Patrick Wallis',\n",
       "     'url': 'https://www.semanticscholar.org/author/144362430'}],\n",
       "   'doi': '10.1136/pmj.78.924.612',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'cbcdbc7068635be3417a8aa2d5ba006f9dbe24ea',\n",
       "   'title': \"Use of complementary therapies and non-prescribed medication in patients with Parkinson's disease.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/cbcdbc7068635be3417a8aa2d5ba006f9dbe24ea',\n",
       "   'venue': 'Postgraduate medical journal',\n",
       "   'year': 2002},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47740942',\n",
       "     'name': 'Peter P. Richter',\n",
       "     'url': 'https://www.semanticscholar.org/author/47740942'},\n",
       "    {'authorId': '48844026',\n",
       "     'name': 'Jacob Werner',\n",
       "     'url': 'https://www.semanticscholar.org/author/48844026'},\n",
       "    {'authorId': '48717954',\n",
       "     'name': 'Andrés Heerlein',\n",
       "     'url': 'https://www.semanticscholar.org/author/48717954'},\n",
       "    {'authorId': '49244965',\n",
       "     'name': 'A. Kraus',\n",
       "     'url': 'https://www.semanticscholar.org/author/49244965'},\n",
       "    {'authorId': '50990637',\n",
       "     'name': 'Heinrich Sauer',\n",
       "     'url': 'https://www.semanticscholar.org/author/50990637'}],\n",
       "   'doi': '10.1159/000066239',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f3a98b7b852b8629665a32babe49c258d13be878',\n",
       "   'title': 'On the Validity of the Beck Depression Inventory',\n",
       "   'url': 'https://www.semanticscholar.org/paper/f3a98b7b852b8629665a32babe49c258d13be878',\n",
       "   'venue': 'Psychopathology',\n",
       "   'year': 1998},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47190591',\n",
       "     'name': 'Margaret M. Hoehn',\n",
       "     'url': 'https://www.semanticscholar.org/author/47190591'},\n",
       "    {'authorId': '4144556',\n",
       "     'name': 'Melvin D. Yahr',\n",
       "     'url': 'https://www.semanticscholar.org/author/4144556'}],\n",
       "   'doi': None,\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'dddef7f00b6969bd402361043abb1ad1451378ae',\n",
       "   'title': 'Parkinsonism: onset, progression, and mortality. 1967.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/dddef7f00b6969bd402361043abb1ad1451378ae',\n",
       "   'venue': 'Neurology',\n",
       "   'year': 1998},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1704488',\n",
       "     'name': 'Erwin B. Montgomery',\n",
       "     'url': 'https://www.semanticscholar.org/author/1704488'}],\n",
       "   'doi': '10.1212/01.wnl.0000254819.77545.da',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e1c845f99ad01ea8e0f0d7701bbad29e8729d295',\n",
       "   'title': 'Practice Parameter: neuroprotective strategies and alternative therapies for Parkinson disease (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e1c845f99ad01ea8e0f0d7701bbad29e8729d295',\n",
       "   'venue': 'Neurology',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6108551',\n",
       "     'name': 'Kathryn R. Sharp',\n",
       "     'url': 'https://www.semanticscholar.org/author/6108551'},\n",
       "    {'authorId': '14498481',\n",
       "     'name': 'Jonathan Hewitt',\n",
       "     'url': 'https://www.semanticscholar.org/author/14498481'}],\n",
       "   'doi': '10.1016/j.neubiorev.2014.09.009',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': True,\n",
       "   'paperId': '147ab9d1682d3d342b873873e961d45c7ffc4f4b',\n",
       "   'title': \"Dance as an intervention for people with Parkinson's disease: A systematic review and meta-analysis\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/147ab9d1682d3d342b873873e961d45c7ffc4f4b',\n",
       "   'venue': 'Neuroscience & Biobehavioral Reviews',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6657129',\n",
       "     'name': 'Cees J. T. de Goede',\n",
       "     'url': 'https://www.semanticscholar.org/author/6657129'},\n",
       "    {'authorId': '144987636',\n",
       "     'name': 'Samyra H J Keus',\n",
       "     'url': 'https://www.semanticscholar.org/author/144987636'},\n",
       "    {'authorId': '3409412',\n",
       "     'name': 'Gert Kwakkel',\n",
       "     'url': 'https://www.semanticscholar.org/author/3409412'},\n",
       "    {'authorId': '8991214',\n",
       "     'name': 'Robert C. Wagenaar',\n",
       "     'url': 'https://www.semanticscholar.org/author/8991214'}],\n",
       "   'doi': '10.1053/apmr.2001.22352',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '6af812a94f6467619dc06c10c1377406d11f5c31',\n",
       "   'title': \"The effects of physical therapy in Parkinson's disease: a research synthesis.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/6af812a94f6467619dc06c10c1377406d11f5c31',\n",
       "   'venue': 'Archives of physical medicine and rehabilitation',\n",
       "   'year': 2001},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '7547066',\n",
       "     'name': 'Toni M. Cutson',\n",
       "     'url': 'https://www.semanticscholar.org/author/7547066'},\n",
       "    {'authorId': '46497567',\n",
       "     'name': 'K C Laub',\n",
       "     'url': 'https://www.semanticscholar.org/author/46497567'},\n",
       "    {'authorId': '4110873',\n",
       "     'name': 'Margaret Schenkman',\n",
       "     'url': 'https://www.semanticscholar.org/author/4110873'}],\n",
       "   'doi': '10.1093/ptj/75.5.363',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f10be548eefd8ec19d3ffe4487bddc1e625cb104',\n",
       "   'title': \"Pharmacological and nonpharmacological interventions in the treatment of Parkinson's disease.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/f10be548eefd8ec19d3ffe4487bddc1e625cb104',\n",
       "   'venue': 'Physical therapy',\n",
       "   'year': 1995},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '34344513',\n",
       "     'name': 'Myeong Soo Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/34344513'},\n",
       "    {'authorId': '1947095',\n",
       "     'name': 'Paul Tc Lam',\n",
       "     'url': 'https://www.semanticscholar.org/author/1947095'},\n",
       "    {'authorId': '1846334',\n",
       "     'name': 'Edzard Ernst',\n",
       "     'url': 'https://www.semanticscholar.org/author/1846334'}],\n",
       "   'doi': '10.1016/j.parkreldis.2008.02.003',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'cc27d9bde411bac9648222f1690b5e9c9b9a3b96',\n",
       "   'title': \"Effectiveness of tai chi for Parkinson's disease: a critical review.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/cc27d9bde411bac9648222f1690b5e9c9b9a3b96',\n",
       "   'venue': 'Parkinsonism & related disorders',\n",
       "   'year': 2008},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '35032848',\n",
       "     'name': 'Lisa M. Shulman',\n",
       "     'url': 'https://www.semanticscholar.org/author/35032848'},\n",
       "    {'authorId': '1398016841',\n",
       "     'name': 'Ann L. Gruber-Baldini',\n",
       "     'url': 'https://www.semanticscholar.org/author/1398016841'},\n",
       "    {'authorId': '145528142',\n",
       "     'name': 'Karen E Anderson',\n",
       "     'url': 'https://www.semanticscholar.org/author/145528142'},\n",
       "    {'authorId': '2884077',\n",
       "     'name': 'Paul S. Fishman',\n",
       "     'url': 'https://www.semanticscholar.org/author/2884077'},\n",
       "    {'authorId': '34862999',\n",
       "     'name': 'Stephen G. Reich',\n",
       "     'url': 'https://www.semanticscholar.org/author/34862999'},\n",
       "    {'authorId': '3470022',\n",
       "     'name': 'William J. Weiner',\n",
       "     'url': 'https://www.semanticscholar.org/author/3470022'}],\n",
       "   'doi': '10.1001/archneurol.2009.295',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '56c20fadc0d67a29fbaae94f2e069c883b232d3e',\n",
       "   'title': \"The clinically important difference on the unified Parkinson's disease rating scale.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/56c20fadc0d67a29fbaae94f2e069c883b232d3e',\n",
       "   'venue': 'Archives of neurology',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '35136090',\n",
       "     'name': 'Louis C. S. Tan',\n",
       "     'url': 'https://www.semanticscholar.org/author/35136090'},\n",
       "    {'authorId': '122917549',\n",
       "     'name': 'Puay-Ngoh Lau',\n",
       "     'url': 'https://www.semanticscholar.org/author/122917549'},\n",
       "    {'authorId': '3915766',\n",
       "     'name': 'Roland Dominic G Jamora',\n",
       "     'url': 'https://www.semanticscholar.org/author/3915766'},\n",
       "    {'authorId': '46774337',\n",
       "     'name': 'Edwin Shih-Yen Chan',\n",
       "     'url': 'https://www.semanticscholar.org/author/46774337'}],\n",
       "   'doi': '10.1002/mds.20662',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'db4bf17af745e612d6885d0e7ecc63478765dcc1',\n",
       "   'title': \"Use of complementary therapies in patients with Parkinson's disease in Singapore.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/db4bf17af745e612d6885d0e7ecc63478765dcc1',\n",
       "   'venue': 'Movement disorders : official journal of the Movement Disorder Society',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '34344513',\n",
       "     'name': 'Myeong Soo Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/34344513'},\n",
       "    {'authorId': '6701610',\n",
       "     'name': 'Max H. Pittler',\n",
       "     'url': 'https://www.semanticscholar.org/author/6701610'},\n",
       "    {'authorId': '1397985004',\n",
       "     'name': 'Ruth E Taylor-Piliae',\n",
       "     'url': 'https://www.semanticscholar.org/author/1397985004'},\n",
       "    {'authorId': '47420535',\n",
       "     'name': 'Edzard Ernst',\n",
       "     'url': 'https://www.semanticscholar.org/author/47420535'}],\n",
       "   'doi': '10.1097/HJH.0b013e32828cc8cd',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '2e0a644ccd9731278def1b888c83cd8346de456a',\n",
       "   'title': 'Tai chi for cardiovascular disease and its risk factors: a systematic review.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/2e0a644ccd9731278def1b888c83cd8346de456a',\n",
       "   'venue': 'Journal of hypertension',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '49640122',\n",
       "     'name': 'Ji Hua Zhou',\n",
       "     'url': 'https://www.semanticscholar.org/author/49640122'},\n",
       "    {'authorId': '143849254',\n",
       "     'name': 'Tao Yin',\n",
       "     'url': 'https://www.semanticscholar.org/author/143849254'},\n",
       "    {'authorId': '1701997',\n",
       "     'name': 'Qian Gao',\n",
       "     'url': 'https://www.semanticscholar.org/author/1701997'},\n",
       "    {'authorId': '4062478',\n",
       "     'name': 'Xiao Cun Yang',\n",
       "     'url': 'https://www.semanticscholar.org/author/4062478'}],\n",
       "   'doi': '10.1155/2015/593263',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '1f29c6efb6d82c573fba4f18fe8c45830a638526',\n",
       "   'title': \"A Meta-Analysis on the Efficacy of Tai Chi in Patients with Parkinson's Disease between 2008 and 2014\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/1f29c6efb6d82c573fba4f18fe8c45830a638526',\n",
       "   'venue': 'Evidence-based complementary and alternative medicine : eCAM',\n",
       "   'year': 2015},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '121059987',\n",
       "     'name': 'A G de Boer',\n",
       "     'url': 'https://www.semanticscholar.org/author/121059987'},\n",
       "    {'authorId': '7370843',\n",
       "     'name': 'W. Wijker',\n",
       "     'url': 'https://www.semanticscholar.org/author/7370843'},\n",
       "    {'authorId': '2085111',\n",
       "     'name': 'Johannes D. Speelman',\n",
       "     'url': 'https://www.semanticscholar.org/author/2085111'},\n",
       "    {'authorId': '5140949',\n",
       "     'name': 'J. C. de Haes',\n",
       "     'url': 'https://www.semanticscholar.org/author/5140949'}],\n",
       "   'doi': '10.1136/jnnp.61.1.70',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '3765abbfc62fb628dd79d314aef7c5decb98f222',\n",
       "   'title': \"Quality of life in patients with Parkinson's disease: development of a questionnaire.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/3765abbfc62fb628dd79d314aef7c5decb98f222',\n",
       "   'venue': 'Journal of neurology, neurosurgery, and psychiatry',\n",
       "   'year': 1996},\n",
       "  {'arxivId': None,\n",
       "   'authors': [],\n",
       "   'doi': '10.1002/mds.10473',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '6e7888de30397d96c6db01e659e4407c8f33fb11',\n",
       "   'title': \"The Unified Parkinson's Disease Rating Scale (UPDRS): status and recommendations.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/6e7888de30397d96c6db01e659e4407c8f33fb11',\n",
       "   'venue': 'Movement disorders : official journal of the Movement Disorder Society',\n",
       "   'year': 2003},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '143847364',\n",
       "     'name': 'Carl E Clarke',\n",
       "     'url': 'https://www.semanticscholar.org/author/143847364'},\n",
       "    {'authorId': '153725851',\n",
       "     'name': 'A Peter Moore',\n",
       "     'url': 'https://www.semanticscholar.org/author/153725851'}],\n",
       "   'doi': '10.4135/9781412950510.n613',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c4e0c83f2437ae233b5b1321431bbc64845d2a68',\n",
       "   'title': \"Parkinson's disease.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/c4e0c83f2437ae233b5b1321431bbc64845d2a68',\n",
       "   'venue': 'Clinical evidence',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6530668',\n",
       "     'name': 'K Berg',\n",
       "     'url': 'https://www.semanticscholar.org/author/6530668'},\n",
       "    {'authorId': '1411962774',\n",
       "     'name': 'Sharon Wood-Dauphinée',\n",
       "     'url': 'https://www.semanticscholar.org/author/1411962774'},\n",
       "    {'authorId': '122472664',\n",
       "     'name': 'Joyce I Williams',\n",
       "     'url': 'https://www.semanticscholar.org/author/122472664'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9c538e37ed828e87beb83e897254baa465484ea7',\n",
       "   'title': 'The Balance Scale: reliability assessment with elderly residents and patients with an acute stroke.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9c538e37ed828e87beb83e897254baa465484ea7',\n",
       "   'venue': 'Scandinavian journal of rehabilitation medicine',\n",
       "   'year': 1995},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47735071',\n",
       "     'name': 'Sara Houston',\n",
       "     'url': 'https://www.semanticscholar.org/author/47735071'},\n",
       "    {'authorId': '46832735',\n",
       "     'name': 'Ashley McGill',\n",
       "     'url': 'https://www.semanticscholar.org/author/46832735'}],\n",
       "   'doi': '10.1080/17533015.2012.745580',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '74a6bb2420669df1ebd6f535da0c2b1e83fe6a81',\n",
       "   'title': \"A mixed-methods study into ballet for people living with Parkinson's\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/74a6bb2420669df1ebd6f535da0c2b1e83fe6a81',\n",
       "   'venue': '',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144416840',\n",
       "     'name': 'Steven L. Wolf',\n",
       "     'url': 'https://www.semanticscholar.org/author/144416840'},\n",
       "    {'authorId': '12261364',\n",
       "     'name': 'H X Barnhart',\n",
       "     'url': 'https://www.semanticscholar.org/author/12261364'},\n",
       "    {'authorId': '145564732',\n",
       "     'name': 'Nancy G. Kutner',\n",
       "     'url': 'https://www.semanticscholar.org/author/145564732'},\n",
       "    {'authorId': '3514554',\n",
       "     'name': 'Eileen McNeely',\n",
       "     'url': 'https://www.semanticscholar.org/author/3514554'},\n",
       "    {'authorId': '48402571',\n",
       "     'name': 'Carol Coogler',\n",
       "     'url': 'https://www.semanticscholar.org/author/48402571'},\n",
       "    {'authorId': '47361911',\n",
       "     'name': 'Tingsen Xu',\n",
       "     'url': 'https://www.semanticscholar.org/author/47361911'}],\n",
       "   'doi': '10.1111/j.1532-5415.1996.tb01432.x',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '6e374661be63cd5201d0231e67e5e889f231c868',\n",
       "   'title': 'Reducing frailty and falls in older persons: an investigation of Tai Chi and computerized balance training. Atlanta FICSIT Group. Frailty and Injuries: Cooperative Studies of Intervention Techniques.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/6e374661be63cd5201d0231e67e5e889f231c868',\n",
       "   'venue': 'Journal of the American Geriatrics Society',\n",
       "   'year': 1996},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '49393913',\n",
       "     'name': 'Debora Burini',\n",
       "     'url': 'https://www.semanticscholar.org/author/49393913'},\n",
       "    {'authorId': '9961323',\n",
       "     'name': 'Bruno Farabollini',\n",
       "     'url': 'https://www.semanticscholar.org/author/9961323'},\n",
       "    {'authorId': '15462530',\n",
       "     'name': 'S Iacucci',\n",
       "     'url': 'https://www.semanticscholar.org/author/15462530'},\n",
       "    {'authorId': '51918417',\n",
       "     'name': 'C Rimatori',\n",
       "     'url': 'https://www.semanticscholar.org/author/51918417'},\n",
       "    {'authorId': '144662429',\n",
       "     'name': 'Gabriele Riccardi',\n",
       "     'url': 'https://www.semanticscholar.org/author/144662429'},\n",
       "    {'authorId': '3115269',\n",
       "     'name': 'Marianna Capecci',\n",
       "     'url': 'https://www.semanticscholar.org/author/3115269'},\n",
       "    {'authorId': '2688959',\n",
       "     'name': 'Leandro Provinciali',\n",
       "     'url': 'https://www.semanticscholar.org/author/2688959'},\n",
       "    {'authorId': '50440559',\n",
       "     'name': 'Maria Grazia Ceravolo',\n",
       "     'url': 'https://www.semanticscholar.org/author/50440559'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4045324565054bb80536ff522bed1dfec1447453',\n",
       "   'title': \"A randomised controlled cross-over trial of aerobic training versus Qigong in advanced Parkinson's disease.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/4045324565054bb80536ff522bed1dfec1447453',\n",
       "   'venue': 'Europa medicophysica',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '8217038',\n",
       "     'name': 'L. Heiberger',\n",
       "     'url': 'https://www.semanticscholar.org/author/8217038'},\n",
       "    {'authorId': '144049125',\n",
       "     'name': 'Christoph Maurer',\n",
       "     'url': 'https://www.semanticscholar.org/author/144049125'},\n",
       "    {'authorId': '3864299',\n",
       "     'name': 'Florian Amtage',\n",
       "     'url': 'https://www.semanticscholar.org/author/3864299'},\n",
       "    {'authorId': '1414082484',\n",
       "     'name': 'Ignacio Méndez-Balbuena',\n",
       "     'url': 'https://www.semanticscholar.org/author/1414082484'},\n",
       "    {'authorId': '1399376820',\n",
       "     'name': 'Jürgen Schulte-Moenting',\n",
       "     'url': 'https://www.semanticscholar.org/author/1399376820'},\n",
       "    {'authorId': '1398389123',\n",
       "     'name': 'Marie-Claude Hepp-Reymond',\n",
       "     'url': 'https://www.semanticscholar.org/author/1398389123'},\n",
       "    {'authorId': '48268009',\n",
       "     'name': 'Rumyana Kristeva',\n",
       "     'url': 'https://www.semanticscholar.org/author/48268009'}],\n",
       "   'doi': '10.3389/fnagi.2011.00014',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'd08abaa6fdc8f4bc9b68a994f7a9f92b6b2aacb6',\n",
       "   'title': 'Impact of a Weekly Dance Class on the Functional Mobility and on the Quality of Life of Individuals with Parkinson’s Disease',\n",
       "   'url': 'https://www.semanticscholar.org/paper/d08abaa6fdc8f4bc9b68a994f7a9f92b6b2aacb6',\n",
       "   'venue': 'Front. Ag. Neurosci.',\n",
       "   'year': 2011},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '40448483',\n",
       "     'name': 'Talia Herman',\n",
       "     'url': 'https://www.semanticscholar.org/author/40448483'},\n",
       "    {'authorId': '144701716',\n",
       "     'name': 'Nir Giladi',\n",
       "     'url': 'https://www.semanticscholar.org/author/144701716'},\n",
       "    {'authorId': '4108504',\n",
       "     'name': 'Leor Gruendlinger',\n",
       "     'url': 'https://www.semanticscholar.org/author/4108504'},\n",
       "    {'authorId': '7766661',\n",
       "     'name': 'Jeffrey M. Hausdorff',\n",
       "     'url': 'https://www.semanticscholar.org/author/7766661'}],\n",
       "   'doi': '10.1016/j.apmr.2007.05.015',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '78c53b2616398412673515f586f356b04e8025fa',\n",
       "   'title': \"Six weeks of intensive treadmill training improves gait and quality of life in patients with Parkinson's disease: a pilot study.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/78c53b2616398412673515f586f356b04e8025fa',\n",
       "   'venue': 'Archives of physical medicine and rehabilitation',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47190591',\n",
       "     'name': 'Margaret M. Hoehn',\n",
       "     'url': 'https://www.semanticscholar.org/author/47190591'},\n",
       "    {'authorId': '4144556',\n",
       "     'name': 'Melvin D. Yahr',\n",
       "     'url': 'https://www.semanticscholar.org/author/4144556'}],\n",
       "   'doi': '10.1212/WNL.17.5.427',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9c505fdd165d4220d15a96c86debab59bc775dbb',\n",
       "   'title': 'Parkinsonism: onset, progression and mortality.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9c505fdd165d4220d15a96c86debab59bc775dbb',\n",
       "   'venue': 'Neurology',\n",
       "   'year': 1967},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5863706',\n",
       "     'name': 'Xiaojia Ni',\n",
       "     'url': 'https://www.semanticscholar.org/author/5863706'},\n",
       "    {'authorId': '5078004',\n",
       "     'name': 'Shaonan Liu',\n",
       "     'url': 'https://www.semanticscholar.org/author/5078004'},\n",
       "    {'authorId': '5648477',\n",
       "     'name': 'Fuchang Lu',\n",
       "     'url': 'https://www.semanticscholar.org/author/5648477'},\n",
       "    {'authorId': '6345553',\n",
       "     'name': 'Xiaogeng Shi',\n",
       "     'url': 'https://www.semanticscholar.org/author/6345553'},\n",
       "    {'authorId': '8395339',\n",
       "     'name': 'Xinfeng Guo',\n",
       "     'url': 'https://www.semanticscholar.org/author/8395339'}],\n",
       "   'doi': '10.1371/journal.pone.0099377',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'aabcdc6120af76136fa5766454c0dc9a33a6d273',\n",
       "   'title': \"Efficacy and Safety of Tai Chi for Parkinson's Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/aabcdc6120af76136fa5766454c0dc9a33a6d273',\n",
       "   'venue': 'PloS one',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5666886',\n",
       "     'name': 'Marshal F. Folstein',\n",
       "     'url': 'https://www.semanticscholar.org/author/5666886'},\n",
       "    {'authorId': '5428872',\n",
       "     'name': 'Susan E B Folstein',\n",
       "     'url': 'https://www.semanticscholar.org/author/5428872'},\n",
       "    {'authorId': '2063285',\n",
       "     'name': 'Paul R. McHugh',\n",
       "     'url': 'https://www.semanticscholar.org/author/2063285'}],\n",
       "   'doi': '10.1016/0022-3956(75)90026-6',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'cf1d949e5197753c304e2e57ef83d51543eb176f',\n",
       "   'title': '\"Mini-mental state\". A practical method for grading the cognitive state of patients for the clinician.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/cf1d949e5197753c304e2e57ef83d51543eb176f',\n",
       "   'venue': 'Journal of psychiatric research',\n",
       "   'year': 1975},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50103617',\n",
       "     'name': 'Shibley Rahman',\n",
       "     'url': 'https://www.semanticscholar.org/author/50103617'},\n",
       "    {'authorId': '2940869',\n",
       "     'name': 'Harry J. Griffin',\n",
       "     'url': 'https://www.semanticscholar.org/author/2940869'},\n",
       "    {'authorId': '3256817',\n",
       "     'name': 'Niall Patrick Quinn',\n",
       "     'url': 'https://www.semanticscholar.org/author/3256817'},\n",
       "    {'authorId': '144568109',\n",
       "     'name': 'Marjan Jahanshahi',\n",
       "     'url': 'https://www.semanticscholar.org/author/144568109'}],\n",
       "   'doi': '10.1002/mds.21667',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f4cc8d857d1c711db9e308d344694bffdadd1489',\n",
       "   'title': \"Quality of life in Parkinson's disease: the relative importance of the symptoms.\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/f4cc8d857d1c711db9e308d344694bffdadd1489',\n",
       "   'venue': 'Movement disorders : official journal of the Movement Disorder Society',\n",
       "   'year': 2008}],\n",
       " 'title': \"Turo (Qi Dance) Program for Parkinson's Disease Patients: Randomized, Assessor Blind, Waiting‐List Control, Partial Crossover Study\",\n",
       " 'topics': [{'topic': 'Parkinson Disease',\n",
       "   'topicId': '3556',\n",
       "   'url': 'https://www.semanticscholar.org/topic/3556'},\n",
       "  {'topic': 'Parkinsonian Disorders',\n",
       "   'topicId': '13136',\n",
       "   'url': 'https://www.semanticscholar.org/topic/13136'},\n",
       "  {'topic': 'Tai Ji',\n",
       "   'topicId': '1075',\n",
       "   'url': 'https://www.semanticscholar.org/topic/1075'},\n",
       "  {'topic': 'Beck depression inventory',\n",
       "   'topicId': '24617',\n",
       "   'url': 'https://www.semanticscholar.org/topic/24617'},\n",
       "  {'topic': 'Training Programs',\n",
       "   'topicId': '14616',\n",
       "   'url': 'https://www.semanticscholar.org/topic/14616'},\n",
       "  {'topic': 'Visually Impaired Persons',\n",
       "   'topicId': '471815',\n",
       "   'url': 'https://www.semanticscholar.org/topic/471815'},\n",
       "  {'topic': 'Equilibration disorder',\n",
       "   'topicId': '94954',\n",
       "   'url': 'https://www.semanticscholar.org/topic/94954'},\n",
       "  {'topic': 'Qigong',\n",
       "   'topicId': '1065',\n",
       "   'url': 'https://www.semanticscholar.org/topic/1065'},\n",
       "  {'topic': \"Unified Parkinson's Disease Rating Scale Questionnaire\",\n",
       "   'topicId': '109571',\n",
       "   'url': 'https://www.semanticscholar.org/topic/109571'},\n",
       "  {'topic': 'Patients',\n",
       "   'topicId': '153409',\n",
       "   'url': 'https://www.semanticscholar.org/topic/153409'},\n",
       "  {'topic': 'PersonNameUse - assigned',\n",
       "   'topicId': '9798943',\n",
       "   'url': 'https://www.semanticscholar.org/topic/9798943'}],\n",
       " 'url': 'https://www.semanticscholar.org/paper/ba02eb4199691ddf8974a5ff8c776ea1b37fb2a0',\n",
       " 'venue': 'Explore',\n",
       " 'year': 2018}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'abstract': 'Cancer constitutes the major health problem both in human and veterinary medicine. Comparative oncology as an integrative approach offers to learn more about naturally occurring cancers across different species. Canine models have many advantages as they experience spontaneous disease, have many genes similar to human genes, five to seven-fold accelerated ageing compared to humans, respond to treatments similarly as humans do and health care levels second only to humans. Also, the clinical trials in canines could generate more robust data, as their spontaneous nature mimics real-life situations and could be translated to humans.',\n",
       " 'arxivId': None,\n",
       " 'authors': [{'authorId': '47472141',\n",
       "   'name': 'Faheem Sultan',\n",
       "   'url': 'https://www.semanticscholar.org/author/47472141'},\n",
       "  {'authorId': '35458013',\n",
       "   'name': 'Bilal Ahmad Ganaie',\n",
       "   'url': 'https://www.semanticscholar.org/author/35458013'}],\n",
       " 'citationVelocity': 0,\n",
       " 'citations': [{'arxivId': None,\n",
       "   'authors': [{'authorId': '51886607',\n",
       "     'name': 'Chris Weir',\n",
       "     'url': 'https://www.semanticscholar.org/author/51886607'},\n",
       "    {'authorId': '51901969',\n",
       "     'name': 'Annika Oksa',\n",
       "     'url': 'https://www.semanticscholar.org/author/51901969'},\n",
       "    {'authorId': '1400940655',\n",
       "     'name': 'Jennifer Millar',\n",
       "     'url': 'https://www.semanticscholar.org/author/1400940655'},\n",
       "    {'authorId': '39092374',\n",
       "     'name': 'Miles J. Alexander',\n",
       "     'url': 'https://www.semanticscholar.org/author/39092374'},\n",
       "    {'authorId': '1402266841',\n",
       "     'name': 'Nicola Kynoch',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402266841'},\n",
       "    {'authorId': '1402266803',\n",
       "     'name': 'Zoe Walton-Weitz',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402266803'},\n",
       "    {'authorId': '1402266810',\n",
       "     'name': 'Peter Mackenzie-Wood',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402266810'},\n",
       "    {'authorId': '1402267665',\n",
       "     'name': 'Felicia Tam',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402267665'},\n",
       "    {'authorId': '83293160',\n",
       "     'name': 'H. Meredith Richards',\n",
       "     'url': 'https://www.semanticscholar.org/author/83293160'},\n",
       "    {'authorId': '52105642',\n",
       "     'name': 'Richard J. Naylor',\n",
       "     'url': 'https://www.semanticscholar.org/author/52105642'},\n",
       "    {'authorId': '1406015897',\n",
       "     'name': 'Katrina Cheng',\n",
       "     'url': 'https://www.semanticscholar.org/author/1406015897'},\n",
       "    {'authorId': '50249919',\n",
       "     'name': 'Peter F Bennett',\n",
       "     'url': 'https://www.semanticscholar.org/author/50249919'},\n",
       "    {'authorId': '144604624',\n",
       "     'name': 'Nikolai Petrovsky',\n",
       "     'url': 'https://www.semanticscholar.org/author/144604624'},\n",
       "    {'authorId': '7469391',\n",
       "     'name': 'R E Allavena',\n",
       "     'url': 'https://www.semanticscholar.org/author/7469391'}],\n",
       "   'doi': '10.3390/vetsci5040087',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e2cef67ca136f4b301736031caac1aa1085abcb6',\n",
       "   'title': 'The Safety of an Adjuvanted Autologous Cancer Vaccine Platform in Canine Cancer Patients',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e2cef67ca136f4b301736031caac1aa1085abcb6',\n",
       "   'venue': 'Veterinary sciences',\n",
       "   'year': 2018},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '134112175',\n",
       "     'name': 'Ieva Palubeckaitė',\n",
       "     'url': 'https://www.semanticscholar.org/author/134112175'}],\n",
       "   'doi': '10.7190/shu-thesis-00179',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'af5575e50574f454386d0240ca10dc8eba60c85e',\n",
       "   'title': 'Analysis of three dimensional cell cultures using mass spectrometry imaging',\n",
       "   'url': 'https://www.semanticscholar.org/paper/af5575e50574f454386d0240ca10dc8eba60c85e',\n",
       "   'venue': '',\n",
       "   'year': 2018},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1402168967',\n",
       "     'name': 'Munsef A. Al-Mansour',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402168967'},\n",
       "    {'authorId': '47244968',\n",
       "     'name': 'Mahir A.G. Kubba',\n",
       "     'url': 'https://www.semanticscholar.org/author/47244968'},\n",
       "    {'authorId': '1402168793',\n",
       "     'name': 'Seham A. Al-Azreg',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402168793'},\n",
       "    {'authorId': '1402168756',\n",
       "     'name': 'Sausen A. Dribika',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402168756'}],\n",
       "   'doi': '10.4314/ovj.v8i3.3',\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f26108ccef81998684ffb68173af954abe1b494f',\n",
       "   'title': 'Comparative histopathology and immunohistochemistry of human and canine mammary tumors',\n",
       "   'url': 'https://www.semanticscholar.org/paper/f26108ccef81998684ffb68173af954abe1b494f',\n",
       "   'venue': 'Open veterinary journal',\n",
       "   'year': 2018},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1581904822',\n",
       "     'name': 'João Manuel Ribeiro Jordão',\n",
       "     'url': 'https://www.semanticscholar.org/author/1581904822'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4cc1ac02646c8091bbd5774bec7b5100af85de27',\n",
       "   'title': 'Raman Spectroscopy in tumoral tissues',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4cc1ac02646c8091bbd5774bec7b5100af85de27',\n",
       "   'venue': '',\n",
       "   'year': 2019}],\n",
       " 'corpusId': 3365224,\n",
       " 'doi': '10.4314/ovj.v8i1.5',\n",
       " 'fieldsOfStudy': ['Medicine'],\n",
       " 'influentialCitationCount': 0,\n",
       " 'is_open_access': True,\n",
       " 'is_publisher_licensed': True,\n",
       " 'paperId': 'dd81361979e154e6d2ecfda702436bbd1ab8c0b4',\n",
       " 'references': [{'arxivId': None,\n",
       "   'authors': [{'authorId': '1843138',\n",
       "     'name': 'Konstantin Strauch',\n",
       "     'url': 'https://www.semanticscholar.org/author/1843138'},\n",
       "    {'authorId': '6491150',\n",
       "     'name': 'Rolf Fimmers',\n",
       "     'url': 'https://www.semanticscholar.org/author/6491150'},\n",
       "    {'authorId': '38541662',\n",
       "     'name': 'Max P. Baur',\n",
       "     'url': 'https://www.semanticscholar.org/author/38541662'},\n",
       "    {'authorId': '2661355',\n",
       "     'name': 'Thomas F. Wienker',\n",
       "     'url': 'https://www.semanticscholar.org/author/2661355'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '899c45cc28cbeb230df7229f8cee5535e5efbad2',\n",
       "   'title': 'How to model a complex trait. 1. General considerations and suggestions.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/899c45cc28cbeb230df7229f8cee5535e5efbad2',\n",
       "   'venue': 'Human heredity',\n",
       "   'year': 2003},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50439366',\n",
       "     'name': 'Deli Liu',\n",
       "     'url': 'https://www.semanticscholar.org/author/50439366'},\n",
       "    {'authorId': '50275931',\n",
       "     'name': 'Huan Xiong',\n",
       "     'url': 'https://www.semanticscholar.org/author/50275931'},\n",
       "    {'authorId': '34299151',\n",
       "     'name': 'Angela E. Ellis',\n",
       "     'url': 'https://www.semanticscholar.org/author/34299151'},\n",
       "    {'authorId': '52228748',\n",
       "     'name': 'N Northrup',\n",
       "     'url': 'https://www.semanticscholar.org/author/52228748'},\n",
       "    {'authorId': '144901219',\n",
       "     'name': 'Carlos O. Rodriguez',\n",
       "     'url': 'https://www.semanticscholar.org/author/144901219'},\n",
       "    {'authorId': '1398108932',\n",
       "     'name': \"Ruth M O'Regan\",\n",
       "     'url': 'https://www.semanticscholar.org/author/1398108932'},\n",
       "    {'authorId': '144806331',\n",
       "     'name': 'Stephen Dalton',\n",
       "     'url': 'https://www.semanticscholar.org/author/144806331'},\n",
       "    {'authorId': '50623874',\n",
       "     'name': 'Shaying Zhao',\n",
       "     'url': 'https://www.semanticscholar.org/author/50623874'}],\n",
       "   'doi': '10.1158/0008-5472.CAN-14-0392',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '5ecba90c9e9385b390537698bb62a64b39ecdad3',\n",
       "   'title': 'Molecular homology and difference between spontaneous canine mammary cancer and human breast cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/5ecba90c9e9385b390537698bb62a64b39ecdad3',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144438404',\n",
       "     'name': 'Takashi Higuchi',\n",
       "     'url': 'https://www.semanticscholar.org/author/144438404'},\n",
       "    {'authorId': '4911811',\n",
       "     'name': 'Grant N. Burcham',\n",
       "     'url': 'https://www.semanticscholar.org/author/4911811'},\n",
       "    {'authorId': '144149927',\n",
       "     'name': 'M O Childress',\n",
       "     'url': 'https://www.semanticscholar.org/author/144149927'},\n",
       "    {'authorId': '14405668',\n",
       "     'name': 'Jacob J Rohleder',\n",
       "     'url': 'https://www.semanticscholar.org/author/14405668'},\n",
       "    {'authorId': '6766884',\n",
       "     'name': 'Patty L. Bonney',\n",
       "     'url': 'https://www.semanticscholar.org/author/6766884'},\n",
       "    {'authorId': '1400752997',\n",
       "     'name': 'José Antonio Ramos-Vara',\n",
       "     'url': 'https://www.semanticscholar.org/author/1400752997'},\n",
       "    {'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'}],\n",
       "   'doi': '10.2460/javma.242.4.499',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '0ee0a804c0c2a359db16fa5d8ab5e0246b49e28a',\n",
       "   'title': 'Characterization and treatment of transitional cell carcinoma of the abdominal wall in dogs: 24 cases (1985-2010).',\n",
       "   'url': 'https://www.semanticscholar.org/paper/0ee0a804c0c2a359db16fa5d8ab5e0246b49e28a',\n",
       "   'venue': 'Journal of the American Veterinary Medical Association',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6842040',\n",
       "     'name': 'Frances R. Balkwill',\n",
       "     'url': 'https://www.semanticscholar.org/author/6842040'},\n",
       "    {'authorId': '47351682',\n",
       "     'name': 'Kellie A. Charles',\n",
       "     'url': 'https://www.semanticscholar.org/author/47351682'},\n",
       "    {'authorId': '143679013',\n",
       "     'name': 'Alberto Mantovani',\n",
       "     'url': 'https://www.semanticscholar.org/author/143679013'}],\n",
       "   'doi': '10.1016/j.ccr.2005.02.013',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '2f16874fd78cd3c46a441c28f200e500738af840',\n",
       "   'title': 'Smoldering and polarized inflammation in the initiation and promotion of malignant disease.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/2f16874fd78cd3c46a441c28f200e500738af840',\n",
       "   'venue': 'Cancer cell',\n",
       "   'year': 2005},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3496667',\n",
       "     'name': 'Lisa Mirabello',\n",
       "     'url': 'https://www.semanticscholar.org/author/3496667'},\n",
       "    {'authorId': '35104252',\n",
       "     'name': 'Rebecca Troisi',\n",
       "     'url': 'https://www.semanticscholar.org/author/35104252'},\n",
       "    {'authorId': '2114991',\n",
       "     'name': 'Sharon A Savage',\n",
       "     'url': 'https://www.semanticscholar.org/author/2114991'}],\n",
       "   'doi': '10.1002/cncr.24121',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '27e47e274fcd0d19f2943b3f73db3e04bce964b4',\n",
       "   'title': 'Osteosarcoma incidence and survival rates from 1973 to 2004: data from the Surveillance, Epidemiology, and End Results Program.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/27e47e274fcd0d19f2943b3f73db3e04bce964b4',\n",
       "   'venue': 'Cancer',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144299056',\n",
       "     'name': 'E. Gregory Macewen',\n",
       "     'url': 'https://www.semanticscholar.org/author/144299056'},\n",
       "    {'authorId': '35133226',\n",
       "     'name': 'Robert C. Rosenthal',\n",
       "     'url': 'https://www.semanticscholar.org/author/35133226'},\n",
       "    {'authorId': '6477667',\n",
       "     'name': 'Leslie E. Fox',\n",
       "     'url': 'https://www.semanticscholar.org/author/6477667'},\n",
       "    {'authorId': '9830184',\n",
       "     'name': 'Aimee Loar',\n",
       "     'url': 'https://www.semanticscholar.org/author/9830184'},\n",
       "    {'authorId': '6375292',\n",
       "     'name': 'Ilene D. Kurzman',\n",
       "     'url': 'https://www.semanticscholar.org/author/6375292'}],\n",
       "   'doi': '10.1111/j.1939-1676.1992.tb00344.x',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c79ed63b7cea901e16bf858368aca392ca7ef406',\n",
       "   'title': 'Evaluation of L-asparaginase: polyethylene glycol conjugate versus native L-asparaginase combined with chemotherapy. A randomized double-blind study in canine lymphoma.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/c79ed63b7cea901e16bf858368aca392ca7ef406',\n",
       "   'venue': 'Journal of veterinary internal medicine',\n",
       "   'year': 1992},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '114576550',\n",
       "     'name': 'Chi-Hau Chen',\n",
       "     'url': 'https://www.semanticscholar.org/author/114576550'},\n",
       "    {'authorId': '13211357',\n",
       "     'name': 'R J Chen',\n",
       "     'url': 'https://www.semanticscholar.org/author/13211357'}],\n",
       "   'doi': '10.1016/S0929-6646(11)60043-0',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '567311f4e08787160a053d73cbb5776821adad7a',\n",
       "   'title': 'Prevalence of telomerase activity in human cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/567311f4e08787160a053d73cbb5776821adad7a',\n",
       "   'venue': 'Journal of the Formosan Medical Association = Taiwan yi zhi',\n",
       "   'year': 2011},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144753610',\n",
       "     'name': 'Natalie Cook',\n",
       "     'url': 'https://www.semanticscholar.org/author/144753610'},\n",
       "    {'authorId': '6305419',\n",
       "     'name': 'Duncan I. Jodrell',\n",
       "     'url': 'https://www.semanticscholar.org/author/6305419'},\n",
       "    {'authorId': '4530161',\n",
       "     'name': 'David A Tuveson',\n",
       "     'url': 'https://www.semanticscholar.org/author/4530161'}],\n",
       "   'doi': '10.1016/j.drudis.2012.02.003',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '0e32db4aec39a1ea2e462515e06166dba62fb07e',\n",
       "   'title': 'Predictive in vivo animal models and translation to clinical trials.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/0e32db4aec39a1ea2e462515e06166dba62fb07e',\n",
       "   'venue': 'Drug discovery today',\n",
       "   'year': 2012},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '48638391',\n",
       "     'name': 'Ira K. Gordon',\n",
       "     'url': 'https://www.semanticscholar.org/author/48638391'},\n",
       "    {'authorId': '34588572',\n",
       "     'name': 'Melissa Paoloni',\n",
       "     'url': 'https://www.semanticscholar.org/author/34588572'},\n",
       "    {'authorId': '6777667',\n",
       "     'name': 'Christina Mazcko',\n",
       "     'url': 'https://www.semanticscholar.org/author/6777667'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1371/journal.pmed.1000161',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'a2be58059a4e48dff151eac444a92a92bf18e353',\n",
       "   'title': 'The Comparative Oncology Trials Consortium: Using Spontaneously Occurring Cancers in Dogs to Inform the Cancer Drug Development Pathway',\n",
       "   'url': 'https://www.semanticscholar.org/paper/a2be58059a4e48dff151eac444a92a92bf18e353',\n",
       "   'venue': 'PLoS medicine',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '35599476',\n",
       "     'name': 'Brian J Cummings',\n",
       "     'url': 'https://www.semanticscholar.org/author/35599476'},\n",
       "    {'authorId': '144530552',\n",
       "     'name': 'Elizabeth Head',\n",
       "     'url': 'https://www.semanticscholar.org/author/144530552'},\n",
       "    {'authorId': '47230954',\n",
       "     'name': 'William W. Ruehl',\n",
       "     'url': 'https://www.semanticscholar.org/author/47230954'},\n",
       "    {'authorId': '5733956',\n",
       "     'name': 'Norton W. Milgram',\n",
       "     'url': 'https://www.semanticscholar.org/author/5733956'},\n",
       "    {'authorId': '145975173',\n",
       "     'name': 'Carl Cotman',\n",
       "     'url': 'https://www.semanticscholar.org/author/145975173'}],\n",
       "   'doi': '10.1016/0197-4580(95)02060-8',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b449f90c30e8d0521511e1ec42fb1efeda62a376',\n",
       "   'title': 'The canine as an animal model of human aging and dementia',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b449f90c30e8d0521511e1ec42fb1efeda62a376',\n",
       "   'venue': 'Neurobiology of Aging',\n",
       "   'year': 1996},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1400499854',\n",
       "     'name': 'Jesús Aguirre-Hernández',\n",
       "     'url': 'https://www.semanticscholar.org/author/1400499854'},\n",
       "    {'authorId': '49492843',\n",
       "     'name': 'Bruce S Milne',\n",
       "     'url': 'https://www.semanticscholar.org/author/49492843'},\n",
       "    {'authorId': '39534306',\n",
       "     'name': 'Chris Queen',\n",
       "     'url': 'https://www.semanticscholar.org/author/39534306'},\n",
       "    {'authorId': '1400643392',\n",
       "     'name': \"Patricia C. O'brien\",\n",
       "     'url': 'https://www.semanticscholar.org/author/1400643392'},\n",
       "    {'authorId': '3864512',\n",
       "     'name': 'Tess Hoather',\n",
       "     'url': 'https://www.semanticscholar.org/author/3864512'},\n",
       "    {'authorId': '40601946',\n",
       "     'name': 'Sean Haugland',\n",
       "     'url': 'https://www.semanticscholar.org/author/40601946'},\n",
       "    {'authorId': '1392320176',\n",
       "     'name': 'Malcolm A. Ferguson-Smith',\n",
       "     'url': 'https://www.semanticscholar.org/author/1392320176'},\n",
       "    {'authorId': '9544333',\n",
       "     'name': 'Jane M. Dobson',\n",
       "     'url': 'https://www.semanticscholar.org/author/9544333'},\n",
       "    {'authorId': '4071251',\n",
       "     'name': 'David Sargan',\n",
       "     'url': 'https://www.semanticscholar.org/author/4071251'}],\n",
       "   'doi': '10.1186/1746-6148-5-27',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '23c7e0e34a6afc248b6dc92429184168f1eaad40',\n",
       "   'title': 'Disruption of chromosome 11 in canine fibrosarcomas highlights an unusual variability of CDKN2B in dogs',\n",
       "   'url': 'https://www.semanticscholar.org/paper/23c7e0e34a6afc248b6dc92429184168f1eaad40',\n",
       "   'venue': 'BMC veterinary research',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '143874433',\n",
       "     'name': 'Erik Teske',\n",
       "     'url': 'https://www.semanticscholar.org/author/143874433'},\n",
       "    {'authorId': '6445172',\n",
       "     'name': 'Gerard R. Rutteman',\n",
       "     'url': 'https://www.semanticscholar.org/author/6445172'},\n",
       "    {'authorId': '13028967',\n",
       "     'name': 'T S vd Ingh',\n",
       "     'url': 'https://www.semanticscholar.org/author/13028967'},\n",
       "    {'authorId': '12214359',\n",
       "     'name': 'Ronald van Noort',\n",
       "     'url': 'https://www.semanticscholar.org/author/12214359'},\n",
       "    {'authorId': '7656234',\n",
       "     'name': 'Wim Misdorp',\n",
       "     'url': 'https://www.semanticscholar.org/author/7656234'}],\n",
       "   'doi': None,\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c9ce5ee82b596ed34e2de22ef6cae78471c4b075',\n",
       "   'title': 'Liposome-encapsulated muramyl tripeptide phosphatidylethanolamine (L-MTP-PE): a randomized clinical trial in dogs with mammary carcinoma.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/c9ce5ee82b596ed34e2de22ef6cae78471c4b075',\n",
       "   'venue': 'Anticancer research',\n",
       "   'year': 1998},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5597148',\n",
       "     'name': 'Kenneth Paigen',\n",
       "     'url': 'https://www.semanticscholar.org/author/5597148'}],\n",
       "   'doi': '10.1038/nm0395-215',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c6d9471b5e9da590ca5842a6fd2313a6052f03e8',\n",
       "   'title': 'A miracle enough: the power of mice',\n",
       "   'url': 'https://www.semanticscholar.org/paper/c6d9471b5e9da590ca5842a6fd2313a6052f03e8',\n",
       "   'venue': 'Nature Medicine',\n",
       "   'year': 1995},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1382417671',\n",
       "     'name': 'Kerstin Lindblad-Toh',\n",
       "     'url': 'https://www.semanticscholar.org/author/1382417671'},\n",
       "    {'authorId': '2366861',\n",
       "     'name': 'Claire M. Wade',\n",
       "     'url': 'https://www.semanticscholar.org/author/2366861'},\n",
       "    {'authorId': '31586471',\n",
       "     'name': 'Tarjei S. Mikkelsen',\n",
       "     'url': 'https://www.semanticscholar.org/author/31586471'},\n",
       "    {'authorId': '7543407',\n",
       "     'name': 'Elinor K Karlsson',\n",
       "     'url': 'https://www.semanticscholar.org/author/7543407'},\n",
       "    {'authorId': '144094478',\n",
       "     'name': 'David B. Jaffe',\n",
       "     'url': 'https://www.semanticscholar.org/author/144094478'},\n",
       "    {'authorId': '21739725',\n",
       "     'name': 'Michael Kamal',\n",
       "     'url': 'https://www.semanticscholar.org/author/21739725'},\n",
       "    {'authorId': '2197104',\n",
       "     'name': 'Michele Clamp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2197104'},\n",
       "    {'authorId': '50319707',\n",
       "     'name': 'Jean L. Chang',\n",
       "     'url': 'https://www.semanticscholar.org/author/50319707'},\n",
       "    {'authorId': '3693855',\n",
       "     'name': 'Edward J. Kulbokas',\n",
       "     'url': 'https://www.semanticscholar.org/author/3693855'},\n",
       "    {'authorId': '2505257',\n",
       "     'name': 'Michael C. Zody',\n",
       "     'url': 'https://www.semanticscholar.org/author/2505257'},\n",
       "    {'authorId': '2675621',\n",
       "     'name': 'Evan Mauceli',\n",
       "     'url': 'https://www.semanticscholar.org/author/2675621'},\n",
       "    {'authorId': '47907087',\n",
       "     'name': 'Xiaohui Xie',\n",
       "     'url': 'https://www.semanticscholar.org/author/47907087'},\n",
       "    {'authorId': '153422181',\n",
       "     'name': 'Matthew Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/153422181'},\n",
       "    {'authorId': '144295924',\n",
       "     'name': 'Robert K Wayne',\n",
       "     'url': 'https://www.semanticscholar.org/author/144295924'},\n",
       "    {'authorId': '3252069',\n",
       "     'name': 'Elaine A. Ostrander',\n",
       "     'url': 'https://www.semanticscholar.org/author/3252069'},\n",
       "    {'authorId': '2500371',\n",
       "     'name': 'Chris P. Ponting',\n",
       "     'url': 'https://www.semanticscholar.org/author/2500371'},\n",
       "    {'authorId': '2261447',\n",
       "     'name': 'Francis Galibert',\n",
       "     'url': 'https://www.semanticscholar.org/author/2261447'},\n",
       "    {'authorId': '47387088',\n",
       "     'name': 'Douglas R. Smith',\n",
       "     'url': 'https://www.semanticscholar.org/author/47387088'},\n",
       "    {'authorId': '144580644',\n",
       "     'name': 'Pieter J. Dejong',\n",
       "     'url': 'https://www.semanticscholar.org/author/144580644'},\n",
       "    {'authorId': '3088257',\n",
       "     'name': 'Ewen F. Kirkness',\n",
       "     'url': 'https://www.semanticscholar.org/author/3088257'},\n",
       "    {'authorId': '152599359',\n",
       "     'name': 'Pablo Álvarez',\n",
       "     'url': 'https://www.semanticscholar.org/author/152599359'},\n",
       "    {'authorId': '47364238',\n",
       "     'name': 'Tara Biagi',\n",
       "     'url': 'https://www.semanticscholar.org/author/47364238'},\n",
       "    {'authorId': '2557185',\n",
       "     'name': 'William W. Brockman',\n",
       "     'url': 'https://www.semanticscholar.org/author/2557185'},\n",
       "    {'authorId': '151043980',\n",
       "     'name': 'Jonathan Butler',\n",
       "     'url': 'https://www.semanticscholar.org/author/151043980'},\n",
       "    {'authorId': '1402558866',\n",
       "     'name': 'Chee-Wye Chin',\n",
       "     'url': 'https://www.semanticscholar.org/author/1402558866'},\n",
       "    {'authorId': '152144443',\n",
       "     'name': 'A. Cook',\n",
       "     'url': 'https://www.semanticscholar.org/author/152144443'},\n",
       "    {'authorId': '3068396',\n",
       "     'name': 'James Cuff',\n",
       "     'url': 'https://www.semanticscholar.org/author/3068396'},\n",
       "    {'authorId': '144524355',\n",
       "     'name': 'Mark J. Daly',\n",
       "     'url': 'https://www.semanticscholar.org/author/144524355'},\n",
       "    {'authorId': '1973239',\n",
       "     'name': 'David DeCaprio',\n",
       "     'url': 'https://www.semanticscholar.org/author/1973239'},\n",
       "    {'authorId': '5441351',\n",
       "     'name': 'Sante Gnerre',\n",
       "     'url': 'https://www.semanticscholar.org/author/5441351'},\n",
       "    {'authorId': '51370067',\n",
       "     'name': 'Manfred Grabherr',\n",
       "     'url': 'https://www.semanticscholar.org/author/51370067'},\n",
       "    {'authorId': '2809250',\n",
       "     'name': 'Manolis Kellis',\n",
       "     'url': 'https://www.semanticscholar.org/author/2809250'},\n",
       "    {'authorId': '12511546',\n",
       "     'name': 'Michael Kleber',\n",
       "     'url': 'https://www.semanticscholar.org/author/12511546'},\n",
       "    {'authorId': '4036531',\n",
       "     'name': 'Carolyne Bardeleben',\n",
       "     'url': 'https://www.semanticscholar.org/author/4036531'},\n",
       "    {'authorId': '1892378',\n",
       "     'name': 'Leo Goodstadt',\n",
       "     'url': 'https://www.semanticscholar.org/author/1892378'},\n",
       "    {'authorId': '2715131',\n",
       "     'name': 'Andreas Heger',\n",
       "     'url': 'https://www.semanticscholar.org/author/2715131'},\n",
       "    {'authorId': '1403229413',\n",
       "     'name': 'Christophe Hitte',\n",
       "     'url': 'https://www.semanticscholar.org/author/1403229413'},\n",
       "    {'authorId': '47714398',\n",
       "     'name': 'Lisa Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/47714398'},\n",
       "    {'authorId': '47316737',\n",
       "     'name': 'Klaus-Peter Koepfli',\n",
       "     'url': 'https://www.semanticscholar.org/author/47316737'},\n",
       "    {'authorId': '35436305',\n",
       "     'name': 'Heidi G Parker',\n",
       "     'url': 'https://www.semanticscholar.org/author/35436305'},\n",
       "    {'authorId': '4798563',\n",
       "     'name': 'John P Pollinger',\n",
       "     'url': 'https://www.semanticscholar.org/author/4798563'},\n",
       "    {'authorId': '1788227',\n",
       "     'name': 'Stephen M. J. Searle',\n",
       "     'url': 'https://www.semanticscholar.org/author/1788227'},\n",
       "    {'authorId': '8629036',\n",
       "     'name': 'Nathan B. Sutter',\n",
       "     'url': 'https://www.semanticscholar.org/author/8629036'},\n",
       "    {'authorId': '143987432',\n",
       "     'name': 'Rachael Thomas',\n",
       "     'url': 'https://www.semanticscholar.org/author/143987432'},\n",
       "    {'authorId': '152707679',\n",
       "     'name': 'Caleb Webber',\n",
       "     'url': 'https://www.semanticscholar.org/author/152707679'},\n",
       "    {'authorId': '9311320',\n",
       "     'name': 'Eric S. Lander',\n",
       "     'url': 'https://www.semanticscholar.org/author/9311320'}],\n",
       "   'doi': '10.1038/nature04338',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b01f00857b9abec4f245c665c5042824f5430711',\n",
       "   'title': 'Genome sequence, comparative analysis and haplotype structure of the domestic dog',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b01f00857b9abec4f245c665c5042824f5430711',\n",
       "   'venue': 'Nature',\n",
       "   'year': 2005},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5563282',\n",
       "     'name': 'William A. Priester',\n",
       "     'url': 'https://www.semanticscholar.org/author/5563282'}],\n",
       "   'doi': '10.1002/1097-0142(197710)40:4+<1845::AID-CNCR2820400812>3.0.CO;2-8',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '2f292a5303f14fb0a4997768c978405e08c5de2f',\n",
       "   'title': 'Multiple primary tumors in domestic animals: a preliminary view with particular emphasis on tumors in dogs.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/2f292a5303f14fb0a4997768c978405e08c5de2f',\n",
       "   'venue': 'Cancer',\n",
       "   'year': 1977},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '7543407',\n",
       "     'name': 'Elinor K Karlsson',\n",
       "     'url': 'https://www.semanticscholar.org/author/7543407'},\n",
       "    {'authorId': '1382417671',\n",
       "     'name': 'Kerstin Lindblad-Toh',\n",
       "     'url': 'https://www.semanticscholar.org/author/1382417671'}],\n",
       "   'doi': '10.1038/nrg2382',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '249b5eb6398605e339eccfb50960d99cf0baeb10',\n",
       "   'title': 'Leader of the pack: gene mapping in dogs and other model organisms',\n",
       "   'url': 'https://www.semanticscholar.org/paper/249b5eb6398605e339eccfb50960d99cf0baeb10',\n",
       "   'venue': 'Nature Reviews Genetics',\n",
       "   'year': 2008},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145150036',\n",
       "     'name': 'Mike Starkey',\n",
       "     'url': 'https://www.semanticscholar.org/author/145150036'},\n",
       "    {'authorId': '3854860',\n",
       "     'name': 'Timothy J. Scase',\n",
       "     'url': 'https://www.semanticscholar.org/author/3854860'},\n",
       "    {'authorId': '3593066',\n",
       "     'name': 'Cathryn Mellersh',\n",
       "     'url': 'https://www.semanticscholar.org/author/3593066'},\n",
       "    {'authorId': '50019425',\n",
       "     'name': 'Sue Murphy',\n",
       "     'url': 'https://www.semanticscholar.org/author/50019425'}],\n",
       "   'doi': '10.1093/bfgp/4.2.112',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'eedcd1b964fcccbc15315140c96de907924a47ad',\n",
       "   'title': \"Dogs really are man's best friend--canine genomics has applications in veterinary and human medicine!\",\n",
       "   'url': 'https://www.semanticscholar.org/paper/eedcd1b964fcccbc15315140c96de907924a47ad',\n",
       "   'venue': 'Briefings in functional genomics & proteomics',\n",
       "   'year': 2005},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2415653',\n",
       "     'name': 'Brenda N. Bonnett',\n",
       "     'url': 'https://www.semanticscholar.org/author/2415653'},\n",
       "    {'authorId': '6154094',\n",
       "     'name': 'Agneta Egenvall',\n",
       "     'url': 'https://www.semanticscholar.org/author/6154094'}],\n",
       "   'doi': '10.1016/j.jcpa.2009.10.008',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'ab28af5da0a227674eff8e949c5563ecca45622b',\n",
       "   'title': 'Age patterns of disease and death in insured Swedish dogs, cats and horses.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/ab28af5da0a227674eff8e949c5563ecca45622b',\n",
       "   'venue': 'Journal of comparative pathology',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47232724',\n",
       "     'name': 'Konstantin Strauch',\n",
       "     'url': 'https://www.semanticscholar.org/author/47232724'},\n",
       "    {'authorId': '6491150',\n",
       "     'name': 'Rolf Fimmers',\n",
       "     'url': 'https://www.semanticscholar.org/author/6491150'},\n",
       "    {'authorId': '38541662',\n",
       "     'name': 'Max P. Baur',\n",
       "     'url': 'https://www.semanticscholar.org/author/38541662'},\n",
       "    {'authorId': '2661355',\n",
       "     'name': 'Thomas F. Wienker',\n",
       "     'url': 'https://www.semanticscholar.org/author/2661355'}],\n",
       "   'doi': '10.1159/000076394',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'ceb6e64d26056de8b4d59f48bc024f87e7e8dab1',\n",
       "   'title': 'How to Model a Complex Trait',\n",
       "   'url': 'https://www.semanticscholar.org/paper/ceb6e64d26056de8b4d59f48bc024f87e7e8dab1',\n",
       "   'venue': 'Human Heredity',\n",
       "   'year': 2003},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2662985',\n",
       "     'name': 'Pablo Perel',\n",
       "     'url': 'https://www.semanticscholar.org/author/2662985'},\n",
       "    {'authorId': '144582098',\n",
       "     'name': 'Ian Roberts',\n",
       "     'url': 'https://www.semanticscholar.org/author/144582098'},\n",
       "    {'authorId': '4218353',\n",
       "     'name': 'Emily S Sena',\n",
       "     'url': 'https://www.semanticscholar.org/author/4218353'},\n",
       "    {'authorId': '47349652',\n",
       "     'name': 'Philipa Wheble',\n",
       "     'url': 'https://www.semanticscholar.org/author/47349652'},\n",
       "    {'authorId': '39924238',\n",
       "     'name': 'Catherine Briscoe',\n",
       "     'url': 'https://www.semanticscholar.org/author/39924238'},\n",
       "    {'authorId': '144007987',\n",
       "     'name': 'Peter Sandercock',\n",
       "     'url': 'https://www.semanticscholar.org/author/144007987'},\n",
       "    {'authorId': '145788858',\n",
       "     'name': 'Malcolm Robert Macleod',\n",
       "     'url': 'https://www.semanticscholar.org/author/145788858'},\n",
       "    {'authorId': '47920235',\n",
       "     'name': 'Luciano E Mignini',\n",
       "     'url': 'https://www.semanticscholar.org/author/47920235'},\n",
       "    {'authorId': '40054103',\n",
       "     'name': 'Pradeep M Jayaram',\n",
       "     'url': 'https://www.semanticscholar.org/author/40054103'},\n",
       "    {'authorId': '145229618',\n",
       "     'name': 'Khalid S Khan',\n",
       "     'url': 'https://www.semanticscholar.org/author/145229618'}],\n",
       "   'doi': '10.1136/bmj.39048.407928.BE',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '646d5daad7626dcec97589a027f5d1f69aeeeec4',\n",
       "   'title': 'Comparison of treatment effects between animal experiments and clinical trials: systematic review.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/646d5daad7626dcec97589a027f5d1f69aeeeec4',\n",
       "   'venue': 'BMJ',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6445172',\n",
       "     'name': 'Gerard R. Rutteman',\n",
       "     'url': 'https://www.semanticscholar.org/author/6445172'},\n",
       "    {'authorId': '7656234',\n",
       "     'name': 'Wim Misdorp',\n",
       "     'url': 'https://www.semanticscholar.org/author/7656234'},\n",
       "    {'authorId': '4233865',\n",
       "     'name': 'Marinus A. Blankenstein',\n",
       "     'url': 'https://www.semanticscholar.org/author/4233865'},\n",
       "    {'authorId': '5768571',\n",
       "     'name': 'Walter E van den Brom',\n",
       "     'url': 'https://www.semanticscholar.org/author/5768571'}],\n",
       "   'doi': '10.1038/bjc.1988.266',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '00e1cab13ffe77bcc422e8c96b99ad9894d647fb',\n",
       "   'title': 'Oestrogen (ER) and progestin receptors (PR) in mammary tissue of the female dog: different receptor profile in non-malignant and malignant states.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/00e1cab13ffe77bcc422e8c96b99ad9894d647fb',\n",
       "   'venue': 'British Journal of Cancer',\n",
       "   'year': 1988},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4459370',\n",
       "     'name': 'Stephen J. Withrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/4459370'},\n",
       "    {'authorId': '33556864',\n",
       "     'name': 'Ross M. Wilkins',\n",
       "     'url': 'https://www.semanticscholar.org/author/33556864'}],\n",
       "   'doi': '10.1093/ilar.51.3.208',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '432c8decc52b701d54f51d5e71fc24daa28e22a4',\n",
       "   'title': 'Cross talk from pets to people: translational osteosarcoma treatments.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/432c8decc52b701d54f51d5e71fc24daa28e22a4',\n",
       "   'venue': 'ILAR journal',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '121674598',\n",
       "     'name': 'Isabella W Y Mak',\n",
       "     'url': 'https://www.semanticscholar.org/author/121674598'},\n",
       "    {'authorId': '4746348',\n",
       "     'name': 'Nathan Evaniew',\n",
       "     'url': 'https://www.semanticscholar.org/author/4746348'},\n",
       "    {'authorId': '5646156',\n",
       "     'name': 'Michelle A. Ghert',\n",
       "     'url': 'https://www.semanticscholar.org/author/5646156'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '801e15e4a14ad21295cd1c93112f4c457ea1f8d1',\n",
       "   'title': 'Lost in translation: animal models and clinical trials in cancer treatment.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/801e15e4a14ad21295cd1c93112f4c457ea1f8d1',\n",
       "   'venue': 'American journal of translational research',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4218353',\n",
       "     'name': 'Emily S Sena',\n",
       "     'url': 'https://www.semanticscholar.org/author/4218353'},\n",
       "    {'authorId': '3481496',\n",
       "     'name': 'H. Bart van der Worp',\n",
       "     'url': 'https://www.semanticscholar.org/author/3481496'},\n",
       "    {'authorId': '40570150',\n",
       "     'name': 'Philip M Bath',\n",
       "     'url': 'https://www.semanticscholar.org/author/40570150'},\n",
       "    {'authorId': '3085016',\n",
       "     'name': 'David William Howells',\n",
       "     'url': 'https://www.semanticscholar.org/author/3085016'},\n",
       "    {'authorId': '145788858',\n",
       "     'name': 'Malcolm Robert Macleod',\n",
       "     'url': 'https://www.semanticscholar.org/author/145788858'}],\n",
       "   'doi': '10.1371/journal.pbio.1000344',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c8389db9ce7b323b5fdec1d075b016f0cddc8675',\n",
       "   'title': 'Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy',\n",
       "   'url': 'https://www.semanticscholar.org/paper/c8389db9ce7b323b5fdec1d075b016f0cddc8675',\n",
       "   'venue': 'PLoS biology',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '39321693',\n",
       "     'name': 'Sumanta K Goswami',\n",
       "     'url': 'https://www.semanticscholar.org/author/39321693'},\n",
       "    {'authorId': '1829691',\n",
       "     'name': 'Erik Sahai',\n",
       "     'url': 'https://www.semanticscholar.org/author/1829691'},\n",
       "    {'authorId': '4497477',\n",
       "     'name': 'Jeffrey B Wyckoff',\n",
       "     'url': 'https://www.semanticscholar.org/author/4497477'},\n",
       "    {'authorId': '6141038',\n",
       "     'name': 'Michael Cammer',\n",
       "     'url': 'https://www.semanticscholar.org/author/6141038'},\n",
       "    {'authorId': '2332855',\n",
       "     'name': 'Dianne Cox',\n",
       "     'url': 'https://www.semanticscholar.org/author/2332855'},\n",
       "    {'authorId': '5380310',\n",
       "     'name': 'Fiona J Pixley',\n",
       "     'url': 'https://www.semanticscholar.org/author/5380310'},\n",
       "    {'authorId': '145329084',\n",
       "     'name': 'E Richard Stanley',\n",
       "     'url': 'https://www.semanticscholar.org/author/145329084'},\n",
       "    {'authorId': '3299296',\n",
       "     'name': 'Jeffrey E Segall',\n",
       "     'url': 'https://www.semanticscholar.org/author/3299296'},\n",
       "    {'authorId': '145495075',\n",
       "     'name': 'John S Condeelis',\n",
       "     'url': 'https://www.semanticscholar.org/author/145495075'}],\n",
       "   'doi': '10.1158/0008-5472.CAN-04-1853',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '320cdc40340fa371a35c5436f9219c1b94fd3399',\n",
       "   'title': 'Macrophages promote the invasion of breast carcinoma cells via a colony-stimulating factor-1/epidermal growth factor paracrine loop.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/320cdc40340fa371a35c5436f9219c1b94fd3399',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 2005},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3523881',\n",
       "     'name': 'Susan M. Larue',\n",
       "     'url': 'https://www.semanticscholar.org/author/3523881'},\n",
       "    {'authorId': '4459370',\n",
       "     'name': 'Stephen J. Withrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/4459370'},\n",
       "    {'authorId': '6569611',\n",
       "     'name': 'Barbara E. Powers',\n",
       "     'url': 'https://www.semanticscholar.org/author/6569611'},\n",
       "    {'authorId': '3153140',\n",
       "     'name': 'Robert H. Wrigley',\n",
       "     'url': 'https://www.semanticscholar.org/author/3153140'},\n",
       "    {'authorId': '2115791',\n",
       "     'name': 'Edward L. Gillette',\n",
       "     'url': 'https://www.semanticscholar.org/author/2115791'},\n",
       "    {'authorId': '144474840',\n",
       "     'name': 'P D Schwarz',\n",
       "     'url': 'https://www.semanticscholar.org/author/144474840'},\n",
       "    {'authorId': '4037722',\n",
       "     'name': 'Rod C. Straw',\n",
       "     'url': 'https://www.semanticscholar.org/author/4037722'},\n",
       "    {'authorId': '47225881',\n",
       "     'name': 'S L Richter',\n",
       "     'url': 'https://www.semanticscholar.org/author/47225881'}],\n",
       "   'doi': None,\n",
       "   'intent': ['methodology'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9363f01654b7b6ef34c9598c89b85e35f05f212f',\n",
       "   'title': 'Limb-sparing treatment for osteosarcoma in dogs.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9363f01654b7b6ef34c9598c89b85e35f05f212f',\n",
       "   'venue': 'Journal of the American Veterinary Medical Association',\n",
       "   'year': 1989},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145681905',\n",
       "     'name': 'Giuseppe Ru',\n",
       "     'url': 'https://www.semanticscholar.org/author/145681905'},\n",
       "    {'authorId': '5431143',\n",
       "     'name': 'Benedetto Terracini',\n",
       "     'url': 'https://www.semanticscholar.org/author/5431143'},\n",
       "    {'authorId': '12902236',\n",
       "     'name': 'Larry T. Glickman',\n",
       "     'url': 'https://www.semanticscholar.org/author/12902236'}],\n",
       "   'doi': '10.1016/S1090-0233(98)80059-2',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'd6a9eeed9f4cd0f154433e49c2c1a78390c5906a',\n",
       "   'title': 'Host related risk factors for canine osteosarcoma.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/d6a9eeed9f4cd0f154433e49c2c1a78390c5906a',\n",
       "   'venue': 'Veterinary journal',\n",
       "   'year': 1998},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2096074',\n",
       "     'name': 'Matthew Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/2096074'},\n",
       "    {'authorId': '3740943',\n",
       "     'name': 'Jaime F Modiano',\n",
       "     'url': 'https://www.semanticscholar.org/author/3740943'}],\n",
       "   'doi': '10.1007/s10577-007-1212-4',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '04b0bd2f73c1840c15e5d2b5744e62131b3a3b1c',\n",
       "   'title': 'Evolutionarily conserved cytogenetic changes in hematological malignancies of dogs and humans – man and his best friend share more than companionship',\n",
       "   'url': 'https://www.semanticscholar.org/paper/04b0bd2f73c1840c15e5d2b5744e62131b3a3b1c',\n",
       "   'venue': 'Chromosome Research',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4994098',\n",
       "     'name': 'Kyubo Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/4994098'},\n",
       "    {'authorId': '3971574',\n",
       "     'name': 'Eui Kyu Chie',\n",
       "     'url': 'https://www.semanticscholar.org/author/3971574'},\n",
       "    {'authorId': '143702426',\n",
       "     'name': 'Wonshik Han',\n",
       "     'url': 'https://www.semanticscholar.org/author/143702426'},\n",
       "    {'authorId': '1916796',\n",
       "     'name': 'Dong-Young Noh',\n",
       "     'url': 'https://www.semanticscholar.org/author/1916796'},\n",
       "    {'authorId': '5365246',\n",
       "     'name': 'In Ae Park',\n",
       "     'url': 'https://www.semanticscholar.org/author/5365246'},\n",
       "    {'authorId': '3880128',\n",
       "     'name': 'Do-Youn Oh',\n",
       "     'url': 'https://www.semanticscholar.org/author/3880128'},\n",
       "    {'authorId': '143784365',\n",
       "     'name': 'Seock-Ah Im',\n",
       "     'url': 'https://www.semanticscholar.org/author/143784365'},\n",
       "    {'authorId': '46760149',\n",
       "     'name': 'Tae-You Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/46760149'},\n",
       "    {'authorId': '144693153',\n",
       "     'name': 'Yung-Jue Bang',\n",
       "     'url': 'https://www.semanticscholar.org/author/144693153'},\n",
       "    {'authorId': '4450042',\n",
       "     'name': 'Sung W Ha',\n",
       "     'url': 'https://www.semanticscholar.org/author/4450042'}],\n",
       "   'doi': '10.3346/jkms.2010.25.2.235',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'bead5a68b8011791205180b9e234734698f13e6b',\n",
       "   'title': 'Prognostic Value of p53 and bcl-2 Expression in Patients Treated with Breast Conservative Therapy',\n",
       "   'url': 'https://www.semanticscholar.org/paper/bead5a68b8011791205180b9e234734698f13e6b',\n",
       "   'venue': 'Journal of Korean medical science',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47293585',\n",
       "     'name': 'Tsutomu Shirahama',\n",
       "     'url': 'https://www.semanticscholar.org/author/47293585'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '1f7fedd4458cda0068b01e3fca4983424c6e7b31',\n",
       "   'title': 'Cyclooxygenase-2 expression is up-regulated in transitional cell carcinoma and its preneoplastic lesions in the human urinary bladder.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/1f7fedd4458cda0068b01e3fca4983424c6e7b31',\n",
       "   'venue': 'Clinical cancer research : an official journal of the American Association for Cancer Research',\n",
       "   'year': 2000},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2391548',\n",
       "     'name': 'Sulma Ibrahim Mohammed',\n",
       "     'url': 'https://www.semanticscholar.org/author/2391548'},\n",
       "    {'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'},\n",
       "    {'authorId': '2202192',\n",
       "     'name': 'David G. Bostwick',\n",
       "     'url': 'https://www.semanticscholar.org/author/2202192'},\n",
       "    {'authorId': '5455571',\n",
       "     'name': 'Richard S. Foster',\n",
       "     'url': 'https://www.semanticscholar.org/author/5455571'},\n",
       "    {'authorId': '46581247',\n",
       "     'name': 'Kanwar Nasir M. Khan',\n",
       "     'url': 'https://www.semanticscholar.org/author/46581247'},\n",
       "    {'authorId': '3226022',\n",
       "     'name': 'Jaime L. Masferrer',\n",
       "     'url': 'https://www.semanticscholar.org/author/3226022'},\n",
       "    {'authorId': '6921952',\n",
       "     'name': 'Bryan M. Woerner',\n",
       "     'url': 'https://www.semanticscholar.org/author/6921952'},\n",
       "    {'authorId': '50386825',\n",
       "     'name': 'Paul W. Snyder',\n",
       "     'url': 'https://www.semanticscholar.org/author/50386825'},\n",
       "    {'authorId': '5608129',\n",
       "     'name': 'Alane T. Koki',\n",
       "     'url': 'https://www.semanticscholar.org/author/5608129'}],\n",
       "   'doi': None,\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '672df6e426ae1acc81d0766a6610a9ce11a5be3f',\n",
       "   'title': 'Expression of cyclooxygenase-2 (COX-2) in human invasive transitional cell carcinoma (TCC) of the urinary bladder.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/672df6e426ae1acc81d0766a6610a9ce11a5be3f',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 1999},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4396138',\n",
       "     'name': 'Cheryl A London',\n",
       "     'url': 'https://www.semanticscholar.org/author/4396138'},\n",
       "    {'authorId': '35894242',\n",
       "     'name': 'Alison L. Hannah',\n",
       "     'url': 'https://www.semanticscholar.org/author/35894242'},\n",
       "    {'authorId': '10183180',\n",
       "     'name': 'Regina Zadovoskaya',\n",
       "     'url': 'https://www.semanticscholar.org/author/10183180'},\n",
       "    {'authorId': '30292316',\n",
       "     'name': 'May B. Chien',\n",
       "     'url': 'https://www.semanticscholar.org/author/30292316'},\n",
       "    {'authorId': '1417906048',\n",
       "     'name': 'Cynthia Kollias-Baker',\n",
       "     'url': 'https://www.semanticscholar.org/author/1417906048'},\n",
       "    {'authorId': '49007792',\n",
       "     'name': 'Mona Rosenberg',\n",
       "     'url': 'https://www.semanticscholar.org/author/49007792'},\n",
       "    {'authorId': '49260164',\n",
       "     'name': 'Sue Downing',\n",
       "     'url': 'https://www.semanticscholar.org/author/49260164'},\n",
       "    {'authorId': '1797930',\n",
       "     'name': 'Gerald Post',\n",
       "     'url': 'https://www.semanticscholar.org/author/1797930'},\n",
       "    {'authorId': '40427048',\n",
       "     'name': 'Joseph F. Boucher',\n",
       "     'url': 'https://www.semanticscholar.org/author/40427048'},\n",
       "    {'authorId': '36040922',\n",
       "     'name': 'Narmada Shenoy',\n",
       "     'url': 'https://www.semanticscholar.org/author/36040922'},\n",
       "    {'authorId': '49922125',\n",
       "     'name': 'Dirk B. Mendel',\n",
       "     'url': 'https://www.semanticscholar.org/author/49922125'},\n",
       "    {'authorId': '34760291',\n",
       "     'name': 'Gerald Mcmahon',\n",
       "     'url': 'https://www.semanticscholar.org/author/34760291'},\n",
       "    {'authorId': '5445181',\n",
       "     'name': 'Julie M. Cherrington',\n",
       "     'url': 'https://www.semanticscholar.org/author/5445181'}],\n",
       "   'doi': None,\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b75b881bbbbf364192f16c63ef75af0ad90693e4',\n",
       "   'title': 'Phase I dose-escalating study of SU11654, a small molecule receptor tyrosine kinase inhibitor, in dogs with spontaneous malignancies.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b75b881bbbbf364192f16c63ef75af0ad90693e4',\n",
       "   'venue': 'Clinical cancer research : an official journal of the American Association for Cancer Research',\n",
       "   'year': 2003},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5294957',\n",
       "     'name': 'Anthony W Rusk',\n",
       "     'url': 'https://www.semanticscholar.org/author/5294957'},\n",
       "    {'authorId': '34849813',\n",
       "     'name': 'Elizabeth M Cozzi',\n",
       "     'url': 'https://www.semanticscholar.org/author/34849813'},\n",
       "    {'authorId': '5004265',\n",
       "     'name': 'Marty E Stebbins',\n",
       "     'url': 'https://www.semanticscholar.org/author/5004265'},\n",
       "    {'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '50604063',\n",
       "     'name': 'J. Graham',\n",
       "     'url': 'https://www.semanticscholar.org/author/50604063'},\n",
       "    {'authorId': '8081135',\n",
       "     'name': 'Victor E. O. Valli',\n",
       "     'url': 'https://www.semanticscholar.org/author/8081135'},\n",
       "    {'authorId': '2356680',\n",
       "     'name': 'Jack Henkin',\n",
       "     'url': 'https://www.semanticscholar.org/author/2356680'},\n",
       "    {'authorId': '38585888',\n",
       "     'name': 'Rick Sharpee',\n",
       "     'url': 'https://www.semanticscholar.org/author/38585888'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1158/1078-0432.CCR-06-0110',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '8aece2450b79e07eed79d095d34906e81cf8dcbc',\n",
       "   'title': 'Cooperative activity of cytotoxic chemotherapy with antiangiogenic thrombospondin-I peptides, ABT-526 in pet dogs with relapsed lymphoma.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/8aece2450b79e07eed79d095d34906e81cf8dcbc',\n",
       "   'venue': 'Clinical cancer research : an official journal of the American Association for Cancer Research',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2517209',\n",
       "     'name': 'Jong Chul Park',\n",
       "     'url': 'https://www.semanticscholar.org/author/2517209'},\n",
       "    {'authorId': '40643859',\n",
       "     'name': 'Noah M. Hahn',\n",
       "     'url': 'https://www.semanticscholar.org/author/40643859'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'af4f8c494f82a1e1d7bbbc3ed2b94389f86f41ec',\n",
       "   'title': 'Bladder cancer: a disease ripe for major advances.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/af4f8c494f82a1e1d7bbbc3ed2b94389f86f41ec',\n",
       "   'venue': 'Clinical advances in hematology & oncology : H&O',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1398837771',\n",
       "     'name': 'Xabier García-Albéniz',\n",
       "     'url': 'https://www.semanticscholar.org/author/1398837771'},\n",
       "    {'authorId': '49704607',\n",
       "     'name': 'Rosa Mª Prieto Gallego',\n",
       "     'url': 'https://www.semanticscholar.org/author/49704607'},\n",
       "    {'authorId': '1420214410',\n",
       "     'name': 'Ralf Dieter Hofheinz',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420214410'},\n",
       "    {'authorId': '1420213984',\n",
       "     'name': 'Glòria Fernández-Esparrach',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420213984'},\n",
       "    {'authorId': '1420177335',\n",
       "     'name': 'Juan Ramón Ayuso-Colella',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420177335'},\n",
       "    {'authorId': '133707008',\n",
       "     'name': 'Josep Antoni Bombí',\n",
       "     'url': 'https://www.semanticscholar.org/author/133707008'},\n",
       "    {'authorId': '153353111',\n",
       "     'name': 'Carles Conill',\n",
       "     'url': 'https://www.semanticscholar.org/author/153353111'},\n",
       "    {'authorId': '1420177198',\n",
       "     'name': 'Miriam Cuatrecasas',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420177198'},\n",
       "    {'authorId': '47063516',\n",
       "     'name': 'Salvadora Delgado',\n",
       "     'url': 'https://www.semanticscholar.org/author/47063516'},\n",
       "    {'authorId': '152763382',\n",
       "     'name': 'Àngels Ginés',\n",
       "     'url': 'https://www.semanticscholar.org/author/152763382'},\n",
       "    {'authorId': '47578196',\n",
       "     'name': 'Rosa Miquel',\n",
       "     'url': 'https://www.semanticscholar.org/author/47578196'},\n",
       "    {'authorId': '143840320',\n",
       "     'name': 'Mario Pagés',\n",
       "     'url': 'https://www.semanticscholar.org/author/143840320'},\n",
       "    {'authorId': '144396979',\n",
       "     'name': 'Estela Pineda',\n",
       "     'url': 'https://www.semanticscholar.org/author/144396979'},\n",
       "    {'authorId': '50367971',\n",
       "     'name': 'Verônica Aparecida Pereira',\n",
       "     'url': 'https://www.semanticscholar.org/author/50367971'},\n",
       "    {'authorId': '49117099',\n",
       "     'name': 'Aarón Sosa',\n",
       "     'url': 'https://www.semanticscholar.org/author/49117099'},\n",
       "    {'authorId': '47213632',\n",
       "     'name': 'Òscar Reig',\n",
       "     'url': 'https://www.semanticscholar.org/author/47213632'},\n",
       "    {'authorId': '1420163427',\n",
       "     'name': 'Iván Victoria',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420163427'},\n",
       "    {'authorId': '51313516',\n",
       "     'name': 'Luis Féliz',\n",
       "     'url': 'https://www.semanticscholar.org/author/51313516'},\n",
       "    {'authorId': '3881910',\n",
       "     'name': 'Antonio María de Lacy',\n",
       "     'url': 'https://www.semanticscholar.org/author/3881910'},\n",
       "    {'authorId': '144850359',\n",
       "     'name': 'Antoni Castells',\n",
       "     'url': 'https://www.semanticscholar.org/author/144850359'},\n",
       "    {'authorId': '1413073649',\n",
       "     'name': 'Iris Burkholder',\n",
       "     'url': 'https://www.semanticscholar.org/author/1413073649'},\n",
       "    {'authorId': '3215981',\n",
       "     'name': 'Andreas Hochhaus',\n",
       "     'url': 'https://www.semanticscholar.org/author/3215981'},\n",
       "    {'authorId': '1420164193',\n",
       "     'name': 'Joan Maurel',\n",
       "     'url': 'https://www.semanticscholar.org/author/1420164193'}],\n",
       "   'doi': '10.1007/978-3-540-47648-1_4777',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '6ba370cb6858cff211578f80822d85e07ef9cc3d',\n",
       "   'title': 'PROSPECTIVE STUDY',\n",
       "   'url': 'https://www.semanticscholar.org/paper/6ba370cb6858cff211578f80822d85e07ef9cc3d',\n",
       "   'venue': '',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3904662',\n",
       "     'name': 'Laurel J. Gershwin',\n",
       "     'url': 'https://www.semanticscholar.org/author/3904662'}],\n",
       "   'doi': '10.1196/annals.1398.013',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9cd792dbe84e5bfca1bfc30c09c7e7ba47ebf6e1',\n",
       "   'title': 'Veterinary autoimmunity: autoimmune diseases in domestic animals.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9cd792dbe84e5bfca1bfc30c09c7e7ba47ebf6e1',\n",
       "   'venue': 'Annals of the New York Academy of Sciences',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '39670777',\n",
       "     'name': 'John Arrowsmith',\n",
       "     'url': 'https://www.semanticscholar.org/author/39670777'}],\n",
       "   'doi': '10.1038/nrd3375',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'a4d08a3e5012bf400b643bd2e8d3977754d419c1',\n",
       "   'title': 'Trial watch: Phase III and submission failures: 2007–2010',\n",
       "   'url': 'https://www.semanticscholar.org/paper/a4d08a3e5012bf400b643bd2e8d3977754d419c1',\n",
       "   'venue': 'Nature Reviews Drug Discovery',\n",
       "   'year': 2011},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '119288862',\n",
       "     'name': 'Karen L. Overall',\n",
       "     'url': 'https://www.semanticscholar.org/author/119288862'}],\n",
       "   'doi': '10.1016/S0278-5846(00)00104-4',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '65a037a6b0dbfad9c73b54bf8488711663ecc726',\n",
       "   'title': 'Natural animal models of human psychiatric conditions: assessment of mechanism and validity',\n",
       "   'url': 'https://www.semanticscholar.org/paper/65a037a6b0dbfad9c73b54bf8488711663ecc726',\n",
       "   'venue': 'Progress in Neuro-Psychopharmacology and Biological Psychiatry',\n",
       "   'year': 2000},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145740331',\n",
       "     'name': 'S. Mcchesney Gillette',\n",
       "     'url': 'https://www.semanticscholar.org/author/145740331'},\n",
       "    {'authorId': '3080678',\n",
       "     'name': 'Mark W. Dewhirst',\n",
       "     'url': 'https://www.semanticscholar.org/author/3080678'},\n",
       "    {'authorId': '2115791',\n",
       "     'name': 'Edward L. Gillette',\n",
       "     'url': 'https://www.semanticscholar.org/author/2115791'},\n",
       "    {'authorId': '6057701',\n",
       "     'name': 'Donald E. Thrall',\n",
       "     'url': 'https://www.semanticscholar.org/author/6057701'},\n",
       "    {'authorId': '30707796',\n",
       "     'name': 'Rodney López Page',\n",
       "     'url': 'https://www.semanticscholar.org/author/30707796'},\n",
       "    {'authorId': '6569611',\n",
       "     'name': 'Barbara E. Powers',\n",
       "     'url': 'https://www.semanticscholar.org/author/6569611'},\n",
       "    {'authorId': '4459370',\n",
       "     'name': 'Stephen J. Withrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/4459370'},\n",
       "    {'authorId': '103337715',\n",
       "     'name': 'Gary Rosner',\n",
       "     'url': 'https://www.semanticscholar.org/author/103337715'},\n",
       "    {'authorId': '50455697',\n",
       "     'name': 'Ching-yee Wong',\n",
       "     'url': 'https://www.semanticscholar.org/author/50455697'},\n",
       "    {'authorId': '1917287',\n",
       "     'name': 'Dalice A. Sim',\n",
       "     'url': 'https://www.semanticscholar.org/author/1917287'}],\n",
       "   'doi': '10.3109/02656739209021786',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '892431b3ae238ef82aec2f2b4aae6f9dfdf519a2',\n",
       "   'title': 'Response of canine soft tissue sarcomas to radiation or radiation plus hyperthermia: a randomized phase II study.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/892431b3ae238ef82aec2f2b4aae6f9dfdf519a2',\n",
       "   'venue': 'International journal of hyperthermia : the official journal of European Society for Hyperthermic Oncology, North American Hyperthermia Group',\n",
       "   'year': 1992},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '143836149',\n",
       "     'name': 'Nam Woo Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/143836149'},\n",
       "    {'authorId': '5267817',\n",
       "     'name': 'Mieczyslaw A. Piatyszek',\n",
       "     'url': 'https://www.semanticscholar.org/author/5267817'},\n",
       "    {'authorId': '49460041',\n",
       "     'name': 'Karen R. Prowse',\n",
       "     'url': 'https://www.semanticscholar.org/author/49460041'},\n",
       "    {'authorId': '3352995',\n",
       "     'name': 'Calvin B. Harley',\n",
       "     'url': 'https://www.semanticscholar.org/author/3352995'},\n",
       "    {'authorId': '145309241',\n",
       "     'name': 'Michael D. West',\n",
       "     'url': 'https://www.semanticscholar.org/author/145309241'},\n",
       "    {'authorId': '143972391',\n",
       "     'name': 'Paulo Lee Ho',\n",
       "     'url': 'https://www.semanticscholar.org/author/143972391'},\n",
       "    {'authorId': '50786164',\n",
       "     'name': 'Giancarlo Coviello',\n",
       "     'url': 'https://www.semanticscholar.org/author/50786164'},\n",
       "    {'authorId': '144651557',\n",
       "     'name': 'Woodring E Wright',\n",
       "     'url': 'https://www.semanticscholar.org/author/144651557'},\n",
       "    {'authorId': '48303120',\n",
       "     'name': 'Scott L. Weinrich',\n",
       "     'url': 'https://www.semanticscholar.org/author/48303120'},\n",
       "    {'authorId': '2982625',\n",
       "     'name': 'Jerry W. Shay',\n",
       "     'url': 'https://www.semanticscholar.org/author/2982625'}],\n",
       "   'doi': '10.1126/science.7605428',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'be533cf27f96121f29ea6b8b45ab4871642f1a05',\n",
       "   'title': 'Specific association of human telomerase activity with immortal cells and cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/be533cf27f96121f29ea6b8b45ab4871642f1a05',\n",
       "   'venue': 'Science',\n",
       "   'year': 1994},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1388510485',\n",
       "     'name': 'E. Gregory MacEwen',\n",
       "     'url': 'https://www.semanticscholar.org/author/1388510485'}],\n",
       "   'doi': '10.1007/BF00046339',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e99f8bdafaa9b5e6f9dcc692055fd9bde2c8df08',\n",
       "   'title': 'Spontaneous tumors in dogs and cats: Models for the study of cancer biology and treatment',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e99f8bdafaa9b5e6f9dcc692055fd9bde2c8df08',\n",
       "   'venue': 'Cancer and Metastasis Reviews',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '6715331',\n",
       "     'name': 'Abraham M. Lilienfeld',\n",
       "     'url': 'https://www.semanticscholar.org/author/6715331'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '7b6b9ffd99efac5382d9ff281eb73b15dc2a8828',\n",
       "   'title': 'THE EPIDEMIOLOGY OF BREAST CANCER.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/7b6b9ffd99efac5382d9ff281eb73b15dc2a8828',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 1963},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '153004368',\n",
       "     'name': 'Wolfgang Löscher',\n",
       "     'url': 'https://www.semanticscholar.org/author/153004368'},\n",
       "    {'authorId': '1413120103',\n",
       "     'name': 'Dorothea Schwartz-Porsche',\n",
       "     'url': 'https://www.semanticscholar.org/author/1413120103'},\n",
       "    {'authorId': '4425262',\n",
       "     'name': 'H. H. Frey',\n",
       "     'url': 'https://www.semanticscholar.org/author/4425262'},\n",
       "    {'authorId': '48207388',\n",
       "     'name': 'Donna Schmidt',\n",
       "     'url': 'https://www.semanticscholar.org/author/48207388'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '804541c5a5981b65ec265a493f0e7cb73bc6c15f',\n",
       "   'title': 'Evaluation of epileptic dogs as an animal model of human epilepsy.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/804541c5a5981b65ec265a493f0e7cb73bc6c15f',\n",
       "   'venue': 'Arzneimittel-Forschung',\n",
       "   'year': 1985},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '48858454',\n",
       "     'name': 'Melissa Paoloni',\n",
       "     'url': 'https://www.semanticscholar.org/author/48858454'},\n",
       "    {'authorId': '32555980',\n",
       "     'name': 'Sean R. Davis',\n",
       "     'url': 'https://www.semanticscholar.org/author/32555980'},\n",
       "    {'authorId': '3594869',\n",
       "     'name': 'Susan E. Lana',\n",
       "     'url': 'https://www.semanticscholar.org/author/3594869'},\n",
       "    {'authorId': '4459370',\n",
       "     'name': 'Stephen J. Withrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/4459370'},\n",
       "    {'authorId': '3078043',\n",
       "     'name': 'Luca Sangiorgi',\n",
       "     'url': 'https://www.semanticscholar.org/author/3078043'},\n",
       "    {'authorId': '2943369',\n",
       "     'name': 'Piero Picci',\n",
       "     'url': 'https://www.semanticscholar.org/author/2943369'},\n",
       "    {'authorId': '1820083',\n",
       "     'name': 'Stephen M. Hewitt',\n",
       "     'url': 'https://www.semanticscholar.org/author/1820083'},\n",
       "    {'authorId': '2135404',\n",
       "     'name': 'Timothy Triche',\n",
       "     'url': 'https://www.semanticscholar.org/author/2135404'},\n",
       "    {'authorId': '1738356',\n",
       "     'name': 'Paul S. Meltzer',\n",
       "     'url': 'https://www.semanticscholar.org/author/1738356'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1186/1471-2164-10-625',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '28a946e01a1e4ec06aee17617eb0f3965efe9278',\n",
       "   'title': 'Canine tumor cross-species genomics uncovers targets linked to osteosarcoma progression',\n",
       "   'url': 'https://www.semanticscholar.org/paper/28a946e01a1e4ec06aee17617eb0f3965efe9278',\n",
       "   'venue': 'BMC Genomics',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'},\n",
       "    {'authorId': '1400752997',\n",
       "     'name': 'José Antonio Ramos-Vara',\n",
       "     'url': 'https://www.semanticscholar.org/author/1400752997'},\n",
       "    {'authorId': '118630767',\n",
       "     'name': 'George Moore',\n",
       "     'url': 'https://www.semanticscholar.org/author/118630767'},\n",
       "    {'authorId': '5516383',\n",
       "     'name': 'Deepika Dhawan',\n",
       "     'url': 'https://www.semanticscholar.org/author/5516383'},\n",
       "    {'authorId': '6766884',\n",
       "     'name': 'Patty L. Bonney',\n",
       "     'url': 'https://www.semanticscholar.org/author/6766884'},\n",
       "    {'authorId': '49539844',\n",
       "     'name': 'K E Young',\n",
       "     'url': 'https://www.semanticscholar.org/author/49539844'}],\n",
       "   'doi': '10.1093/ilar/ilu018',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '79323b0911026c62ac689ba085c49fdaeeaa710f',\n",
       "   'title': 'Urinary bladder cancer in dogs, a naturally occurring model for cancer biology and drug development.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/79323b0911026c62ac689ba085c49fdaeeaa710f',\n",
       "   'venue': 'ILAR journal',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '72939859',\n",
       "     'name': 'F. Manfred',\n",
       "     'url': 'https://www.semanticscholar.org/author/72939859'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '884ee4616b7190655a016b0586e02ab07f239d0c',\n",
       "   'title': 'About the generation of a',\n",
       "   'url': 'https://www.semanticscholar.org/paper/884ee4616b7190655a016b0586e02ab07f239d0c',\n",
       "   'venue': '',\n",
       "   'year': 1979},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'},\n",
       "    {'authorId': '2692645',\n",
       "     'name': 'Ralph C. Richardson',\n",
       "     'url': 'https://www.semanticscholar.org/author/2692645'},\n",
       "    {'authorId': '12768992',\n",
       "     'name': 'Gerald D. Bottoms',\n",
       "     'url': 'https://www.semanticscholar.org/author/12768992'},\n",
       "    {'authorId': '50635849',\n",
       "     'name': 'Robert Teclaw',\n",
       "     'url': 'https://www.semanticscholar.org/author/50635849'},\n",
       "    {'authorId': '26969986',\n",
       "     'name': 'Thomas C.-K. Chan',\n",
       "     'url': 'https://www.semanticscholar.org/author/26969986'}],\n",
       "   'doi': '10.1007/BF00686255',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '76ffb4e240ab8460216e2b3cab9f02511dc68198',\n",
       "   'title': 'Phase I trial of piroxicam in 62 dogs bearing naturally occurring tumors',\n",
       "   'url': 'https://www.semanticscholar.org/paper/76ffb4e240ab8460216e2b3cab9f02511dc68198',\n",
       "   'venue': 'Cancer Chemotherapy and Pharmacology',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '39760536',\n",
       "     'name': 'JoAnn C L Schuh',\n",
       "     'url': 'https://www.semanticscholar.org/author/39760536'}],\n",
       "   'doi': '10.1080/01926230490424770',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '3c13f964a51dc22b807a7dbef641d2ae939227e6',\n",
       "   'title': 'Trials, Tribulations, and Trends in Tumor Modeling in Mice',\n",
       "   'url': 'https://www.semanticscholar.org/paper/3c13f964a51dc22b807a7dbef641d2ae939227e6',\n",
       "   'venue': 'Toxicologic pathology',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '34588572',\n",
       "     'name': 'Melissa Paoloni',\n",
       "     'url': 'https://www.semanticscholar.org/author/34588572'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1038/nrc2273',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '057a521a408ad8c8f558966caf143533ee591701',\n",
       "   'title': 'Translation of new cancer treatments from pet dogs to humans',\n",
       "   'url': 'https://www.semanticscholar.org/paper/057a521a408ad8c8f558966caf143533ee591701',\n",
       "   'venue': 'Nature Reviews Cancer',\n",
       "   'year': 2008},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3917118',\n",
       "     'name': 'Ruthanne Chun',\n",
       "     'url': 'https://www.semanticscholar.org/author/3917118'},\n",
       "    {'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'},\n",
       "    {'authorId': '5323413',\n",
       "     'name': 'William R. Widmer',\n",
       "     'url': 'https://www.semanticscholar.org/author/5323413'},\n",
       "    {'authorId': '8325617',\n",
       "     'name': 'Domenic Denicola',\n",
       "     'url': 'https://www.semanticscholar.org/author/8325617'},\n",
       "    {'authorId': '4690975',\n",
       "     'name': 'Nita W. Glickman',\n",
       "     'url': 'https://www.semanticscholar.org/author/4690975'},\n",
       "    {'authorId': '5676302',\n",
       "     'name': 'Thomas Kuczek',\n",
       "     'url': 'https://www.semanticscholar.org/author/5676302'},\n",
       "    {'authorId': '9852227',\n",
       "     'name': 'A Degortari',\n",
       "     'url': 'https://www.semanticscholar.org/author/9852227'},\n",
       "    {'authorId': '145799775',\n",
       "     'name': 'C M Han',\n",
       "     'url': 'https://www.semanticscholar.org/author/145799775'}],\n",
       "   'doi': '10.1111/j.1939-1676.1997.tb00465.x',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '230e4086b18d5700dc8a93cafe1f202d63b5b5ee',\n",
       "   'title': 'Phase II clinical trial of carboplatin in canine transitional cell carcinoma of the urinary bladder.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/230e4086b18d5700dc8a93cafe1f202d63b5b5ee',\n",
       "   'venue': 'Journal of veterinary internal medicine',\n",
       "   'year': 1997},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50416065',\n",
       "     'name': 'Enrico Vezzali',\n",
       "     'url': 'https://www.semanticscholar.org/author/50416065'},\n",
       "    {'authorId': '2816496',\n",
       "     'name': 'André Laurent Parodi',\n",
       "     'url': 'https://www.semanticscholar.org/author/2816496'},\n",
       "    {'authorId': '4405618',\n",
       "     'name': 'Paolo Stefano Marcato',\n",
       "     'url': 'https://www.semanticscholar.org/author/4405618'},\n",
       "    {'authorId': '4543042',\n",
       "     'name': 'Giuliano Bettini',\n",
       "     'url': 'https://www.semanticscholar.org/author/4543042'}],\n",
       "   'doi': '10.1111/j.1476-5829.2009.00201.x',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4ce272b085f9b5b91d529d466e3d7043f746610a',\n",
       "   'title': 'Histopathologic classification of 171 cases of canine and feline non-Hodgkin lymphoma according to the WHO.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4ce272b085f9b5b91d529d466e3d7043f746610a',\n",
       "   'venue': 'Veterinary and comparative oncology',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5919451',\n",
       "     'name': 'Kathryn L Schwertfeger',\n",
       "     'url': 'https://www.semanticscholar.org/author/5919451'},\n",
       "    {'authorId': '35046614',\n",
       "     'name': 'Jeffrey M. Rosen',\n",
       "     'url': 'https://www.semanticscholar.org/author/35046614'},\n",
       "    {'authorId': '1870119',\n",
       "     'name': 'Donald Allen Cohen',\n",
       "     'url': 'https://www.semanticscholar.org/author/1870119'}],\n",
       "   'doi': '10.1007/s10911-006-9028-y',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4ae4ef9fc010570de8758141d124057ec1b0864b',\n",
       "   'title': 'Mammary Gland Macrophages: Pleiotropic Functions in Mammary Development',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4ae4ef9fc010570de8758141d124057ec1b0864b',\n",
       "   'venue': 'Journal of Mammary Gland Biology and Neoplasia',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3150702',\n",
       "     'name': 'Sanaa Eissa',\n",
       "     'url': 'https://www.semanticscholar.org/author/3150702'},\n",
       "    {'authorId': '144393791',\n",
       "     'name': 'Tarek M. K. Motawi',\n",
       "     'url': 'https://www.semanticscholar.org/author/144393791'},\n",
       "    {'authorId': '40297084',\n",
       "     'name': 'Soheir Badr',\n",
       "     'url': 'https://www.semanticscholar.org/author/40297084'},\n",
       "    {'authorId': '15808987',\n",
       "     'name': 'Ashraf Zaghlool',\n",
       "     'url': 'https://www.semanticscholar.org/author/15808987'},\n",
       "    {'authorId': '48451493',\n",
       "     'name': 'Ahmed Maher',\n",
       "     'url': 'https://www.semanticscholar.org/author/48451493'}],\n",
       "   'doi': '10.7754/Clin.Lab.2012.120507',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '940897c60e25f00fd5a65bfb5f7065f694d07adc',\n",
       "   'title': 'Evaluation of urinary human telomerase reverse transcriptase mRNA and scatter factor protein as urine markers for diagnosis of bladder cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/940897c60e25f00fd5a65bfb5f7065f694d07adc',\n",
       "   'venue': 'Clinical laboratory',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '83639637',\n",
       "     'name': 'David J Smith',\n",
       "     'url': 'https://www.semanticscholar.org/author/83639637'},\n",
       "    {'authorId': '10426448',\n",
       "     'name': 'Caren Broadhead',\n",
       "     'url': 'https://www.semanticscholar.org/author/10426448'},\n",
       "    {'authorId': '10400908',\n",
       "     'name': 'Gérard Descotes',\n",
       "     'url': 'https://www.semanticscholar.org/author/10400908'},\n",
       "    {'authorId': '5157924',\n",
       "     'name': 'Richard T. Fosse',\n",
       "     'url': 'https://www.semanticscholar.org/author/5157924'},\n",
       "    {'authorId': '6186534',\n",
       "     'name': 'Ruediger Hack',\n",
       "     'url': 'https://www.semanticscholar.org/author/6186534'},\n",
       "    {'authorId': '10384010',\n",
       "     'name': 'Klaus Krauser',\n",
       "     'url': 'https://www.semanticscholar.org/author/10384010'},\n",
       "    {'authorId': '2511820',\n",
       "     'name': 'Rudolf Pfister',\n",
       "     'url': 'https://www.semanticscholar.org/author/2511820'},\n",
       "    {'authorId': '49359431',\n",
       "     'name': 'Barry Phillips',\n",
       "     'url': 'https://www.semanticscholar.org/author/49359431'},\n",
       "    {'authorId': '4331887',\n",
       "     'name': 'Yvon Rabemampianina',\n",
       "     'url': 'https://www.semanticscholar.org/author/4331887'},\n",
       "    {'authorId': '144181833',\n",
       "     'name': 'James Sanders',\n",
       "     'url': 'https://www.semanticscholar.org/author/144181833'},\n",
       "    {'authorId': '46468864',\n",
       "     'name': 'Sue Sparrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/46468864'},\n",
       "    {'authorId': '1409361245',\n",
       "     'name': 'Markus Stephan-Gueldner',\n",
       "     'url': 'https://www.semanticscholar.org/author/1409361245'},\n",
       "    {'authorId': '152972150',\n",
       "     'name': 'Soren Dyring Jacobsen',\n",
       "     'url': 'https://www.semanticscholar.org/author/152972150'}],\n",
       "   'doi': '10.1093/ilar.43.Suppl_1.S39',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'bf40667e964a38b074d76bbfda9ba9c67ecd8577',\n",
       "   'title': 'Preclinical safety evaluation using nonrodent species: an industry/welfare project to minimize dog use.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/bf40667e964a38b074d76bbfda9ba9c67ecd8577',\n",
       "   'venue': 'ILAR journal',\n",
       "   'year': 2002},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4837771',\n",
       "     'name': 'Heidi Ledford',\n",
       "     'url': 'https://www.semanticscholar.org/author/4837771'}],\n",
       "   'doi': '10.1038/477526a',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4c9c8918e82815a93c164b04f0f3a55987adc248',\n",
       "   'title': 'Translational research: 4 ways to fix the clinical trial',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4c9c8918e82815a93c164b04f0f3a55987adc248',\n",
       "   'venue': 'Nature',\n",
       "   'year': 2011},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144385961',\n",
       "     'name': 'Richard M. Simon',\n",
       "     'url': 'https://www.semanticscholar.org/author/144385961'}],\n",
       "   'doi': '10.1016/j.ejca.2008.09.009',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '8e544ad961e929cd2e8df4cdd45745807fc284ce',\n",
       "   'title': 'Lost in translation: problems and pitfalls in translating laboratory observations to clinical utility.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/8e544ad961e929cd2e8df4cdd45745807fc284ce',\n",
       "   'venue': 'European journal of cancer',\n",
       "   'year': 2008},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '6375292',\n",
       "     'name': 'Ilene D. Kurzman',\n",
       "     'url': 'https://www.semanticscholar.org/author/6375292'},\n",
       "    {'authorId': '14734632',\n",
       "     'name': 'Phyllis C. Glawe',\n",
       "     'url': 'https://www.semanticscholar.org/author/14734632'},\n",
       "    {'authorId': '1410304889',\n",
       "     'name': \"Maura G. O'Brien\",\n",
       "     'url': 'https://www.semanticscholar.org/author/1410304889'},\n",
       "    {'authorId': '3917118',\n",
       "     'name': 'Ruthanne Chun',\n",
       "     'url': 'https://www.semanticscholar.org/author/3917118'},\n",
       "    {'authorId': '36338583',\n",
       "     'name': 'Laura D Garrett',\n",
       "     'url': 'https://www.semanticscholar.org/author/36338583'},\n",
       "    {'authorId': '10170566',\n",
       "     'name': 'Joyce E. Obradovich',\n",
       "     'url': 'https://www.semanticscholar.org/author/10170566'},\n",
       "    {'authorId': '16693238',\n",
       "     'name': 'Rogers M Fred',\n",
       "     'url': 'https://www.semanticscholar.org/author/16693238'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'},\n",
       "    {'authorId': '47615271',\n",
       "     'name': 'Gail T. Colbern',\n",
       "     'url': 'https://www.semanticscholar.org/author/47615271'},\n",
       "    {'authorId': '6661615',\n",
       "     'name': 'Peter K. Working',\n",
       "     'url': 'https://www.semanticscholar.org/author/6661615'}],\n",
       "   'doi': '10.1007/s00280-002-0469-8',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'd149f54001e6dbc215aaa6f9bea4f99366b045fa',\n",
       "   'title': 'STEALTH liposome-encapsulated cisplatin (SPI-77) versus carboplatin as adjuvant therapy for spontaneously arising osteosarcoma (OSA) in the dog: a randomized multicenter clinical trial',\n",
       "   'url': 'https://www.semanticscholar.org/paper/d149f54001e6dbc215aaa6f9bea4f99366b045fa',\n",
       "   'venue': 'Cancer Chemotherapy and Pharmacology',\n",
       "   'year': 2002},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '40058029',\n",
       "     'name': 'Jared S. Fowles',\n",
       "     'url': 'https://www.semanticscholar.org/author/40058029'},\n",
       "    {'authorId': '39625196',\n",
       "     'name': 'Deanna D. Dailey',\n",
       "     'url': 'https://www.semanticscholar.org/author/39625196'},\n",
       "    {'authorId': '2037488',\n",
       "     'name': 'Daniel L. Gustafson',\n",
       "     'url': 'https://www.semanticscholar.org/author/2037488'},\n",
       "    {'authorId': '2465310',\n",
       "     'name': 'Douglas H. Thamm',\n",
       "     'url': 'https://www.semanticscholar.org/author/2465310'},\n",
       "    {'authorId': '37575680',\n",
       "     'name': 'Dawn L. Duval',\n",
       "     'url': 'https://www.semanticscholar.org/author/37575680'}],\n",
       "   'doi': '10.1111/vco.12192',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '959a488d9875eb6ceedf3e15638f0f669792399d',\n",
       "   'title': 'The Flint Animal Cancer Center (FACC) Canine Tumour Cell Line Panel: a resource for veterinary drug discovery, comparative oncology and translational medicine',\n",
       "   'url': 'https://www.semanticscholar.org/paper/959a488d9875eb6ceedf3e15638f0f669792399d',\n",
       "   'venue': 'Veterinary and comparative oncology',\n",
       "   'year': 2017},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1879669',\n",
       "     'name': 'Serena Marchetti',\n",
       "     'url': 'https://www.semanticscholar.org/author/1879669'},\n",
       "    {'authorId': '1903718',\n",
       "     'name': 'Jan H M Schellens',\n",
       "     'url': 'https://www.semanticscholar.org/author/1903718'}],\n",
       "   'doi': '10.1038/sj.bjc.6603925',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '8ae3d343de5c5caf93288e31748d52e547539ec9',\n",
       "   'title': 'The impact of FDA and EMEA guidelines on drug development in relation to Phase 0 trials',\n",
       "   'url': 'https://www.semanticscholar.org/paper/8ae3d343de5c5caf93288e31748d52e547539ec9',\n",
       "   'venue': 'British Journal of Cancer',\n",
       "   'year': 2007},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2816496',\n",
       "     'name': 'André Laurent Parodi',\n",
       "     'url': 'https://www.semanticscholar.org/author/2816496'},\n",
       "    {'authorId': '7656234',\n",
       "     'name': 'Wim Misdorp',\n",
       "     'url': 'https://www.semanticscholar.org/author/7656234'},\n",
       "    {'authorId': '10115518',\n",
       "     'name': 'J-P. Mialot',\n",
       "     'url': 'https://www.semanticscholar.org/author/10115518'},\n",
       "    {'authorId': '14377385',\n",
       "     'name': 'Martine Mialot',\n",
       "     'url': 'https://www.semanticscholar.org/author/14377385'},\n",
       "    {'authorId': '145546187',\n",
       "     'name': 'Augustinus A. M. Hart',\n",
       "     'url': 'https://www.semanticscholar.org/author/145546187'},\n",
       "    {'authorId': '8094544',\n",
       "     'name': 'M. Hurtrel',\n",
       "     'url': 'https://www.semanticscholar.org/author/8094544'},\n",
       "    {'authorId': '3754941',\n",
       "     'name': 'Jeffrey C. Salomon',\n",
       "     'url': 'https://www.semanticscholar.org/author/3754941'}],\n",
       "   'doi': '10.1007/BF00199160',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '401596412a7ad4a8b940664cd3d81e35997557ec',\n",
       "   'title': 'Intratumoral BCG and Corynebacterium parvum therapy of canine mammary tumours before radical mastectomy',\n",
       "   'url': 'https://www.semanticscholar.org/paper/401596412a7ad4a8b940664cd3d81e35997557ec',\n",
       "   'venue': 'Cancer Immunology, Immunotherapy',\n",
       "   'year': 1983},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145024832',\n",
       "     'name': 'Leslie Bernstein',\n",
       "     'url': 'https://www.semanticscholar.org/author/145024832'}],\n",
       "   'doi': '10.1023/A:1015714305420',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '86df6b5a2d93333662db61db9110c0cf1bda9ff6',\n",
       "   'title': 'Epidemiology of Endocrine-Related Risk Factors for Breast Cancer',\n",
       "   'url': 'https://www.semanticscholar.org/paper/86df6b5a2d93333662db61db9110c0cf1bda9ff6',\n",
       "   'venue': 'Journal of Mammary Gland Biology and Neoplasia',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144530104',\n",
       "     'name': 'Roderick A F Macleod',\n",
       "     'url': 'https://www.semanticscholar.org/author/144530104'},\n",
       "    {'authorId': '2477736',\n",
       "     'name': 'Wilhelm G Dirks',\n",
       "     'url': 'https://www.semanticscholar.org/author/2477736'},\n",
       "    {'authorId': '2044140',\n",
       "     'name': 'Yoshinobu Matsuo',\n",
       "     'url': 'https://www.semanticscholar.org/author/2044140'},\n",
       "    {'authorId': '143918408',\n",
       "     'name': 'Manfred Kaufmann',\n",
       "     'url': 'https://www.semanticscholar.org/author/143918408'},\n",
       "    {'authorId': '3874791',\n",
       "     'name': 'Hedda Milch',\n",
       "     'url': 'https://www.semanticscholar.org/author/3874791'},\n",
       "    {'authorId': '2370341',\n",
       "     'name': 'Hans Guenter Drexler',\n",
       "     'url': 'https://www.semanticscholar.org/author/2370341'}],\n",
       "   'doi': '10.1002/(SICI)1097-0215(19991112)83:4<555::AID-IJC19>3.0.CO;2-2',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '0b9964ce4a9b98b6a5858cbb43e35f8aff55a486',\n",
       "   'title': 'Widespread intraspecies cross-contamination of human tumor cell lines arising at source.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/0b9964ce4a9b98b6a5858cbb43e35f8aff55a486',\n",
       "   'venue': 'International journal of cancer',\n",
       "   'year': 1999},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '35436305',\n",
       "     'name': 'Heidi G Parker',\n",
       "     'url': 'https://www.semanticscholar.org/author/35436305'},\n",
       "    {'authorId': '4490629',\n",
       "     'name': 'Kathryn M Meurs',\n",
       "     'url': 'https://www.semanticscholar.org/author/4490629'},\n",
       "    {'authorId': '3252069',\n",
       "     'name': 'Elaine A. Ostrander',\n",
       "     'url': 'https://www.semanticscholar.org/author/3252069'}],\n",
       "   'doi': '10.1016/j.jvc.2006.04.002',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '92bdfc6ab5aaad37efcde96723de41031f229868',\n",
       "   'title': 'Finding cardiovascular disease genes in the dog.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/92bdfc6ab5aaad37efcde96723de41031f229868',\n",
       "   'venue': 'Journal of veterinary cardiology : the official journal of the European Society of Veterinary Cardiology',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '20619484',\n",
       "     'name': 'Yoichi Gondo',\n",
       "     'url': 'https://www.semanticscholar.org/author/20619484'},\n",
       "    {'authorId': '48948691',\n",
       "     'name': 'Ryutaro Fukumura',\n",
       "     'url': 'https://www.semanticscholar.org/author/48948691'},\n",
       "    {'authorId': '2722183',\n",
       "     'name': 'Takuya Murata',\n",
       "     'url': 'https://www.semanticscholar.org/author/2722183'},\n",
       "    {'authorId': '46620134',\n",
       "     'name': 'Shigeru Makino',\n",
       "     'url': 'https://www.semanticscholar.org/author/46620134'}],\n",
       "   'doi': '10.5483/BMBRep.2009.42.6.315',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '60fb4fdefa84bfdb671d74963489138eaa6c1702',\n",
       "   'title': 'Next-generation gene targeting in the mouse for functional genomics.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/60fb4fdefa84bfdb671d74963489138eaa6c1702',\n",
       "   'venue': 'BMB reports',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '144299056',\n",
       "     'name': 'E. Gregory Macewen',\n",
       "     'url': 'https://www.semanticscholar.org/author/144299056'}],\n",
       "   'doi': '10.3109/07357900009012210',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': True,\n",
       "   'paperId': 'e72731131cdf66712c02fe1ef1c1b9c205a1d6aa',\n",
       "   'title': 'Spontaneously occurring tumors of companion animals as models for human cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e72731131cdf66712c02fe1ef1c1b9c205a1d6aa',\n",
       "   'venue': 'Cancer investigation',\n",
       "   'year': 2000},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '48775386',\n",
       "     'name': 'Isabelle Donnay',\n",
       "     'url': 'https://www.semanticscholar.org/author/48775386'},\n",
       "    {'authorId': '13748022',\n",
       "     'name': 'J. Rauïs',\n",
       "     'url': 'https://www.semanticscholar.org/author/13748022'},\n",
       "    {'authorId': '12689903',\n",
       "     'name': 'Nicole Devleeschouwer',\n",
       "     'url': 'https://www.semanticscholar.org/author/12689903'},\n",
       "    {'authorId': '1404808507',\n",
       "     'name': 'P. Wouters-Ballman',\n",
       "     'url': 'https://www.semanticscholar.org/author/1404808507'},\n",
       "    {'authorId': '3745613',\n",
       "     'name': 'Guy Leclercq',\n",
       "     'url': 'https://www.semanticscholar.org/author/3745613'},\n",
       "    {'authorId': '7316005',\n",
       "     'name': 'John P Verstegen',\n",
       "     'url': 'https://www.semanticscholar.org/author/7316005'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b684d14963fabbfbc7d4f2f1a3b3e671cf5b5c32',\n",
       "   'title': 'Comparison of estrogen and progesterone receptor expression in normal and tumor mammary tissues from dogs.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b684d14963fabbfbc7d4f2f1a3b3e671cf5b5c32',\n",
       "   'venue': 'American journal of veterinary research',\n",
       "   'year': 1995},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '49549102',\n",
       "     'name': 'Dimitrios Krikelis',\n",
       "     'url': 'https://www.semanticscholar.org/author/49549102'},\n",
       "    {'authorId': '51126327',\n",
       "     'name': 'Ian Judson',\n",
       "     'url': 'https://www.semanticscholar.org/author/51126327'}],\n",
       "   'doi': '10.1586/era.09.176',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'a46fb4abeabf00aad7bda1e4563055245119b1c0',\n",
       "   'title': 'Role of chemotherapy in the management of soft tissue sarcomas.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/a46fb4abeabf00aad7bda1e4563055245119b1c0',\n",
       "   'venue': 'Expert review of anticancer therapy',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1760037',\n",
       "     'name': 'Domenico Franco Merlo',\n",
       "     'url': 'https://www.semanticscholar.org/author/1760037'},\n",
       "    {'authorId': '50531651',\n",
       "     'name': 'Léa Rossi',\n",
       "     'url': 'https://www.semanticscholar.org/author/50531651'},\n",
       "    {'authorId': '46762599',\n",
       "     'name': 'Christine M. Pellegrino',\n",
       "     'url': 'https://www.semanticscholar.org/author/46762599'},\n",
       "    {'authorId': '48014580',\n",
       "     'name': 'Maurizio Ceppi',\n",
       "     'url': 'https://www.semanticscholar.org/author/48014580'},\n",
       "    {'authorId': '14004015',\n",
       "     'name': 'Umberto Cardellino',\n",
       "     'url': 'https://www.semanticscholar.org/author/14004015'},\n",
       "    {'authorId': '1686894',\n",
       "     'name': 'Claudia Capurro',\n",
       "     'url': 'https://www.semanticscholar.org/author/1686894'},\n",
       "    {'authorId': '49357652',\n",
       "     'name': 'Alessandra Ratto',\n",
       "     'url': 'https://www.semanticscholar.org/author/49357652'},\n",
       "    {'authorId': '14936597',\n",
       "     'name': 'P L Sambucco',\n",
       "     'url': 'https://www.semanticscholar.org/author/14936597'},\n",
       "    {'authorId': '47898567',\n",
       "     'name': 'Vincenzo Sestito',\n",
       "     'url': 'https://www.semanticscholar.org/author/47898567'},\n",
       "    {'authorId': '15117201',\n",
       "     'name': 'G. Tanara',\n",
       "     'url': 'https://www.semanticscholar.org/author/15117201'},\n",
       "    {'authorId': '47255010',\n",
       "     'name': 'Vittorio Bocchini',\n",
       "     'url': 'https://www.semanticscholar.org/author/47255010'}],\n",
       "   'doi': '10.1111/j.1939-1676.2008.0133.x',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '9757c2fbd6f447bc89810dfd0f98abd3c7e66cbd',\n",
       "   'title': 'Cancer incidence in pet dogs: findings of the Animal Tumor Registry of Genoa, Italy.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/9757c2fbd6f447bc89810dfd0f98abd3c7e66cbd',\n",
       "   'venue': 'Journal of veterinary internal medicine',\n",
       "   'year': 2008},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '36456243',\n",
       "     'name': 'Joshua D Schiffman',\n",
       "     'url': 'https://www.semanticscholar.org/author/36456243'},\n",
       "    {'authorId': '1877585',\n",
       "     'name': 'Matthew Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/1877585'}],\n",
       "   'doi': '10.1098/rstb.2014.0231',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '1d3a6a8a7c99ca98405085c5179a4722532d358a',\n",
       "   'title': 'Comparative oncology: what dogs and other species can teach us about humans with cancer',\n",
       "   'url': 'https://www.semanticscholar.org/paper/1d3a6a8a7c99ca98405085c5179a4722532d358a',\n",
       "   'venue': 'Philosophical Transactions of the Royal Society B: Biological Sciences',\n",
       "   'year': 2015},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '49559358',\n",
       "     'name': 'Sylvanie Surget',\n",
       "     'url': 'https://www.semanticscholar.org/author/49559358'},\n",
       "    {'authorId': '4746552',\n",
       "     'name': 'Marie P. Khoury',\n",
       "     'url': 'https://www.semanticscholar.org/author/4746552'},\n",
       "    {'authorId': '39433798',\n",
       "     'name': 'Jean-Christophe Bourdon',\n",
       "     'url': 'https://www.semanticscholar.org/author/39433798'}],\n",
       "   'doi': '10.2147/OTT.S53876',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e08027c5b8dd8813576b29c9659d3a919571b718',\n",
       "   'title': 'Uncovering the role of p53 splice variants in human malignancy: a clinical perspective',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e08027c5b8dd8813576b29c9659d3a919571b718',\n",
       "   'venue': 'OncoTargets and therapy',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '144299056',\n",
       "     'name': 'E. Gregory Macewen',\n",
       "     'url': 'https://www.semanticscholar.org/author/144299056'},\n",
       "    {'authorId': '6817604',\n",
       "     'name': 'Amiya K. Patnaik',\n",
       "     'url': 'https://www.semanticscholar.org/author/6817604'},\n",
       "    {'authorId': '144871374',\n",
       "     'name': 'H. Jay Harvey',\n",
       "     'url': 'https://www.semanticscholar.org/author/144871374'},\n",
       "    {'authorId': '2430630',\n",
       "     'name': 'W. B. Panko',\n",
       "     'url': 'https://www.semanticscholar.org/author/2430630'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'a0eb80b091159dc8cb1815d0a8e2aa1b3de51cf9',\n",
       "   'title': 'Estrogen receptors in canine mammary tumors.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/a0eb80b091159dc8cb1815d0a8e2aa1b3de51cf9',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 1982},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '79794521',\n",
       "     'name': 'Gayathri T Selvarajah',\n",
       "     'url': 'https://www.semanticscholar.org/author/79794521'},\n",
       "    {'authorId': '4115295',\n",
       "     'name': 'Jolle Kirpensteijn',\n",
       "     'url': 'https://www.semanticscholar.org/author/4115295'},\n",
       "    {'authorId': '117231451',\n",
       "     'name': 'Monique E. van Wolferen',\n",
       "     'url': 'https://www.semanticscholar.org/author/117231451'},\n",
       "    {'authorId': '34406708',\n",
       "     'name': 'N A S Rao',\n",
       "     'url': 'https://www.semanticscholar.org/author/34406708'},\n",
       "    {'authorId': '3902108',\n",
       "     'name': 'Hille Fieten',\n",
       "     'url': 'https://www.semanticscholar.org/author/3902108'},\n",
       "    {'authorId': '1782610',\n",
       "     'name': 'Jan A Mol',\n",
       "     'url': 'https://www.semanticscholar.org/author/1782610'}],\n",
       "   'doi': '10.1186/1476-4598-8-72',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': True,\n",
       "   'paperId': '8b50178e43ce42724fef6f1544234a9f6fc1a3fa',\n",
       "   'title': 'Gene expression profiling of canine osteosarcoma reveals genes associated with short and long survival times',\n",
       "   'url': 'https://www.semanticscholar.org/paper/8b50178e43ce42724fef6f1544234a9f6fc1a3fa',\n",
       "   'venue': 'Molecular Cancer',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3694732',\n",
       "     'name': 'Russell C. Hovey',\n",
       "     'url': 'https://www.semanticscholar.org/author/3694732'},\n",
       "    {'authorId': '1854810',\n",
       "     'name': 'Thomas B. Mcfadden',\n",
       "     'url': 'https://www.semanticscholar.org/author/1854810'},\n",
       "    {'authorId': '5041323',\n",
       "     'name': 'Robert Michael Akers',\n",
       "     'url': 'https://www.semanticscholar.org/author/5041323'}],\n",
       "   'doi': '10.1023/A:1018704603426',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '456af14ba9543a69844ef8f7c6d1fc380ee95d50',\n",
       "   'title': 'Regulation of Mammary Gland Growth and Morphogenesis by the Mammary Fat Pad: A Species Comparison',\n",
       "   'url': 'https://www.semanticscholar.org/paper/456af14ba9543a69844ef8f7c6d1fc380ee95d50',\n",
       "   'venue': 'Journal of Mammary Gland Biology and Neoplasia',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '35475748',\n",
       "     'name': 'Stephen A. Ramsey',\n",
       "     'url': 'https://www.semanticscholar.org/author/35475748'},\n",
       "    {'authorId': '8132329',\n",
       "     'name': 'Tanjin Xu',\n",
       "     'url': 'https://www.semanticscholar.org/author/8132329'},\n",
       "    {'authorId': '1927631',\n",
       "     'name': 'Cheri P. Goodall',\n",
       "     'url': 'https://www.semanticscholar.org/author/1927631'},\n",
       "    {'authorId': '39411328',\n",
       "     'name': 'Adelaide C. Rhodes',\n",
       "     'url': 'https://www.semanticscholar.org/author/39411328'},\n",
       "    {'authorId': '46360508',\n",
       "     'name': 'Amita Kashyap',\n",
       "     'url': 'https://www.semanticscholar.org/author/46360508'},\n",
       "    {'authorId': '144400645',\n",
       "     'name': 'Jun He',\n",
       "     'url': 'https://www.semanticscholar.org/author/144400645'},\n",
       "    {'authorId': '5914535',\n",
       "     'name': 'Shay Bracha',\n",
       "     'url': 'https://www.semanticscholar.org/author/5914535'}],\n",
       "   'doi': '10.1002/gcc.22441',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '7aca496942d4274073875df8784c7a25c14c2876',\n",
       "   'title': 'Cross‐species analysis of the canine and human bladder cancer transcriptome and exome',\n",
       "   'url': 'https://www.semanticscholar.org/paper/7aca496942d4274073875df8784c7a25c14c2876',\n",
       "   'venue': 'Genes, chromosomes & cancer',\n",
       "   'year': 2017},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50190103',\n",
       "     'name': 'K Hansen',\n",
       "     'url': 'https://www.semanticscholar.org/author/50190103'},\n",
       "    {'authorId': '145926592',\n",
       "     'name': 'Chesta Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926592'}],\n",
       "   'doi': '10.1016/j.ejca.2003.11.031',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': True,\n",
       "   'paperId': 'ebb5f4d6d1c162effa86f756c0e43095551902c0',\n",
       "   'title': 'Spontaneous and genetically engineered animal models; use in preclinical cancer drug development.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/ebb5f4d6d1c162effa86f756c0e43095551902c0',\n",
       "   'venue': 'European journal of cancer',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '3955173',\n",
       "     'name': 'Kevin A. Hahn',\n",
       "     'url': 'https://www.semanticscholar.org/author/3955173'},\n",
       "    {'authorId': '46470402',\n",
       "     'name': 'Lou Bravo',\n",
       "     'url': 'https://www.semanticscholar.org/author/46470402'},\n",
       "    {'authorId': '144444961',\n",
       "     'name': 'W. H. Adams',\n",
       "     'url': 'https://www.semanticscholar.org/author/144444961'},\n",
       "    {'authorId': '3285286',\n",
       "     'name': 'Donita L. Frazier',\n",
       "     'url': 'https://www.semanticscholar.org/author/3285286'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'aa388eb8363b25d3156965594b573b35b051e4da',\n",
       "   'title': 'Naturally occurring tumors in dogs as comparative models for cancer therapy research.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/aa388eb8363b25d3156965594b573b35b051e4da',\n",
       "   'venue': 'In vivo',\n",
       "   'year': 1994},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '15184326',\n",
       "     'name': 'Andrea Y. Angstadt',\n",
       "     'url': 'https://www.semanticscholar.org/author/15184326'},\n",
       "    {'authorId': '3580156',\n",
       "     'name': 'Venugopal Thayanithy',\n",
       "     'url': 'https://www.semanticscholar.org/author/3580156'},\n",
       "    {'authorId': '47302220',\n",
       "     'name': 'Subbaya Subramanian',\n",
       "     'url': 'https://www.semanticscholar.org/author/47302220'},\n",
       "    {'authorId': '3740943',\n",
       "     'name': 'Jaime F Modiano',\n",
       "     'url': 'https://www.semanticscholar.org/author/3740943'},\n",
       "    {'authorId': '1877585',\n",
       "     'name': 'Matthew Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/1877585'}],\n",
       "   'doi': '10.1016/j.cancergen.2012.09.005',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f3202fa10985aaf7638f28c0e19995e0ed434bc8',\n",
       "   'title': 'A genome-wide approach to comparative oncology: high-resolution oligonucleotide aCGH of canine and human osteosarcoma pinpoints shared microaberrations.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/f3202fa10985aaf7638f28c0e19995e0ed434bc8',\n",
       "   'venue': 'Cancer genetics',\n",
       "   'year': 2012},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '153608230',\n",
       "     'name': 'Vernon Jeremay Adams',\n",
       "     'url': 'https://www.semanticscholar.org/author/153608230'},\n",
       "    {'authorId': '46745456',\n",
       "     'name': 'Katy M. Evans',\n",
       "     'url': 'https://www.semanticscholar.org/author/46745456'},\n",
       "    {'authorId': '122583582',\n",
       "     'name': 'Jenny Sampson',\n",
       "     'url': 'https://www.semanticscholar.org/author/122583582'},\n",
       "    {'authorId': '145558251',\n",
       "     'name': 'J L N Wood',\n",
       "     'url': 'https://www.semanticscholar.org/author/145558251'}],\n",
       "   'doi': '10.1111/j.1748-5827.2010.00974.x',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e1a912fce98a444b840f82d3a4ea950ba84162de',\n",
       "   'title': 'Methods and mortality results of a health survey of purebred dogs in the UK.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e1a912fce98a444b840f82d3a4ea950ba84162de',\n",
       "   'venue': 'The Journal of small animal practice',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '11621763',\n",
       "     'name': 'Brian G Caserto',\n",
       "     'url': 'https://www.semanticscholar.org/author/11621763'}],\n",
       "   'doi': '10.1177/0300985813476069',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'da04d5a26db3904d925f2daa8ee0a1894e60cde9',\n",
       "   'title': 'A Comparative Review of Canine and Human Rhabdomyosarcoma With Emphasis on Classification and Pathogenesis',\n",
       "   'url': 'https://www.semanticscholar.org/paper/da04d5a26db3904d925f2daa8ee0a1894e60cde9',\n",
       "   'venue': 'Veterinary pathology',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5034623',\n",
       "     'name': 'Maria Cekanova',\n",
       "     'url': 'https://www.semanticscholar.org/author/5034623'},\n",
       "    {'authorId': '50487027',\n",
       "     'name': 'Md Jashim Uddin',\n",
       "     'url': 'https://www.semanticscholar.org/author/50487027'},\n",
       "    {'authorId': '3664829',\n",
       "     'name': 'Joseph W. Bartges',\n",
       "     'url': 'https://www.semanticscholar.org/author/3664829'},\n",
       "    {'authorId': '2026007',\n",
       "     'name': 'Amanda Callens',\n",
       "     'url': 'https://www.semanticscholar.org/author/2026007'},\n",
       "    {'authorId': '4673636',\n",
       "     'name': 'Alfred M. Legendre',\n",
       "     'url': 'https://www.semanticscholar.org/author/4673636'},\n",
       "    {'authorId': '4275275',\n",
       "     'name': 'Kusum Rathore',\n",
       "     'url': 'https://www.semanticscholar.org/author/4275275'},\n",
       "    {'authorId': '50451500',\n",
       "     'name': 'Laura Wright',\n",
       "     'url': 'https://www.semanticscholar.org/author/50451500'},\n",
       "    {'authorId': '39749316',\n",
       "     'name': 'Amanda J. Carter',\n",
       "     'url': 'https://www.semanticscholar.org/author/39749316'},\n",
       "    {'authorId': '6176086',\n",
       "     'name': 'Lawrence J Marnett',\n",
       "     'url': 'https://www.semanticscholar.org/author/6176086'}],\n",
       "   'doi': '10.1158/1940-6207.CAPR-12-0358',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '1de7115e8510ecd5a4f5cf0e4372a2f81d811a50',\n",
       "   'title': 'Molecular imaging of cyclooxygenase-2 in canine transitional cell carcinomas in vitro and in vivo.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/1de7115e8510ecd5a4f5cf0e4372a2f81d811a50',\n",
       "   'venue': 'Cancer prevention research',\n",
       "   'year': 2013},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '79401936',\n",
       "     'name': 'John R. W. Masters',\n",
       "     'url': 'https://www.semanticscholar.org/author/79401936'}],\n",
       "   'doi': '10.1038/nrc775',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '4795634206dfb581980127d0bd729bbd6fa86fba',\n",
       "   'title': 'HeLa cells 50 years on: the good, the bad and the ugly',\n",
       "   'url': 'https://www.semanticscholar.org/paper/4795634206dfb581980127d0bd729bbd6fa86fba',\n",
       "   'venue': 'Nature Reviews Cancer',\n",
       "   'year': 2002},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '1399539279',\n",
       "     'name': 'Angela L. McCleary-Wheeler',\n",
       "     'url': 'https://www.semanticscholar.org/author/1399539279'},\n",
       "    {'authorId': '49535569',\n",
       "     'name': 'Laurel E Williams',\n",
       "     'url': 'https://www.semanticscholar.org/author/49535569'},\n",
       "    {'authorId': '39741171',\n",
       "     'name': 'Paul R Hess',\n",
       "     'url': 'https://www.semanticscholar.org/author/39741171'},\n",
       "    {'authorId': '34802891',\n",
       "     'name': 'Steven E Suter',\n",
       "     'url': 'https://www.semanticscholar.org/author/34802891'}],\n",
       "   'doi': '10.2460/ajvr.71.12.1468',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '63f06954296979431c7e60ef24666164941b8ac0',\n",
       "   'title': 'Evaluation of an in vitro telomeric repeat amplification protocol assay to detect telomerase activity in canine urine.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/63f06954296979431c7e60ef24666164941b8ac0',\n",
       "   'venue': 'American journal of veterinary research',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '123159873',\n",
       "     'name': 'Chung-Ho Lee',\n",
       "     'url': 'https://www.semanticscholar.org/author/123159873'},\n",
       "    {'authorId': '9834042',\n",
       "     'name': 'Wan-hee Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/9834042'},\n",
       "    {'authorId': '97882643',\n",
       "     'name': 'Ji-hey Lim',\n",
       "     'url': 'https://www.semanticscholar.org/author/97882643'},\n",
       "    {'authorId': '120434416',\n",
       "     'name': 'Min-soo Kang',\n",
       "     'url': 'https://www.semanticscholar.org/author/120434416'},\n",
       "    {'authorId': '48693394',\n",
       "     'name': 'Dae-yong Kim',\n",
       "     'url': 'https://www.semanticscholar.org/author/48693394'},\n",
       "    {'authorId': '144747380',\n",
       "     'name': 'Oh-Kyeong Kweon',\n",
       "     'url': 'https://www.semanticscholar.org/author/144747380'}],\n",
       "   'doi': '10.4142/jvs.2004.5.1.63',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '85e1e63a5f63b9890264b754ec0fdf11afe37053',\n",
       "   'title': 'Mutation and overexpression of p53 as a prognostic factor in canine mammary tumors.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/85e1e63a5f63b9890264b754ec0fdf11afe37053',\n",
       "   'venue': 'Journal of veterinary science',\n",
       "   'year': 2004},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '48237496',\n",
       "     'name': 'Josef Singer',\n",
       "     'url': 'https://www.semanticscholar.org/author/48237496'},\n",
       "    {'authorId': '1885949',\n",
       "     'name': 'Judit Fazekas',\n",
       "     'url': 'https://www.semanticscholar.org/author/1885949'},\n",
       "    {'authorId': '46314813',\n",
       "     'name': 'Wei Wang',\n",
       "     'url': 'https://www.semanticscholar.org/author/46314813'},\n",
       "    {'authorId': '5171084',\n",
       "     'name': 'Marlene Weichselbaumer',\n",
       "     'url': 'https://www.semanticscholar.org/author/5171084'},\n",
       "    {'authorId': '40134483',\n",
       "     'name': 'Miroslawa Matz',\n",
       "     'url': 'https://www.semanticscholar.org/author/40134483'},\n",
       "    {'authorId': '6471556',\n",
       "     'name': 'Alexander Oliver Mader',\n",
       "     'url': 'https://www.semanticscholar.org/author/6471556'},\n",
       "    {'authorId': '4378296',\n",
       "     'name': 'Willibald Steinfellner',\n",
       "     'url': 'https://www.semanticscholar.org/author/4378296'},\n",
       "    {'authorId': '9719217',\n",
       "     'name': 'Sarah Meitz',\n",
       "     'url': 'https://www.semanticscholar.org/author/9719217'},\n",
       "    {'authorId': '5582863',\n",
       "     'name': 'Diana Mechtcheriakova',\n",
       "     'url': 'https://www.semanticscholar.org/author/5582863'},\n",
       "    {'authorId': '47441968',\n",
       "     'name': 'Yuri Sobanov',\n",
       "     'url': 'https://www.semanticscholar.org/author/47441968'},\n",
       "    {'authorId': '2823430',\n",
       "     'name': 'Michael Willmann',\n",
       "     'url': 'https://www.semanticscholar.org/author/2823430'},\n",
       "    {'authorId': '2619638',\n",
       "     'name': 'Thomas Stockner',\n",
       "     'url': 'https://www.semanticscholar.org/author/2619638'},\n",
       "    {'authorId': '4991889',\n",
       "     'name': 'Edzard Spillner',\n",
       "     'url': 'https://www.semanticscholar.org/author/4991889'},\n",
       "    {'authorId': '3497376',\n",
       "     'name': 'Renate Kunert',\n",
       "     'url': 'https://www.semanticscholar.org/author/3497376'},\n",
       "    {'authorId': '1382101721',\n",
       "     'name': 'Erika Jensen-Jarolim',\n",
       "     'url': 'https://www.semanticscholar.org/author/1382101721'}],\n",
       "   'doi': '10.1158/1535-7163.MCT-13-0288',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'c87de5c4c480e559f6b2fea4b4dee9fb42a425ff',\n",
       "   'title': 'Generation of a canine anti-EGFR (ErbB-1) antibody for passive immunotherapy in dog cancer patients.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/c87de5c4c480e559f6b2fea4b4dee9fb42a425ff',\n",
       "   'venue': 'Molecular cancer therapeutics',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5866492',\n",
       "     'name': 'Beverly A. Teicher',\n",
       "     'url': 'https://www.semanticscholar.org/author/5866492'}],\n",
       "   'doi': '10.1177/0192623308329473',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b879c0050e43612c09cc5dae842c4ffd46256eac',\n",
       "   'title': 'In Vivo/Ex Vivo and In Situ Assays Used in Cancer Research: A Brief Review',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b879c0050e43612c09cc5dae842c4ffd46256eac',\n",
       "   'venue': 'Toxicologic pathology',\n",
       "   'year': 2009},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5075359',\n",
       "     'name': 'Gary J. Patronek',\n",
       "     'url': 'https://www.semanticscholar.org/author/5075359'},\n",
       "    {'authorId': '49300266',\n",
       "     'name': 'David J. Waters',\n",
       "     'url': 'https://www.semanticscholar.org/author/49300266'},\n",
       "    {'authorId': '2273920',\n",
       "     'name': 'Lawrence T. Glickman',\n",
       "     'url': 'https://www.semanticscholar.org/author/2273920'}],\n",
       "   'doi': '10.1093/gerona/52A.3.B171',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'f8dc62d1d78489a7a2079980a8e8dfec92379f54',\n",
       "   'title': 'Comparative longevity of pet dogs and humans: implications for gerontology research.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/f8dc62d1d78489a7a2079980a8e8dfec92379f54',\n",
       "   'venue': 'The journals of gerontology. Series A, Biological sciences and medical sciences',\n",
       "   'year': 1997},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '49460041',\n",
       "     'name': 'Karen R. Prowse',\n",
       "     'url': 'https://www.semanticscholar.org/author/49460041'},\n",
       "    {'authorId': '5840987',\n",
       "     'name': 'Carol W Greider',\n",
       "     'url': 'https://www.semanticscholar.org/author/5840987'}],\n",
       "   'doi': '10.1073/pnas.92.11.4818',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'e675e90f0a702215cbb3555bf53fda97b8b973d9',\n",
       "   'title': 'Developmental and tissue-specific regulation of mouse telomerase and telomere length.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/e675e90f0a702215cbb3555bf53fda97b8b973d9',\n",
       "   'venue': 'Proceedings of the National Academy of Sciences of the United States of America',\n",
       "   'year': 1995},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '15729799',\n",
       "     'name': 'Alexandra Sahora',\n",
       "     'url': 'https://www.semanticscholar.org/author/15729799'},\n",
       "    {'authorId': '5294957',\n",
       "     'name': 'Anthony W Rusk',\n",
       "     'url': 'https://www.semanticscholar.org/author/5294957'},\n",
       "    {'authorId': '2356680',\n",
       "     'name': 'Jack Henkin',\n",
       "     'url': 'https://www.semanticscholar.org/author/2356680'},\n",
       "    {'authorId': '5338386',\n",
       "     'name': 'Evelyn M. Mckeegan',\n",
       "     'url': 'https://www.semanticscholar.org/author/5338386'},\n",
       "    {'authorId': '5428838',\n",
       "     'name': 'Yongxiang Shi',\n",
       "     'url': 'https://www.semanticscholar.org/author/5428838'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1111/j.1939-1676.2012.00966.x',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '8e911d343f59a42ceea78f14ac022873ba567830',\n",
       "   'title': 'Prospective study of thrombospondin-1 mimetic peptides, ABT-510 and ABT-898, in dogs with soft tissue sarcoma.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/8e911d343f59a42ceea78f14ac022873ba567830',\n",
       "   'venue': 'Journal of veterinary internal medicine',\n",
       "   'year': 2012},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '47346730',\n",
       "     'name': 'Melissa C. Paoloni',\n",
       "     'url': 'https://www.semanticscholar.org/author/47346730'},\n",
       "    {'authorId': '6777667',\n",
       "     'name': 'Christina Mazcko',\n",
       "     'url': 'https://www.semanticscholar.org/author/6777667'},\n",
       "    {'authorId': '3017950',\n",
       "     'name': 'Elizabeth Fox',\n",
       "     'url': 'https://www.semanticscholar.org/author/3017950'},\n",
       "    {'authorId': '4079371',\n",
       "     'name': 'Timothy M Fan',\n",
       "     'url': 'https://www.semanticscholar.org/author/4079371'},\n",
       "    {'authorId': '34893853',\n",
       "     'name': 'Susan Lana',\n",
       "     'url': 'https://www.semanticscholar.org/author/34893853'},\n",
       "    {'authorId': '4755616',\n",
       "     'name': 'William C. Kisseberth',\n",
       "     'url': 'https://www.semanticscholar.org/author/4755616'},\n",
       "    {'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '46347072',\n",
       "     'name': 'Kaylee Nuckolls',\n",
       "     'url': 'https://www.semanticscholar.org/author/46347072'},\n",
       "    {'authorId': '39257719',\n",
       "     'name': 'Tanasa Osborne',\n",
       "     'url': 'https://www.semanticscholar.org/author/39257719'},\n",
       "    {'authorId': '6337024',\n",
       "     'name': 'Samuel Yalkowsy',\n",
       "     'url': 'https://www.semanticscholar.org/author/6337024'},\n",
       "    {'authorId': '47771931',\n",
       "     'name': 'Daniel J. Gustafson',\n",
       "     'url': 'https://www.semanticscholar.org/author/47771931'},\n",
       "    {'authorId': '6594328',\n",
       "     'name': 'Yunkai Yu',\n",
       "     'url': 'https://www.semanticscholar.org/author/6594328'},\n",
       "    {'authorId': '145366281',\n",
       "     'name': 'Liang Hong Cao',\n",
       "     'url': 'https://www.semanticscholar.org/author/145366281'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1371/journal.pone.0011013',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'b94c18c34b1ab737280c5771182fb8e22824b0b3',\n",
       "   'title': 'Rapamycin Pharmacokinetic and Pharmacodynamic Relationships in Osteosarcoma: A Comparative Oncology Study in Dogs',\n",
       "   'url': 'https://www.semanticscholar.org/paper/b94c18c34b1ab737280c5771182fb8e22824b0b3',\n",
       "   'venue': 'PloS one',\n",
       "   'year': 2010},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'},\n",
       "    {'authorId': '1382417671',\n",
       "     'name': 'Kerstin Lindblad-Toh',\n",
       "     'url': 'https://www.semanticscholar.org/author/1382417671'},\n",
       "    {'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '5287792',\n",
       "     'name': 'Cheryl London',\n",
       "     'url': 'https://www.semanticscholar.org/author/5287792'},\n",
       "    {'authorId': '50404089',\n",
       "     'name': 'Philip Bergman',\n",
       "     'url': 'https://www.semanticscholar.org/author/50404089'},\n",
       "    {'authorId': '47723616',\n",
       "     'name': 'Lisa Barber',\n",
       "     'url': 'https://www.semanticscholar.org/author/47723616'},\n",
       "    {'authorId': '1877585',\n",
       "     'name': 'Matthew Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/1877585'},\n",
       "    {'authorId': '47959301',\n",
       "     'name': 'Barbara S. Kitchell',\n",
       "     'url': 'https://www.semanticscholar.org/author/47959301'},\n",
       "    {'authorId': '144456668',\n",
       "     'name': 'Elizabeth McNeil',\n",
       "     'url': 'https://www.semanticscholar.org/author/144456668'},\n",
       "    {'authorId': '3740943',\n",
       "     'name': 'Jaime F Modiano',\n",
       "     'url': 'https://www.semanticscholar.org/author/3740943'},\n",
       "    {'authorId': '46511076',\n",
       "     'name': 'Steven M. Niemi',\n",
       "     'url': 'https://www.semanticscholar.org/author/46511076'},\n",
       "    {'authorId': '6121318',\n",
       "     'name': 'Kenine E. Comstock',\n",
       "     'url': 'https://www.semanticscholar.org/author/6121318'},\n",
       "    {'authorId': '46373698',\n",
       "     'name': 'Elaine Ostrander',\n",
       "     'url': 'https://www.semanticscholar.org/author/46373698'},\n",
       "    {'authorId': '47943642',\n",
       "     'name': 'Susan Westmoreland',\n",
       "     'url': 'https://www.semanticscholar.org/author/47943642'},\n",
       "    {'authorId': '4459370',\n",
       "     'name': 'Stephen J. Withrow',\n",
       "     'url': 'https://www.semanticscholar.org/author/4459370'}],\n",
       "   'doi': '10.1038/nbt0906-1065b',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'ed388bc4e317cc7aaceeaff7c12c9b1d41651ada',\n",
       "   'title': 'The dog as a cancer model',\n",
       "   'url': 'https://www.semanticscholar.org/paper/ed388bc4e317cc7aaceeaff7c12c9b1d41651ada',\n",
       "   'venue': 'Nature Biotechnology',\n",
       "   'year': 2006},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '5528655',\n",
       "     'name': 'Karin U Sorenmo',\n",
       "     'url': 'https://www.semanticscholar.org/author/5528655'}],\n",
       "   'doi': '10.1016/S0195-5616(03)00020-2',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'd36244572047f877b2f868a5ee345ffaeb2441d7',\n",
       "   'title': 'Canine mammary gland tumors.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/d36244572047f877b2f868a5ee345ffaeb2441d7',\n",
       "   'venue': 'The Veterinary clinics of North America. Small animal practice',\n",
       "   'year': 2003},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '145566721',\n",
       "     'name': 'Brian W Davis',\n",
       "     'url': 'https://www.semanticscholar.org/author/145566721'},\n",
       "    {'authorId': '3252069',\n",
       "     'name': 'Elaine A. Ostrander',\n",
       "     'url': 'https://www.semanticscholar.org/author/3252069'}],\n",
       "   'doi': '10.1093/ilar/ilu017',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '0ec1bb1ca0d4d0f09469692a2cf17f9ea190f340',\n",
       "   'title': 'Domestic dogs and cancer research: a breed-based genomics approach.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/0ec1bb1ca0d4d0f09469692a2cf17f9ea190f340',\n",
       "   'venue': 'ILAR journal',\n",
       "   'year': 2014},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '40236436',\n",
       "     'name': 'Heather L Gardner',\n",
       "     'url': 'https://www.semanticscholar.org/author/40236436'},\n",
       "    {'authorId': '40425427',\n",
       "     'name': 'Joelle M Fenger',\n",
       "     'url': 'https://www.semanticscholar.org/author/40425427'},\n",
       "    {'authorId': '4396138',\n",
       "     'name': 'Cheryl A London',\n",
       "     'url': 'https://www.semanticscholar.org/author/4396138'}],\n",
       "   'doi': '10.1146/annurev-animal-022114-110911',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': 'cc19316e1ea87a093624d78fa65cf2372de1d46c',\n",
       "   'title': 'Dogs as a Model for Cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/cc19316e1ea87a093624d78fa65cf2372de1d46c',\n",
       "   'venue': 'Annual review of animal biosciences',\n",
       "   'year': 2016},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '2038296',\n",
       "     'name': 'George E. Block',\n",
       "     'url': 'https://www.semanticscholar.org/author/2038296'},\n",
       "    {'authorId': '3205817',\n",
       "     'name': 'Elwood V. Jensen',\n",
       "     'url': 'https://www.semanticscholar.org/author/3205817'},\n",
       "    {'authorId': '4092163',\n",
       "     'name': 'Theodore Z. Polley',\n",
       "     'url': 'https://www.semanticscholar.org/author/4092163'}],\n",
       "   'doi': '10.1097/00000658-197509000-00017',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '79b1834c1c5459fd75989076cbd27e9673d87d6b',\n",
       "   'title': 'The prediction of hormonal dependency of mammary cancer.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/79b1834c1c5459fd75989076cbd27e9673d87d6b',\n",
       "   'venue': 'Annals of surgery',\n",
       "   'year': 1975},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '4920847',\n",
       "     'name': 'Amy K Leblanc',\n",
       "     'url': 'https://www.semanticscholar.org/author/4920847'},\n",
       "    {'authorId': '50860878',\n",
       "     'name': 'Matthew S Breen',\n",
       "     'url': 'https://www.semanticscholar.org/author/50860878'},\n",
       "    {'authorId': '143999992',\n",
       "     'name': 'Peter L. Choyke',\n",
       "     'url': 'https://www.semanticscholar.org/author/143999992'},\n",
       "    {'authorId': '35068825',\n",
       "     'name': 'Mark Dewhirst',\n",
       "     'url': 'https://www.semanticscholar.org/author/35068825'},\n",
       "    {'authorId': '4079371',\n",
       "     'name': 'Timothy M Fan',\n",
       "     'url': 'https://www.semanticscholar.org/author/4079371'},\n",
       "    {'authorId': '2037488',\n",
       "     'name': 'Daniel L. Gustafson',\n",
       "     'url': 'https://www.semanticscholar.org/author/2037488'},\n",
       "    {'authorId': '4169821',\n",
       "     'name': 'Lee J. Helman',\n",
       "     'url': 'https://www.semanticscholar.org/author/4169821'},\n",
       "    {'authorId': '5633163',\n",
       "     'name': 'Michael B. Kastan',\n",
       "     'url': 'https://www.semanticscholar.org/author/5633163'},\n",
       "    {'authorId': '2062572',\n",
       "     'name': 'Deborah W Knapp',\n",
       "     'url': 'https://www.semanticscholar.org/author/2062572'},\n",
       "    {'authorId': '153273057',\n",
       "     'name': 'Wendy J. Levin',\n",
       "     'url': 'https://www.semanticscholar.org/author/153273057'},\n",
       "    {'authorId': '4396138',\n",
       "     'name': 'Cheryl A London',\n",
       "     'url': 'https://www.semanticscholar.org/author/4396138'},\n",
       "    {'authorId': '48162520',\n",
       "     'name': 'Nicola C Mason',\n",
       "     'url': 'https://www.semanticscholar.org/author/48162520'},\n",
       "    {'authorId': '6777667',\n",
       "     'name': 'Christina Mazcko',\n",
       "     'url': 'https://www.semanticscholar.org/author/6777667'},\n",
       "    {'authorId': '4549008',\n",
       "     'name': 'Patricia N. Olson',\n",
       "     'url': 'https://www.semanticscholar.org/author/4549008'},\n",
       "    {'authorId': '30707796',\n",
       "     'name': 'Rodney López Page',\n",
       "     'url': 'https://www.semanticscholar.org/author/30707796'},\n",
       "    {'authorId': '5866492',\n",
       "     'name': 'Beverly A. Teicher',\n",
       "     'url': 'https://www.semanticscholar.org/author/5866492'},\n",
       "    {'authorId': '2465310',\n",
       "     'name': 'Douglas H. Thamm',\n",
       "     'url': 'https://www.semanticscholar.org/author/2465310'},\n",
       "    {'authorId': '78367986',\n",
       "     'name': 'Jeffrey M. Trent',\n",
       "     'url': 'https://www.semanticscholar.org/author/78367986'},\n",
       "    {'authorId': '2244975',\n",
       "     'name': 'David M. Vail',\n",
       "     'url': 'https://www.semanticscholar.org/author/2244975'},\n",
       "    {'authorId': '145926594',\n",
       "     'name': 'Chand Khanna',\n",
       "     'url': 'https://www.semanticscholar.org/author/145926594'}],\n",
       "   'doi': '10.1126/scitranslmed.aaf0746',\n",
       "   'intent': [],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '0862aec72973c7844905fcbeec5ac6e9511f91bf',\n",
       "   'title': 'Perspectives from man’s best friend: National Academy of Medicine’s Workshop on Comparative Oncology',\n",
       "   'url': 'https://www.semanticscholar.org/paper/0862aec72973c7844905fcbeec5ac6e9511f91bf',\n",
       "   'venue': 'Science Translational Medicine',\n",
       "   'year': 2016},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50368588',\n",
       "     'name': 'Deidre Cohen',\n",
       "     'url': 'https://www.semanticscholar.org/author/50368588'},\n",
       "    {'authorId': '6665100',\n",
       "     'name': 'John S. Reif',\n",
       "     'url': 'https://www.semanticscholar.org/author/6665100'},\n",
       "    {'authorId': '8348131',\n",
       "     'name': 'Robert S. Brodey',\n",
       "     'url': 'https://www.semanticscholar.org/author/8348131'},\n",
       "    {'authorId': '1832923',\n",
       "     'name': 'Harry R. Keiser',\n",
       "     'url': 'https://www.semanticscholar.org/author/1832923'}],\n",
       "   'doi': None,\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '182459ed92b5b07a8f5df80d78c87319f23afca8',\n",
       "   'title': 'Epidemiological analysis of the most prevalent sites and types of canine neoplasia observed in a veterinary hospital.',\n",
       "   'url': 'https://www.semanticscholar.org/paper/182459ed92b5b07a8f5df80d78c87319f23afca8',\n",
       "   'venue': 'Cancer research',\n",
       "   'year': 1974},\n",
       "  {'arxivId': None,\n",
       "   'authors': [{'authorId': '50319629',\n",
       "     'name': 'Jenny Chang',\n",
       "     'url': 'https://www.semanticscholar.org/author/50319629'},\n",
       "    {'authorId': '33147794',\n",
       "     'name': 'Richard M. Elledge',\n",
       "     'url': 'https://www.semanticscholar.org/author/33147794'}],\n",
       "   'doi': '10.1023/A:1012203917104',\n",
       "   'intent': ['background'],\n",
       "   'isInfluential': False,\n",
       "   'paperId': '40350e7419254c730c21aa8926975386425d2459',\n",
       "   'title': 'Clinical management of women with genomic BRCA1 and BRCA2 mutations*',\n",
       "   'url': 'https://www.semanticscholar.org/paper/40350e7419254c730c21aa8926975386425d2459',\n",
       "   'venue': 'Breast Cancer Research and Treatment',\n",
       "   'year': 2004}],\n",
       " 'title': 'Comparative oncology: Integrating human and veterinary medicine',\n",
       " 'topics': [{'topic': 'Veterinary Medicine (discipline)',\n",
       "   'topicId': '15486',\n",
       "   'url': 'https://www.semanticscholar.org/topic/15486'},\n",
       "  {'topic': 'Canis familiaris',\n",
       "   'topicId': '40476',\n",
       "   'url': 'https://www.semanticscholar.org/topic/40476'},\n",
       "  {'topic': 'Aging',\n",
       "   'topicId': '947',\n",
       "   'url': 'https://www.semanticscholar.org/topic/947'},\n",
       "  {'topic': 'Malignant Neoplasms',\n",
       "   'topicId': '2672',\n",
       "   'url': 'https://www.semanticscholar.org/topic/2672'}],\n",
       " 'url': 'https://www.semanticscholar.org/paper/dd81361979e154e6d2ecfda702436bbd1ab8c0b4',\n",
       " 'venue': 'Open veterinary journal',\n",
       " 'year': 2018}"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "requests.get(api_url + '10.4314/ovj.v8i1.5').json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join(scraper_dir, 'doi2paper.json'), 'w') as f:\n",
    "    json.dump(doi2paper, f)\n",
    "with open(os.path.join(scraper_dir, 'errors.json'), 'w') as f:\n",
    "    json.dump(list(errors), f)\n",
    "with open(os.path.join(scraper_dir, 'notfound.json'), 'w') as f:\n",
    "    json.dump(list(notfound), f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "65309"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(doi2paper) # 22938"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "264"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(errors) # 98"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4147"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(notfound) # 1526"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Citation network dedup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cit_df = pd.read_parquet(os.path.join(cord19_dir, 't60_citation_index.parquet'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "for idx, row in cit_df.iterrows():\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "citation_id                     ff5b04afab04334df136f78f8fee03af5dd565c5\n",
       "approx_citation_ids    [4073b3fad31dd01667e794bb7db1dccce400ea92, 1bc...\n",
       "near_duplicates                                                     1068\n",
       "rank                                                                   1\n",
       "Name: 0, dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['4073b3fad31dd01667e794bb7db1dccce400ea92',\n",
       " '1bcfcc6d113f9225467954ed880dc9d1dddc584d',\n",
       " 'b91c07de17f79cfbdb4046602c377e590393cc8e',\n",
       " '2074224e10bf884718b7373665ff372525cbb830',\n",
       " '1936c179aecd5b3895a9aac32221ff03230a7a63',\n",
       " '95497926021f939c17b2c267773cd20756370fed',\n",
       " '90308637bf6ffb3a626dd19975bb4ee287a3ce62',\n",
       " 'ab1c38d2c087c20f6784984a23604cc882b4a8ac',\n",
       " 'e02547b34463accb464d91d8de93ae95f50d4f66',\n",
       " '70fdc2c9713c71c74591bda72ee9af638e4dc695',\n",
       " 'bedeaaeee17a6166ef5bca20b4d65f10ad6d47dc',\n",
       " 'b5b84516e3e2154ebbf424298223e98512d7f76b',\n",
       " '13ee2b0843adeab2103dd67ec4d65ca793cf5687',\n",
       " 'e49191b299e7434174401161c9860eae8c5ae481',\n",
       " '0fd656403484903391493e46c1e1047f27c8a4da',\n",
       " 'b075f06f8b901152c10206a84dbff1d267c88063',\n",
       " '25757bba0841229b6690343b20ce252e5e3a1af4',\n",
       " 'd8515865b23baeaccda2068b54a07abb8fa41ac6',\n",
       " 'd442a361e7d777e8675f7a9464feec176515f26c',\n",
       " 'c9732ee56d2491f1acc37bb65c165b4d4f1dc6c7',\n",
       " 'b97eed61f6edee125a19346a63be5a319c77a0df',\n",
       " 'ed642bdbec29cc1738fcf628e58cac77257b475f',\n",
       " 'a0d5def2613899c3e2687af5563093492257c666',\n",
       " '59454f71166de30da3ff7b54738868b5b1c50a73',\n",
       " '9dabede0b5de01394ff9683c8de09002093649f0',\n",
       " 'b94c11a55e47ec3630254b2586fcb53534495472',\n",
       " '9c4e3d662df9c2a6215636fa3742cd0be46cf30c',\n",
       " 'a812f696b0182f5b9be4d0a2f7c6ba8d74a34c1d',\n",
       " 'ad3945331187c36be2fb7f0f841f76d601e9c99d',\n",
       " '5a1ed9fa94832d92b9d58b8ae0a80a1170691207',\n",
       " 'c6f1ca2830aeac43cd7bd48c2e75cc9cf52595a9',\n",
       " 'dce75dd0c405c800bd8bb016f08375be2fa3f38d',\n",
       " '321efd6b6a41810c8174e8c4632e193bad34a6c8',\n",
       " 'f2975b3eecad42ca95cf48b1f4b7bff1bd1cff05',\n",
       " '940680513730b4e8ec090ed7c4557bef571b0916',\n",
       " '51c279ebfcdbccc0c257acc56e1cbb368983eca6',\n",
       " '71d1bc9b4c52d289c1947cd82799b84bb64118a6',\n",
       " 'c1a795e77751b372ba4db538d3672100253904e9',\n",
       " '2b838ea6876cf1100b0ed676d8ea185674a05e2a',\n",
       " 'e18d6d81146a386d44f17326bc572eed8ac44256',\n",
       " 'c49e65eaef0fe196f817b006bfee89b730bb2cb0',\n",
       " '320b0732897b59757b223c8311895288c221f506',\n",
       " 'b4d054715854f26849be6bd0e4c6cd040a671cc3',\n",
       " '8b6edec5bee9bbe7ec3e2bb46661ae0f444839d7',\n",
       " '0cd6e8138baa003eba260d6bf9c69316d9be987f',\n",
       " '62f9da75f20c029ce11dc7ecb3f3d85488fcd003',\n",
       " 'f94c51cc894fd8b081a78bcec60a95569fea9832',\n",
       " 'a56b41a24999fcb35101de5d48edf24b2fb970c1',\n",
       " 'cf42a5030882e5e847ceb6b58404c4f99019672a',\n",
       " 'e08259e3d693fe87586c391b1468d1f5b0f9e008',\n",
       " '2bf7719fc01306a77691d3708cb68ad6caf09f94',\n",
       " 'cbdd9b8c9045fbbe13c951ac756ee2bd1cae82a1',\n",
       " 'ce3bda56e720b03c77fd24ec433d2b2dcdb8e761',\n",
       " '16efa1520d3f1aebfe32ddc2bf00a37486097247',\n",
       " '272615937af3536c040f1013f1c48e7e7bef7998',\n",
       " '39daa18692ee08fce67061a41fa30fe2bc7e5c74',\n",
       " 'ec4516c1a5105658b55e9846bcf363e588d2165e',\n",
       " 'd65b963a976ae02badbd10617c6c4d3e59795b80',\n",
       " 'd29d5e0fcec647b764cdbc0c00dfcca09b8ae99b',\n",
       " '796cb0152939771aaefd0f260f00d2a6f18b607d',\n",
       " '283a030c693ba32cc5b74fee83df1702b65d6c8a',\n",
       " '80e2feffaffd30eb411176626eac3ed9a9086e27',\n",
       " 'deeb89351c875cea208331f4a74158db21dc11a7',\n",
       " '337ac06deaebcca32a237fc4a8469ef1f3da4f2e',\n",
       " '01c02b75cbc042e73a7c758f21f96b054ff3f9b4',\n",
       " 'd6b81da1fcaa20b76c786ebb3680bad28335b8cf',\n",
       " '804f9387e6a067d8d47d8c36c0f2514d8fea82c3',\n",
       " 'ea93df2c83fb2556ef2fb0e346eca65d0cd9d209',\n",
       " 'c95e49ea55de9f9f0298f236a2860b0e9fd0f25e',\n",
       " 'bfd2cbaeb6fa1a95df6f165c7038e329708fa79a',\n",
       " 'bce3d134f38e530faa993fd57d02add228c74a07',\n",
       " '190041f4a28a9d7983eaa701298e8450c3b19fe2',\n",
       " 'cc7f01c4b6921ef5bf5896cb085c69570848e611',\n",
       " '1e569cb7e71ad66e1896fc3ef739b9b7c670c554',\n",
       " '0131369e429c1903d8e7cb97641b866f919f66fd',\n",
       " '40082570cc911235097602a7aec3dd1661d417b6',\n",
       " 'dac1c7b21692c7f68b1db9b7a869ad9264c06945',\n",
       " 'a181a52a2893c9f812478a6bcf2ab2f63f3c7709',\n",
       " '1cfb4b5cb47769c5779077dac1026da6d870e856',\n",
       " 'ecd70c8dc7a3655bc51bfb17ab31140d7614ffcc',\n",
       " 'ae9036e258cb4a4b7d9c059b438636d4d974df3a',\n",
       " 'e8c6ca000632a8f9d6277057c8d6f66d4c52f4d5',\n",
       " 'a24ee9d15ac4f91fd5b647c972deaf94806300a8',\n",
       " '7a44fb9c41d3ac51e95978e0f13905b428e52cb2',\n",
       " 'c0b7e0e88d37b2dfcbdd5e80f6ff0df4680a62eb',\n",
       " 'a7d0fd2762a4624c7b4c335d66ce384f7f59a93c',\n",
       " 'd31ad1be6879f0b97da2d650debd42d5d1c6e61a',\n",
       " 'ecfc40eef9008c2ae6ae1775241b594cbb9b304a',\n",
       " '3f158e1ca2013d3849ef7e1a31298d50da537d00',\n",
       " '9095032c5afd0afe83934172907b7734c34f4a55',\n",
       " '59961390d05d42132a923223a16aaa35bffd9a86',\n",
       " 'ae0402f211a11965b4890947f7975aff8eaac552',\n",
       " '2e886322ad9af61c92815c5863e7a634551b1cd0',\n",
       " '28a797805549b02241efe7d5cc10a8b94c7724d3',\n",
       " '07a6fbd404a4f85783342e30c8c36c84fc1e132c',\n",
       " '347bb55c9d7d2301e4dc266f492605a21f8f2f1c',\n",
       " '52df146fa0362dd24512ed7d9e2366a1ac9a5ff3',\n",
       " '96f0b5ff037d79269d4d159a58484c52f57a903a',\n",
       " '806ef16307ee611cb6ca638e453733abfc89a11c',\n",
       " '424db4561f18c28bbcdbd482f9a24022660395cd',\n",
       " '9d747c585651333902331fd1bfe8ebc19a5f4196',\n",
       " '8c1494524c64f6325a2791900fdbeb90092c5b4c',\n",
       " 'aadbd7911887926bb2228495ffeb8d825b3a9555',\n",
       " 'cd6bc1cc9b562150e756fb681a3931c718fc2f97',\n",
       " 'deb06f74a30b6e867fffc2ef2cea876a4c2bd202',\n",
       " '92b8cffe8bb08a6429b2eec66f887e7e4b71c2d3',\n",
       " '22a0500c5434d95fdd6b9bf7bf671fc7322f08f3',\n",
       " 'fd08bc3e4242ae4bbd7f5cda5f85ccc884c57aaa',\n",
       " 'b7920c3c10bd76cf3e375162426eac2486b8d0c4',\n",
       " '1db15f1134f55adca77cc7ef100e603ceec5535f',\n",
       " '97b47ed946c14d402610dd7e6d898e5913a16bf1',\n",
       " 'e4aa1ecdc211da7262cabaaa7ea93b120f3ac1d1',\n",
       " 'bd5404ab563075a93c41d5c80cd4c60d14a54722',\n",
       " 'c9262d6b1760fe25a4a8b424fd7d51f3aa797f8c',\n",
       " '3c1ac50652858d2c603bb498194ba686cf2ba878',\n",
       " '7228a1e8914c09328728eb5c17800334f6407ba8',\n",
       " '04ca261c7fefb77a1e2827db79e104f78c2067c1',\n",
       " 'bb241ab81489aa165b79643a73d89575785efc99',\n",
       " '56f65f5949296849e754488c47a76f48f93f8e33',\n",
       " 'c6e6c1a4c5d1221f0930c4d925e4a8eaa8b6a5e5',\n",
       " '89184a8157c90241f1da73da85b82556596ae58e',\n",
       " 'd858ec8a31518882c8ed39ac1159ef5f04569da6',\n",
       " '2a3aebb9fdab45fefafd7204622c46c2aff3631b',\n",
       " 'b313ab51b64b3ae78348d7e30991b27a71e46806',\n",
       " '3bdd1a3ccd52a94248197ca222fcdd76c74a6750',\n",
       " '11ad2310fc108deb13261eb803a27aefb05b8522',\n",
       " '4a3a851edc1dc9a76c24953b3b11e59838a41977',\n",
       " 'a598503270557222cc643e8b1eca674f5c2cc2b5',\n",
       " '473e485638ff4e8758f82c1c524d38c7120d8fdd',\n",
       " 'bdd08dd634fbde081dcd560122bfbab78c75e987',\n",
       " '26e73a2ae7a73abb96a106bdfe629f82a9aea15d',\n",
       " '1ecf7c6159473cb30e39698241381ad038734d79',\n",
       " '86294febd8c8ba2c605de4a9d3614edb1c36102a',\n",
       " 'ddfcd4655d2f5bb9e499861ad249f3b02198b2fd',\n",
       " 'a7d487dc6975b08be8dbd3998e53a1b9c5e4615f',\n",
       " '1abfbfa555906c47bd8b523be19885abe6897cf3',\n",
       " 'da3350c174b7e5ba5d972cb47fddc05d726a0516',\n",
       " '702dde08e29a3bacb67fe69f14965200cedc862a',\n",
       " '25c0abbf960cc4fb3f9830be44b5bed614dc083e',\n",
       " 'ca2363b1edaaae2abe6509aada5918025fea0a6d',\n",
       " 'cf435a567e9896fa2f1111ba95ab734f4be5a502',\n",
       " '1c2f40701d62b7eb049e1a9d61da20584c6dba01',\n",
       " 'c6ebd647a5c26e2906acf8f60ce0264eacf26ddd',\n",
       " '0f6f85b5a1952686a9a0345352bd3f482914819a',\n",
       " '7007b5f6799fefea7992844bf052cbab75bf1dc7',\n",
       " '6f85385381814b69fe3f772eb34eaf0798445273',\n",
       " '20d2dbf233e85c078f3d5549eaf222738911aa9f',\n",
       " 'fe07b0cd319bc94745e3210d2c5fec343114ea1b',\n",
       " '97f1e71b72f670c158403eb16364fe3a9ca53a1b',\n",
       " 'e3b4fa3fe45a9968f6b358a4cd189a4e973a4123',\n",
       " '302082d161c292e90df4c00fd0526a24a81d8548',\n",
       " 'ab36348fdc90790aee6ba2e393df8f9bff01446d',\n",
       " 'd6de7ab7ca5a64cb5cc7b43014155e8181e5de2f',\n",
       " '1352f3e88615f6af7ddf4cc15909792a8aa887a9',\n",
       " '70ada7573e2553beb8d45f88439e4e7fda862954',\n",
       " '4d64788039d0c2c9e2c19d21238e01c0b8844b9f',\n",
       " 'a87a6c1c0fa024f0ff972c7b2cd6ddfa9717aff0',\n",
       " '911fafa1036a207fe8f19d2d96ee4635bcbf847b',\n",
       " '0d5a76b58eb306eed10de7cb4a1ba499eed36943',\n",
       " '78524dcd99bf9a5cbb66989216e7e3cd65a94ac0',\n",
       " '4919ce8ffdc981277dd6a370ca60b143a1d20ff6',\n",
       " 'fdc1eb8402869cd20950acd54f81683573e18048',\n",
       " 'ac35448e9369d24b53a1e180d4ad2fc0b7cd5c0b',\n",
       " '864b6f1036ae92228d7234fb28acc7d2d8cf17d7',\n",
       " '156153888e9c785699fef19a6d1c3d7692a0657e',\n",
       " '665622f232675703797ad870c6246b96a8cf0fb6',\n",
       " 'e708b10ee8c149df3e2e994c805359d4ca81b2b8',\n",
       " '0e11f9c9f6e808947628a69310849d622b91ca33',\n",
       " '8a866bbf3d86d69d060f59a8faecc0c614929c9e',\n",
       " '4bd6ceeacae44e781c985f3b5a4b9b7462593e62',\n",
       " '444553604e5ec7379516b860a0122ea290fac3e7',\n",
       " '8fff34fd5fb4952e6255a95f2dc22a93560dc126',\n",
       " 'fc4fcf9387269cc1c517b2a05f6eca25d9ee626f',\n",
       " 'bf7dd683766d90b270a941018e1d7de180890d74',\n",
       " '741861c075f73b914322ebb94df08a77e11a476e',\n",
       " '009619b1fbbae7270a5be15bda91e5d6b48f2068',\n",
       " 'f66be4888a8ab678f0ee88dd9e06cc0dfaefdd85',\n",
       " 'a02822d330b3708d9b8574127f2309feb6a258be',\n",
       " 'f7057c659bee597cbcc44899989688c46000852b',\n",
       " '76cf1db0e57dde04422a88353ecbc8db58889219',\n",
       " 'd9c4b10129522e7d011e3a15e6f9dcfc4b3fbd2b',\n",
       " '5c4e93e0a23077a5b0a0d31f8c0e47fc5df6cf0e',\n",
       " 'b7d99dbff4373698a7797c19952ea7835b30a724',\n",
       " 'cbf7f5b89db3f5fe33bf2bc577c1bc30587dd824',\n",
       " '6e9aa00f3c1c302918f7e6b10391f99705c284c9',\n",
       " 'ba6301aa7ce58ad636d389ea12111dae19a78bec',\n",
       " 'fcb8b0aa4a2e2926b9a6e26dee84653af9538555',\n",
       " 'd0bd690b4c8a5644e16f0b0f3e80c37d8d998d69',\n",
       " 'b40867dc8a327031ea4f4e829fe0036ce7db3bb1',\n",
       " 'b8b35780f8b13c2b36834116387d8511e5ea2433',\n",
       " 'f8fa4ade0d66985024ce1a65e61982b52b88e3d1',\n",
       " 'df3a930bc94a2ffc4e9990b1562e98448dd60d12',\n",
       " '25344277812e504067555b4a85c4f1e42a295509',\n",
       " '1cc5496ed990f595fa914c92ae905f5571db3dd7',\n",
       " 'bc490a74f52ba13cb4e6272f3da382f04fc777e6',\n",
       " '2bd2ef02b479954885ed4971dd9df3fb6eb0fe5a',\n",
       " '0e4e70f399342a0cda843939c00d1b4d69151abe',\n",
       " '9d74c93fb30412fad29eb78440efc34c46c7eaf5',\n",
       " '81da26a1588de1ba332450ce984e86846b0bb347',\n",
       " 'b9fd0687e48780f9face2cc252910389ea1c4625',\n",
       " '11774c56b1763fcc568a169a049e06abcebdd419',\n",
       " '19c0d589690e9bd1c30a8730fd8ded851a97beb2',\n",
       " '1077dfe9620565c5f2a3b7a323454ea0d585e1c3',\n",
       " 'ab0543f2497fd34654211260fe80b422f34be957',\n",
       " 'cfefcb4c958569d1f61ef7a528124084ba507651',\n",
       " '78da15c508dda50fb40d3cd4e959a053411d700b',\n",
       " '7c59fb9ec62d6d6328287a410808aed90d0b742b',\n",
       " 'bbf87b7c410b44b4f8a39937db2cfe1c8d8e9cb0',\n",
       " '54e4a3925bc99765fe6621680e397fd1a579d842',\n",
       " '13e6bb792c50d2cc0d2c8e51959edf90a7577850',\n",
       " '8aa6d08d4b79481bb12221f9118606f513d49be0',\n",
       " '0b2dc741daebc41569ee33ebed96c6d432719cbc',\n",
       " '36c653a19c8ad28cbbc0b75d47b82e202dc876ed',\n",
       " '1f6595fe4ad1aa1818e7b99a05ca77f71f048593',\n",
       " '7b905a029aba80ebea41a71abccd355946fa768a',\n",
       " '26f3163569c573ac6306227bc5593f135f869eab',\n",
       " 'e247879efe529cb930780ea0164b622c26d0c155',\n",
       " '0c37942c3a14d0073395a8f8d08d8fdad72ea66e',\n",
       " 'c2112f02dea8ad53498a9c2532e829d11b521c16',\n",
       " '9e841b79433b7f4393702b24eb7e6d9716f12188',\n",
       " '5d367ead62ac8f33f61bfbab04f4b7970f64f8c1',\n",
       " '54deef2c59f214fc9d9893038d21c0225a0f6d55',\n",
       " '2ecf05d90b7fd5ffffac48c3c7bed04d3cb2dcd5',\n",
       " 'b541fbe5fb57785d85a226041a15d61a414bcb6d',\n",
       " '3a6e0bfed399b468d985acdf83598fc61a607676',\n",
       " '2f44d4bd55811f9d86c5e1a9d8e41fbe0cabb5b9',\n",
       " 'd5822e9a11ac860580aad41b97ff100d929c49a0',\n",
       " 'a24ac0d1cbac833225de4fd387e2fe39cf4f48df',\n",
       " 'b6d497d5d6a03b78e87676632590011c205048fe',\n",
       " '9764de17b6cf10520037377c638a1c783e3e996e',\n",
       " '38cb9ae0ac4a669a8174b9c99fbd70a55a169915',\n",
       " '677e860ff37ca5846e1e24fbc9d47d1b031cb06a',\n",
       " 'e41d0a820c56afa10313058b10835f2ceb2c39b6',\n",
       " '031a9180e0fca531c8e1fa8d7a81523e23f4c240',\n",
       " '883c4b5b615cb4be537d71227a34768c6b43f26b',\n",
       " '3cd62e6ac94207f57cdc95846a93c5efac445fab',\n",
       " 'a69291428cd3d6bfb6bb11df5ea12393da6a8634',\n",
       " '00ea39646438002e6b8730f8c29d064522e32f39',\n",
       " 'b8b9637bd843652d5b8716b3929b5ec7f26d11e2',\n",
       " '5a92abefcbf077c7414ab0fceebae6113b170bc7',\n",
       " '07fbb4da6e9eaafcc9df31ef69ed62ffd3a437c3',\n",
       " '6ede257f380d99723e6b6749ef540f12f739f648',\n",
       " '3334bcb63a041b256ceb42c9d9fd616ed88d3ef9',\n",
       " '57664d40bceb1bf83b3b0ea319becd839ec1e555',\n",
       " '0944f4b44789bef2d085c91f72fcde554f0ee876',\n",
       " 'f5588eb4819688637c6ae5f4b81c89d6e6f21349',\n",
       " '23e35c3cf39808893baad8d3f84d6f573151cdff',\n",
       " 'ddb729b0c5d3f845ded3dc086bc71648300dd0db',\n",
       " 'e2df958cfbbd48b5476fc148fbb69719ea22a3ba',\n",
       " 'ef6e5a96e24878093959d2c8a07e8c9dd54709da',\n",
       " '4118e8c4b68798655737f1dde71722036a848cab',\n",
       " 'fc775f2bf40c3a3b6f8765f16eae81dbdc4f9c01',\n",
       " '524173305c3ff9cafccc4e12ef3c8c0335dd527d',\n",
       " 'f2f1688bc25d4d3ca12d90a54b9e2b246877c1e5',\n",
       " '9675db4165405ac8b02d63780edabd0ea06c9674',\n",
       " '021b1921a9c2288b2cb9d242c24826be2fc74312',\n",
       " 'c3e955c087c0dcf45e0ad1e3c1a249cb88cccf16',\n",
       " 'a951dd30c066ccc4079e025b6532d3a1fe490438',\n",
       " '5f5f475f2a0c346ef6ba3c0fe6165a844262bb47',\n",
       " '8898d3fcd66f7fcaf7b43c044c2baf0870225782',\n",
       " '9343af5988e741a8c348ce533b2daa93bfde8100',\n",
       " 'b8839d8664571d6be1e13f5aa0f88addec662ce9',\n",
       " '2fde38837f384036059b6adcba767d875f0519f4',\n",
       " 'c48850793e3ed4ab48df297ed49ba380c17ad5c5',\n",
       " '5e00e8b65eb128f9cec395209f03bf0a65801221',\n",
       " '7ff0c6cd89725acb9877bebe976671f2d3594b10',\n",
       " 'f5eda19d79c5db541dcc589f51257ab977446f2c',\n",
       " '14a5088cc4077110817c2a12448ccf62dc0dc28b',\n",
       " '016ee2808b38d109c818017070d92598843cb9cf',\n",
       " '0f569d9d63b989ddd86bae1123eb0135043124e8',\n",
       " '754d832ee5ce652c3259c089c9c2b530e0de82af',\n",
       " '123c59139aa3d323810713e8ad15f5162a263eaa',\n",
       " '36c9d6f5c783f2cf2c90933fbfbbf25487f34966',\n",
       " '2511a9239803a82e4740af9e234d22b26d4012f9',\n",
       " '2c924de6049f5f675cee75ff1ce7760b629f8079',\n",
       " 'f83779bde74274f59daf5ffa3693aded965c94cf',\n",
       " 'f29a0c05e1e3fa4c3027dcc6722ec24c80eae890',\n",
       " '86572996372331ed3f7ba397ff52a39f9bf5af84',\n",
       " '6e8ee08b98753d1918c1559adb0aaa46bc434640',\n",
       " '631847be0289a65c49a717d22f8ee2bae8a3bff7',\n",
       " '5e2e163f6301837669b09337afe658eaeb2a8da6',\n",
       " 'c073791065069dfc9d3d0db624dfefa1cf94b3c0',\n",
       " '655cd45b8ec75edc64380113626219b3b3ff9a3a',\n",
       " 'e60da6e4a2addd36140a9ca776e4ee4d3ef1a1cf',\n",
       " 'aff81dbc120a539764f771d08f4e6e35e64c380e',\n",
       " '2f8a18d331f83b956298c8b88ce08c1cbbfa47dc',\n",
       " 'd42ec3558709091b9ea92db2e35cc7d17fd53600',\n",
       " '438b199600501deda150461983cead7b8d47befc',\n",
       " '8cd81b45afdc279bda1c3c24f5b430050c3cb0f2',\n",
       " '40f58dd82485fdd18e1b7e9aa24000540197289d',\n",
       " '78b674a480356b7e2f54ca374816ad22062bc139',\n",
       " 'eb1d64a3a5bd9f17a0d9e8959dd4010377de1f8a',\n",
       " '9ecd5f15ab8024d0d2f9b9a122872889e614ae8d',\n",
       " '4402520eb501a8c98b5ef09b24233c4cf1053bfd',\n",
       " '318caa4203d8d237dbc9120f30fb2827bc6dad10',\n",
       " 'c9978811545fdbbb0e973c3229d5f87ba91f11f8',\n",
       " '3de91945346009c3ff75a088c8c248743cc6215f',\n",
       " 'd63d76fe10665ebffcb7d1e46b790955041048ff',\n",
       " '8a96c9a7bd38c6ecfe0a66f8c54b987f8eb877aa',\n",
       " 'a29badfe1f0ae440d3e75a5a6b7bab9a08d3be66',\n",
       " '50b3de6c1337510ef4e59066665490d4af05438d',\n",
       " '278a4cc67cd5ff581db4a1cde2a21691de6604d3',\n",
       " 'a14f61665f35fd2835446056561154636ad9fbfa',\n",
       " '5050304388db4823c35aec23cbe977823ee6ebfb',\n",
       " '7989d5264f37eb113d633d15951b12bc17d9479a',\n",
       " '114d258d8ee1fc312e315d2c351b95e4c2d67082',\n",
       " 'f1cf8d2256085b4e2ecb03a5a69fafda0aea247a',\n",
       " '3f4147179a2c570952f6e756017b9a83ec50f344',\n",
       " 'd3bb1c5e0ef0120459b3192445a8b5e2702633c3',\n",
       " 'dcec53706943af982d68741dddc5834bfef1f719',\n",
       " '41b3a3fc46c764e5c107393b972f2e0806774aff',\n",
       " '12bcca43904451f231bc5cb4e578aca95bf2d572',\n",
       " 'f2a09cb43f40678a1554e7790f7aa0d87b7f9804',\n",
       " '22127b608fec97f41eda809a13cabaf880136899',\n",
       " '8aec75181f2e52472e19a15b9b0c3b9b54247f8a',\n",
       " 'c126d574148d8bd4000a0542b5f26cc3e9bbb656',\n",
       " 'f28a339c7f41d212e1001e79d1e9dde405b565a6',\n",
       " '767004b20115e754d1f74914761b285ea047d5e4',\n",
       " '8320f32d9b3fa32ea20e58b8168d11b5792a020c',\n",
       " '8b17523e63d6b0b664b55067d9e352180aef7b48',\n",
       " 'e016c1acfe91aeff2fe0b6ff44b07b49a08f72d3',\n",
       " '7ac8b6c23e9d2c30060894779a7809c17114ff5a',\n",
       " '902dfeeafc9f115eaabcdd14164795da6ab51d1f',\n",
       " '87719ce630aab24efe299acb18e5b687b82ba217',\n",
       " '892f76e39317c4b0a02969d7f888896f81f0d99a',\n",
       " '863711174591d9fa2e9f266574dde8f6e5838398',\n",
       " '0681c581be64853a331a2c1671fe1b77fd1aac8c',\n",
       " '89548267e9686069840b3e74c689dd5d7f05fc6c',\n",
       " '09ece59b8c699c0edff243c43be7b7aaaee20fa0',\n",
       " 'fcabb56ee66d110bd23bd45adbe21b4728ec15fb',\n",
       " '1738332323d2a77cfcb7e2ffa6ebcad5bf61532e',\n",
       " '28ec7c4ed9604486ecf01ebc856c46e78a7b80d1',\n",
       " '20e4a83eeb10c3d40ce3b452e0e4b75d62c5688c',\n",
       " 'd0c50e44d7b0ff0f4af7035d2bdfb0825d5c37e6',\n",
       " 'a4fe0f05ee3453d7418f23aa30ea39602352e08f',\n",
       " 'df04062829db1e7f5fbf0ae9b5889a4c0cc1b374',\n",
       " '7621971ff05b651629c179fefaca2e473e36651c',\n",
       " '704bfdf5d71fec2dbda1fe2cf47ca26e4cd5ef87',\n",
       " 'ee77eda7bc3e0c797e42bbf8fc14494a2dc4b663',\n",
       " '0177c43cddba4fab09b7be68417c91139e60c21a',\n",
       " '952ef49797510e5abdd0c594e045ed48397e3a97',\n",
       " '5e931af9117af89abece4fc30c529760667c49ce',\n",
       " '4dc725abea0988fc463cb23eba066a5cbf5beab0',\n",
       " 'f447a9ce90559c27b13a6dcdc90dc1ee0c61e746',\n",
       " '6f6542fbb9887ade73a95ab8b5e120418ff9a886',\n",
       " '6e7eee764864b6b19087f7ad9b6de5974852bfda',\n",
       " '18eef79efdd87c881c22262ba3980f1bb3ca9cf6',\n",
       " '6fdf9416777880caed00f8653257165171c42bc0',\n",
       " '49fd4f168530945bbebc22e0111e377adb677c80',\n",
       " 'c0bdca07b4125d4743dcd8d74098bfba34f1e01e',\n",
       " 'f4a86b97060b1b771a327b0c4110459ab411e45b',\n",
       " '663d1496e04fb27378f30f92e22fe165015291a5',\n",
       " 'a46169836da6bdd756eac0af65400878c0bf2ab8',\n",
       " '8d6ad4d6932cd89a204dcbf44471eef87f1eb2c9',\n",
       " '8a6aa10e01adfc9e8308d468e1060627ffed14ad',\n",
       " 'ad210c0fb97862a472245ba8f55c9ff6509f7da6',\n",
       " 'ecdf0e30f7a3284a9a722ba86a1b64fe7a449e9a',\n",
       " '789b414168fc9b5d7af4948adabc52f84e4113ac',\n",
       " '6702185b2b47d598e4fd02658e6522ba337354b5',\n",
       " '144db56966a4c1fefe792ec4057db9376a580bdb',\n",
       " '976360228f44354af751f164fe580099ea949e47',\n",
       " '0f1635aae9aea8bbcc2f9a0f325b2ec860e73ea9',\n",
       " 'e2f588b6906c3c2601e27f89e3cb129052db2c06',\n",
       " '128cb02f97783505441fe2d2a8b6e6eb4da54d3f',\n",
       " '2dd912bb5afde976b0dd3313fc4f9c0102524bbf',\n",
       " 'dc7303a80fdafdbfadfb05e59282be3065eccb5d',\n",
       " '85ed45651d033e60961f969266fe436190ff78b0',\n",
       " '1ee553c16dae020454ba8d861b1ff4f93babe852',\n",
       " '64c14ad65ee9d4d6119492dd5bbfb129ff2de81b',\n",
       " '9c565f82e45373eeff287c556a39ce58bd543d0c',\n",
       " '1b0d1f3c950efe3f8f24affb1547daefe23f59b1',\n",
       " 'c2970e92ea9da9618c439812b320b7c141f8424d',\n",
       " '3e21e167fa4e498838474de9e40442501f7c86fc',\n",
       " '72fc47893dada04d5d376185b8ded661242abbe6',\n",
       " '69a4e14099e18bb406369ce1a562d6e5d71f3e7c',\n",
       " '7f6fcfe112303babc33735ceab37ce8e284ca2f8',\n",
       " '378de5050248dae18f18cbf702f4f30b833a3f65',\n",
       " '9b0fd2e4669b9359dfe98c353db7e1e05e6e5ad5',\n",
       " 'aad7b85a82a06aad961e31d0451f003a5c107f75',\n",
       " '5c11b88e348952cbd9d0579646d735116be1679c',\n",
       " 'c8bf8a99fd2766f8e04a7bded8c57b90f63bec8b',\n",
       " 'ac3e809d6bb8d404a9d4fdb5147937764bed9d6e',\n",
       " '53476ebb5c85da608d4f4df8c2e3d57ad2e40586',\n",
       " 'b8e01a92b6bfd60ebcf8ee5d121778403e82aaa0',\n",
       " '9a8d8d9307d7cf7aaef807c0fdca5c665eec5383',\n",
       " '56672d6eaa70213c606fcc253ca740aed5007c7d',\n",
       " 'd75c6562a8f92820eac9a1fda58893df9dbf4001',\n",
       " '5b617eaa37802cb0399b15be938a962ca5cdc509',\n",
       " 'a997e0a56128d6d45e725dec1576d5be480a8583',\n",
       " 'd871192bd07368aa5ff27b364130cbd54c9e2109',\n",
       " '7adc0daf2117b67d05fde5d8e047d0d4d38c4fa6',\n",
       " 'dfa4e578292f55f2f21c437929b69928af654f2d',\n",
       " '7d189bdc0954cf7dfc177397540be3ac0babc0bb',\n",
       " '054384bf9a573361b3c7eb00bfe99106c564b11f',\n",
       " '8b3a554fb2d6bcc620bcb68f3e65a08e9ed47d14',\n",
       " 'f538b82b0a66d42a887c445d1e39a5e3d43481b3',\n",
       " '92f3f4ca7095ccd9618b274122678e5c5958fd29',\n",
       " 'ecbed0b2fed6bf7c9b29856fe14851aed5fb2de0',\n",
       " '2ed1befeeeb14fc7a8418dc99057ff6fb2a3ca4b',\n",
       " '8c779edf93e01f4e6dc859b535ed307ceb98c357',\n",
       " '411ebc001a244d79d0c4bbd0c2bbc64b893c6d5f',\n",
       " '88300410cfde8f858ff585c38c87a8a23b593536',\n",
       " '98ca3746ac3713f36b31e768a4c9f9bb504135bc',\n",
       " '830624e592836e9d064f88caf2ef08263b3a7245',\n",
       " '4eafa1a5f1dc5a75d039e7e7188a373e1f131920',\n",
       " '648ff5677ef9a60b1d55d745e13ca97fe5b88137',\n",
       " '47d1156cd8b2b8a788ee265d3d0ac78197fcb232',\n",
       " 'eaaf9b310dd09bc826dbecfee1410f14aebb6411',\n",
       " 'bf380a4c42a65b0023993c8828f504a36668cf54',\n",
       " '8222895f2650a3126ec9599397646eea9c3d31a0',\n",
       " 'dedb1c6bfe55577a83941c0e4622a1b74cc402ec',\n",
       " 'cd10a08e37b6b9cfbd0cc5aab0e8ae55424eb539',\n",
       " '2ad6d4361d4addfa30b71a4a90972cfe9de03fec',\n",
       " '97640fbeb61a46070d984180e096dac41664251a',\n",
       " '425b1520c3f70df8e822a81d713c9f7c0045aa97',\n",
       " 'acab539408f7bc7528c918891235f87c82f1c65c',\n",
       " '5a12cea72d29622d3475f818d07159b9607dbfb0',\n",
       " 'ab085e3ff5c51f1d2d83ddc6db9ecbe29c7e40e3',\n",
       " '8e4144fa3b480bca42923c779782a96cf81cb5cb',\n",
       " '2ede704e842f4108b0ef7792dfbcd7a9c1bb4d32',\n",
       " '4f607ec2628a0c4891aae82d3d6ea81242669a6b',\n",
       " '994cab45f76458cf985b7593e2371124eac72a6d',\n",
       " '3ec7b52727312efe0af6dc201bdc6486766f4f65',\n",
       " '5478f06105c5403ada85cac06fd9d859cd375f27',\n",
       " '896bd03eec02bddb97db9fd757fc6bff77196f31',\n",
       " 'fa7b4e3d0fcc666fb69afbbca7272985ee5cf425',\n",
       " 'e350213864b0281225b4a1d006277afa65b7d236',\n",
       " '0fd4cc18823ea1cd53691c5f20c2c1d55feeee9a',\n",
       " 'ab04f4772e5fa69dfcecf3f4fcad872b5f65ec25',\n",
       " 'ad276944dcdafbdabe85cf3d12827034e776cea4',\n",
       " '14ebc6ce392772dea7c19ac23a3ce2f8d36a4f0c',\n",
       " 'dcd3684022858683bd6336cc796aeba11d6de4e7',\n",
       " 'beb5a9c813bee3a059345b01db90f7cbebbcb762',\n",
       " '9da4c7b0983c263442419c79251844742a8ddd76',\n",
       " 'ee70c8a3f92670e7b605499b5516d4569234340d',\n",
       " 'd007fd5e8983104b6532158b3d921e55c8c0ce75',\n",
       " 'bf0bdcea061e682f0ff68c242ce1ef3d94e85ddc',\n",
       " '76627383bf8bad65468dd4619f275b2834d47541',\n",
       " '3d0d972fea1d4b89bbbbe714a3067b18f905f2e8',\n",
       " '81e30cd8972c9dd584e359a33d9ed61e642dfa63',\n",
       " '654f5441e45da361f8fd8272e17a072aa2327a20',\n",
       " 'b2ded9d5e924cc8311539388d3a8d86e01147607',\n",
       " 'dfb1d79bfe0f123411e7f0b84b70426d7f8dd083',\n",
       " 'dbf1276d21967ebf325b7b820f976a7080119301',\n",
       " '947b9e88812b2f7f5b125df96bd9e8c6fd485961',\n",
       " 'a3948d82a52cbad87d9474c92de72fc14b10ff4e',\n",
       " 'dbd775a560601ee770104715dcffb5c18ad0480b',\n",
       " 'f503a34caa210a0c2152d9519d09da951c8fa71d',\n",
       " '4dbd2dde3503558f9abc1bfa9cf72ddaef098347',\n",
       " '1f25da9afb42e98669f3e417ab0d1175a1002c7a',\n",
       " '1f41aed6c6f4ef054f9422312e96fa9baaa1d8dc',\n",
       " '19641003d495f2dbb2f5876dcd5fdd3ce65e13fb',\n",
       " '999cea9bb146bdeb1c65267e472d30864e0ca925',\n",
       " 'c6d30315e4cdabccdb7139bb390c47666e51f3cb',\n",
       " '59e9d414ace0570172d7db360babab208e3c51ad',\n",
       " '85c5d58d80958cc87aca65a6fcd13a91c3dcac23',\n",
       " 'e09e1e5f9c9d5b9e158e376dd2b5e6b8fd42f33e',\n",
       " '96961f898ed04ec881d1bb3776f1b67381fc7bf1',\n",
       " 'eb297536587c51a03625cd75e761e27682abce00',\n",
       " '219c67cf668e1b727fcafbe74bbc29d703aa585f',\n",
       " 'c7d041659eb23ed3f398d89e0d02955e0ba6d325',\n",
       " 'b60061b95ac3f4c327c96f760f5bad47900d69cc',\n",
       " '935678315bf23da65d4c87a55856a314a64e593d',\n",
       " '03a50742fe1efecf771561895fdbe4fa18642d93',\n",
       " '139b9743ea0cbef4455f3b45c6e6ddabb8794b01',\n",
       " '63c61e9a37c4f9234c168d071451b5ce0ba314be',\n",
       " '9b475ac0e95433ee42cae0f22824c358fd956490',\n",
       " '74e6cbb248f62de3800c27060a220e649970faa2',\n",
       " '911bed2d13933495e24cda2d34f8acb5fbde5d36',\n",
       " 'a6733dd4e5c28226ac3f21ab79af4c13a7e24a05',\n",
       " 'b41725d00586be0fc44affa4adf2c0be2b1b5611',\n",
       " '5a74ba9dbd90ab22e9ad9a1cddd125efd662370b',\n",
       " 'fb0c38b206fa918bff4bae6f91ea1d0d7e4191ea',\n",
       " '7c1b91c18231338115bb85b25fddc22227c10c77',\n",
       " '497c93ae0a5982854d5cb17823047dd9ad7d20db',\n",
       " '310910fcd8b5b9c0693b38518acbc671ed81e9d4',\n",
       " '688260e0cac92f8f2df9a65e468139f1abd1c0a5',\n",
       " 'e9da6fe0a16117183efc6971eab8056713de828a',\n",
       " '1e26aec7cb49ec42a488c6d04b5119a6e349f025',\n",
       " '495c54083a1be73f2ce77ba79af3b6801494daaa',\n",
       " 'c1bc3c5496a77406e02b4ec888b947c3fee5c548',\n",
       " 'ede937904a2ff946a97adf519a774184c04156ed',\n",
       " '79acda48335c4733a7711391086dd8911e071a35',\n",
       " '8c4c48e3b080575777bc531c9d3f80234535239d',\n",
       " '65225c1e0930bc2901c9007747a536c0e1dff12b',\n",
       " '00e3e3bcb46dca174c78f595fbcdcb4d05357c0e',\n",
       " '9fd6e01a5ee8253cd72487804f868d137de65517',\n",
       " '9f603a6b785dd0637a3d69903231eda70da615a3',\n",
       " '8acdf0941255ae2f532f9ff3d288e988b06b04d1',\n",
       " 'aba674ec77975d37ccca3cd891eea06d9789520d',\n",
       " 'c20e1c3ad7484b057a194849b88856b01af71dc1',\n",
       " 'f5dd7d338cc5366b4eadb7e1bdedd059d773fe27',\n",
       " '1b3d352fdf73c0f33123adac3b657b21a3c984d3',\n",
       " 'c409097e1d2a3d6c020e19968491b8eedbf95851',\n",
       " '37c27bd13af62e6da4d573073308ddc5100c7225',\n",
       " '33ba83f5f92c884e769bd61d41a732a7b19213b4',\n",
       " 'a31066b35b81ed23145dc245b2de2f43e78051c1',\n",
       " '03edcd072dec728cd5734e9c618003d6c5143013',\n",
       " 'cef28de0bf879b9221ae39765f82049cdd32b699',\n",
       " 'ebbbf0fbd1397dd5682817ecc104ba396afaf617',\n",
       " '8760aacfe7e44a2e76d3be0faa84bcce1d6d2287',\n",
       " '761205155f259ee432f07d4309f6e701c9d1bdb9',\n",
       " '8f4684c6294ae37d60b67eeb9071daf3b8834141',\n",
       " '1ca218515b21b396a643a60b05f3df561b6de854',\n",
       " '73ab5e97bdceed7877fa94d12ef6d28e42968e0e',\n",
       " 'b0d6f50530aaf96f0dda4794b4e8d88284aab7d4',\n",
       " '749ee26a2b32a65c30ab55acf617df0a1af1faf4',\n",
       " '0a5dd76651f2816fab2e679a7de9bffa78dc8540',\n",
       " '3807136c1cc4081896ac281a0f888beb68d74294',\n",
       " '6f987286a8d3a62f813901555e20cafba97c05f8',\n",
       " '37637cd72fcac436c28f29a118dfd5dd5aa205a2',\n",
       " 'f300ef574989deaed9b2704fcdda64fd722dd4b0',\n",
       " 'e8283130f7a968e7c76533876db37d3c1d05d316',\n",
       " 'b6393c9e8ab1c947dab29260f62f27c68a03d0b8',\n",
       " '9b366fab69c550b7792580c1ee339e45ea63abd7',\n",
       " '03e0700e4ea60b6fbdf0236e6d050e2e8948c778',\n",
       " '38009514f2e1c9b38dac6e833cce7099e5067c42',\n",
       " '4a16618d9f6463c241e7ff6aa11853eee46605be',\n",
       " '3f5dc17ef379097f124b77a9b7a75faa0576258e',\n",
       " '4f2022a05fc00b24a1638a203a0b404a48f88f74',\n",
       " 'f3250e71e97f436dd02a012b054261021ecd236a',\n",
       " '4cad0441dc24f0833bec139eef93b97b6a91443b',\n",
       " '9319a01c793c08f095bb882a32151561290605da',\n",
       " 'cf873e826330f56f9fd3e262057a37170638e415',\n",
       " '426bd6cc83959dddce32818e462e5ba01a2afe15',\n",
       " '6390450d16b95d964d3c561a1f80e3cd0e034831',\n",
       " '785251e5b5ff6c7c1a2abd1e3bb46b9fca77701a',\n",
       " 'f042caee4113207319fcec8e8eb5d88e2cbe25f4',\n",
       " 'ba87f5395f8edad6bb7aa6924d1c0d4474600f7a',\n",
       " '94e5291cc0069dcbf4f53931d8a71c8a2674d324',\n",
       " '7425ad9bc05eb130c225298a25fa7b67199275f3',\n",
       " 'f81906ac27f8c528ea5275e68ff5d0103a2fc76c',\n",
       " '36656d7e64f3f6c4fad00e09eadf9d39693dfc76',\n",
       " 'f649084f64cac59bd6a18382b5fdf608d76acb49',\n",
       " '5c00a0624c127710a08f4673d1767c54ac89df9c',\n",
       " 'd06058c18c459a2d0af57c1e191db8d8b50ab416',\n",
       " 'bb226e3e740426798b6939a825261011cec8fbc1',\n",
       " '78ba3bdf11d8a4ef558dc1c0d7f07feb1313ad82',\n",
       " 'ae2e82d12952277dc9ae823d8194f07be47a865e',\n",
       " '50a3367b898961bafaddfb4609f1f9f2a544c522',\n",
       " '43ce6bb5d6551a574d00ae24fab5d0fdc907eb43',\n",
       " '57d825c1ec0a30272948d593c3fc6fd054042a36',\n",
       " 'f1ae98c86319b28f9469c17bb00844749fc6232f',\n",
       " 'a2e84ecc2ff3bab4b85a9a4e73e1f38818fa7d3d',\n",
       " '983039dfc2c45de7a8474a20f842b21225d0b00c',\n",
       " '1bf7583bb5ab0a883c59c1c63d09efcb51b58c76',\n",
       " 'fa5df3df14d305b5e4f051660006f79fdb975150',\n",
       " '160e19b778bd65ffd7eab254fd6d634ee61cc889',\n",
       " 'aa8be72317b24e65ec84d70c6564757ce48f0106',\n",
       " '5e21965d8ee9d4c87e5dc3d64ce563715d474d41',\n",
       " '0b3d52b820733440420549827c7c2a47c7477b85',\n",
       " '87cc0198a60622964cefb8bddf94ec1308072f48',\n",
       " '2f35b34e4f04273c831868831dc332ad5e95821c',\n",
       " '67397f93565d47b98f8de8c6a5a5bbfd529102ec',\n",
       " 'dfcc83093a657d3dfa265a2a07056ec368199e4c',\n",
       " 'f2e0f53b0350fa7dd564c94622e843e283c5948f',\n",
       " 'd96deec33c7fc50728bfaee89b33a36422b5c3e3',\n",
       " 'd15ea306a8333fe0bc450fb19f49dc390cce6524',\n",
       " '6f54be07852f841a6cc9afb73672e12d8143b8ac',\n",
       " '7a2db4eb09275af9f0d3c261c5df92c2bec801ec',\n",
       " 'd22cd926627056df7d795ba225b9f256544392bb',\n",
       " '04998a1140d8f2068f9a84051b2732d4a3dca770',\n",
       " '7a21b3c4e08ac92d6bece7ffa240c3fcb7f7637e',\n",
       " '56a022e885406a277baf3cca8d2d424841ce7c9d',\n",
       " 'd4dc3d2116d7e575fbb0ddf034d1ec1ab789cf29',\n",
       " '9c5f4c9fdbc5e13c84d583df1348d4c2cb6233ee',\n",
       " '617138ed9867e3178f96b267e793af690829889f',\n",
       " '65ff98da632539c98ce05106b949623cb250864e',\n",
       " '1eb17235dffce3f14793e55b7dcc57917ecfcbfa',\n",
       " '016bd0921b20aa75bf3ceadf6ece0e7b39c0d532',\n",
       " '968f9cba92ee7160b6f654b5e93fceea78180277',\n",
       " '325b607b8bbc3f1ab1b604a9390a6205ebe0c0a5',\n",
       " 'c4b8cad39f9a1bcaa657a427768838c47738132f',\n",
       " '5258974bdc09129fce21e051073e175c5223d618',\n",
       " 'f3a293b22ccdde009414943b86c3487c13ab1cff',\n",
       " 'd314c2a36072d8a94cabb6cf0fcba8d8edce18fa',\n",
       " '15e72682ab72b0c94834b045c8ec263683aa775b',\n",
       " 'f003f39e8c0e79ed0f9894adf901e83862736ec1',\n",
       " 'ff5b04afab04334df136f78f8fee03af5dd565c5',\n",
       " '76c7f1ead42ad57d7281e5d5f142c99e8ed121d3',\n",
       " 'da4a58b08605149cbd647aec970d789c542b02e7',\n",
       " '76a864145b51985146013089921fb9ce294644e2',\n",
       " 'f468f2744c007350592b2c13f0ba26d465e34aee',\n",
       " '78a7ae1b22f35541255cf18aab340e42891a9e38',\n",
       " '2b30fdd087366554bd9a71f2459f3656147dd3c2',\n",
       " 'd098eedef9fc235659341d9ea587d9349ecb5a8e',\n",
       " 'ba30f374ff33bbdab018949e2d259590aa0304d1',\n",
       " '14967e9365a39f4532cfaacc2be5dc6cc2c29781',\n",
       " '1629c4a0ad273f5bf49b9558e53c8350e568d7c8',\n",
       " 'f83e1650b998192270feb1cbd59a01b1f46e811f',\n",
       " 'e4f9289b629b482a1deecf81cf7638543e0eef9f',\n",
       " 'b32a94b1a389038e5a93d061ee6006d7dc568367',\n",
       " 'a414a300bf0f4eabf07d09a8d9e3817974e88ff5',\n",
       " '5b391b96529563a7c3524a9b9110622c226b1eec',\n",
       " 'fd1f0700d37cc141f672fb85378e7e67fa725d69',\n",
       " '749324a41b915e4f84c1ea0271dcfa27b88ccd51',\n",
       " '7e88578dcb2ebc43faab4cb201a11c2d79f19f18',\n",
       " '3049328a0a3d8cffad37b45f1bd416dc0e06dbd2',\n",
       " 'e571ed5c487eaee0ed7e18caa10eece9646dae67',\n",
       " 'b16fca5e827bfec4f0e7dc44c6149897cd744cbc',\n",
       " '3e64ccbb92abd9870e7a609c67177ec19994c6b0',\n",
       " 'a870c8b6195fac1a1db768062f28beaee539e12c',\n",
       " '2a77bc815090cf759b4d1ca0c3a8167dd341478c',\n",
       " '7991631b776c48d23919845abb09b2ed0fb56f48',\n",
       " '15e1dde81fa16a5348010e30f90e606dc7d3d435',\n",
       " 'ed29bd88a60382048470a3adee75ba647ed4907a',\n",
       " '7dfeb03034c8dcbe1b3a7d02b7ae278d939d12c5',\n",
       " 'ad062b50a6608ca04a981af5bffa4d92e0b7524c',\n",
       " 'e30c1f01ebf6762438b72cda58be9f1129a5639e',\n",
       " '50c20a6cd579ad891a3a26c1353b0d11524040d1',\n",
       " '0f8b030e6b4760c0f36c07553ac5d525a67af7c9',\n",
       " '37934b3095d6d3f4029a6d40150bb9de46faf0a9',\n",
       " 'e6d88a631d3fbb9ce2b337ab39c51080f0e4f9d0',\n",
       " '3559d9524fc57a0f2555c5d800621eb6b29e4c93',\n",
       " '3f797e5e21a400f6ef0cac7312808727f6ede2ed',\n",
       " '9fdb779a53465e9ea6eed7b328be0f925c1b2459',\n",
       " '20e07e460e1f4513ea34ccff10f1ba391187660a',\n",
       " '032673cbfa41f2eeb3127f1f5bf65629466ff748',\n",
       " 'db1d9b1108b76be54bb6bb8323685faffc3c72cb',\n",
       " 'f3a54f44684e532386646cc0e25b85217ef407ab',\n",
       " 'a000405dda474bba71c806bb2ea0b1e1eb75abc9',\n",
       " '4f00ee3a6e564228cb332b089f12ab71de93ab0a',\n",
       " '255b88419f1e6fc2a1cb86fca3c1da0df4f49709',\n",
       " '6b7f500a297f6c37dc570fee98fb660a9f9a0c28',\n",
       " '73f825b47b39ca717e835ab76cede7afc833791e',\n",
       " '57421d0b0827c1b740eccf56115bc997724ca481',\n",
       " '2754ba005f2f189f402b3f09ba3c959be8d8bd10',\n",
       " 'e6ca8496909d56156cd7127b402fb84c6e911800',\n",
       " '531d964c04ceb2f767ffff7807887764caafb60e',\n",
       " '71fec1b693a2551ad7359d46fa3d59dc594eae78',\n",
       " 'ad8339a61b4f665981d68459a36c35078dd4f8e9',\n",
       " '85457bc8cb50eba0ff9e202230357c2fb80dbac7',\n",
       " '55a1d6bde15ade3a77085cc3f686849c6550ac3d',\n",
       " '4f1349de23d4c2a68c75aaec318f5ad025371a90',\n",
       " '9afad3f15faf2302fff2daecebbea847ecbf0178',\n",
       " '9e83a3ebcc06929d4dbbbc612a45c220864544e8',\n",
       " '9fb5c1fab821a4a07d5e72786b7938ed3e3da7ba',\n",
       " '305a49eb35002dd38b9a908186f5a6837b73fbe9',\n",
       " '9e40d28c110148a692fda4e351a70e00ad1c291b',\n",
       " '6408bdafc31f4d5421dbf221d5bf060e80368ad9',\n",
       " 'e5da0160ff5fc43b5a4ecb557f32f408aed41b5c',\n",
       " '9ea87e5a9a02f5d29c8c1267a0f40747bf82d0ac',\n",
       " '2c713f664c2d4bfde75b629fa25b63f3b1b388cf',\n",
       " 'be6a003e0e45e60c2bc689fbdc0c8ea4e5e5d67b',\n",
       " '25018900edc266895a805740d105c256228d78fa',\n",
       " 'd734f1b72379d7b2ba539465b71b1aa65c7e47b1',\n",
       " 'bfac001b1f1726f5713d405b9652506aab89fa49',\n",
       " '0dacb45e771e824292e839c3c29ce38d9e36c66f',\n",
       " '1c16063cf91cef0919e3e305b0a2c3f460a33ff6',\n",
       " '4b076b4b16613bb468bd96dffcf5eb1638fe5be0',\n",
       " 'cc82a5472f1dec8cdca770993e3822341f220d73',\n",
       " '3a86ed4ad922baca0db82f4d8b4dacf4abbf665b',\n",
       " '69fd7c161428793f2bb5cbc389d7d175a6916136',\n",
       " 'bf29f2769f9142b5f57834b15b0c47f1bebd0440',\n",
       " '96dda7966535ba17bede5ad1af77a060eabb4811',\n",
       " '9332721943a8f542a6ed17d9963b4fbe5ecbd6ad',\n",
       " '068b7003479b58e85f96a9870063dfcbfbb9b008',\n",
       " '4cae687c42ad72f0c95ba77bc554d48bc7d6b3a9',\n",
       " '26dfbbf71393b2f8908945686279427c371896d8',\n",
       " '7aefa3645db407d29ca3676d9c23d29f230852a7',\n",
       " '5682277591fcde492cf0134113538c06bbb5c6fa',\n",
       " '1e788ddc6b0271e75b8f380cbbd6021b3e41c6dc',\n",
       " '48aaaba376fa8fd235dd69c465bd46afd30f5612',\n",
       " '788980ea76b8b12fccbe0f29622e4dea99ef023d',\n",
       " '91c6b71ed0a0064862f88aa3756bff3c276b30f9',\n",
       " '37fd76ae7d723200b1eb0e87e8d7efdedf9714f1',\n",
       " 'b7b44b2849a84d488a410d481e32215d66c1023c',\n",
       " '524bbf9f8207524e590db11f1679d405ba05d69e',\n",
       " '9b22c8bfee5b8b485853bd11426a4dff6a6d7f57',\n",
       " 'b71805c6c775401638ec5af146ec69f17113a55d',\n",
       " '194a39fde0cbe2642669583778e4e6fe63e9d79f',\n",
       " '1b242264e51896a7f14ac911e8db57eefa1794bc',\n",
       " '24a52dc875329a4117864d97003b46631cf23c6d',\n",
       " 'b7893f2b18622bc3cca68b608aaf8a18c4450a91',\n",
       " '290362a5f81c2cbb097ce4f4f14b11eb1cab7069',\n",
       " '51ee8148f65bca443e3294cd1df0f37aef950e20',\n",
       " '7e10f12cf4473792fe9331b2a9a766992268eef6',\n",
       " '1dfddde04a03df2167cdc70cd79253a931761837',\n",
       " '798a5bbc4700525ebaea31e36fed6a44661736b3',\n",
       " '5b420775ac2adc0ff48ac3e227e468eeac31ccd4',\n",
       " '2534d44d582c26fceedfcd4dba5f96ad3fcfab20',\n",
       " '5b9ee3ae772ef8ab5f9a6de953a41990ba77a8a4',\n",
       " '6cb939acc038668b6cbf8c26d36880fbdec00622',\n",
       " 'd31507d9b6fcc3b11e5e4d86a0fb7d433e52da65',\n",
       " '0e0b3a94e5a5db0ef97f119ede022de3b8172174',\n",
       " 'aacefdee52a287fd28a103fdb4636405908054f5',\n",
       " '32dd3bfc96a3bb004a64456e8e986dc61e4bdd69',\n",
       " 'cb5eec32b08f8f88506b12cc778378721f6743f3',\n",
       " '1c1e38a6150e70c3f6cd85f83f8c3bfeca7598d5',\n",
       " '00dddea795dd0bf923f043a692c106e6f41376bb',\n",
       " 'dd1d527e921c2f727b020945d457b06d79dbd2a0',\n",
       " '2ad3e239b68ed394c372de1389b436f65d593f99',\n",
       " '4d352475061c2afc674b637e8597a9d3579dec3f',\n",
       " '1730e2b096c514dabbcbd768f5f036fe3a9d0a4f',\n",
       " '0f59cc45662b2b4d5c7e5ea0c087969d1240b175',\n",
       " '2a6340368f3ee70f7972ea0077bab2dd7bf6b67a',\n",
       " '2889a1e776aeec7c4b930fbe3493a7d5576aed9d',\n",
       " '6cd11e64b25e79ab0c0f1ba4ee580e333690c2ea',\n",
       " 'c1381b2d6132c89c1d23ef823d57071470978926',\n",
       " 'efa6665b0dd379cacd002d221803c622103b41f4',\n",
       " '52dcdadb3ab21d5075a96300b2f488826c212797',\n",
       " '7f66db60879c478c111b6b78b8cffd7f2eb859a1',\n",
       " '68cd22b2340cb541b700162b0783377abb4d6327',\n",
       " 'eb7329001f139dd21055ffc4e9dc6d007a1255b8',\n",
       " '2e0e43fd41698018b59783e0b34cf562bb492632',\n",
       " '86b903582ba9c53a7d327e48183a8cbb6c870b4a',\n",
       " '74e496332050da7a278645e49b3d4d1a1aa78bd0',\n",
       " '5951126c9536e8bded765f67d630bf62e20135e9',\n",
       " 'ce438a3c4488f990a9a2faa09774108e4748e2a6',\n",
       " '8d0984f79eebfdee5a570224ad25461c29be59c4',\n",
       " '1f1dab5e26fa275ca0138d333d74c855cf932a2e',\n",
       " 'f745f3e6ef510204816e20a365f821b0f681a911',\n",
       " '1411311fe7d0b4a21a808eb0dcfdf379021b21c6',\n",
       " 'c2be950c01cb447f2be506449a084db9b95e77ca',\n",
       " '8ead15e0f946e52fec58950338a3e60324831787',\n",
       " '9c1afc747c1e74b3d0665df61d4f9fdda82f527c',\n",
       " 'ae58db99009f51938a9d0bb461b5d1f957c05aa4',\n",
       " '302ee6018e606fc73d8d7f3473b21edeb0477680',\n",
       " '200ea44c826b22379ab461dca43be99220a496ff',\n",
       " '202bfa51982cb6f1667c11e2e795c60d25678352',\n",
       " 'b5f71f17bdf1d1857f7765d305fe7c3951b06159',\n",
       " 'ce264404fd12f7fb90dad9af5e843830f6698447',\n",
       " '43d6dd501c7350c73d302fac762477dca789bb92',\n",
       " 'f8dcb21a6f9aa7d0d87533bfac3d10c5b5896118',\n",
       " 'c4470a9c896429cfa9ef06ef808e04b507c17537',\n",
       " '611a7f491c042244375a831bd3d5b9f4dae652b5',\n",
       " '5ee54054517c79aed1d4a370a97b7dd100757dd2',\n",
       " '864807fceff50b6e9ed9ddf34a84a39242e3a904',\n",
       " '8ec206c3d02c51a6b005eeb71b48aa671c2ce084',\n",
       " '073956da47dd206813e4380362007cced3f57b69',\n",
       " 'ee6fa98c5eb5195e9b6b29d7cb59d59374eda0fb',\n",
       " 'c6ff2dba571c6b9f482df9b52aec8d9880103c76',\n",
       " '9b4addab0c07b39bd9f3263456c9cc99cc0e054f',\n",
       " '77c4b0bd743730279ed1599ea59c2b309f065048',\n",
       " 'd303869578175c1ac5a0c9e05bdd1d5c27008bb6',\n",
       " 'ef50cb7a9351dc0ee27939a768bf0697bc385d31',\n",
       " 'a31ffbff1f4439f56133a0f81faa970eef46f968',\n",
       " '07a9f1d1eeb357639527e0fb32635c938391774b',\n",
       " 'cc71d80729b76dd4fb8bd4790dc5f2e0f43696f5',\n",
       " '96e32e78da8c3608d6ff58c0041e5dd7835ba738',\n",
       " '400f9c1bdcbbd6ddeb464279958c89bf98225b41',\n",
       " 'df0c71ccd362637c89226464b265811aefa7f72d',\n",
       " 'a742842d9dc7e91020b354556acf2c515488b662',\n",
       " '095162f781f9f0eb7f1d66c9d294ad9c816bc3ea',\n",
       " '2e36ee377beb0653913e51c55d6ad8f028a723f5',\n",
       " 'e6bcd55dd65916d56232990e685aac1d72982e3b',\n",
       " '12fb1e74ce8c3481077bbe60f3e9ca8fa78e39d9',\n",
       " 'a7e06966b73112f80c3f17d07b81c6ec2fc76bce',\n",
       " 'b79c81e258f6e273d51b613c6d0a9e8743247273',\n",
       " 'd27906219b7b1f3003129169962dbabd00e072c0',\n",
       " '18a6b87fd080eacd0d1a5ecd97b14b04e1093d53',\n",
       " 'e29415a7d660cfe5138f2a891e327a401b713cbf',\n",
       " 'b967ab7497980d9eff95cb665d102b88a0850cf7',\n",
       " 'c63a4e8f1ae8426edd81cddf85bc3c6c5f681b53',\n",
       " '1403b47890f40a3c28037552c769c82a4b82631e',\n",
       " '2b0868991f81267dd6ed2d5376cd3f6576394398',\n",
       " 'cadf5f6b603386883909ebae7b7dd9ade41d960e',\n",
       " '920124bdf67520ccd28cf3532d8531226330f8d9',\n",
       " 'edb3958765701f7042389e55fa3d086f17ec2c5c',\n",
       " '6ee6154b3e1a41ec640c3187b9eddb4795bed9fc',\n",
       " 'b0bf86a41b623aa0a5ec23032981fd2717489df4',\n",
       " '97e7c9387c229830a374286f5b6252259cd42c1d',\n",
       " '5895ed537c5f204ff850c37b738f79be8f150ad5',\n",
       " '42cbaaa6418d359c97949f59e8d87e39c8918baf',\n",
       " 'cece00bcc272b69315bc007962060b111a625950',\n",
       " '9fd7db181f75e695198b9e5bacce2807f8086e75',\n",
       " 'c76211d10a5b3700cb592d77b4deb493d4859672',\n",
       " 'df4255ff6c134c0ddf3824accf5f950480f5773d',\n",
       " 'c6613bb3e1439da40b54ca6322a6431603e54d29',\n",
       " 'e7656fdb4e9f742f6bc487afa46404c9e8266c3a',\n",
       " '248128ab929ce302475bc620add3c39f14f8245a',\n",
       " '0fdcc57e759f651ef5e8ec81f23c99e940f1d750',\n",
       " 'a97988ec7e827360da04741391544d7278e36738',\n",
       " '87613e7e6b89e1307bd30c7c2cc77f15af70f810',\n",
       " '63fd05cf02b7722f2589ae0594a3f65ca9e0e07e',\n",
       " '215570e89b0681efa22ee602e5906b243a6ceb9d',\n",
       " 'f2701aa563186360e1a201fb1cbee84ca68ab6e1',\n",
       " '5bc62d4fdfc9db827b849f656569e7061c79277e',\n",
       " '0b1a2c9acef39e68987691ef96fa6469975252cc',\n",
       " '367c730735a8c5d91899141072c1ed4a0f5b2717',\n",
       " '2022a3f1380ba4589965d7599f47a3e1eaf8482c',\n",
       " 'b96fbb954985a56116a963339d341c7ceea69efe',\n",
       " '0ec8bc38e5ec83fe2af30e614cb246f3c173fac3',\n",
       " 'ce41dab6bfd38560e387529a174252454e8cf227',\n",
       " 'b2955702ec0125113aee03b2b691a027b64340c5',\n",
       " '21066371b2a60e8bc09accd0da22eeadbd8d4920',\n",
       " 'fc7ab38e72f67da0f47ae3d51bc9978258eb83c6',\n",
       " '0c5f13a885b09a4d638ba1b305d6e4c6ca3f0f8b',\n",
       " '253876157d3785c6f6358db144cbd12dedf82e27',\n",
       " 'b06c55785f0499270531d6b8d9e581d4302626f4',\n",
       " 'd01a1b56cdda85ae66e90236384ea19ede2ea919',\n",
       " '78e0ed67cc9e70d75f18552f158d790a2e894363',\n",
       " '45cb8e1cf1e9202503ed70b2e638b3a33c409f0d',\n",
       " '1c3f05dcf54d50e15e294692f217c9f85fc83777',\n",
       " '3b6f8655676b5aface2ca182eeaa67cfabe51bad',\n",
       " '23d32223987dbabdfe21036ae79d3bd1252e6064',\n",
       " '51b6c1458a4763813192e87b49dc9733b86d6b44',\n",
       " '325b19d28881364db8981e21d070b167b34b9a27',\n",
       " 'a0c64cbd4270edfa8b76497eda1dc110da6794d2',\n",
       " '566b9fae76613df98eb5cd8bc79341f14dd0c150',\n",
       " '5221015e90ee0a8f186ff23202276017c0621ff1',\n",
       " 'c40c66c628a5a695169a82d56657ebcb4bfd5e89',\n",
       " '56e96201849e12f59d4bf6d58fb3e321a67eadff',\n",
       " '34c940e659a8ca9a8cf7df58baa34599202b6fc4',\n",
       " '7ee1d689bafc5a7b1ba90554547269ec2e9064d7',\n",
       " '927b44fed3ff1348726ec91d800324c3aee7a95c',\n",
       " 'e370d0b556e4e45d490a6c3d67624e8af039a7e1',\n",
       " '9abfa72b2ef87914ff0133b9ca7f71de8a295c88',\n",
       " 'd14d1f73a7b19f4587ffcf45463a1eda71e181a1',\n",
       " '77418438a02a444ce223e4f8adf90a6198fec40e',\n",
       " 'ab0d9e0d1de2a0ca06765233dcaae412624deb53',\n",
       " '5e17ce2f4e4bdb4eea350d10976cf036319cc5fd',\n",
       " '46277989dce0668d9d9d8df55ee908a5f8066b41',\n",
       " 'da5169cddbe5a729af8ec97f4650e7d7eb725401',\n",
       " '24d8799a70ef055d18e64bc885b0867e90cc7018',\n",
       " 'd3d719d2fb53b985575f73a996e1ad4cacb17efa',\n",
       " '91e729551ef17191a723e5c0e56efc2d531c607c',\n",
       " '5d55857a995a548319d20b5cb295e76b5af5990b',\n",
       " 'b270630e5ef04eee7747206b7317f64a37a9c92d',\n",
       " '2d2313c5ad8a164e7e19ffde20d9043b62f82d60',\n",
       " '74855a160d90324d9c9fda55d9a30ab6deb7b5d2',\n",
       " '4c1d9474765ba696f5edf76f786fc9ef8d6ed749',\n",
       " '720c27c04aa7b014eea4e898aaefb9826bfe5e31',\n",
       " '6e08bc275c8d27d0984e7bc5ceeb70cbab57341d',\n",
       " '17830781b4bb97d350a196e3ff3b09d3d859e001',\n",
       " 'c01ff41eaef9e565e1c5659d926fe7b980bc3142',\n",
       " '710e6f224c8d540c19e9ecbc01384c9984c7a217',\n",
       " 'a92a1509576c83780ef0742aab9477b9db697c54',\n",
       " 'c0d6cbc2230c483bf3d032d8a8ad1fd4f1e6bb66',\n",
       " '18840769653348d59dcc893d7b74f60105c14fb5',\n",
       " '2c467af665e3a7f983c184fdc8811a7d77c3b96b',\n",
       " '7605b9870ee41680b0eca7aaf5f5cc2b8cb68ae7',\n",
       " '98097a9a1b82ce65daed61f3682e02a6f9728541',\n",
       " '13df007421cc7049b7ef83ae3edb4a6329d33cfe',\n",
       " 'be85460ed2e710d8906474cac6760976cf48bf29',\n",
       " '53663d48fdc9afe6f0da7855c34432ee144a4461',\n",
       " '3d11568622e93d48f28a6ff23b30b8040da4ca38',\n",
       " 'da9e0085e2f71792737304490fc7d6d629d03196',\n",
       " '1ea1897d52046df9e038caa2d49d5c182df3793b',\n",
       " 'c11753e0ce74a2562af643372a221009a70820d3',\n",
       " '168b25c67b8b68c7dd8d7642cbe918f071007811',\n",
       " '9a2972c9ac453e9a50ac306ff6c9f003f0b25528',\n",
       " '9a03d0a40311d8b895822bb319001a3530c8598e',\n",
       " 'b2d87de639e9282dfa82c90e94d27ef13725bcd3',\n",
       " '695ab0d48248da3b5f3cb41ecddf270e3009063e',\n",
       " 'bda81c77ed2b1e624e6798e5eab7c1aef9fde7cd',\n",
       " '73fb05ad2eb8c01350553bd13c6ec34f8b628ef8',\n",
       " 'ab09a311932b8a4ae068c917d0abe6bbb8819bff',\n",
       " 'd75a3ca464485a09d8c1901985b5f036c51202de',\n",
       " 'b5c42b33f76e8079f89979be77a51c165f6f5859',\n",
       " '5d033310b0e6616f8ecbf198916979d3c31fc2c5',\n",
       " '495392524d73528b10a119280647e139e6e1d9ea',\n",
       " 'd394730e43afc06f587d76ce33bce95b51a375c6',\n",
       " 'b54b0c84e1c9e97954b5cb23aeb300f2837dad57',\n",
       " '668962ccf9019d70c5eff365834b50964fb1e5f7',\n",
       " '515fbe7aacb842258801dc22c38f8d632e853f6f',\n",
       " '7fbc325fb465cfc4e23e27f714b45942d8014593',\n",
       " '85cb3187e758ba881406b780c97fc391aa76313a',\n",
       " '938694dee112d3725254c69f98970661e1667dac',\n",
       " 'efde912b57b586a5814caaf5cc902385d68a5b1a',\n",
       " '487d10771636e9d70cddc4cc9691651d466cdf68',\n",
       " '3babda06fab2dd06de8ac4416712613b3dd4c7e8',\n",
       " 'f2c382abdc1ba6fe9afb8869b6bc0a046b5f4aa2',\n",
       " '5ab13ef99442890d7112f134432a57518b8eba70',\n",
       " '7e62ae99a0ac7dfd2f7135f6acdebf6d0a3ace8f',\n",
       " '5e14128159af25aff3c9e51833c7e870237ae86a',\n",
       " '7e8b7c5386a74ff33a839b9bb4da78742d3f6a27',\n",
       " '17386f64af842ba3e45287016bb6c3bbc1b6fa77',\n",
       " 'e33d904fd10e1aca1ea0a46905cff370fffd9cce',\n",
       " '0de2ddfef706ac0cbaf8379ed458689faf183701',\n",
       " '62b689932e55e0d47e40d2b134aa8233da82a07e',\n",
       " '5ba22cd47a700266dfc19fde9fa24e276df40c8c',\n",
       " 'fa114338fcaf51503dbf30c3b1e570f96eff2f27',\n",
       " 'e8dbcdda29f8e32e119c7d9dbd5a44070f20dada',\n",
       " '210e78ab35e9d47f0f0033928eba851b4d87eb30',\n",
       " 'd192313dcfdf5b43425b7bec6fec1afa54463f28',\n",
       " 'dcd2f3604838eb318e06687ce99ce01e016fdf93',\n",
       " '9c488300844440afa0cfd5b5e7985a92d4560610',\n",
       " 'd4bc71431fee4375d785cd86c31cca79eacbc65f',\n",
       " '4d0d2c22190d86a489e7067592524c8cac3a5ac5',\n",
       " 'a02d7c6c79b3a9c156f83d41617b2b618bb89336',\n",
       " 'c097a2f4517e4c5d549107f9a2e64cdb0d315464',\n",
       " '81e8b98ff454575a3739deca9d38e51ca8ee1faa',\n",
       " '72d060dda6706119a7a133c44967359a9c6990da',\n",
       " '5864fb57e68ca4cf6f7eb43da9dd0e1df7fbe0e1',\n",
       " 'c1180af9b16a94502d4a432d3423b5dd4ab6bd90',\n",
       " '376366eb944c7a5713b9df2a77e9bc731117e8a8',\n",
       " '5f993f30567604330bdd461e2c1444105ef5327f',\n",
       " '23540996e0af98784b4f90133be96ed31c4cf2ca',\n",
       " '1659e838ef89399e1cbb50a68f1408e861ddc12b',\n",
       " '1a3ed268c88fa049daaa07ffcfd70bda11eec495',\n",
       " '33cb64a9db68167ade5705d72d585d51d02cdcde',\n",
       " '840d2a7465ce1e3bb18d427a2dbecf385f987069',\n",
       " '52cd27779e83d9905d334c09ba7818a8326b8f9e',\n",
       " '04ae4932c094af01ee4b55749e74af5c1bb91cc5',\n",
       " '01fafcef314edd58431f5f56e30c6d8e04df8180',\n",
       " 'ec3790ca60d71834bde8cffdc001ad03d6c45060',\n",
       " 'f284366add0bef2e180e15cd864fe0a45cbe29fd',\n",
       " '4a38486d50eb988a21dc44ceda7b98bd25c2cf01',\n",
       " '708a216b6acf8a6efb60983d0b72c7172eb329e4',\n",
       " '8c81e38214299ebe13749de179733577c4eedd1f',\n",
       " 'eee4c1d317d819a639e7051c3f583c2dd01238d5',\n",
       " '5a4bac65172c8bbfabd662c4db93308a9132ac30',\n",
       " '40212a2ecce6ca8225a5398175ff49dd612fa3f2',\n",
       " 'bf3d3dc4afc0388c24423497e1dcddc0641e3664',\n",
       " '3144e597ca0bdd56a06b7e312f90e66e8acb3dcb',\n",
       " '4773b8eacea081a6c2c30aef525f35dad416d81b',\n",
       " '525450878a25f500c1f2642bc7f39c3dd2920723',\n",
       " 'f2b99342f7b2b9981fbdf8f3bd75c132149d3fb1',\n",
       " '493958ecfbebb2552d337eda89cb0d7f5085529c',\n",
       " '32d7a258764dba06b8470c56914ae16cb495390c',\n",
       " 'f95705398b45ac9353b50f22a20f600baf34ff01',\n",
       " '7e0eb21bca4555c0386b18884e9f064bda983e81',\n",
       " '3e3b8f34ee083dc7a9d6b8fab256563687ef92bd',\n",
       " 'b0aa3bbfe7192f3b275901dfc5691bec935dba2b',\n",
       " '2c510cfaf5af9c0f98ddc88c601f95f27ab41b06',\n",
       " '4119cf8dc2ea01fc5973e0d8c07f2fba6c51cddd',\n",
       " '713ff3a1fb9074f8bd453936aa009cb56f45ddd0',\n",
       " '7f72373272d72b6a9fa1ca4c7f4572e39d48cd0d',\n",
       " 'ef312aa193640302da8c3f0e15e530fdb9a916f9',\n",
       " 'dbf35cd5dca33096a6838f963b51c0d756e85662',\n",
       " '1e2bdf763e359a157d91563247492031876e7a0e',\n",
       " '8835b22aacdbb4f0c94b64329f61eb75acd40f02',\n",
       " '6a4dc3dbf89136af8628ac64db2213cfca492bf7',\n",
       " '731ec6a58f6d9e3275f102c4c5142ae5664582c3',\n",
       " '96de6520ed1d4647078658f91a85dee817e793d1',\n",
       " 'dc269ad0525c920c23456c6ef1598c358ff5aceb',\n",
       " '23f9981a862dcc91e9483b2e89393fd9f3c78f71',\n",
       " '752e7620fc24d5756bfc087fc257b4da2686925d',\n",
       " '390a140699fd2378910d141fcc1159e51f9e20c0',\n",
       " 'd10c45667b66d35d5c16c380e35ad8c5a63ba052',\n",
       " '74f63b86aba7bec04b377114858bd9da5ecc4025',\n",
       " 'bcdafa9fd76f68fa42a51cb4ddcc8bb577ed4e0b',\n",
       " '7c6b313edd08ec94567856ccb8e5309aebc0ce2e',\n",
       " '24c04e006849afe972733a4f79e35a708ed7d089',\n",
       " '6008db538acb292cf4eca84337626dba1014dd26',\n",
       " 'f08cc4d9a905f58e0a7719034ed58ef40ceba452',\n",
       " '59b4ec24035b1b1137c658ad6dffffc7fe114591',\n",
       " '131eaddb9d8efe252fa7480991b91b89830c0a49',\n",
       " '6fba6ad19a1891f71838b7bb9100c56b484c4648',\n",
       " 'a6e407d200545fba9a71527dddbd0950747187f7',\n",
       " '2d08277af83b79a8e2a77781869c1ca0dffaf5aa',\n",
       " '27dfac1311ab07ae2d029176cbf34656b4083213',\n",
       " 'd3a5e0151c5673d07ad295244d16ba7c28c21a60',\n",
       " '19bf2404741ad5f6fb90acfbbe6d6d4b6b80304f',\n",
       " '962af6a175bce3b600ff7a9b2106a9adf7091b7f',\n",
       " '893cbf9642eb2917422a91af1a7c6b4abde88ab9',\n",
       " 'd8c33a28dad7c19d863df5d90668a3d1d926cdb1',\n",
       " '738b59601c427d8fa4ae400abb90f67c8ee62746',\n",
       " 'fa00247b18998e85df041a4dcdbee8d686094c3b',\n",
       " '0e46131591492f256c2fb7968cc37ffc62afe15b',\n",
       " 'e2618e94d5b5c02542e531c08073d1a54c50014e',\n",
       " '589304fa822ed2b14de2ff16721ecb7fae8e09cc',\n",
       " '0149fc0169af445e75d7a6709934c8a4be6eb8eb',\n",
       " '2966f1e3f585896fa2883b3b973c97da2a10b135',\n",
       " 'b88df16d0337e26737b3cc5485de7cb9860f2872',\n",
       " '43d1f9b1be1fe9c2e5a4854e8fe396deefda6a6e',\n",
       " '823524ade4c9fbcd1572ee2da75c91b3139b6356',\n",
       " '160854abd47b2e1ab718c5c240f4453e42d743df',\n",
       " '47f6ee29573ea649d04369d994da9c663fbc1e34',\n",
       " 'eae2fdd272dd884ae1e63233f3aede3b5f04fd74',\n",
       " 'c319367083cd0b0ffcb6f4577024f31cf5dd19e2',\n",
       " 'a25d619e1404eaedbb763596d11c0e0bae4061b7',\n",
       " '1528b6336aa37a3d0cf5fc01710149b3f41d8b7b',\n",
       " '344f15c6d4dd26e667a1ac36c2afaee2026e7768',\n",
       " '950329ad9f07ebe06ccf0b2160f728f68d7ad459',\n",
       " '43319a40bbf26d93f5947bac2f829b59612f0858',\n",
       " '567d91120d3f65d5761710232709dc6c9150f237',\n",
       " '43a1459328841ceb985b644e6e234823c3c253a6',\n",
       " '3f688f46c4df0d2d9e4806de933550473403d54e',\n",
       " 'cfa381e204dbac829f88f6c974d478bcede32ec1',\n",
       " '6dbd0a18750cfb6d9a7b5ea6bca026cfbdbc7943',\n",
       " '191a738a8e07e38e8b6e4e48e7976f0de8164e72',\n",
       " 'a90d5a168609db668baf353344be09efaf28fb30',\n",
       " 'a325c19f4de53c0998b5142ac01c6ca558e77248',\n",
       " 'c5cc7c36db9b85e930082e622de3de94a25eb2b0',\n",
       " '22b9cd9bab85685e990bb1d65f6492f10f5a02cd',\n",
       " '2ea677e00a6053980c36181bc0724cedc25f27b4',\n",
       " '4c2e4a16832cc4642a8acd2badfbf50f5041f62c',\n",
       " '765742d27f26a850c51219e0bc5aac50692d567d',\n",
       " 'bbda42bf194084374317cd8e6b1dd3de4e7822f6',\n",
       " 'a684eca60c54786b7aa074afb1cead5985dd558a',\n",
       " 'efda1fd69a280dae8b6efa53306dbda862e33648',\n",
       " '3b192e67a3802a1d23102b66d950a3f0d02164c6',\n",
       " '1a5d58bd6b4ae13ae9f5dc571d7a93a47a9f8307',\n",
       " '80c7255b00ff047d870453fdac8e23db96564712',\n",
       " 'fe4451a9cba804c3eca5e708f0147d4dbd7e3842',\n",
       " 'c027451f5ece8a8360210f60bed671cc40a3f10c',\n",
       " '7607cfd27c8eea92ae63756130912abac4364d08',\n",
       " '41ebbf015a5880826a4864b06a2ceab1abc74389',\n",
       " '869f09e171cb02a4e248b209b4848f27f19d05ce',\n",
       " '17c75c8297d0d67c30d8d2773f11783c390101ec',\n",
       " 'e02f1285a17e0003a43296323e0c56ebcfe9c218',\n",
       " '848f90d9743e0596c0295b38ae6c0525fd905f95',\n",
       " 'cb0bea643cabf9040d25247e54e794cef89a41b3',\n",
       " '9dcef4ed659fc997b00eb3244faf57d3b7c168d6',\n",
       " 'afcc0035957f95e22dbc45775f57192a08fefb32',\n",
       " '865ee50f92df52324487fd461fdbf32548ba5c1b',\n",
       " 'aeeb1823a0bac96be8f1f7e62897cb2212352ffb',\n",
       " '75e7394df328d8e941f11fced45b450e6f0daa7f',\n",
       " ...]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "row.approx_citation_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "has_shared_authors(a,b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['colizza', 'barrat', 'barthelemy', 'vespignani']"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_names = [a['name'].split()[-1].lower() for a in s2paper['authors']]\n",
    "last_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "for doi, paper in doi2paper.items():\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'paper_id': 'f056da9c64fbf00a4645ae326e8a4339d015d155',\n",
       " 'metadata': {'title': 'SIANN: Strain Identification by Alignment to Near Neighbors',\n",
       "  'authors': [{'first': 'Samuel',\n",
       "    'middle': ['S'],\n",
       "    'last': 'Minot',\n",
       "    'suffix': '',\n",
       "    'affiliation': {},\n",
       "    'email': ''},\n",
       "   {'first': 'Stephen',\n",
       "    'middle': ['D'],\n",
       "    'last': 'Turner',\n",
       "    'suffix': '',\n",
       "    'affiliation': {},\n",
       "    'email': ''},\n",
       "   {'first': 'Krista',\n",
       "    'middle': ['L'],\n",
       "    'last': 'Ternus',\n",
       "    'suffix': '',\n",
       "    'affiliation': {},\n",
       "    'email': ''},\n",
       "   {'first': 'Dana',\n",
       "    'middle': ['R'],\n",
       "    'last': 'Kadavy',\n",
       "    'suffix': '',\n",
       "    'affiliation': {},\n",
       "    'email': ''}]},\n",
       " 'abstract': [{'text': 'Next-generation sequencing is increasingly being used to study samples composed of mixtures of organisms, such as in clinical applications where the presence of a pathogen at very low abundance may be highly important. We present an analytical method (SIANN: Strain Identification by Alignment to Near Neighbors) specifically designed to rapidly detect a set of target organisms in mixed samples that achieves a high degree of species-and strain-specificity by aligning short sequence reads to the genomes of near neighbor organisms, as well as that of the target. Empirical benchmarking alongside the current state-of-the-art methods shows an extremely high Positive Predictive Value, even at very low abundances of the target organism in a mixed sample. SIANN is available as an Illumina BaseSpace app, as well as through Signature Science, LLC. SIANN results are presented in a streamlined report designed to be comprehensible to the non-specialist user, providing a powerful tool for rapid species detection in a mixed sample. By focusing on a set of (customizable) target organisms and their near neighbors, SIANN can operate quickly and with low computational requirements while delivering highly accurate results.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Abstract'}],\n",
       " 'body_text': [{'text': 'There are many different methods that characterize the mixture of organisms present within a metagenomic dataset. Such datasets are generated when a complex environmental sample is processed by a \"next-generation\" high-throughput genome sequencing protocol, and they consist of large numbers of short nucleotide sequences. Each sequence represents a small fragment of a randomly selected genome from the very large collection of genomes present in the source sample. Those sequences indicate the presence of one organism or another according to their similarity to a set of known reference genomes. While a given sequence may be unique to one species, it also may be found in diverse organisms across the tree of life. Therefore, one analytical challenge (among many) is to take that collection of sequences (likely numbering in the millions) and accurately determine what species are present in the sample. Here we describe a novel method (SIANN: Strain Identification by Alignment to Near Neighbors) that is specifically designed to rapidly detect a set of targeted organisms from a metagenomic dataset by aligning reads to genomic regions that are unique at the strain or species level.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Introduction'},\n",
       "  {'text': \"The analytical question motivating a particular piece of metagenomic bioinformatic analysis may vary widely by user and sample type (Segata, et al., 2013) . For example, the function of the human gut microbiome may depend on the relative abundance of hundreds of species of bacteria and the types of metabolic genes they contain (Wu, et al., 2011; Schloissnig, et al., 2013) . In contrast, the clinical treatment of a patient may depend on whether or not a particular virus, or a consortium of co-infecting pathogens, is/are detected in their blood. It is this second class of presence/absence questions that SIANN is designed to address. SIANN is appropriate for situations in which a user wants to know whether a particular organism or set of organisms is present in a sample, but isn't interested in the functions encoded in their genomes, the relative abundance of each organism, or any other more in-depth analysis.\",\n",
       "   'cite_spans': [{'start': 132,\n",
       "     'end': 154,\n",
       "     'text': '(Segata, et al., 2013)',\n",
       "     'ref_id': None},\n",
       "    {'start': 329, 'end': 347, 'text': '(Wu, et al., 2011;', 'ref_id': None},\n",
       "    {'start': 348,\n",
       "     'end': 374,\n",
       "     'text': 'Schloissnig, et al., 2013)',\n",
       "     'ref_id': 'BIBREF10'}],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Introduction'},\n",
       "  {'text': 'Metagenomic classification methods are based on a wide variety of theoretical underpinnings. The basic varieties include alignment of reads to various nucleotide databases or exact matching to nucleotide or protein signature sequences (or kmers). A representative set of recent methods are described in Table 1 (also see Bazinet & Cummings 2012) .',\n",
       "   'cite_spans': [{'start': 321,\n",
       "     'end': 345,\n",
       "     'text': 'Bazinet & Cummings 2012)',\n",
       "     'ref_id': 'BIBREF1'}],\n",
       "   'ref_spans': [{'start': 303,\n",
       "     'end': 310,\n",
       "     'text': 'Table 1',\n",
       "     'ref_id': None}],\n",
       "   'section': 'Approach'},\n",
       "  {'text': 'Alignment to large nucleotide database Huson, et al., 2011 PhymmBL Alignment to large nucleotide database with interpolated Markov models Brady & Salzberg, 2011 Metaphyler Alignment to clade-specific marker genes Liu, et al., Overall, these methods are designed to either classify individual reads to, and/or predict the total abundance of, clades (e.g. genus or species) across the entire tree of life. They generally require reference databases that are very large and/or require a large amount of processing to generate. The gap SIANN is designed to fill is when the entire tree of life is irrelevant, and only predefined subsets of organisms need to be detected. For an underlying method we chose read alignment to diagnostic genomic regions because the algorithms for read alignment are highly parallelizable and have been optimized heavily by the community at large (the current implementation of SIANN uses bowtie2 [Langmead & Salzberg, 2012] for the alignment function, but can be adapted to any alignment algorithm). This approach is distinct from using cladespecific marker genes (Segata, et al., 2012) because unique regions that are larger, smaller, or outside of genes can also be used. Furthermore, this approach supports the rapid construction of custom databases using reference genome sets that require only minimal user-supplied structure.',\n",
       "   'cite_spans': [{'start': 39,\n",
       "     'end': 66,\n",
       "     'text': 'Huson, et al., 2011 PhymmBL',\n",
       "     'ref_id': None},\n",
       "    {'start': 138,\n",
       "     'end': 160,\n",
       "     'text': 'Brady & Salzberg, 2011',\n",
       "     'ref_id': None},\n",
       "    {'start': 213, 'end': 225, 'text': 'Liu, et al.,', 'ref_id': None},\n",
       "    {'start': 922,\n",
       "     'end': 949,\n",
       "     'text': '[Langmead & Salzberg, 2012]',\n",
       "     'ref_id': 'BIBREF7'},\n",
       "    {'start': 1090,\n",
       "     'end': 1112,\n",
       "     'text': '(Segata, et al., 2012)',\n",
       "     'ref_id': None}],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'To understand the principle at work, consider a set of reads that have been aligned to the genomes of several strains belonging to two species. Some regions of those genomes are species-specific, some are strain-specific, and some are shared ( Figure  1a ). When a set of reads is aligned to those genomes such that each read is placed in as many locations as it has a match (at a reasonably stringent threshold), visual inspection of the distribution of reads yields an intuitive understanding of the true source organism as Species I/Strain B (Figure 1b ). If Strain B were not present in the reference database, it would still be clear that the organism was an unknown strain of Species I.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 244,\n",
       "     'end': 254,\n",
       "     'text': 'Figure  1a',\n",
       "     'ref_id': None},\n",
       "    {'start': 545, 'end': 555, 'text': '(Figure 1b', 'ref_id': None}],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The unique identification of a species or strain is quantified by the proportion of the genome that is determined to be species-or strain-specific (defined as reads that are aligned to regions that are species-or strain-specific). Each species and strain is then assigned a numerical measure of the proportion that is covered by these diagnostic reads, and that proportional measure is compared to the ideal case, where sequences from a single organism (generated in silico) are aligned against the database in an identical manner. After that normalization factor is applied, the resulting score indicates whether the source sample contained any of the organisms in the reference database. Figure 1 . A) For a group of strains belonging to two different species, some regions may be unique to each species (region 1), while other regions may be unique to strains within each species (regions 2 and 3). B) A set of reads are aligned to these genomes, and the ones that align in a species-or strain-specific manner are identified by the combination of genomes to which they align. In this example, Strain B of Species I is the organism identified.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 690,\n",
       "     'end': 698,\n",
       "     'text': 'Figure 1',\n",
       "     'ref_id': None}],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The analysis is conducted independently on both the species and the strain level, so that if the true strain is not present in the database, the species of origin will still be identified. While many methods consider the complete taxonomic tree and assign reads to the least common ancestor, SIANN considers only two taxonomic levels: species and strain, throwing out anything that is not unique at one of those levels and thus obviating many of the confounding factors introduced by manually curated taxonomies.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The example shown in Figure 1b indicates that species-specific reads are identified as reads that align to one species (Species I, in that case) but not the other. If Species II were not present in the example shown in Figure 1b , a much larger number of reads would be assigned as \"species-specific,\" when in fact those regions are shared with other species. Therefore, the ability of this method to identify strain-and species-specific sequences is a direct function of the inclusion of near neighbors in the reference database. This characteristic is shared among many classification algorithms, but it is of particular note for this method when users have an opportunity to construct their own database.. In order to detect a target species with a high degree of specificity (reducing false positives), it is necessary to include other related species in the reference database. Only by parallel alignment to those near neighbors can the redundant sequences be separated from the species-specific ones. For example, in order to detect Bacillus anthracis in a sample, it would be necessary to include other species of Bacilli in the reference database so that the presence of B. cereus or B. thuringiensis in a sample does not lead to a false call for B. anthracis.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 21, 'end': 30, 'text': 'Figure 1b', 'ref_id': None},\n",
       "    {'start': 219, 'end': 228, 'text': 'Figure 1b', 'ref_id': None}],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The nomenclature of genus, species, and strain is potentially problematic because it does not correspond to a consistent degree of evolutionary distance or genomic distinctiveness. The ability to distinguish two organisms by any method using genomic sequence data is proportional to the amount of each genome that is shared or unique. One might assume that any two organisms of the same species will have a relatively predictable amount of shared genomic identity. However, some pairs of organisms from the same species may have less in common than other pairs of organisms from different species or even genera. This ambiguity impacts SIANN in two ways. If two organisms have very little genomic sequence to distinguish them, the sensitivity of SIANN to detect either one will diminish (the rate of false negatives will increase as the likelihood of sequencing unique regions decreases). Conversely, if an organism is extremely dissimilar to the near neighbors selected for the database, the specificity with which SIANN detects that organism will decline (the rate of false positives will increase as the number of related genomes available in the database decreases). For example, if a database contained only E. coli and B.anthracis, a sample containing B. cereus would be misidentified as contraining B. anthracis. In the intended use case, a database targeting B. anthracis would contain B. cereus and a number of other near neighbors to prevent that kind of misidentification. It would be convenient to say that an ideal database can be made by calculating the ideal genetic distance between all references and then finding an ideal set of organisms to make up that database, but the behavior of any database will be governed by the particular genomes of the organisms it encounters in the wild. Because not all organisms evolve in the same manner (differences in mutation rate, horizontal gene transfer, recombination, etc), the suitability of a database and method to detect a given organism can only be determined by thorough validation and benchmarking, as well as updating the reference database as needed. Users of SIANN may construct their own custom databases to include newly identified genomes or specific subsets of genomes that best suit their research interests.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'Steps to construct a custom database:',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': '1. Select a set of target organisms 2. Gather a set of genome sequences for those target organisms as well as a matched set of near neighbors 3. Using those reference genome sequences as an input, SIANN will: a. Construct a reference index for alignment b. Simulate a set of reads from each genome c. Align each of those simulated read sets to all of the reference genomes d. Calculate the proportion of each reference genome that is strain-or species-specific e. [If two organisms do not have a minimal amount of unique sequences that exceeds the rate of sequencing error, SIANN asks that all but one of those organisms are removed from the database to eliminate redundancy. Note that the user can provide a single representative genome with multiple strain names so that the redundant strain names are not lost.]',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The files contained within each SIANN database are a compressed genomic index and a list containing the proportion of each reference genome that was found to be strainor species-specific during database construction.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'To run SIANN:',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': '1. Select a pre-made SIANN database and a set of sequences to be analyzed, and 2. SIANN will: a. Align each of the reads against the reference genomes b. Calculate the proportion of each reference genome that is strain-or species-specific within those reads c. Compare that proportion to the simulated ideal case generated during database creation . CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint d. Calculate the probability that the given results could be generated by random chance e. Report the normalized proportion and non-parametric statistic of likelihood for each strain and species in the reference database. The normalized proportion of the genome covered by strain-or speciesspecific reads is the primary statistic reported by this tool.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Name Method Reference MEGAN'},\n",
       "  {'text': 'The performance of SIANN (version 1.6) was tested in comparison to the following stateof-the-art metagenomic classification programs: LMAT (version 1.2), MetaPhlAn (version 1.7.7), and Kraken (version 0.9.1b). All of the programs in Table 1 were investigated for this effort, and three were chosen based on their ability to run on our high-performance computing cluster with an execution time and memory requirement that would be suitable to a clinical lab. Each program was run on a set of 600 simulated datasets generated by MetaSim (Richter, et al., 2008) . Each dataset consisted of 15,000,000 reads (100bp single-ended) with Illumina-simulated error (fourth-degree polynomial) (Korbel, et al., 2009) . The 600 datasets were broken into 12 sets of 50 replicates. Each of the 12 sets contained organisms at different levels of abundance as shown in Table 2 .',\n",
       "   'cite_spans': [{'start': 535,\n",
       "     'end': 558,\n",
       "     'text': '(Richter, et al., 2008)',\n",
       "     'ref_id': 'BIBREF9'},\n",
       "    {'start': 682,\n",
       "     'end': 704,\n",
       "     'text': '(Korbel, et al., 2009)',\n",
       "     'ref_id': 'BIBREF5'}],\n",
       "   'ref_spans': [{'start': 233, 'end': 240, 'text': 'Table 1', 'ref_id': None},\n",
       "    {'start': 852, 'end': 859, 'text': 'Table 2', 'ref_id': None}],\n",
       "   'section': 'Benchmarking'},\n",
       "  {'text': 'Organisms were specifically chosen in pairs so that the ability to distinguish these near neighbors could be determined. The abundances were staggered at 4-fold intervals so that a wide range could be evaluated. All known species of near neighbors for each of the 12 target organisms were included in the reference database used by SIANN for this benchmarking (\"Target Pathogen Database\") and are shown in Appendix 1.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Benchmarking'},\n",
       "  {'text': 'Each program outputs a distinct measure. Kraken and LMAT both count the reads assigned to each taxon, MetaPhlAn calculates the abundance, and SIANN outputs a measure of the proportion of diagnostic genomic regions present. To put these measures on an even footing, we empirically calculated the false positive rate for each method over all 600 samples, at each possible measure of output. Because each dataset is made up of known organisms, any result can be classified as true or false. Therefore, for any possible result (say, 513 reads classified by LMAT or 1.6% abundance assigned by MetaPhlAn), one can calculate the proportion of calls with at least the same amount of support that were correct (True Positives/[True Positives+False Positives]), over all of the 600 datasets. That measure is commonly given as Positive Predictive Value (PPV). For each program, the results can be translated from the raw value into a PPV that is based on this empirical measure of error. The key item of interest is the PPV value for the results that we know to be true positives, the defined spike organisms. Another way of describing this approach is to say that the results of each program have been normalized to the false positive error rate that was empirically observed. If another set of samples were generated, the PPV vs. raw value curve ( Figure 2 ) would likely fall differently, but in this case it gives us a means of comparing a diverse set of methods against the same ground truth. If method 1 detects an organism with a higher PPV than method 2 does, it means that method 1 has fewer false positives in the range that it reports true positives, which is the definition of utility in this setting.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 1339,\n",
       "     'end': 1347,\n",
       "     'text': 'Figure 2',\n",
       "     'ref_id': 'FIGREF0'}],\n",
       "   'section': 'Benchmarking'},\n",
       "  {'text': 'For each method, PPV was calculated as a function of raw output value. Briefly, this was done by compiling the output for all 600 samples, labeling each result as false or true based on the sample set that it came from, and then calculating (at each possible value of output) what the proportion of TP/[TP+FP] was for results with at least that level of raw output. Some simplification steps were taken, such as focusing on the specieslevel assignments (for comparison with methods that do not perform strain assignment), and only taking the top hit for each species from each dataset. Custom R and BASH scripts were used for the data compilation and analysis.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Benchmarking'},\n",
       "  {'text': 'The relationship of raw output value to PPV is shown for each of the four methods in Figure 2 . The point at which PPV is very close to 1 (where 95% of results are true positives) is ~41,000 reads for Kraken, ~2,800 reads for LMAT, ~38% abundance for MetaPhlAn, and 0.21 for SIANN. For SIANN this means that having 38% of the species-unique genome covered by reads resulted in the vast majority of calls being accurate. . CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint For readassignment methods (such as LMAT and Kraken), manual inspection of the results may yield a different understanding of confidence than is presented here, or in any automated analysis.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 85,\n",
       "     'end': 93,\n",
       "     'text': 'Figure 2',\n",
       "     'ref_id': 'FIGREF0'}],\n",
       "   'section': 'Results'},\n",
       "  {'text': 'For example, while each read that is assigned by LMAT and Kraken fall above a certain cutoff for speciesspecificity, some individual reads may be much more specific than others. One could identify a read that aligns to a single species of bacteria with 100% accuracy over its 300bp length, with the next closest match being only 90% similar. It is extremely unlikely that a 300bp exact match would arise due to random chance, and so the user could say with confidence that the organism of interest is found within the sequence data (not considering contamination, horizontal gene transfer, etc). However, such an approach is not currently implemented in an automated method, and many of the steps needed to make that assertion are performed manually by a domain expert, including alignment to near neighbors and ensuring that the read does not fall within a transposon, plasmid, etc. Therefore, while one could say that a single read is all that is needed to state with high PPV that an organism is present, the amount of reads assigned in an automated manner needed to achieve that level of PPV will number in the thousands (Fig 2) . Table 2 ), and each program (boxes at top), across a maximum of 50 replicates (indicated by the size of each point). Note that the reference database for MetaPhlAn does not include viruses, and the reference database for Kraken does not include RNA viruses (e.g., Hanta virus).',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 1125,\n",
       "     'end': 1132,\n",
       "     'text': '(Fig 2)',\n",
       "     'ref_id': 'FIGREF0'},\n",
       "    {'start': 1135, 'end': 1142, 'text': 'Table 2', 'ref_id': None}],\n",
       "   'section': 'Results'},\n",
       "  {'text': '. CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint The next phase of benchmarking was to determine how many raw input reads were needed to achieve the threshold for high PPV. To demonstrate this we plotted the known abundance of each spike organism against the PPV value generated by each method (Figure 3 ). Each point (an organism at a known level of abundance) is comprised of a maximum of 50 replicates, where the diameter of each point increases with an increasing number of replicates. For demonstration purposes we are showing two pairs of bacteria and three viruses. Recall that for each of the pairs of bacteria (and the two poxviruses) any sample containing one did not contain the other (as shown in Table 1 ). The empty boxes result from the organisms not being called at any abundance. For MetaPhlAn, that is a result of no viruses being included in the version of the reference database available for this analysis. Kraken assigned no reads to Hanta virus because viral RNA genomes were not included in this version of the reference database (personal communication with D. Wood). This emphasizes the point that a) the ability to create custom databases targeting organisms of interest can be valuable, and b) the performance of any method must be benchmarked against each potential target of interest.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 460,\n",
       "     'end': 469,\n",
       "     'text': '(Figure 3',\n",
       "     'ref_id': 'FIGREF1'},\n",
       "    {'start': 875, 'end': 882, 'text': 'Table 1', 'ref_id': None}],\n",
       "   'section': 'Results'},\n",
       "  {'text': 'All methods were able identify the bulk of organisms in their databases at high abundances (75% and 18%, Figure 3 ), however performance varied considerably at lower abundances and depended on the particular organism and method used. SIANN detected each organism at high confidence, even at levels as low as 0.3% and 0.07% of the total.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [{'start': 105,\n",
       "     'end': 113,\n",
       "     'text': 'Figure 3',\n",
       "     'ref_id': 'FIGREF1'}],\n",
       "   'section': 'Results'},\n",
       "  {'text': \"The process of detecting trace amounts of a specific organism in a complex mixture of DNA is challenging enough for an expert, but that pales in comparison to the difficulty of accomplishing the same certainty of detection in an automated manner. The results presented here show that SIANN rapidly detects the presence of a given set of organisms with a high degree of specificity and sensitivity. For example, at the 95% confidence (PPV) cutoff of 0.2, SIANN reliably detects all of the organisms tested here at as low as 0.3% abundance. This strong performance is likely due to the fact that SIANN is able to use a method (read alignment to whole genomes) that would be far too computationally costly if it were applied to the entire collection of known genomes. By focusing on a set of (customizable) target organisms and their near neighbors, SIANN can operate quickly and with low computational requirements while delivering highly accurate results. SIANN is available on Illumina's BaseSpace (www.basespace.illumina.com) as a NativeApp, with the database tested here (Appendix 1), as well as a database made from the NCBI representative set of prokaryotic genomes (ftp://ftp.ncbi.nlm.nih.gov/genomes/genome_reports/) (Appendix 2) and the complete set of NCBI viral genomes (ftp://ftp.ncbi.nlm.nih.gov/refseq/release/viral/) (Appendix 3).\",\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Discussion'},\n",
       "  {'text': \". CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint BaseSpace was chosen as an appropriate release platform because while the entire set of software and dependencies can be deployed by the user from within a graphical user interface, the actual computation takes place in a controlled 'cloud' environment. Such a distribution strategy obviates the need to satisfy the multiple software or OS dependencies that often arises with academic computational methods. Results for SIANN are compiled into a report format, showing both the organisms that surpass 95% confidence, as well as the closest strain match for each species. The default view masks the raw data output, so that the results are human-readable and do not present extraneous information. While the code for execution and databaseconstruction on a users system is available from Signature Science, LLC, additional databases on the BaseSpace platform can be made available upon request.\",\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Discussion'},\n",
       "  {'text': 'There is a neverending list of questions that one could ask of metagenomic sequencing data generated from important samples. Instead of answering them all, we demonstrate a technique with a very narrow focus that is able to report with a high degree of confidence whether a given set of organisms is present in a sample. These results are presented to the user in a comprehensible format, and accessible on a commonly-used web platform. The world of bioinformatics will continue to progress and develop more sophisticated tools for metagenomic analysis, and we hope that the utility of SIANN will convince others to package and benchmark their tools in a way that they can be used with confidence by the larger public, as well as the research community.',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Discussion'},\n",
       "  {'text': '. CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint',\n",
       "   'cite_spans': [],\n",
       "   'ref_spans': [],\n",
       "   'section': 'Discussion'}],\n",
       " 'bib_entries': {'BIBREF0': {'ref_id': 'b0',\n",
       "   'title': 'Scalable metagenomic taxonomy classification using a reference genome database',\n",
       "   'authors': [{'first': 'S', 'middle': ['K'], 'last': 'Ames', 'suffix': ''},\n",
       "    {'first': 'D', 'middle': ['A'], 'last': 'Hysom', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': ['N'], 'last': 'Gardner', 'suffix': ''},\n",
       "    {'first': 'G', 'middle': ['S'], 'last': 'Lloyd', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': ['B'], 'last': 'Gokhale', 'suffix': ''},\n",
       "    {'first': 'Allen', 'middle': [], 'last': 'Je', 'suffix': ''}],\n",
       "   'year': 2013,\n",
       "   'venue': 'Bioinformatics',\n",
       "   'volume': '29',\n",
       "   'issn': '18',\n",
       "   'pages': '2253--60',\n",
       "   'other_ids': {'DOI': ['10.1093/bioinformatics/btt389']}},\n",
       "  'BIBREF1': {'ref_id': 'b1',\n",
       "   'title': 'A comparative evaluation of sequence classification programs',\n",
       "   'authors': [{'first': 'A',\n",
       "     'middle': ['L'],\n",
       "     'last': 'Bazinet',\n",
       "     'suffix': ''},\n",
       "    {'first': 'M', 'middle': ['P'], 'last': 'Cummings', 'suffix': ''}],\n",
       "   'year': 2012,\n",
       "   'venue': 'BMC Bioinformatics',\n",
       "   'volume': '13',\n",
       "   'issn': '',\n",
       "   'pages': '',\n",
       "   'other_ids': {'DOI': ['10.1186/1471-2105-13-92']}},\n",
       "  'BIBREF2': {'ref_id': 'b2',\n",
       "   'title': 'Rapid phylogenetic and functional classification of short genomic fragments with signature peptides',\n",
       "   'authors': [{'first': 'J', 'middle': [], 'last': 'Berendzen', 'suffix': ''},\n",
       "    {'first': 'W', 'middle': ['J'], 'last': 'Bruno', 'suffix': ''},\n",
       "    {'first': 'J', 'middle': ['D'], 'last': 'Cohn', 'suffix': ''},\n",
       "    {'first': 'N', 'middle': ['W'], 'last': 'Hengartner', 'suffix': ''},\n",
       "    {'first': 'C', 'middle': ['R'], 'last': 'Kuske', 'suffix': ''},\n",
       "    {'first': 'B', 'middle': ['H'], 'last': 'Mcmahon', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': ['A'], 'last': 'Wolinsky', 'suffix': ''},\n",
       "    {'first': 'G', 'middle': [], 'last': 'Xie', 'suffix': ''}],\n",
       "   'year': 2012,\n",
       "   'venue': 'BMC Research Notes',\n",
       "   'volume': '5',\n",
       "   'issn': '',\n",
       "   'pages': '',\n",
       "   'other_ids': {'DOI': ['10.1186/1756-0500-5-460']}},\n",
       "  'BIBREF3': {'ref_id': 'b3',\n",
       "   'title': 'Phymm and PhymmBL: metagenomic phylogenetic classification with interpolated Markov models',\n",
       "   'authors': [{'first': 'A', 'middle': [], 'last': 'Brady', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': ['L'], 'last': 'Salzberg', 'suffix': ''}],\n",
       "   'year': 2009,\n",
       "   'venue': 'Nature Methods',\n",
       "   'volume': '6',\n",
       "   'issn': '9',\n",
       "   'pages': '673--679',\n",
       "   'other_ids': {'DOI': ['10.1038/nmeth.1358']}},\n",
       "  'BIBREF4': {'ref_id': 'b4',\n",
       "   'title': 'Integrative analysis of environmental sequences using MEGAN4',\n",
       "   'authors': [{'first': 'D', 'middle': ['H'], 'last': 'Huson', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': [], 'last': 'Mitra', 'suffix': ''},\n",
       "    {'first': 'N', 'middle': [], 'last': 'Weber', 'suffix': ''},\n",
       "    {'first': 'H', 'middle': [], 'last': 'Ruscheweyh', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': ['C'], 'last': 'Schuster', 'suffix': ''}],\n",
       "   'year': 2011,\n",
       "   'venue': 'Genome Research',\n",
       "   'volume': '21',\n",
       "   'issn': '',\n",
       "   'pages': '1552--1560',\n",
       "   'other_ids': {}},\n",
       "  'BIBREF5': {'ref_id': 'b5',\n",
       "   'title': 'PEMer: a computational framework with simulation-based error models for inferring genomic structural variants from massive paired-end sequencing data',\n",
       "   'authors': [{'first': 'J', 'middle': ['O'], 'last': 'Korbel', 'suffix': ''},\n",
       "    {'first': 'A', 'middle': [], 'last': 'Abyzov', 'suffix': ''},\n",
       "    {'first': 'X', 'middle': ['J'], 'last': 'Mu', 'suffix': ''},\n",
       "    {'first': 'N', 'middle': [], 'last': 'Carriero', 'suffix': ''},\n",
       "    {'first': 'P', 'middle': [], 'last': 'Cayting', 'suffix': ''},\n",
       "    {'first': 'Z', 'middle': [], 'last': 'Zhang', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': [], 'last': 'Snyder', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': ['B'], 'last': 'Gerstein', 'suffix': ''}],\n",
       "   'year': 2009,\n",
       "   'venue': '',\n",
       "   'volume': '',\n",
       "   'issn': '',\n",
       "   'pages': '',\n",
       "   'other_ids': {}},\n",
       "  'BIBREF7': {'ref_id': 'b7',\n",
       "   'title': 'Fast gapped-read alignment with Bowtie 2',\n",
       "   'authors': [{'first': 'B', 'middle': [], 'last': 'Langmead', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': [], 'last': 'Salzberg', 'suffix': ''}],\n",
       "   'year': 2012,\n",
       "   'venue': 'Nature Methods',\n",
       "   'volume': '9',\n",
       "   'issn': '',\n",
       "   'pages': '357--359',\n",
       "   'other_ids': {}},\n",
       "  'BIBREF8': {'ref_id': 'b8',\n",
       "   'title': 'Accurate and fast estimation of taxonomic profiles from metagenomic shotgun sequences',\n",
       "   'authors': [{'first': 'B', 'middle': [], 'last': 'Liu', 'suffix': ''},\n",
       "    {'first': 'T', 'middle': [], 'last': 'Gibbons', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': [], 'last': 'Ghodsi', 'suffix': ''},\n",
       "    {'first': 'T', 'middle': [], 'last': 'Treangen', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': [], 'last': 'Pop', 'suffix': ''}],\n",
       "   'year': 2011,\n",
       "   'venue': 'BMC Genomics',\n",
       "   'volume': '12',\n",
       "   'issn': '2',\n",
       "   'pages': '',\n",
       "   'other_ids': {}},\n",
       "  'BIBREF9': {'ref_id': 'b9',\n",
       "   'title': 'MetaSim: a sequencing simulator for genomics and metagenomics',\n",
       "   'authors': [{'first': 'D',\n",
       "     'middle': ['C'],\n",
       "     'last': 'Richter',\n",
       "     'suffix': ''},\n",
       "    {'first': 'F', 'middle': [], 'last': 'Ott', 'suffix': ''},\n",
       "    {'first': 'A', 'middle': ['F'], 'last': 'Auch', 'suffix': ''},\n",
       "    {'first': 'R', 'middle': [], 'last': 'Schmid', 'suffix': ''},\n",
       "    {'first': 'D', 'middle': ['H'], 'last': 'Huson', 'suffix': ''}],\n",
       "   'year': 2008,\n",
       "   'venue': 'PLoS One',\n",
       "   'volume': '3',\n",
       "   'issn': '10',\n",
       "   'pages': '',\n",
       "   'other_ids': {'DOI': ['10.1371/journal.pone.0003373']}},\n",
       "  'BIBREF10': {'ref_id': 'b10',\n",
       "   'title': 'Genomic variation landscape of the human gut microbiome',\n",
       "   'authors': [{'first': 'S',\n",
       "     'middle': [],\n",
       "     'last': 'Schloissnig',\n",
       "     'suffix': ''},\n",
       "    {'first': 'M', 'middle': [], 'last': 'Arumugam', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': [], 'last': 'Sunagawa', 'suffix': ''},\n",
       "    {'first': 'M', 'middle': [], 'last': 'Mitreva', 'suffix': ''},\n",
       "    {'first': 'J', 'middle': [], 'last': 'Tap', 'suffix': ''},\n",
       "    {'first': 'A', 'middle': [], 'last': 'Zhu', 'suffix': ''},\n",
       "    {'first': 'A', 'middle': [], 'last': 'Waller', 'suffix': ''},\n",
       "    {'first': 'D', 'middle': ['R'], 'last': 'Mende', 'suffix': ''},\n",
       "    {'first': 'J', 'middle': ['R'], 'last': 'Kultima', 'suffix': ''},\n",
       "    {'first': 'J', 'middle': [], 'last': 'Martin', 'suffix': ''},\n",
       "    {'first': 'K', 'middle': [], 'last': 'Kota', 'suffix': ''},\n",
       "    {'first': 'S', 'middle': ['R'], 'last': 'Sunyaev', 'suffix': ''},\n",
       "    {'first': 'G', 'middle': ['M'], 'last': 'Weinstock', 'suffix': ''},\n",
       "    {'first': 'P', 'middle': [], 'last': 'Bork', 'suffix': ''}],\n",
       "   'year': 2013,\n",
       "   'venue': 'Nature',\n",
       "   'volume': '493',\n",
       "   'issn': '7430',\n",
       "   'pages': '45--50',\n",
       "   'other_ids': {'DOI': ['10.1038/nature11711']}}},\n",
       " 'ref_entries': {'FIGREF0': {'text': 'Relationship of reported value for each program (horizontal axis, log scale) to the empirically-determined Positive Predictive Value (PPV), shown on the vertical axis. While the exact values depend on the test data used, the general values at significant cutoff values (0.8, 0.9, 0.95 PPV) remain relatively constant across different datasets (data not shown).',\n",
       "   'latex': None,\n",
       "   'type': 'figure'},\n",
       "  'FIGREF1': {'text': 'The Positive Predictive Value (PPV, vertical axis) is shown for each organism (boxes on right), at each level of known abundance (horizontal axis, see',\n",
       "   'latex': None,\n",
       "   'type': 'figure'},\n",
       "  'TABREF0': {'text': '. Summary of methods for metagenomic classification.',\n",
       "   'latex': None,\n",
       "   'type': 'table'},\n",
       "  'TABREF1': {'text': 'Table 2. The abundance of each target organism in each set of simulated datasets. Each set is indicated by the number in the top row, and was generated with 50 replicates.',\n",
       "   'latex': None,\n",
       "   'type': 'table'},\n",
       "  'TABREF2': {'text': 'Segata N, Waldron L, Ballarini A, Narasimhan V,Jousson O, and Huttenhower C (2012). Metagenomic microbial community profiling using unique clade-specific marker genes. NatureMethods, 9(8):811-4. doi: 10.1038/nmeth.2066. Sunagawa S, Mende DR, Zeller G, Izquierdo-Carrasco F, Berger SA, Kultima JR, Coelho LP, Arumugam M, Tap J, Nielsen HB, Rasmussen S, Brunak S, Pedersen O, Guarner F, de Vos WM, Wang J, Li J, Doré J, Ehrlich SD, Stamatakis A & Bork P (2013). Metagenomic species profiling using universal phylogenetic marker genes. Nature Methods, 10, 1196-1199 (2013) doi:10.1038/nmeth.2693 Wood DE and Salzberg SL. Ultrafast metagenomic sequence classification using exact alignments. In submission. Wu GD, Chen J, Hoffmann C, Bittinger K, Chen YY, Keilbaugh SA, Bewtra M, Knights D, Walters WA, Knight R, Sinha R, Gilroy E, Gupta K, Baldassano R, Nessel L, Li H, Bushman FD, Lewis JD (2011). Linking long-term dietary patterns with gut microbial enterotypes. Science. 334(6052):105-8. doi: 10.1126/science.1208344. The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint Appendix 2: Viral Database Abaca bunchy top virus DNA-C Abaca bunchy top virus DNA-M Abaca bunchy top virus DNA-The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101Circovirus-like genome BBC-A Circovirus-like genome CB-A Circovirus-like genome CB-B Circovirus-like genome RW-A Circovirus-like genome RW-B Circovirus-like genome RW-C Circovirus-like genome RW-D Circovirus-like genome RW-E Circovirus-like genome SAR-A Circovirus-like genome SAR-B Citrus psorosis virus RNA1 Citrus psorosis virus RNA2 Citrus psorosis virus RNA3The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprintThe copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprintThe copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint . CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint . CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder.It . https://doi.org/10.1101/001727 doi: bioRxiv preprint Thermovirga lienii DSM 17291 . CC-BY-NC-ND 4.0 International license is made available under a The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It . https://doi.org/10.1101/001727 doi: bioRxiv preprint Thermus scotoductus SA 01 Thermus sp CCB US3 UF1 Thermus thermophilus HB27 Vibrio cholerae O1 biovar El Tor str N16961 Vibrio fischeri ES114 Vibrio furnissii NCTC 11218 Vibrio harveyi ATCC BAA 1116 . CC-BY-NC-ND 4.0 International license is made available under a',\n",
       "   'latex': None,\n",
       "   'type': 'table'}},\n",
       " 'back_matter': [],\n",
       " '_meta': {'sha': 'f056da9c64fbf00a4645ae326e8a4339d015d155',\n",
       "  'source_x': 'biorxiv',\n",
       "  'title': 'SIANN: Strain Identification by Alignment to Near Neighbors',\n",
       "  'doi': '10.1101/001727',\n",
       "  'pmcid': nan,\n",
       "  'pubmed_id': nan,\n",
       "  'license': 'biorxiv',\n",
       "  'abstract': 'Next-generation sequencing is increasingly being used to study samples composed of mixtures of organisms, such as in clinical applications where the presence of a pathogen at very low abundance may be highly important. We present an analytical method (SIANN: Strain Identification by Alignment to Near Neighbors) specifically designed to rapidly detect a set of target organisms in mixed samples that achieves a high degree of species- and strain-specificity by aligning short sequence reads to the genomes of near neighbor organisms, as well as that of the target. Empirical benchmarking alongside the current state-of-the-art methods shows an extremely high Positive Predictive Value, even at very low abundances of the target organism in a mixed sample. SIANN is available as an Illumina BaseSpace app, as well as through Signature Science, LLC. SIANN results are presented in a streamlined report designed to be comprehensible to the non-specialist user, providing a powerful tool for rapid species detection in a mixed sample. By focusing on a set of (customizable) target organisms and their near neighbors, SIANN can operate quickly and with low computational requirements while delivering highly accurate results.',\n",
       "  'publish_time': '2014-01-10',\n",
       "  'authors': 'Samuel Minot; Stephen D Turner; Krista L Ternus; Dana R Kadavy',\n",
       "  'journal': nan,\n",
       "  'Microsoft Academic Paper ID': nan,\n",
       "  'WHO #Covidence': nan,\n",
       "  'has_full_text': True,\n",
       "  'full_text_file': 'biorxiv_medrxiv',\n",
       "  'url': 'https://doi.org/10.1101/001727'}}"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['minot', 'turner', 'ternus', 'kadavy']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_names = [a['last'].lower() for a in paper['metadata']['authors']]\n",
    "last_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s2paper['venue']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "math.isnan(paper['_meta']['journal'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Validate train/test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error_pairs = []\n",
    "\n",
    "for k in range(1, n_splits+1):\n",
    "    \n",
    "    split_dir = os.path.join(cord19_dir, 'splits', str(k))\n",
    "    split_train_df = pd.read_csv(os.path.join(split_dir, 'train.tsv'), sep='\\t')\n",
    "    split_test_df = pd.read_csv(os.path.join(split_dir, 'test.tsv'), sep='\\t')\n",
    "\n",
    "    train_pairs = split_train_df[['text', 'text_b']].values.tolist()\n",
    "    test_pairs = split_test_df[['text', 'text_b']].values.tolist()\n",
    "\n",
    "    for p in test_pairs:\n",
    "        if p in train_pairs:\n",
    "            #raise ValueError('ERROR - Test pair exists also in train!')\n",
    "            error_pairs.append(p)\n",
    "            \n",
    "len(error_pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "error_pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Rapid communication',\n",
       " 'Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany.\\n2019-nCoV Transmission from Asymptomatic Patient In this report, investigators in Germany detected the spread of the novel coronavirus (2019-nCoV) from a person who had recently traveled from China...']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Meta data\n",
    "\n",
    "- title\n",
    "- year\n",
    "- venue\n",
    "- authors\n",
    "- arxiv_id\n",
    "- s2_id\n",
    "- doi\n",
    "- cord19_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doi</th>\n",
       "      <th>title</th>\n",
       "      <th>authors</th>\n",
       "      <th>year</th>\n",
       "      <th>venue</th>\n",
       "      <th>s2_id</th>\n",
       "      <th>arxiv_id</th>\n",
       "      <th>in_citations_count</th>\n",
       "      <th>out_citations_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1101/001727</td>\n",
       "      <td>SIANN: Strain Identification by Alignment to N...</td>\n",
       "      <td>Samuel Minot; Stephen D Turner; Krista L Ternu...</td>\n",
       "      <td>2014</td>\n",
       "      <td></td>\n",
       "      <td>f056da9c64fbf00a4645ae326e8a4339d015d155</td>\n",
       "      <td></td>\n",
       "      <td>-1</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.1101/003889</td>\n",
       "      <td>Spatial epidemiology of networked metapopulati...</td>\n",
       "      <td>Lin WANG; Xiang Li</td>\n",
       "      <td>2014</td>\n",
       "      <td></td>\n",
       "      <td>daf32e013d325a6feb80e83d15aabc64a48fae33</td>\n",
       "      <td></td>\n",
       "      <td>-1</td>\n",
       "      <td>149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.1101/006866</td>\n",
       "      <td>Sequencing of the human IG light chain loci fr...</td>\n",
       "      <td>Corey T Watson; Karyn Meltz Steinberg; Tina A ...</td>\n",
       "      <td>2014</td>\n",
       "      <td></td>\n",
       "      <td>f33c6d94b0efaa198f8f3f20e644625fa3fe10d2</td>\n",
       "      <td></td>\n",
       "      <td>-1</td>\n",
       "      <td>62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10.1101/007476</td>\n",
       "      <td>Bayesian mixture analysis for metagenomic comm...</td>\n",
       "      <td>Sofia Morfopoulou; Vincent Plagnol</td>\n",
       "      <td>2014</td>\n",
       "      <td></td>\n",
       "      <td>4da8a87e614373d56070ed272487451266dce919</td>\n",
       "      <td></td>\n",
       "      <td>-1</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.1101/010389</td>\n",
       "      <td>Mapping a viral phylogeny onto outbreak trees ...</td>\n",
       "      <td>Stephen P Velsko; Jonathan E Allen</td>\n",
       "      <td>2014</td>\n",
       "      <td></td>\n",
       "      <td>eccef80cfbe078235df22398f195d5db462d8000</td>\n",
       "      <td></td>\n",
       "      <td>-1</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95195</th>\n",
       "      <td>10.1016/S0006-291X(67)80055-X</td>\n",
       "      <td>On the size of the active site in proteases. I...</td>\n",
       "      <td>Israel Schechter; Abe Berger</td>\n",
       "      <td>1967</td>\n",
       "      <td>Biochemical and biophysical research communica...</td>\n",
       "      <td>1ad6237a340cc6be5043f1b63f165512ece17df4</td>\n",
       "      <td></td>\n",
       "      <td>772</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95196</th>\n",
       "      <td>10.1038/srep36160</td>\n",
       "      <td>HCV core protein inhibits polarization and act...</td>\n",
       "      <td>Qianqian Zhang; Yang Wang; Naicui Zhai; Hongxi...</td>\n",
       "      <td>2016</td>\n",
       "      <td>Scientific reports</td>\n",
       "      <td>959a8d9c934bbe74d0d55b6da33f1a712e3eec3f</td>\n",
       "      <td></td>\n",
       "      <td>12</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95197</th>\n",
       "      <td>10.1128/AAC.04447-14</td>\n",
       "      <td>Relationship between azithromycin susceptibili...</td>\n",
       "      <td>Begoña Euba; Javier Moleres; Cristina Viadas; ...</td>\n",
       "      <td>2015</td>\n",
       "      <td>Antimicrobial agents and chemotherapy</td>\n",
       "      <td>9a1e4f5ac24aa47eb958bc493899974840ccc1f8</td>\n",
       "      <td></td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95198</th>\n",
       "      <td>10.1016/j.jep.2012.04.053</td>\n",
       "      <td>Effect of Sophora flavescens Aiton extract on ...</td>\n",
       "      <td>Hyungwoo Kim; Mi Ran Lee; Guem San Lee; Won Gu...</td>\n",
       "      <td>2012</td>\n",
       "      <td>Journal of ethnopharmacology</td>\n",
       "      <td>fc4c38837d0718a0d7c2c0e8bba0457ba6aa3e29</td>\n",
       "      <td></td>\n",
       "      <td>24</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95199</th>\n",
       "      <td>10.1128/MCB.23.7.2614-2622.2003</td>\n",
       "      <td>Disruption of mouse CD46 causes an accelerated...</td>\n",
       "      <td>Naokazu Inoue; Masahito Ikawa; Tomoko Nakanish...</td>\n",
       "      <td>2003</td>\n",
       "      <td>Molecular and cellular biology</td>\n",
       "      <td>3df578bb5aba5af004f32b70aa45b36b2bfd9a27</td>\n",
       "      <td></td>\n",
       "      <td>65</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>95200 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   doi  \\\n",
       "0                       10.1101/001727   \n",
       "1                       10.1101/003889   \n",
       "2                       10.1101/006866   \n",
       "3                       10.1101/007476   \n",
       "4                       10.1101/010389   \n",
       "...                                ...   \n",
       "95195    10.1016/S0006-291X(67)80055-X   \n",
       "95196                10.1038/srep36160   \n",
       "95197             10.1128/AAC.04447-14   \n",
       "95198        10.1016/j.jep.2012.04.053   \n",
       "95199  10.1128/MCB.23.7.2614-2622.2003   \n",
       "\n",
       "                                                   title  \\\n",
       "0      SIANN: Strain Identification by Alignment to N...   \n",
       "1      Spatial epidemiology of networked metapopulati...   \n",
       "2      Sequencing of the human IG light chain loci fr...   \n",
       "3      Bayesian mixture analysis for metagenomic comm...   \n",
       "4      Mapping a viral phylogeny onto outbreak trees ...   \n",
       "...                                                  ...   \n",
       "95195  On the size of the active site in proteases. I...   \n",
       "95196  HCV core protein inhibits polarization and act...   \n",
       "95197  Relationship between azithromycin susceptibili...   \n",
       "95198  Effect of Sophora flavescens Aiton extract on ...   \n",
       "95199  Disruption of mouse CD46 causes an accelerated...   \n",
       "\n",
       "                                                 authors  year  \\\n",
       "0      Samuel Minot; Stephen D Turner; Krista L Ternu...  2014   \n",
       "1                                     Lin WANG; Xiang Li  2014   \n",
       "2      Corey T Watson; Karyn Meltz Steinberg; Tina A ...  2014   \n",
       "3                     Sofia Morfopoulou; Vincent Plagnol  2014   \n",
       "4                     Stephen P Velsko; Jonathan E Allen  2014   \n",
       "...                                                  ...   ...   \n",
       "95195                       Israel Schechter; Abe Berger  1967   \n",
       "95196  Qianqian Zhang; Yang Wang; Naicui Zhai; Hongxi...  2016   \n",
       "95197  Begoña Euba; Javier Moleres; Cristina Viadas; ...  2015   \n",
       "95198  Hyungwoo Kim; Mi Ran Lee; Guem San Lee; Won Gu...  2012   \n",
       "95199  Naokazu Inoue; Masahito Ikawa; Tomoko Nakanish...  2003   \n",
       "\n",
       "                                                   venue  \\\n",
       "0                                                          \n",
       "1                                                          \n",
       "2                                                          \n",
       "3                                                          \n",
       "4                                                          \n",
       "...                                                  ...   \n",
       "95195  Biochemical and biophysical research communica...   \n",
       "95196                                 Scientific reports   \n",
       "95197              Antimicrobial agents and chemotherapy   \n",
       "95198                       Journal of ethnopharmacology   \n",
       "95199                     Molecular and cellular biology   \n",
       "\n",
       "                                          s2_id arxiv_id  in_citations_count  \\\n",
       "0      f056da9c64fbf00a4645ae326e8a4339d015d155                           -1   \n",
       "1      daf32e013d325a6feb80e83d15aabc64a48fae33                           -1   \n",
       "2      f33c6d94b0efaa198f8f3f20e644625fa3fe10d2                           -1   \n",
       "3      4da8a87e614373d56070ed272487451266dce919                           -1   \n",
       "4      eccef80cfbe078235df22398f195d5db462d8000                           -1   \n",
       "...                                         ...      ...                 ...   \n",
       "95195  1ad6237a340cc6be5043f1b63f165512ece17df4                          772   \n",
       "95196  959a8d9c934bbe74d0d55b6da33f1a712e3eec3f                           12   \n",
       "95197  9a1e4f5ac24aa47eb958bc493899974840ccc1f8                            9   \n",
       "95198  fc4c38837d0718a0d7c2c0e8bba0457ba6aa3e29                           24   \n",
       "95199  3df578bb5aba5af004f32b70aa45b36b2bfd9a27                           65   \n",
       "\n",
       "       out_citations_count  \n",
       "0                       10  \n",
       "1                      149  \n",
       "2                       62  \n",
       "3                       29  \n",
       "4                       30  \n",
       "...                    ...  \n",
       "95195                    0  \n",
       "95196                   49  \n",
       "95197                    0  \n",
       "95198                   35  \n",
       "95199                   44  \n",
       "\n",
       "[95200 rows x 9 columns]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "meta_rows = []\n",
    "\n",
    "# Cord 19\n",
    "for i, (doi, p) in enumerate(doi2paper.items()):\n",
    "    m = p['_meta']\n",
    "    meta_rows.append({\n",
    "        'doi': doi,\n",
    "        'title': m['title'],\n",
    "        'authors': m['authors'],\n",
    "        'year': int(m['publish_time'].split('-')[0]),\n",
    "        'venue': '' if isinstance(m['journal'], float) and math.isnan(m['journal']) else m['journal'],\n",
    "        's2_id': p['paper_id'],\n",
    "        'arxiv_id': '',\n",
    "        'in_citations_count': -1,\n",
    "        'out_citations_count': len(p['bib_entries']),\n",
    "    })   \n",
    "\n",
    "# S2 paper\n",
    "for i, (doi, p) in enumerate(doi2s2paper.items()):\n",
    "\n",
    "    meta_rows.append({\n",
    "        'doi': doi,\n",
    "        'title': p['title'],\n",
    "        'authors': '; '.join([a['name'] for a in p['authors']]),\n",
    "        'year': int(p['year'] or 0),\n",
    "        'venue': p['venue'] or '',\n",
    "        's2_id': p['paperId'],\n",
    "        'arxiv_id': p['arxivId'] or '',\n",
    "        'in_citations_count': len(p['citations']),\n",
    "        'out_citations_count': len(p['references']),\n",
    "    })   \n",
    "    \n",
    "    \n",
    "meta_df = pd.DataFrame(meta_rows) #, dtype={'year': 'number'})\n",
    "meta_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2015    7002\n",
       "2014    6936\n",
       "2013    6589\n",
       "2016    6518\n",
       "2017    6058\n",
       "        ... \n",
       "1945       1\n",
       "1935       1\n",
       "1936       1\n",
       "1937       1\n",
       "1938       1\n",
       "Name: year, Length: 91, dtype: int64"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "meta_df['year'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "meta_df.to_csv(os.path.join(cord19_dir, 'meta.csv'), index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[{\"doi\": \"10.1101/001727\", \"title\": \"SIANN: Strain Identification by Alignment to Near Neighbors\", \"authors\": \"Samuel Minot; Stephen D Turner; Krista L Ternus; Dana R Kadavy\", \"year\": 2014, \"venue\": null, \"s2_id\": \"f056da9c64fbf00a4645ae326e8a4339d015d155\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 10}, {\"doi\": \"10.1101/003889\", \"title\": \"Spatial epidemiology of networked metapopulation: An overview\", \"authors\": \"Lin WANG; Xiang Li\", \"year\": 2014, \"venue\": null, \"s2_id\": \"daf32e013d325a6feb80e83d15aabc64a48fae33\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 149}, {\"doi\": \"10.1101/006866\", \"title\": \"Sequencing of the human IG light chain loci from a hydatidiform mole BAC library reveals locus-specific signatures of genetic diversity\", \"authors\": \"Corey T Watson; Karyn Meltz Steinberg; Tina A Graves-Lindsay; Rene L Warren; Maika Malig; Jacqueline E Schein; Richard K Wilson; Rob Holt; Evan Eichler; Felix Breden\", \"year\": 2014, \"venue\": null, \"s2_id\": \"f33c6d94b0efaa198f8f3f20e644625fa3fe10d2\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 62}, {\"doi\": \"10.1101/007476\", \"title\": \"Bayesian mixture analysis for metagenomic community profiling.\", \"authors\": \"Sofia Morfopoulou; Vincent Plagnol\", \"year\": 2014, \"venue\": null, \"s2_id\": \"4da8a87e614373d56070ed272487451266dce919\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 29}, {\"doi\": \"10.1101/010389\", \"title\": \"Mapping a viral phylogeny onto outbreak trees to improve host transmission inference\", \"authors\": \"Stephen P Velsko; Jonathan E Allen\", \"year\": 2014, \"venue\": null, \"s2_id\": \"eccef80cfbe078235df22398f195d5db462d8000\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 30}, {\"doi\": \"10.1101/012070\", \"title\": \"The infant airway microbiome in health and disease impacts later asthma development\", \"authors\": \"Shu Mei Teo; Danny Mok; Kym Pham; Merci Kusel; Michael Serralha; Niamh Troy; Barbara J Holt; Belinda J Hales; Michael L Walker; Elysia Hollams; Yury H Bochkov; Kristine Grindle; Sebastian L Johnston; James E Gern; Peter D Sly; Patrick G Holt; Kathryn E Holt; Michael Inouye\", \"year\": 2014, \"venue\": null, \"s2_id\": \"c41fdb2efd6d61384a92a84cbba3f8233629a41b\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 41}, {\"doi\": \"10.1101/017178\", \"title\": \"Using heterogeneity in the population structure of U.S. swine farms to compare transmission models for porcine epidemic diarrhoea\", \"authors\": \"Eamon B. O\\\\u2019Dea; Harry Snelson; Shweta Bansal\", \"year\": 2015, \"venue\": null, \"s2_id\": \"1dd898b5ca1ae70ec0e3cad89fc87a165002a99e\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 43}, {\"doi\": \"10.1101/018481\", \"title\": \"Big city, small world: Density, contact rates, and transmission of dengue across Pakistan.\", \"authors\": \"Moritz U. G. Kraemer; T. Alex Perkins; Derek A.T. Cummings; Rubeena Zakar; Simon I. Hay; David L. Smith; Robert C. Reiner\", \"year\": 2015, \"venue\": null, \"s2_id\": \"33565294e6bc67fb7ee14dcae6cfdb08148f4ea5\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 3}, {\"doi\": \"10.1101/020834\", \"title\": \"MERS-CoV recombination: implications about the reservoir and potential for adaptation\", \"authors\": \"Gytis Dudas; Andrew Rambaut\", \"year\": 2015, \"venue\": null, \"s2_id\": \"3461d71f6890f7e5ba53bf168be3945cdb16d901\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 71}, {\"doi\": \"10.1101/027722\", \"title\": \"On the causes of evolutionary transition:transversion bias\", \"authors\": \"Arlin Stoltzfus; Ryan W. Norris\", \"year\": 2015, \"venue\": null, \"s2_id\": \"1f9d3f9a1a0e8db6a086e0a2b5ba50cf9f235dae\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 43}, {\"doi\": \"10.1101/029397\", \"title\": \"Viruses are a dominant driver of protein adaptation in mammals\", \"authors\": \"David Enard; Le Cai; Carina Gwenapp; Dmitri A Petrov\", \"year\": 2015, \"venue\": null, \"s2_id\": \"01e3b313e78a352593be2ff64927192af66619b5\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 25}, {\"doi\": \"10.1101/041509\", \"title\": \"How domain growth is implemented determines the long term behaviour of a cell population through its effect on spatial correlations\", \"authors\": \"Robert J. H. Ross; R. E. Baker; C. A. Yates\", \"year\": 2016, \"venue\": null, \"s2_id\": \"3b73c8abb7528c958cec0bae1890828af007b599\", \"arxiv_id\": null, \"in_citations_count\": null, \"out_citations_count\": 46}]'"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.dumps(meta_rows)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'doi': '10.1142/S1793048008000708',\n",
       "  'title': 'EPIDEMIC PREDICTIONS AND PREDICTABILITY IN COMPLEX ENVIRONMENTS',\n",
       "  'authors': 'Vittoria Colizza; Alain Barrat; Marc Barthelemy; Alessandro Vespignani',\n",
       "  'year': 2008,\n",
       "  'venue': '',\n",
       "  's2_id': '9c4f57a877f8bd1eab094d42dd9adef6b7023dde',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 2,\n",
       "  'out_citations_count': 0},\n",
       " {'doi': '10.1186/1743-422X-4-102',\n",
       "  'title': 'Cloning of the canine RNA polymerase I promoter and establishment of reverse genetics for influenza A and B in MDCK cells',\n",
       "  'authors': 'Zhaoti Wang; Gregory M. Duke',\n",
       "  'year': 2007,\n",
       "  'venue': 'Virology Journal',\n",
       "  's2_id': '383311a7b538cde429515869677fbda4826f9faa',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 18,\n",
       "  'out_citations_count': 18},\n",
       " {'doi': '10.1016/S0168-1702(99)00032-5',\n",
       "  'title': 'The S gene of canine coronavirus, strain UCD-1, is more closely related to the S gene of transmissible gastroenteritis virus than to that of feline infectious peritonitis virus.',\n",
       "  'authors': 'Ronald D. Wesley',\n",
       "  'year': 1999,\n",
       "  'venue': 'Virus research',\n",
       "  's2_id': 'a4b80dc3d8382941b450d9c006a5b5225fcba84d',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 28,\n",
       "  'out_citations_count': 20},\n",
       " {'doi': '10.3390/molecules22040657',\n",
       "  'title': 'Diversity Analysis and Bioresource Characterization of Halophilic Bacteria Isolated from a South African Saltpan',\n",
       "  'authors': 'Ramganesh Selvarajan; Timothy Sibanda; Memory Tekere; Hlengilizwe Nyoni; Stephen Meddows-Taylor',\n",
       "  'year': 2017,\n",
       "  'venue': 'Molecules',\n",
       "  's2_id': 'e37fa5ffe72279de2b5afed5647b71c079383b9e',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 12,\n",
       "  'out_citations_count': 81},\n",
       " {'doi': '10.1126/science.1223467',\n",
       "  'title': 'Quantifying the impact of human mobility on malaria.',\n",
       "  'authors': 'Amy Wesolowski; Nathan Eagle; Andrew J. Tatem; David L Smith; A. M. Noor; Robert W. Snow; Caroline O. Buckee',\n",
       "  'year': 2012,\n",
       "  'venue': 'Science',\n",
       "  's2_id': '2d345a375744d4d0dae25d56309a1097de3a20b4',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 356,\n",
       "  'out_citations_count': 0},\n",
       " {'doi': '10.3389/fcimb.2012.00114',\n",
       "  'title': 'Enteric pathogens through life stages',\n",
       "  'authors': 'Glynis L. Kolling; Martin Wu; Richard Littleton Guerrant',\n",
       "  'year': 2012,\n",
       "  'venue': 'Front. Cell. Inf. Microbio.',\n",
       "  's2_id': 'b286207f42621d8aea40ca2628b939f54d128588',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 29,\n",
       "  'out_citations_count': 120},\n",
       " {'doi': '10.1186/s12931-015-0219-1',\n",
       "  'title': 'Measurement of C-reactive protein, procalcitonin and neutrophil elastase in saliva of COPD patients and healthy controls: correlation to self-reported wellbeing parameters',\n",
       "  'authors': 'Neil Patel; John Belcher; Gary Thorpe; Nicholas R Forsyth; Monica A. Spiteri',\n",
       "  'year': 2015,\n",
       "  'venue': 'Respiratory Research',\n",
       "  's2_id': '2149e6dff07a909eeb6b9a435e4d2203b5ab83c2',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 8,\n",
       "  'out_citations_count': 69},\n",
       " {'doi': '10.4315/0362-028X.JFP-11-081',\n",
       "  'title': 'Evaluation of murine norovirus persistence in environments relevant to food production and processing.',\n",
       "  'authors': 'Saman Fallahi; Kirsten Mattison',\n",
       "  'year': 2011,\n",
       "  'venue': 'Journal of food protection',\n",
       "  's2_id': 'e8aed4a3358e9d8824e2cb1bbc031ea1846aa166',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 16,\n",
       "  'out_citations_count': 0},\n",
       " {'doi': '10.1002/chem.201605960',\n",
       "  'title': 'Bis-picolinamide Ruthenium(III) Dihalide Complexes: Dichloride-to-Diiodide Exchange Generates Single trans Isomers with High Potency and Cancer Cell Selectivity.',\n",
       "  'authors': 'Aida M. Basri; Rianne M Lord; Simon J Allison; Andrea Rodríguez-Bárzano; Stephanie J Lucas; Felix D Janeway; Helena J Shepherd; Christopher M Pask; Roger M. Phillips; Patrick C. McGowan',\n",
       "  'year': 2017,\n",
       "  'venue': 'Chemistry',\n",
       "  's2_id': '29990de62a727e4378e3c237404fea3d33ad4787',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 2,\n",
       "  'out_citations_count': 1},\n",
       " {'doi': '10.1016/j.virusres.2015.12.009',\n",
       "  'title': 'Characterization of a new megabirnavirus that confers hypovirulence with the aid of a co-infecting partitivirus to the host fungus, Rosellinia necatrix.',\n",
       "  'authors': 'Atsuko Sasaki; Hitoshi Nakamura; Nobuhiro Suzuki; Satoko Kanematsu',\n",
       "  'year': 2016,\n",
       "  'venue': 'Virus research',\n",
       "  's2_id': 'a59f667d41b5e33b42d2e24a8f7b1b827b88ccb5',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 10,\n",
       "  'out_citations_count': 41},\n",
       " {'doi': '10.1146/annurev-immunol-032713-120142',\n",
       "  'title': 'The interaction between signal regulatory protein alpha (SIRPα) and CD47: structure, function, and therapeutic target.',\n",
       "  'authors': 'A. Neil Barclay; Timo K van den Berg',\n",
       "  'year': 2014,\n",
       "  'venue': 'Annual review of immunology',\n",
       "  's2_id': '3a918f9299ac6f421e0de372bd881a7bd6ccdee3',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 99,\n",
       "  'out_citations_count': 0},\n",
       " {'doi': '10.1111/j.1742-4658.2012.08729.x',\n",
       "  'title': \"The PDZ protein discs-large (DLG): the 'Jekyll and Hyde' of the epithelial polarity proteins.\",\n",
       "  'authors': 'Sally Roberts; Craig P. Delury; Elizabeth Kate Marsh',\n",
       "  'year': 2012,\n",
       "  'venue': 'The FEBS journal',\n",
       "  's2_id': '3c6c985822774c2436fa7a7327a07d629ca55f71',\n",
       "  'arxiv_id': None,\n",
       "  'in_citations_count': 40,\n",
       "  'out_citations_count': 0}]"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#meta_rows = []\n",
    "\n",
    "\n",
    "    if i > 10:\n",
    "        break\n",
    "    \n",
    "#p['abstract'][0]['text']\n",
    "#p['metadata']['title']\n",
    "meta_rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['abstract', 'arxivId', 'authors', 'citationVelocity', 'citations', 'corpusId', 'doi', 'fieldsOfStudy', 'influentialCitationCount', 'is_open_access', 'is_publisher_licensed', 'paperId', 'references', 'title', 'topics', 'url', 'venue', 'year'])"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Sally Roberts, Craig P. Delury, Elizabeth Kate Marsh'"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['paper_id', 'metadata', 'abstract', 'body_text', 'bib_entries', 'ref_entries', 'back_matter', '_meta'])"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sha': 'f056da9c64fbf00a4645ae326e8a4339d015d155',\n",
       " 'source_x': 'biorxiv',\n",
       " 'title': 'SIANN: Strain Identification by Alignment to Near Neighbors',\n",
       " 'doi': '10.1101/001727',\n",
       " 'pmcid': nan,\n",
       " 'pubmed_id': nan,\n",
       " 'license': 'biorxiv',\n",
       " 'abstract': 'Next-generation sequencing is increasingly being used to study samples composed of mixtures of organisms, such as in clinical applications where the presence of a pathogen at very low abundance may be highly important. We present an analytical method (SIANN: Strain Identification by Alignment to Near Neighbors) specifically designed to rapidly detect a set of target organisms in mixed samples that achieves a high degree of species- and strain-specificity by aligning short sequence reads to the genomes of near neighbor organisms, as well as that of the target. Empirical benchmarking alongside the current state-of-the-art methods shows an extremely high Positive Predictive Value, even at very low abundances of the target organism in a mixed sample. SIANN is available as an Illumina BaseSpace app, as well as through Signature Science, LLC. SIANN results are presented in a streamlined report designed to be comprehensible to the non-specialist user, providing a powerful tool for rapid species detection in a mixed sample. By focusing on a set of (customizable) target organisms and their near neighbors, SIANN can operate quickly and with low computational requirements while delivering highly accurate results.',\n",
       " 'publish_time': '2014-01-10',\n",
       " 'authors': 'Samuel Minot; Stephen D Turner; Krista L Ternus; Dana R Kadavy',\n",
       " 'journal': nan,\n",
       " 'Microsoft Academic Paper ID': nan,\n",
       " 'WHO #Covidence': nan,\n",
       " 'has_full_text': True,\n",
       " 'full_text_file': 'biorxiv_medrxiv',\n",
       " 'url': 'https://doi.org/10.1101/001727'}"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p['_meta']['authors']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'f056da9c64fbf00a4645ae326e8a4339d015d155'"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p['paper_id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}