Tips/2016-03-14-Command-Line-tools-in-Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Python 开发命令行工具\n",
    "\n",
    "Python 作为一种脚本语言，可以非常方便地用于系统（尤其是\\*nix系统）命令行工具的开发。Python 自身也集成了一些标准库，专门用于处理命令行相关的问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### 命令行工具的一般结构\n",
    "\n",
    "![CL-in-Python](http://qncdn.rainy.im/CL-in-Python.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**1. 标准输入输出**\n",
    "\n",
    "\\*nix 系统中，一切皆为文件，因此标准输入、输出可以完全可以看做是对文件的操作。标准化输入可以通过管道（pipe）或重定向（redirect）的方式传递："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# script reverse.py\n",
    "#!/usr/bin/env python\n",
    "import sys\n",
    "for l in sys.stdin.readlines():\n",
    "    sys.stdout.write(l[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "保存为 `reverse.py`，通过管道 `|` 传递：\n",
    "\n",
    "```sh\n",
    "chmod +x reverse.py\n",
    "cat reverse.py | ./reverse.py\n",
    "\n",
    "nohtyp vne/nib/rsu/!#\n",
    "sys tropmi\n",
    ":)(senildaer.nidts.sys ni l rof\n",
    ")]1-::[l(etirw.tuodts.sys\n",
    "```\n",
    "\n",
    "通过重定向 `<` 传递：\n",
    "\n",
    "```sh\n",
    "./reverse.py < reverse.py\n",
    "# 输出结果同上\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**2. 命令行参数**\n",
    "\n",
    "一般在命令行后追加的参数可以通过 `sys.argv` 获取， `sys.argv` 是一个列表，其中第一个元素为当前脚本的文件名："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['/Users/rainy/Projects/GitHub/pytips/venv3/lib/python3.5/site-packages/ipykernel/__main__.py', '-f', '/Users/rainy/Library/Jupyter/runtime/kernel-0533e681-bd7c-4c4d-9094-a78fde7fc2ed.json']\n"
     ]
    }
   ],
   "source": [
    "# script argv.py\n",
    "#!/usr/bin/env python\n",
    "import sys\n",
    "print(sys.argv) # 下面返回的是 Jupyter 运行的结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行上面的脚本：\n",
    "\n",
    "```sh\n",
    "chmod +x argv.py\n",
    "./argv.py hello world\n",
    "python argv.py hello world\n",
    "\n",
    "# 返回的结果是相同的\n",
    "# ['./test.py', 'hello', 'world']\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于比较复杂的命令行参数，例如通过 `--option` 传递的选项参数，如果是对 `sys.argv` 逐项进行解析会很麻烦，Python 提供标准库 [`argparse`](https://docs.python.org/3/library/argparse.html)（旧的库为 `optparse`，已经停止维护）专门解析命令行参数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Load config from: config.ini\n",
      "Set theme: default.theme\n"
     ]
    }
   ],
   "source": [
    "# script convert.py\n",
    "#!/usr/bin/env python\n",
    "import argparse as apa\n",
    "def loadConfig(config):\n",
    "    print(\"Load config from: {}\".format(config))\n",
    "def setTheme(theme):\n",
    "    print(\"Set theme: {}\".format(theme))\n",
    "def main():\n",
    "    parser = apa.ArgumentParser(prog=\"convert\") # 设定命令信息，用于输出帮助信息\n",
    "    parser.add_argument(\"-c\", \"--config\", required=False, default=\"config.ini\")\n",
    "    parser.add_argument(\"-t\", \"--theme\", required=False, default=\"default.theme\")\n",
    "    parser.add_argument(\"-f\") # Accept Jupyter runtime option\n",
    "    args = parser.parse_args()\n",
    "    loadConfig(args.config)\n",
    "    setTheme(args.theme)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "利用 `argparse` 可以很方便地解析选项参数，同时可以定义指定参数的相关属性（是否必须、默认值等），同时还可以自动生成帮助文档。执行上面的脚本：\n",
    "\n",
    "```sh\n",
    "./convert.py -h\n",
    "usage: convert [-h] [-c CONFIG] [-t THEME]\n",
    "\n",
    "optional arguments:\n",
    "  -h, --help            show this help message and exit\n",
    "  -c CONFIG, --config CONFIG\n",
    "  -t THEME, --theme THEME\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "**3. 执行系统命令**\n",
    "\n",
    "当 Python 能够准确地解读输入信息或参数之后，就可以通过 Python 去做任何事情了。这里主要介绍通过 Python 调用系统命令，也就是替代 `Shell` 脚本完成系统管理的功能。我以前的习惯是将命令行指令通过 `os.system(command)` 执行，但是更好的做法应该是用 [`subprocess`](https://docs.python.org/3.5/library/subprocess.html) 标准库，它的存在就是为了替代旧的 `os.system; os.spawn*` 。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "`subprocess` 模块提供简便的直接调用系统指令的`call()`方法，以及较为复杂可以让用户更加深入地与系统命令进行交互的`Popen`对象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rw-r--r--  1 rainy  staff   3.4K  3  8 17:36 ./2016-03-06-The-Zen-of-Python.ipynb\n",
      "-rw-r--r--  1 rainy  staff   6.7K  3  8 17:45 ./2016-03-07-iterator-and-generator.ipynb\n",
      "-rw-r--r--  1 rainy  staff   6.0K  3 10 12:35 ./2016-03-08-Functional-Programming-in-Python.ipynb\n",
      "-rw-r--r--  1 rainy  staff   5.9K  3  9 16:28 ./2016-03-09-List-Comprehension.ipynb\n",
      "-rw-r--r--  1 rainy  staff    10K  3 10 14:14 ./2016-03-10-Scope-and-Closure.ipynb\n",
      "-rw-r--r--  1 rainy  staff   8.0K  3 11 16:30 ./2016-03-11-Arguments-and-Unpacking.ipynb\n",
      "-rw-r--r--  1 rainy  staff   8.5K  3 14 19:31 ./2016-03-14-Command-Line-tools-in-Python.ipynb\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# script list_files.py\n",
    "#!/usr/bin/env python\n",
    "import subprocess as sb\n",
    "res = sb.check_output(\"ls -lh ./*.ipynb\", shell=True) # 为了安全起见，默认不通过系统 Shell 执行，因此需要设定 shell=True\n",
    "print(res.decode()) # 默认返回值为 bytes 类型，需要进行解码操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "如果只是简单地执行系统命令还不能满足你的需求，可以使用 `subprocess.Popen` 与生成的子进程进行更多交互："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      \"    \\\"p = sb.Popen(['grep', 'communicate'], stdout=sb.PIPE)\\\\n\\\",\\n\",\n",
      "      \"    \\\"# res = p.communicate(sb.check_output('cat ./*'))\\\"\\n\",\n",
      "    \"p = sb.Popen(['grep', 'communicate'], stdin=sb.PIPE, stdout=sb.PIPE)\\n\",\n",
      "    \"res, err = p.communicate(sb.check_output('cat ./*', shell=True))\\n\",\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import subprocess as sb\n",
    "\n",
    "p = sb.Popen(['grep', 'communicate'], stdin=sb.PIPE, stdout=sb.PIPE)\n",
    "res, err = p.communicate(sb.check_output('cat ./*', shell=True))\n",
    "if not err:\n",
    "    print(res.decode())"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}