Adding New Line and Cell Magics

The datalab package has a number of utilities to help with adding new cell and line magics. In particular, we use argparse for parsing cell magics and have some functions to simplify creating these parsers and dispatching to appropriate handlers.

Note that we monkey-patch the existing Jupyter handlers for line and cell magics to make them more tolerant of '%' vs '%%' errors. This code is in datalab/kernel/__init__.py, in the load_ipython_extension method. We consider it to be a good practice to use just one magic per cell, with no conmingling with other code, in which case the distinction between '%' and '%%' seems unnecessary. If using the datalab package in Jupyter it is worth being aware of this change.

The basic skeleton for adding a new magic is shown below. Assume here that we are adding a line magic %foo with an optional bar argument:

import IPython.core.magic
import datalab.utils.commands


@IPython.core.magic.register_line_magic
def foo(line):
  main_parser = datalab.utils.commands.CommandParser(prog='%foo', description="""
    Do some foo(l)ish stuff here.
  """)
  bar_subparser = main_parser.subcommand('bar', 'Set the bar value.')
  main_parser.set_defaults(func=_foo)
  return datalab.utils.commands.handle_magic_line(line, None, parser)

def _foo(args, cellbody):
  # Handle the args to the %foo magic. args will be a dictionary of argument names and
  # values, and cellbody a string of cell content.

Note that the dispatch to the handler _foo is done indirectly through the handle_magic_line function, which invokes the argument parser and then calls the handler which was set as the default attribute func on the parser. handle_magic_line catches exceptions and prints them out as cell outputs if they occur.

The second argument to handle_magic_line is the cell content; as this is a line magic there is no cell content so we pass None.

It is possible to support $var variable expansion in a magic line. To do this, just change the invocation of handle_magic_line to:

  return datalab.utils.commnds.handle_magic_line(line, None, parser,
      namespace=datalab.utils.commands.notebook_environment())

If this is a cell magic and you want variable expansion in the cell body, that can be done as follows using parse_config. Note that the cell body must be JSON or YAML in this case:

@IPython.core.magic.register_cell_magic
def foo(line, cell):
  main_parser = datalab.utils.commands.CommandParser(prog='%foo', description="""
    Do some foo(l)ish stuff here.
  """)
  bar_subparser = main_parser.subcommand('bar', 'Set the bar value.')
  main_parser.set_defaults(func=_foo)
  config = datalab.utils.commands.parse_config(cell, datalab.utils.commands.notebook_environment())
  return datalab.utils.commands.handle_magic_line(line, config, parser)

def _foo(args, config):
  # Handle the args to the %foo magic. args will be a dictionary of argument names and
  # values, and config a dictionary of names/values.