Support mutating row in --convert without returning it

[simonw]

Currently you have to do this:

$ sqlite-utils insert dogs.db dogs dogs.json --convert '
row["is_good"] = 1
return row'

Would be neat if this worked too:

$ sqlite-utils insert dogs.db dogs dogs.json \
  --convert 'row["is_good"] = 1'

[simonw]

Might be a case of modifying this line:

sqlite-utils/sqlite_utils/cli.py

Line 828 in e0c476b

docs = (fn(doc) for doc in docs)

To:

docs = (fn(doc) or doc for doc in docs)

[simonw]

Also need to update relevant docs for that example.

[simonw]

Tried this test:

    result = CliRunner().invoke(
        cli.cli,
        [
            "insert",
            db_path,
            "rows",
            "-",
            "--convert",
            'row["is_chicken"] = True',
        ],
        input='{"name": "Azi"}',
    )

And got this error:

E + where 1 = <Result SyntaxError('invalid syntax', ('<string>', 2, 30, ' return row["is_chicken"] = True\n'))>.exit_code

The code snippet compilation isn't currently compatible with this.

[simonw]

Here's the code in question:

sqlite-utils/sqlite_utils/utils.py

Lines 288 to 299 in b8c1340

	# If user defined a convert() function, return that
	try:
	exec(code, globals, locals)
	return locals["convert"]
	except (AttributeError, SyntaxError, NameError, KeyError, TypeError):
	pass
	# Try compiling their code as a function instead
	# If single line and no 'return', add the return
	if "\n" not in code and not code.strip().startswith("return "):
	code = "return {}".format(code)

[simonw]

This seems to work:

def _compile_code(code, imports, variable="value"):
    locals = {}
    globals = {"r": recipes, "recipes": recipes}
    # If user defined a convert() function, return that
    try:
        exec(code, globals, locals)
        return locals["convert"]
    except (AttributeError, SyntaxError, NameError, KeyError, TypeError):
        pass

    # Try compiling their code as a function instead
    body_variants = [code]
    # If single line and no 'return', try adding the return
    if "\n" not in code and not code.strip().startswith("return "):
        body_variants.insert(0, "return {}".format(code))

    for variant in body_variants:
        new_code = ["def fn({}):".format(variable)]
        for line in variant.split("\n"):
            new_code.append("    {}".format(line))
        try:
            code_o = compile("\n".join(new_code), "<string>", "exec")
            break
        except SyntaxError:
            # Try another variant, e.g. for 'return row["column"] = 1'
            continue

    for import_ in imports:
        globals[import_.split(".")[0]] = __import__(import_)
    exec(code_o, globals, locals)
    return locals["fn"]

[simonw]

The previous code for highlighting errors in syntax (which was already a bit confused thanks to the added return, see #355 (comment) - isn't compatible with this approach at all. I'm going to ditch it and just show a generic Error: Could not compile code message.