Difference vim.system and plenary.job

[yochem]

Hi nvim-lua team, I wonder what the difference is between vim.system and plenary.job, mostly in performance.

For context: I'm writing a plugin that writes output of jq to a buffer, using vim.system:

local on_exit = function(result)
  vim.schedule(function ()
    local out = result.code == 0 and result.stdout or result.stderr
    local lines = vim.split(out, "\n", { plain = true })
    vim.api.nvim_buf_set_lines(output_bufnr, 0, -1, false, lines)
  end)
end

local ok, _ = pcall(vim.system, cli_args, { stdin = stdin }, on_exit)

The vim.schedule is there because nvim_buf_set_lines "must not be used in a fast context". However, this still hangs the editor for a while on large json input/output. Do you know if plenary.job could help here?

[delphinus]

vim.system has stdout option to capture chunks from STDOUT pipe gradually.

vim.system(cmd, {
  stdout = function(err, data)
    local chunk = data
    -- ……
  end,
})

This chunk may have unfinished line strings. When the whole output is below,

aaa123456
bbb234567
ccc345678

Each chunk may have such a string.

chunk = "aaa123456\nbb"  -- 1st time call
chunk = "b234567\nccc3"  -- 2nd
chunk = "45678"          -- 3rd
chunk = nil              -- the last

You should carefully deal with such cases, then you can implement logic to add outputs into buffers smoothly.

plenary.job also has a similar feature: on_stdout option. You should also need such logic above to deal with unfinished lines.