Document InfluxDB I/O procedures

[matkuliak]

About

Main documentation for CrateDB Toolkit's InfluxDB I/O subsystem, based on influxio.

Preview

• Import data from InfluxDB

References

• https://github.com/crate/tech-content/issues/99

[amotl]

Thanks for adding this article. Apologies that I have quite a few suggestions to reshape it.

1. Please design the tutorial to be self-contained for users approaching it without much ado, similar to what I had suggested on the MQTT tutorial.

   An option could be to use Docker across the board, even for invoking ctk load table, so readers/users will effectively not have to install anything but a container subsystem, because all software will be invoked there, both services and client tools.

2. In the context of the CrateDB Guide, please omit the focus on influxio, but just reference it at the end of the article. If you want to write an article about influxio, it should be a dedicated and separate one, but also not copy its README.

3. As I suggested to remove many fragments from this documentation page, it would be sad if they would be lost. So, I provided guidance over there where those updates should have been submitted to.

   • Documentation: Use Markdown instead of reStructuredText daq-tools/influxio#115

4. The CI job currently fails, because influxdb.md needs to be added to another page's toctree.

   checking consistency... /path/to/docs/integrate/etl/influxdb.md: WARNING: document isn't included in any toctree
   

[amotl]

When aiming to use CrateDB Toolkit, installing influxio separately is not needed. The right installation command would be pip install 'cratedb-toolkit[influxdb]'.

In general, in the main section of the article, I would omit all details about influxio, and only present the corresponding CLI command of CrateDB Toolkit.

At the end of the article, I would link to the documentation of influxio, conveying the message that it offers a few more commands which help in different scenarios like the one you've picked.

[amotl]

Currently, there is no pip install command inside. When aiming for conveying that tutorial "the Docker way", like proposed at #75 (comment), we will probably not need any of those. JFYI.

[amotl]

That command would fail when no InfluxDB service is running.

[amotl]

You can find corresponding commands to start the services using Docker on https://influxio.readthedocs.io/development.html.

[matkuliak]

It may be sensible to add a "start services" section here, which conveys how to run both services using Docker. We can't present it for each and every platform, so Docker just yields the best platform coverage.

Added the Docker variant, but this is only influxio right? That doesn't come with ctk or crash. Which are needed here:

export CRATEDB_SQLALCHEMY_URL=crate://crate@localhost:4200/testdrive/demo
ctk load table influxdb2://example:token123@localhost:8086/testdrive/demo

crash --command "SELECT * FROM testdrive.demo;"

So I should still instruct to install those, yes?

[amotl]

Bonus / Subsequent iteration: CrateDB Toolkit is also available per OCI image.
See also #75 (comment).

[amotl]

I think the OCI image of Toolkit also includes crash out of the box.
See also #75 (comment).

[amotl]

influxio does not come with ctk or crash?

ctk pulls in influxio when installed correspondingly.

pip install 'cratedb-toolkit[influxdb]'

It should be included into the Docker image on behalf of the all extra, no?

• https://github.com/crate-workbench/cratedb-toolkit/blob/main/release/oci/Dockerfile#L22-L24
• https://github.com/crate-workbench/cratedb-toolkit/blob/main/pyproject.toml#L102-L104

$ alias influxio="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest influxio"
$ influxio --version
influxio, version 0.2.0

$ alias crash="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest crash"
$ crash --version
0.31.4

[amotl]

Please also use such aliases to provide optimal UX to readers of the documentation when it comes to invoking tools, and why not just everything. ;]

Those are aliases for all required programs in a nutshell, also alphanumerically sorted:

alias crash="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest crash"
alias ctk="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest ctk"
alias influx="docker run --rm -it --network=host influxdb:2 influx"
alias influxio="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest influxio"

Optionally, some (or just all?) commands would probably also need to use --network=host, like the influx one, to provide maximum conveniency to users.

Feel free to do anything of that on a later iteration, based on your energy and spirits. The current version is more than good enough to have it, and can also be improved later. Don't forget to squash your commits, into a single one with a meaningful message before merging. Thanks!

[matkuliak]

That's good, thank you very much. Running it all on docker would be ideal for sure.
Will continue on this tomorrow and try to wrap it up.

[matkuliak]

Hi again. I think it should be good now. I did it a bit differently than in the docs, although it's still all docker. Looked easier to me this way. What do you think?

[matkuliak]

One thing to mention - On macOS (Apple Silicon) I got docker: no matching manifest for linux/arm64/v8 in the manifest list entries. for both of these:

• https://github.com/crate/cratedb-guide/pull/75/files#diff-a9fd0140e16a6e1b64b4fa6e2ed9c2be66e12e55201944581fd72c30be4e5c29R69
• https://github.com/crate/cratedb-guide/pull/75/files#diff-a9fd0140e16a6e1b64b4fa6e2ed9c2be66e12e55201944581fd72c30be4e5c29R75

[amotl]

Wonderful. Thanks!

[amotl]

Hi. I did not review the HTML preview, apologies. Apparently, there is something wrong on the folding/nesting of elements, see https://cratedb.com/docs/guide/integrate/etl/influxdb.html. Can I ask you to investigate and fix it?

[matkuliak]

I missed that too, thanks for checking. #78 will help

[amotl]

Sorry for eventually merging this too early. It looks like the commits have not been squashed.