Add debugging messages and configurable verbosity

[jameslamb]

While working through rapidsai/rmm#1502, I've found myself struggling to debug problems of the form "rapids-build-backend isn't doing what I want, but also isn't raising any warnings or errors".

It'd be helpful to be able to turn on debug-level logging and see exactly what the library is doing.

For example, I'd like to be able to use logs to inspect what's happening in here:

rapids-build-backend/rapids_build_backend/impls.py

Lines 180 to 205 in bbe5b13

	for file_key, file_config in parsed_config.files.items():
	if (
	rapids_dependency_file_generator.Output.PYPROJECT
	not in file_config.output
	):
	continue
	pyproject_dir = os.path.join(
	os.path.dirname(config.dependencies_file),
	file_config.pyproject_dir,
	)
	if not (
	os.path.exists(pyproject_dir)
	and os.path.samefile(pyproject_dir, ".")
	):
	continue
	matrix = _parse_matrix(config.matrix_entry) or dict(file_config.matrix)
	if cuda_version is not None:
	matrix["cuda"] = [f"{cuda_version[0]}.{cuda_version[1]}"]
	rapids_dependency_file_generator.make_dependency_files(
	parsed_config=parsed_config,
	file_keys=[file_key],
	output={rapids_dependency_file_generator.Output.PYPROJECT},
	matrix=matrix,
	prepend_channels=[],
	to_stdout=False,
	)

References

Some related conversations:

• Better reporting on output from PEP 517 build backends pypa/pip#6634
• doc: Clarify verbosity when using pip scikit-build/scikit-build-core#513

[vyasr]

One way to do this would be to use the logging module and configure log levels using an environment variable. It's tricky because the build frontend has most of the control of how the backend is invoked, so you really have to use an environment variable or something similar to control the log levels because you can't force any information to be passed via the CLI/API.

[jameslamb]

so you really have to use an environment variable or something similar to control the log levels

rapids-build-backend could expose configuration for logging verbosity that's passed through config-settings from pyproject.toml, just like scikit-build-core does with cmake.verbose.

like this

[tool.rapids-build-backend]
log-level = "DEBUG"

[vyasr]

Yup that would work. RBB supports configuration, and ~all config variables in pyproject.toml may be overridden with config settings or an env var.