Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Doxygen CI check and pre-commit hook #177

Merged
merged 12 commits into from
Jul 12, 2022

Conversation

PointKernel
Copy link
Member

@PointKernel PointKernel commented Jun 19, 2022

This work is mainly taken from rapidsai/cudf#11076.

It adds Doxygen CI check and pre-commit hook. All the existing Doxygen warnings are fixed.

@PointKernel PointKernel added type: docs Improvements or additions to documentation type: feature request New feature request topic: CI CI issue labels Jun 19, 2022
Copy link
Contributor

@karthikeyann karthikeyann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!
ci/checks/style.sh runs only clang-format. If you would like to run doxygen check in ci, style.sh needs to be updated.

Also, is there a plan to publish doxygen documentation in github.io pages?

@PointKernel
Copy link
Member Author

PointKernel commented Jun 20, 2022

If you would like to run doxygen check in ci, style.sh needs to be updated.

I noticed in the cudf style.sh there is no command invoking doxygen.sh thus wonder how doxygen check is triggered in CI.

is there a plan to publish doxygen documentation in github.io pages?

Nothing solid yet but yes, it's on my to-do list. Any comments here?

@PointKernel PointKernel force-pushed the doxygen-pre-commit-hook branch from 24ef0fa to f2d9d90 Compare June 20, 2022 15:08
@karthikeyann
Copy link
Contributor

I noticed in the cudf style.sh there is no command invoking doxygen.sh thus wonder how doxygen check is triggered in CI.

https://github.com/rapidsai/cudf/blob/4bbb65876d1d775a8523e416fddae0d805caefb0/ci/checks/style.sh#L23
cudf runs all pre-commit hooks against all files.
pre-commit run --hook-stage manual --all-files

is there a plan to publish doxygen documentation in github.io pages?

Nothing solid yet but yes, it's on my to-do list. Any comments here?

Depends on when documentation should be updated. I don't see releases or tags in this repo.
How about updating documentation on every push to main branch? (last updated July 16, 2020 ! not update for long time!)

@PointKernel PointKernel marked this pull request as ready for review June 20, 2022 22:14
@PointKernel PointKernel requested a review from jrhemstad June 20, 2022 22:14
@PointKernel PointKernel added the Needs Review Awaiting reviews before merging label Jun 20, 2022
@PointKernel
Copy link
Member Author

cudf runs all pre-commit hooks against all files.

Great, I've updated the cuco script to do the same.

@karthikeyann Can you please re-review? Also, thanks for filing #178 to track the doc automation separately.

Copy link
Contributor

@karthikeyann karthikeyann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix for remaining doxygen warnings as patch.
(I don't have write permissions to this repo, I could not push my commit to this PR)

From 63635ee3f967e82b444376b8eab4a56462ace388 Mon Sep 17 00:00:00 2001
From: Karthikeyan Natarajan <[email protected]>
Date: Tue, 21 Jun 2022 10:47:20 +0530
Subject: [PATCH 1/1] fix some doxygen warnings

---
 include/cuco/detail/probe_sequence_impl.cuh | 27 ++++++++++++++-------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/include/cuco/detail/probe_sequence_impl.cuh b/include/cuco/detail/probe_sequence_impl.cuh
index 6fdb32b..c7ad07b 100644
--- a/include/cuco/detail/probe_sequence_impl.cuh
+++ b/include/cuco/detail/probe_sequence_impl.cuh
@@ -47,7 +47,7 @@ class probe_sequence_base {
   static constexpr uint32_t vector_width() noexcept { return 2u; }
 };

-/*
+/**
  * @brief Base class of probe sequence implementation.
  *
  * Hash map operations are generally memory-bandwidth bound. A vector-load loads two consecutive
@@ -70,14 +70,17 @@ template <typename Key,
           uint32_t CGSize>
 class probe_sequence_impl_base {
  protected:
-  using value_type         = cuco::pair_type<Key, Value>;
-  using key_type           = Key;
-  using mapped_type        = Value;
-  using atomic_key_type    = cuda::atomic<key_type, Scope>;
-  using atomic_mapped_type = cuda::atomic<mapped_type, Scope>;
-  using pair_atomic_type   = cuco::pair_type<atomic_key_type, atomic_mapped_type>;
-  using iterator           = pair_atomic_type*;
-  using const_iterator     = pair_atomic_type const*;
+  using value_type         = cuco::pair_type<Key, Value>;       ///< Type of key/value pairs
+  using key_type           = Key;                               ///< Key type
+  using mapped_type        = Value;                             ///< Type of mapped values
+  using atomic_key_type    = cuda::atomic<key_type, Scope>;     ///< Type of atomic keys
+  using atomic_mapped_type = cuda::atomic<mapped_type, Scope>;  ///< Type of atomic mapped values
+  /// Pair type of atomic key and atomic mapped value
+  using pair_atomic_type = cuco::pair_type<atomic_key_type, atomic_mapped_type>;
+  /// Type of the forward iterator to `pair_atomic_type`
+  using iterator = pair_atomic_type*;
+  /// Type of the forward iterator to `const pair_atomic_type`
+  using const_iterator = pair_atomic_type const*;

   /**
    * @brief Returns the number of elements loaded with each vector-load.
@@ -115,6 +118,8 @@ class probe_sequence_impl_base {
  public:
   /**
    * @brief Returns the capacity of the hash map.
+   *
+   * @return The capacity of the hash map.
    */
   __host__ __device__ __forceinline__ std::size_t get_capacity() const noexcept
   {
@@ -123,11 +128,15 @@ class probe_sequence_impl_base {

   /**
    * @brief Returns slots array.
+   *
+   * @return Slots array
    */
   __device__ __forceinline__ iterator get_slots() noexcept { return slots_; }

   /**
    * @brief Returns slots array.
+   *
+   * @return Slots array
    */
   __device__ __forceinline__ const_iterator get_slots() const noexcept { return slots_; }

--
2.36.1

@PointKernel PointKernel force-pushed the doxygen-pre-commit-hook branch from 56910a3 to 6d3763e Compare June 21, 2022 12:43
Copy link
Contributor

@karthikeyann karthikeyann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Comment on lines +395 to +397
* Hash should be callable with both <tt>std::iterator_traits<InputIt>::value_type</tt> and Key
* type. <tt>std::invoke_result<KeyEqual, std::iterator_traits<InputIt>::value_type, Key></tt>
* must be well-formed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the deal with <tt>? afaik, markdown backticks work in Doxygen.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example warning message of using backticks https://gpuci.gpuopenanalytics.com/blue/organizations/jenkins/nvidia%2Fgpuci%2FcuCollections%2Fprb%2FcuCollections-style/detail/cuCollections-style/147/pipeline#log-176. This happened exclusively to all docs containing the keyword "value_type".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which error in particular should I be looking at?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: explicit link request to 'value_type' could not be resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's what I found:

* `std::iterator_traits<InputIt>`

Works fine but adding ::value_type will fail with explicit link request to 'value_type' could not be resolved:

* `std::iterator_traits<InputIt>::value_type`

I thought the problem is that we didn't provide a link to the implementation of "std::iterator_traits" thus the link request to the code fails. So maybe marking a locally defined function with backticks will pass? Following this idea, I tried the below code but had no luck either:

* `cuco::detail::Murmurhash3_32<key_type>::result_type`

Another thing I tried is turning off AUTOLINK_SUPPORT in the doxygen config file. We can use backticks without getting any warnings by doing that but I think our intention is to keep the auto-link on in Doxygen. Any thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect it's expecting docs to exist for std::iterator_traits<InputIt>::value_type to exist and link to or something.

Copy link
Collaborator

@jrhemstad jrhemstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm excited for this!

I'd like to expand the Doxygen section of the README with an example of what our expectations/conventions are for documentation (we can steal this from libcudf).

The change from__forceinline__ to inline seems orthogonal to this PR.

I also am not a fan of having to use <tt></tt> for code font. Whenever I've checked in the past, markdown ticks work in Doxygen.

@PointKernel PointKernel force-pushed the doxygen-pre-commit-hook branch from 97f51c6 to 447ab36 Compare July 5, 2022 19:13
@PointKernel PointKernel requested a review from jrhemstad July 5, 2022 21:18
@PointKernel PointKernel force-pushed the doxygen-pre-commit-hook branch from b60b7ac to 05fb1db Compare July 12, 2022 18:44
@PointKernel PointKernel merged commit 6aa00d2 into NVIDIA:dev Jul 12, 2022
@PointKernel PointKernel deleted the doxygen-pre-commit-hook branch July 12, 2022 22:41
PointKernel pushed a commit that referenced this pull request Aug 2, 2023
While running `pre-commit`, I got this error:

```bash
./ci/checks/doxygen.sh: line 19: : command not found
./ci/checks/doxygen.sh: line 20: : command not found
./ci/checks/doxygen.sh: line 21: : command not found
```

I have doxygen 1.9.7 installed, which isn't a supported version in this
script. The Doxygen check is supposed to pass silently when run locally
with an unsupported version (it runs properly in CI, instead).

It seems like there's an unrecognized symbol in the bash script from PR
#177:
05fb1db

This PR removes that symbol, fixing the error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review Awaiting reviews before merging topic: CI CI issue type: docs Improvements or additions to documentation type: feature request New feature request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants