Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: replace filepath.Walk with filepath.WalkDir #139108

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

Gofastasf
Copy link
Contributor

Replace filepath.Walk with filepath.WalkDir.

filepath.WalkDir, introduced in Go 1.16, is more performant as it avoids creating unnecessary intermediate os.FileInfo objects. While filepath.Walk calls os.Lstat for every file or directory to retrieve os.FileInfo, filepath.WalkDir provides a fs.DirEntry, which includes file type information without requiring a stat call.

This change reduces unnecessary system calls and aligns with modern Go practices for directory traversal.

@Gofastasf Gofastasf requested review from a team as code owners January 15, 2025 06:56
@Gofastasf Gofastasf requested review from srosenberg, golgeek and kev-cao and removed request for a team January 15, 2025 06:56
Copy link

blathers-crl bot commented Jan 15, 2025

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

Before a member of our team reviews your PR, I have some potential action items for you:

  • Please ensure your git commit message contains a release note.
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@Gofastasf Gofastasf requested review from RaduBerinde and wenyihu6 and removed request for a team January 15, 2025 06:56
@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jan 15, 2025

CLA assistant check
All committers have signed the CLA.

@blathers-crl blathers-crl bot added the O-community Originated from the community label Jan 15, 2025
@Gofastasf Gofastasf requested review from kyle-a-wong, arjunmahishi and aa-joshi and removed request for a team January 15, 2025 06:56
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from 67e928f to 74705b0 Compare January 15, 2025 08:00
Copy link

blathers-crl bot commented Jan 15, 2025

Thank you for updating your pull request.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Thanks!

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aa-joshi, @arjunmahishi, @Gofastasf, @golgeek, @kev-cao, @kyle-a-wong, @srosenberg, and @wenyihu6)


pkg/cli/debug_merge_logs.go line 353 at r1 (raw file):

	var files []fileInfo
	for _, p := range paths {
		// NB: come go1.16, we should use WalkDir here as it is more efficient.

[nit] remove comment

@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from 74705b0 to 454d292 Compare January 15, 2025 14:00
Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @aa-joshi, @arjunmahishi, @Gofastasf, @golgeek, @kev-cao, @kyle-a-wong, @srosenberg, and @wenyihu6)

Copy link

blathers-crl bot commented Jan 15, 2025

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

  • We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
  • Please ensure your git commit message contains a release note.
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from 07f5741 to 52c519f Compare January 15, 2025 20:55
@Gofastasf Gofastasf requested a review from a team as a code owner January 15, 2025 20:55
Copy link

blathers-crl bot commented Jan 16, 2025

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

  • We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
  • Please ensure your git commit message contains a release note.
  • When CI has completed, please ensure no errors have appeared.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@Gofastasf
Copy link
Contributor Author

@RaduBerinde Hey there seems to be some build issue but its from Github CI deprecated so I don't need to care ? Is this okay to get merged ?

This is my first PR :)

@RaduBerinde
Copy link
Member

bors r+

craig bot pushed a commit that referenced this pull request Jan 16, 2025
139108: refactor: replace filepath.Walk with filepath.WalkDir r=RaduBerinde a=Gofastasf

Replace filepath.Walk with filepath.WalkDir.

filepath.WalkDir, introduced in Go 1.16, is more performant as it avoids creating unnecessary intermediate os.FileInfo objects. While filepath.Walk calls os.Lstat for every file or directory to retrieve os.FileInfo, filepath.WalkDir provides a fs.DirEntry, which includes file type information without requiring a stat call.

This change reduces unnecessary system calls and aligns with modern Go practices for directory traversal.

Co-authored-by: Gofastasf <[email protected]>
Co-authored-by: gofastasf <[email protected]>
@craig
Copy link
Contributor

craig bot commented Jan 16, 2025

Build failed:

@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from bca1d3f to e6a0bce Compare January 16, 2025 16:49
@RaduBerinde
Copy link
Member

=== CONT  TestLint/TestCrlfmt
    gen-lint_test.go:1787: 
        diff -u old/cmd/whoownsit/whoownsit.go new/cmd/whoownsit/whoownsit.go
        --- old/cmd/whoownsit/whoownsit.go	2025-01-16 16:24:58.609856355 +0000
        +++ new/cmd/whoownsit/whoownsit.go	2025-01-16 16:24:58.609856355 +0000
        @@ -12,8 +12,8 @@
         import (
         	"flag"
         	"fmt"
        -	"log"
         	"io/fs"
        +	"log"
         	"os"
         	"path/filepath"
         	"strings"
    gen-lint_test.go:1802: run the following to fix your formatting:
        
        bin/crlfmt "-fast" "-ignore" "zcgo*|\\.(pb(\\.gw)?)|(\\.[eo]g)\\.go|/testdata/|^sql/parser/sql\\.go$|(_)?generated(_test)?\\.go$|^sql/pgrepl/pgreplparser/pgrepl\\.go$|^sql/plpgsql/parser/plpgsql\\.go$" "-tab" "2" "-w" "/github-actions-runner/_work/cockroach/cockroach/pkg"
        
        Don't forget to add amend the result to the correct commits.
    --- FAIL: TestLint/TestCrlfmt (6.34s)

Yuo need to apply that patch or run crlfmt (which is our own "stronger" gofmt).

@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from e6a0bce to 39739fd Compare January 16, 2025 17:16
Copy link

blathers-crl bot commented Jan 16, 2025

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

Thank you for updating your pull request.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@RaduBerinde
Copy link
Member

Sorry, I guess the command line is outdated, we don't want to reformat tons of generated code. Just run crlfmt on the files you've changed (we usually set it up in the editor).

filepath.WalkDir, introduced in Go 1.16, is more performant as it avoids creating
unnecessary intermediate os.FileInfo objects. While filepath.Walk calls os.Lstat for every file or directory
to retrieve os.FileInfo, filepath.WalkDir provides a fs.DirEntry, which includes file
type information without requiring a stat call.

This change reduces unnecessary system calls and aligns with modern Go practices
for directory traversal.

Epic: None

Release note (performance improvement): Improved directory traversal performance by switching from filepath.Walk to filepath.WalkDir.

fix: solve the lint error by crlfmt
@Gofastasf Gofastasf force-pushed the refactor/walk-to-walkdir branch from 39739fd to f47addb Compare January 16, 2025 17:59
@Gofastasf
Copy link
Contributor Author

@RaduBerinde Fixed the linting error.

@RaduBerinde
Copy link
Member

Thanks, let's try again.

bors r+

@craig craig bot merged commit 0e13eaa into cockroachdb:master Jan 16, 2025
6 of 7 checks passed
@Gofastasf Gofastasf deleted the refactor/walk-to-walkdir branch January 16, 2025 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
O-community Originated from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants