Skip to content

Commit

Permalink
snapshotter: use syncfs system call
Browse files Browse the repository at this point in the history
`sync` system call triggers a full page cache sync which may not always
work, especially in kubernetes environment where it is easy to be
interfered by others. I have seen several cases where a broken nfs mount
is blocking kaniko from doing its job.

With `syncfs`, it only writes cache back to disk for the current
filesystem that is used by kaniko which is supposed to be more reliable.
  • Loading branch information
zhouhaibing089 committed Oct 30, 2023
1 parent caedb08 commit 71c6800
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 1 deletion.
17 changes: 16 additions & 1 deletion pkg/snapshot/snapshot.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"runtime"
"sort"
"syscall"

Expand Down Expand Up @@ -155,7 +156,21 @@ func (s *Snapshotter) scanFullFilesystem() ([]string, []string, error) {
// for example the hashing function that determines if files are equal uses the mtime of the files,
// which can lag if sync is not called. Unfortunately there can still be lag if too much data needs
// to be flushed or the disk does its own caching/buffering.
syscall.Sync()
if runtime.GOOS == "linux" {
dir, err := os.Open(s.directory)
if err != nil {
return nil, nil, err
}
defer dir.Close()
// 306 (linux specific) is the number for sys_syncfs
_, _, errno := syscall.Syscall(SYS_SYNCFS, dir.Fd(), 0, 0)
if errno != 0 {
return nil, nil, errno
}
} else {
// fallback to full page cache sync
syscall.Sync()
}

s.l.Snapshot()

Expand Down
9 changes: 9 additions & 0 deletions pkg/snapshot/syncfs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
//go:build !linux
// +build !linux

package snapshot

const (
// don't use it at all
SYS_SYNCFS = uintptr(0)
)
12 changes: 12 additions & 0 deletions pkg/snapshot/syncfs_linux.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
//go:build linux
// +build linux

package snapshot

import (
"golang.org/x/sys/unix"
)

const (
SYS_SYNCFS = unix.SYS_SYNCFS

Check failure on line 11 in pkg/snapshot/syncfs_linux.go

View workflow job for this annotation

GitHub Actions / tests

don't use ALL_CAPS in Go names; use CamelCase (golint)
)

0 comments on commit 71c6800

Please sign in to comment.