Merge branch 'master' into cvmfs

README.md

@@ -1,12 +1,13 @@
-# Remote Snapshotter (with [stargz format introduced by CRFS](https://github.com/google/crfs))
+# Remote Snapshotter
 
-Related discussion of the snapshotter:
+Pulling image is one of the major performance bottlenecks in container workload. Research shows that time for pulling accounts for 76% of container startup time[[FAST '16]](https://www.usenix.org/node/194431). *Remote snapshotter* is a solution discussed in the containerd community and this implementation is based on it.
+
+Related discussion of the snapshotter in containerd community:
 - [Support remote snapshotter to speed up image pulling#3731@containerd](https://github.com/containerd/containerd/issues/3731)
 - [Support `Prepare` for existing snapshots in Snapshotter interface#2968@containerd](https://github.com/containerd/containerd/issues/2968)
 - [remote filesystem snapshotter#2943@containerd](https://github.com/containerd/containerd/issues/2943)
 
-This is an example implementation of a *remote snapshotter* which can be plugged into [patched version of containerd](https://github.com/ktock/containerd/tree/remote-snapshotter).
-By using this snapshotter, any converted but docker-compatible image can be pulled in several seconds even if the images are huge.
+By using this snapshotter, images(even if they are huge) can be pulled in lightning speed because this skips pulling layers but fetches the contents on demand at runtime.
 ```
 # time ctr images pull --plain-http --skip-download --snapshotter=remote registry2:5000/fedora:30 > /dev/null 
 real	0m0.447s
@@ -21,7 +22,10 @@ real	0m1.231s
 user	0m0.112s
 sys	0m0.008s
 ```
-To achive that we are using [stargz format introduced by CRFS](https://github.com/google/crfs), which is compatible with current docker image format.
+To achive that we supports following [filesystems](filesystems):
+- Filesystem using [stargz formatted image introduced by CRFS](https://github.com/google/crfs), which is compatible with current docker image format.
+
+This snapshotter can be plugged into [patched version of containerd](https://github.com/ktock/containerd/tree/filter-by-snapshotter-test) but we are currently working on containerd side to make this work with containerd without patches.
 
 ## demo
 
@@ -48,7 +52,7 @@ Use [stargzify](https://github.com/google/crfs/tree/master/stargz/stargzify) com
 ```
 # stargzify -insecure ubuntu:18.04 http://registry2:5000/ubuntu:18.04
 ```
-The converted image is still __compatible with a normal docker image__ so you can still pull and run it with a normal tools like docker.
+The converted image is still __compatible with a normal docker image__ so you can still pull and run it with normal tools(e.g. docker).
 
 ### Pull the image without downloading layers(it's sometimes called "lazypull") and run it
 ```
@@ -70,7 +74,7 @@ bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  s
 
 ## Authentication
 
-Remote-snapshotter supports private repository authentication powerd by [go-containerregistry](https://github.com/google/go-containerregistry) which supports `~/.docker/config.json`-based credential management.
+We support private repository authentication powerd by [go-containerregistry](https://github.com/google/go-containerregistry) which supports `~/.docker/config.json`-based credential management.
 You can authenticate yourself with normal operations (i.e. `docker login` command) using `~/.docker/config.json`.
 
 In the example showed above, you can pull images from your private repository on the DockerHub:
@@ -80,7 +84,7 @@ In the example showed above, you can pull images from your private repository on
 # ctr image pull --user <username>:<password> --skip-download --snapshotter remote index.docker.io/<your-repository>/ubuntu:18.04
 ```
 The `--user` option is just for containerd's side which doesn't recognize `~/.docker/config.json`.
-Remote-snapshotter doesn't use credentials specified by this option but uses `~/.docker/config.json` instead.
+We doesn't use credentials specified by this option but uses `~/.docker/config.json` instead.
 If you have no right to access the repository with credentials stored in `~/.docker/config.json`, this pull optration fallbacks to the normal one(i.e. overlayfs).
 
 ## Filesystem integration
@@ -91,11 +95,14 @@ Filesystems can be easily integrated with this snapshotter and containerd by imp
 
 ## General issues:
 - [ ] Completing necessary patches on the containerd.
-- [ ] Contributing CRFS to make it more stable.
+  - [x] Implement the protocol on metadata snapshotter: https://github.com/containerd/containerd/pull/3793
+  - [ ] Skip downloading remote snapshot layers: https://github.com/containerd/containerd/pull/3846
+  - [ ] Deal with ErrUnavailable error and try re-pull layers
 
 ## Snapshotter specific issues:
-- [ ] Resiliency:
-  - [ ] Ensure all mounts are available on every Prepare() and report erros when unavailable.
-  - [ ] Deal with runtime problems(NW disconnection, authn failure and so on).
+- [x] Resiliency:
+  - [x] Ensure all mounts are available on every Prepare() and report erros when unavailable.
+  - [x] Deal with runtime problems(NW disconnection, authn failure and so on).
 - [x] Authn: Implement fundamental private repository authentication using `~/.docker/config.json`.
 - [ ] Performance: READ performance improvement
+- [ ] Documentation: ([architecture.md](architecture.md)) is stale.

architecture.md

@@ -1,3 +1,6 @@
+# NOTICE: This doc is stale. See https://github.com/containerd/containerd/issues/3731 for latest discussion.
+We will update it soon.
+
 # Remote Snapshotter Architecture
 
 This is an example implementation of a containerd's *remote snapshotter*.

cache/cache.go

@@ -1,5 +1,3 @@
-// +build linux
-
 /*
    Copyright The containerd Authors.
 
@@ -114,15 +112,11 @@ func (dc *directoryCache) Add(blobHash string, p []byte) {
 			return
 		}
 		defer f.Close()
-		if n, err := f.Write(p); err == nil && n == len(p) {
-			return
-		} else {
+		if n, err := f.Write(p); err != nil || n != len(p) {
 			fmt.Printf("Warning: failed to write cache: %d(wrote)/%d(expected): %v\n",
 				n, len(p), err)
 		}
 	}()
-
-	return
 }
 
 func NewMemoryCache() BlobCache {
@@ -152,8 +146,6 @@ func (mc *memoryCache) Add(blobHash string, p []byte) {
 	mc.mu.Lock()
 	defer mc.mu.Unlock()
 	mc.membuf[blobHash] = string(p)
-
-	return
 }
 
 // nopCache is a cache implementation which doesn't cache anything.
@@ -167,6 +159,4 @@ func (nc *nopCache) Fetch(blobHash string) ([]byte, error) {
 	return nil, fmt.Errorf("Missed cache: %s", blobHash)
 }
 
-func (nc *nopCache) Add(blobHash string, p []byte) {
-	return
-}
+func (nc *nopCache) Add(blobHash string, p []byte) {}

cache/cache_test.go

@@ -1,3 +1,19 @@
+/*
+   Copyright The containerd Authors.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+*/
+
 package cache
 
 import (

demo/containerd/config.sh

@@ -1,15 +1,10 @@
 #!/bin/bash
 
 CACHE_TYPE="${1}"
-SIZE="${2}"
 CONTAINERD_ROOT=/var/lib/containerd
 CONFIG_FILE=/etc/containerd/config.toml
 
 if [ "${CACHE_TYPE}" != "" ] ; then 
     sed -i 's/http_cache_type = .*/http_cache_type = "'"${CACHE_TYPE}"'"/g' "${CONFIG_FILE}"
     sed -i 's/filesystem_cache_type = .*/filesystem_cache_type = "'"${CACHE_TYPE}"'"/g' "${CONFIG_FILE}"
 fi
-
-if [ "${SIZE}" != "" ] ; then 
-    sed -i 's/prefetch_size = .*/prefetch_size = '"${SIZE}"'/g' "${CONFIG_FILE}"
-fi

demo/containerd/config.toml

@@ -7,7 +7,6 @@ plugin_dir = "/opt/containerd/plugins"
   [plugins.stargz]
     insecure = ["127.0.0.1", "localhost", "registry2"]
     cache_chunk_size = 50000
-    prefetch_size = 5000000
     lru_max_entry = 5000
     http_cache_type = "directory"
     filesystem_cache_type = "directory"

demo/containerd/run.sh

@@ -40,4 +40,5 @@ echo "preparing plugins..."
 prepare_plugin
 
 echo "running containerd..."
-containerd --config="${CONFIG}" >> containerd.log &
+
+containerd --config="${CONFIG}" $@ &

filesystems/plugin.go

@@ -1,5 +1,3 @@
-// +build linux
-
 /*
    Copyright The containerd Authors.
 
@@ -19,6 +17,8 @@
 package plugin
 
 import (
+	"context"
+
 	"github.com/containerd/containerd/plugin"
 )
 
@@ -31,9 +31,9 @@ const RemoteFileSystemPlugin plugin.Type = "io.containerd.snapshotter.v1.remote"
 // Mount() tries to mount a remote snapshot to the specified mount point
 // directory. If succeed, the mountpoint directory will be treated as a layer
 // snapshot.
-// Check() is called to check the connectibity of the existing layer snapshot on
-// each Prepare() operation.
+// Check() is called to check the connectibity of the existing layer snapshot
+// every time the layer is used by containerd.
 type FileSystem interface {
-	Mount(ref, digest, mountpoint string) error
-	Check(mountpoint string) error
+	Mount(ctx context.Context, ref, digest, mountpoint string) error
+	Check(ctx context.Context, mountpoint string) error
 }

filesystems/stargz/filereaderat.go

@@ -1,5 +1,3 @@
-// +build linux
-
 /*
    Copyright The containerd Authors.
 
@@ -83,15 +81,14 @@ func (gr *stargzReader) openFile(name string) (io.ReaderAt, error) {
 	}, nil
 }
 
-func (gr *stargzReader) prefetch(layer *io.SectionReader, size int64) (<-chan struct{}, error) {
+func (gr *stargzReader) prefetch(layer *io.SectionReader) (<-chan struct{}, error) {
 	done := make(chan struct{})
-	if e, ok := gr.r.Lookup(prefetchLandmark); ok {
-		size = e.Offset
-	}
-	if size == 0 {
+	e, ok := gr.r.Lookup(prefetchLandmark)
+	if !ok {
 		close(done)
 		return done, nil
 	}
+	size := e.Offset
 
 	// Prefetch specified range at once
 	raw := make([]byte, size)
@@ -132,7 +129,6 @@ func (gr *stargzReader) prefetch(layer *io.SectionReader, size int64) (<-chan st
 				nr += ce.ChunkSize
 			}
 		}
-		return
 	}()
 
 	return done, nil