From bb31f19671e4cd394239fed6676c4c8491e4e072 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Sun, 7 Jun 2020 23:08:51 +0200
Subject: [PATCH 01/20] feat swift: use ncs/swift and support large files for
 OpenStack Swift

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 CHANGELOG.md                     |   3 +
 docs/storage.md                  |  18 +-
 go.mod                           |   1 +
 pkg/objstore/swift/swift.go      | 463 +++++++++++++++++--------------
 pkg/objstore/swift/swift_test.go |  17 --
 scripts/cfggen/main.go           |   2 +-
 6 files changed, 278 insertions(+), 226 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index fe5777067f..653ad73afe 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -196,6 +196,9 @@ sse_config:
 - [#2739](https://github.com/thanos-io/thanos/pull/2739) Changed `bucket tool bucket verify` `--id-whitelist` flag to `--id`.
 - [#2748](https://github.com/thanos-io/thanos/pull/2748) Upgrade Prometheus to [@66dfb951c4ca](https://github.com/prometheus/prometheus/commit/66dfb951c4ca2c1dd3f266172a48a925403b13a5) which is after v2.19.0.
   - PromQL now allow us to executed concurrent selects.
+- [#TBA](https://github.com/thanos-io/thanos/pull/TBA) Swift: Switched to a new library [ncw/swift]() providing large objects support.
+   By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
+   To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
 ### Added
 
diff --git a/docs/storage.md b/docs/storage.md
index bbc0bbde67..2827337d6b 100644
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -50,7 +50,7 @@ Current object storage client implementations:
 | [Google Cloud Storage](./storage.md#gcs)                                                           | Stable             | Production Usage      | yes               | @bwplotka               |
 | [AWS/S3](./storage.md#s3) (and all S3-compatible storages e.g disk-based [Minio](https://min.io/)) | Stable             | Production Usage      | yes               | @bwplotka               |
 | [Azure Storage Account](./storage.md#azure)                                                        | Stable             | Production Usage      | no                | @vglafirov              |
-| [OpenStack Swift](./storage.md#openstack-swift)                                                    | Beta (working PoC) | Production Usage      | yes               | @sudhi-vm               |
+| [OpenStack Swift](./storage.md#openstack-swift)                                                    | Beta (working PoC) | Production Usage      | yes               | @FUSAKLA                |
 | [Tencent COS](./storage.md#tencent-cos)                                                            | Beta               | Production Usage      | no                | @jojohappy              |
 | [AliYun OSS](./storage.md#aliyun-oss)                                                              | Beta               | Production Usage      | no                | @shaulboozhiao,@wujinhu |
 | [Local Filesystem](./storage.md#filesystem)                                                        | Stable             | Testing and Demo only | yes               | @bwplotka               |
@@ -331,16 +331,25 @@ config:
 
 #### OpenStack Swift
 
-Thanos uses [gophercloud](http://gophercloud.io/) client to upload Prometheus data into [OpenStack Swift](https://docs.openstack.org/swift/latest/).
+Thanos uses [ncw/swift](https://github.com/ncw/swift) client to upload Prometheus data into [OpenStack Swift](https://docs.openstack.org/swift/latest/).
 
 Below is an example configuration file for thanos to use OpenStack swift container as an object store.
 Note that if the `name` of a user, project or tenant is used one must also specify its domain by ID or name.
 Various examples for OpenStack authentication can be found in the [official documentation](https://developer.openstack.org/api-ref/identity/v3/index.html?expanded=password-authentication-with-scoped-authorization-detail#password-authentication-with-unscoped-authorization).
 
+By default, OpenStack Swift has a limit for maximum file size of 5 GiB. Thanos index files are often larger than that.
+To resolve this issue, Thanos uses [Static Large Objects (SLO)](https://docs.openstack.org/swift/latest/overview_large_objects.html)
+which are uploaded as segments. These are by default put into the `segments` directory of the same container.
+Default limit for using SLO is 1 GiB which is also maximal size of the segment.
+If you don't want to use the same container for the segments
+(best practise is to use `<container_name>_segments` to avoid polluting listing of the container objects)
+you can use the `large_file_segments_container_name` option to override the default and put the segments to other container.
+
 [embedmd]:# (flags/config_bucket_swift.txt yaml)
 ```yaml
 type: SWIFT
 config:
+  auth_version: 0
   auth_url: ""
   username: ""
   user_domain_name: ""
@@ -355,6 +364,11 @@ config:
   project_domain_name: ""
   region_name: ""
   container_name: ""
+  large_object_chunk_size: 1073741824
+  large_object_segments_container_name: ""
+  retries: 3
+  connect_timeout: 10s
+  timeout: 5m
 ```
 
 #### Tencent COS
diff --git a/go.mod b/go.mod
index 08358b0590..25b930d8cd 100644
--- a/go.mod
+++ b/go.mod
@@ -37,6 +37,7 @@ require (
 	github.com/minio/minio-go/v7 v7.0.2
 	github.com/mozillazg/go-cos v0.13.0
 	github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
+	github.com/ncw/swift v1.0.52
 	github.com/oklog/run v1.1.0
 	github.com/oklog/ulid v1.3.1
 	github.com/olekukonko/tablewriter v0.0.2
diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 6214579c68..0e91163663 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -8,16 +8,19 @@ import (
 	"context"
 	"fmt"
 	"io"
+	"io/ioutil"
 	"os"
+	"strconv"
 	"strings"
 	"testing"
+	"time"
+
+	"github.com/go-kit/kit/log/level"
+
+	"github.com/thanos-io/thanos/pkg/runutil"
 
 	"github.com/go-kit/kit/log"
-	"github.com/gophercloud/gophercloud"
-	"github.com/gophercloud/gophercloud/openstack"
-	"github.com/gophercloud/gophercloud/openstack/objectstorage/v1/containers"
-	"github.com/gophercloud/gophercloud/openstack/objectstorage/v1/objects"
-	"github.com/gophercloud/gophercloud/pagination"
+	"github.com/ncw/swift"
 	"github.com/pkg/errors"
 	"gopkg.in/yaml.v2"
 
@@ -25,54 +28,174 @@ import (
 )
 
 // DirDelim is the delimiter used to model a directory structure in an object store bucket.
-const DirDelim = "/"
-
-type SwiftConfig struct {
-	AuthUrl           string `yaml:"auth_url"`
-	Username          string `yaml:"username"`
-	UserDomainName    string `yaml:"user_domain_name"`
-	UserDomainID      string `yaml:"user_domain_id"`
-	UserId            string `yaml:"user_id"`
-	Password          string `yaml:"password"`
-	DomainId          string `yaml:"domain_id"`
-	DomainName        string `yaml:"domain_name"`
-	ProjectID         string `yaml:"project_id"`
-	ProjectName       string `yaml:"project_name"`
-	ProjectDomainID   string `yaml:"project_domain_id"`
-	ProjectDomainName string `yaml:"project_domain_name"`
-	RegionName        string `yaml:"region_name"`
-	ContainerName     string `yaml:"container_name"`
+const DirDelim = '/'
+
+var DefaultConfig = Config{
+	ChunkSize:      1024 * 1024 * 1024,
+	Retries:        3,
+	ConnectTimeout: "10s",
+	Timeout:        "5m",
+}
+
+type Config struct {
+	AuthVersion          int    `yaml:"auth_version"`
+	AuthUrl              string `yaml:"auth_url"`
+	Username             string `yaml:"username"`
+	UserDomainName       string `yaml:"user_domain_name"`
+	UserDomainID         string `yaml:"user_domain_id"`
+	UserId               string `yaml:"user_id"`
+	Password             string `yaml:"password"`
+	DomainId             string `yaml:"domain_id"`
+	DomainName           string `yaml:"domain_name"`
+	ProjectID            string `yaml:"project_id"`
+	ProjectName          string `yaml:"project_name"`
+	ProjectDomainID      string `yaml:"project_domain_id"`
+	ProjectDomainName    string `yaml:"project_domain_name"`
+	RegionName           string `yaml:"region_name"`
+	ContainerName        string `yaml:"container_name"`
+	ChunkSize            int64  `yaml:"large_object_chunk_size"`
+	SegmentContainerName string `yaml:"large_object_segments_container_name"`
+	Retries              int    `yaml:"retries"`
+	ConnectTimeout       string `yaml:"connect_timeout"`
+	Timeout              string `yaml:"timeout"`
+}
+
+func parseConfig(conf []byte) (*Config, error) {
+	sc := DefaultConfig
+	err := yaml.UnmarshalStrict(conf, &sc)
+	return &sc, err
+}
+
+func configFromEnv() (*Config, error) {
+	c := swift.Connection{}
+	if err := c.ApplyEnvironment(); err != nil {
+		return nil, err
+	}
+
+	config := Config{
+		AuthVersion:          c.AuthVersion,
+		AuthUrl:              c.AuthUrl,
+		Password:             c.ApiKey,
+		Username:             c.UserName,
+		UserId:               c.UserId,
+		DomainId:             c.DomainId,
+		DomainName:           c.Domain,
+		ProjectID:            c.TenantId,
+		ProjectName:          c.Tenant,
+		ProjectDomainID:      c.TenantDomainId,
+		ProjectDomainName:    c.TenantDomain,
+		RegionName:           c.Region,
+		ContainerName:        os.Getenv("OS_CONTAINER_NAME"),
+		SegmentContainerName: os.Getenv("SWIFT_SEGMENTS_CONTAINER_NAME"),
+		Retries:              c.Retries,
+		ConnectTimeout:       c.ConnectTimeout.String(),
+		Timeout:              c.Timeout.String(),
+	}
+	if os.Getenv("SWIFT_CHUNK_SIZE") != "" {
+		var err error
+		config.ChunkSize, err = strconv.ParseInt(os.Getenv("SWIFT_CHUNK_SIZE"), 10, 64)
+		if err != nil {
+			return nil, errors.Wrap(err, "swift parsing chunk size")
+		}
+	}
+	return &config, nil
+}
+
+func connectionFromConfig(sc *Config) (*swift.Connection, error) {
+	connection := swift.Connection{
+		Domain:         sc.DomainName,
+		DomainId:       sc.DomainId,
+		UserName:       sc.Username,
+		UserId:         sc.UserId,
+		ApiKey:         sc.Password,
+		AuthUrl:        sc.AuthUrl,
+		Retries:        sc.Retries,
+		Region:         sc.RegionName,
+		AuthVersion:    sc.AuthVersion,
+		Tenant:         sc.ProjectName,
+		TenantId:       sc.ProjectID,
+		TenantDomain:   sc.ProjectDomainName,
+		TenantDomainId: sc.ProjectDomainID,
+	}
+	if sc.ConnectTimeout != "" {
+		connectTimeout, err := time.ParseDuration(sc.ConnectTimeout)
+		if err != nil {
+			return nil, errors.Wrap(err, "swift parsing connectionTimeout")
+		}
+		connection.ConnectTimeout = connectTimeout
+	}
+	if sc.Timeout != "" {
+		timeout, err := time.ParseDuration(sc.Timeout)
+		if err != nil {
+			return nil, errors.Wrap(err, "swift parsing timeout")
+		}
+		connection.Timeout = timeout
+	}
+	return &connection, nil
 }
 
 type Container struct {
-	logger log.Logger
-	client *gophercloud.ServiceClient
-	name   string
+	logger            log.Logger
+	name              string
+	connection        *swift.Connection
+	chunkSize         int64
+	segmentsContainer string
 }
 
 func NewContainer(logger log.Logger, conf []byte) (*Container, error) {
 	sc, err := parseConfig(conf)
 	if err != nil {
-		return nil, err
+		return nil, errors.Wrap(err, "swift parse configs")
 	}
+	return NewContainerFromConfig(logger, sc, false)
+}
 
-	provider, err := openstack.AuthenticatedClient(authOptsFromConfig(sc))
-	if err != nil {
-		return nil, err
+func ensureContainer(connection *swift.Connection, name string, createIfNotExist bool) (string, error) {
+	if _, _, err := connection.Container(name); err != nil {
+		if err != swift.ContainerNotFound {
+			return "", errors.Wrapf(err, "swift verify container %s", name)
+		}
+		if !createIfNotExist {
+			return "", fmt.Errorf("unable to find the expected container %s", name)
+		}
+		var newContainer swift.Container
+		if err = connection.ContainerCreate(name, swift.Headers{}); err != nil {
+			return "", errors.Wrapf(err, "create container %s", name)
+		}
+		return newContainer.Name, nil
 	}
+	return "", nil
+}
 
-	client, err := openstack.NewObjectStorageV1(provider, gophercloud.EndpointOpts{
-		Region: sc.RegionName,
-	})
+func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool) (*Container, error) {
+	connection, err := connectionFromConfig(sc)
 	if err != nil {
+		return nil, errors.Wrap(err, "swift load config")
+	}
+
+	if err := connection.Authenticate(); err != nil {
+		return nil, errors.Wrap(err, "swift authentication")
+	}
+
+	if _, err := ensureContainer(connection, sc.ContainerName, createContainer); err != nil {
 		return nil, err
 	}
+	if sc.SegmentContainerName == "" {
+		sc.SegmentContainerName = sc.ContainerName
+	} else {
+		if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
+			return nil, err
+		}
+	}
 
-	return &Container{
-		logger: logger,
-		client: client,
-		name:   sc.ContainerName,
-	}, nil
+	container := Container{
+		logger:            logger,
+		name:              sc.ContainerName,
+		connection:        connection,
+		chunkSize:         sc.ChunkSize,
+		segmentsContainer: sc.SegmentContainerName,
+	}
+	return &container, nil
 }
 
 // Name returns the container name for swift.
@@ -83,214 +206,138 @@ func (c *Container) Name() string {
 // Iter calls f for each entry in the given directory. The argument to f is the full
 // object name including the prefix of the inspected directory.
 func (c *Container) Iter(ctx context.Context, dir string, f func(string) error) error {
-	// Ensure the object name actually ends with a dir suffix. Otherwise we'll just iterate the
-	// object itself as one prefix item.
 	if dir != "" {
-		dir = strings.TrimSuffix(dir, DirDelim) + DirDelim
+		dir = strings.TrimSuffix(dir, string(DirDelim)) + string(DirDelim)
 	}
-
-	options := &objects.ListOpts{Full: true, Prefix: dir, Delimiter: DirDelim}
-	return objects.List(c.client, c.name, options).EachPage(func(page pagination.Page) (bool, error) {
-		objectNames, err := objects.ExtractNames(page)
+	return c.connection.ObjectsWalk(c.name, &swift.ObjectsOpts{Prefix: dir, Delimiter: DirDelim}, func(opts *swift.ObjectsOpts) (interface{}, error) {
+		objects, err := c.connection.ObjectNames(c.name, opts)
 		if err != nil {
-			return false, err
+			return objects, errors.Wrap(err, "swift list object names")
 		}
-		for _, objectName := range objectNames {
-			if err := f(objectName); err != nil {
-				return false, err
+		for _, object := range objects {
+			if err := f(object); err != nil {
+				return objects, errors.Wrap(err, "swift iteration over objects")
 			}
 		}
-
-		return true, nil
+		return objects, nil
 	})
 }
 
-// Get returns a reader for the given object name.
-func (c *Container) Get(ctx context.Context, name string) (io.ReadCloser, error) {
+func (c *Container) get(name string, headers swift.Headers, checkHash bool) (io.ReadCloser, error) {
 	if name == "" {
-		return nil, errors.New("error, empty container name passed")
+		return nil, errors.New("Object name cannot be empty")
+	}
+	file, _, err := c.connection.ObjectOpen(c.name, name, checkHash, headers)
+	if err != nil {
+		return nil, errors.Wrap(err, "swift open object")
 	}
-	response := objects.Download(c.client, c.name, name, nil)
-	return response.Body, response.Err
+	return file, err
+}
+
+// Get returns a reader for the given object name.
+func (c *Container) Get(ctx context.Context, name string) (io.ReadCloser, error) {
+	return c.get(name, swift.Headers{}, true)
 }
 
-// GetRange returns a new range reader for the given object name and range.
 func (c *Container) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error) {
-	lowerLimit := ""
-	upperLimit := ""
-	if off >= 0 {
-		lowerLimit = fmt.Sprintf("%d", off)
-	}
-	if length > 0 {
-		upperLimit = fmt.Sprintf("%d", off+length-1)
-	}
-	options := objects.DownloadOpts{
-		Newest: true,
-		Range:  fmt.Sprintf("bytes=%s-%s", lowerLimit, upperLimit),
+	// Set Range HTTP header, see the docs https://docs.openstack.org/api-ref/object-store/?expanded=show-container-details-and-list-objects-detail,get-object-content-and-metadata-detail#id76.
+	bytesRange := fmt.Sprintf("bytes=%d-", off)
+	if length != -1 {
+		bytesRange = fmt.Sprintf("%s%d", bytesRange, off+length-1)
 	}
-	response := objects.Download(c.client, c.name, name, options)
-	return response.Body, response.Err
+	return c.get(name, swift.Headers{"Range": bytesRange}, false)
 }
 
 // Attributes returns information about the specified object.
 func (c *Container) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error) {
-	response := objects.Get(c.client, c.name, name, nil)
-	headers, err := response.Extract()
+	if name == "" {
+		return objstore.ObjectAttributes{}, errors.New("Object name cannot be empty")
+	}
+	info, _, err := c.connection.Object(c.name, name)
 	if err != nil {
-		return objstore.ObjectAttributes{}, err
+		return objstore.ObjectAttributes{}, errors.Wrap(err, "swift get object attributes")
 	}
-
 	return objstore.ObjectAttributes{
-		Size:         headers.ContentLength,
-		LastModified: headers.LastModified,
+		Size:         info.Bytes,
+		LastModified: info.LastModified,
 	}, nil
 }
 
 // Exists checks if the given object exists.
 func (c *Container) Exists(ctx context.Context, name string) (bool, error) {
-	err := objects.Get(c.client, c.name, name, nil).Err
-	if err == nil {
-		return true, nil
-	}
-
-	if _, ok := err.(gophercloud.ErrDefault404); ok {
-		return false, nil
+	_, _, err := c.connection.Object(c.name, name)
+	if err != nil {
+		if c.IsObjNotFoundErr(err) {
+			err = nil
+		}
+		return false, errors.Wrap(err, "swift check if file exists")
 	}
-
-	return false, err
+	return true, nil
 }
 
 // IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.
 func (c *Container) IsObjNotFoundErr(err error) bool {
-	_, ok := err.(gophercloud.ErrDefault404)
-	return ok
+	return errors.Is(err, swift.ObjectNotFound)
 }
 
 // Upload writes the contents of the reader as an object into the container.
 func (c *Container) Upload(ctx context.Context, name string, r io.Reader) error {
-	options := &objects.CreateOpts{Content: r}
-	res := objects.Create(c.client, c.name, name, options)
-	return res.Err
-}
-
-// Delete removes the object with the given name.
-func (c *Container) Delete(ctx context.Context, name string) error {
-	return objects.Delete(c.client, c.name, name, nil).Err
-}
-
-func (*Container) Close() error {
-	// Nothing to close.
-	return nil
-}
-
-func parseConfig(conf []byte) (*SwiftConfig, error) {
-	var sc SwiftConfig
-	err := yaml.UnmarshalStrict(conf, &sc)
-	return &sc, err
-}
-
-func authOptsFromConfig(sc *SwiftConfig) gophercloud.AuthOptions {
-	authOpts := gophercloud.AuthOptions{
-		IdentityEndpoint: sc.AuthUrl,
-		Username:         sc.Username,
-		UserID:           sc.UserId,
-		Password:         sc.Password,
-		DomainID:         sc.DomainId,
-		DomainName:       sc.DomainName,
-		TenantID:         sc.ProjectID,
-		TenantName:       sc.ProjectName,
-
-		// Allow Gophercloud to re-authenticate automatically.
-		AllowReauth: true,
+	size, err := objstore.TryToGetSize(r)
+	if err != nil {
+		level.Warn(c.logger).Log("msg", "could not guess file size, using large object to avoid issues if the file is larger than limit", "name", name, "err", err)
+		// Anything higher or equal to chunk size so the SLO is used.
+		size = c.chunkSize
 	}
-
-	// Support for cross-domain scoping (user in different domain than project).
-	// If a userDomainName or userDomainID is given, the user is scoped to this domain.
-	switch {
-	case sc.UserDomainName != "":
-		authOpts.DomainName = sc.UserDomainName
-	case sc.UserDomainID != "":
-		authOpts.DomainID = sc.UserDomainID
+	var file io.WriteCloser
+	if size >= c.chunkSize {
+		file, err = c.connection.StaticLargeObjectCreateFile(&swift.LargeObjectOpts{
+			Container:        c.name,
+			ObjectName:       name,
+			ChunkSize:        c.chunkSize,
+			SegmentContainer: c.segmentsContainer,
+			CheckHash:        true,
+		})
+	} else {
+		file, err = c.connection.ObjectCreate(c.name, name, true, "", "", swift.Headers{})
 	}
-
-	// A token can be scoped to a domain or project.
-	// The project can be in another domain than the user, which is indicated by setting either projectDomainName or projectDomainID.
-	switch {
-	case sc.ProjectDomainName != "":
-		authOpts.Scope = &gophercloud.AuthScope{
-			DomainName: sc.ProjectDomainName,
+	if err != nil {
+		return errors.Wrap(err, "swift failed to create file")
+	}
+	defer runutil.CloseWithLogOnErr(c.logger, file, "swift upload obj close")
+	switch f := file.(type) {
+	case io.ReaderFrom:
+		if _, err := f.ReadFrom(r); err != nil {
+			return errors.Wrap(err, "swift failed to write uploaded file")
 		}
-	case sc.ProjectDomainID != "":
-		authOpts.Scope = &gophercloud.AuthScope{
-			DomainID: sc.ProjectDomainID,
+	default:
+		buf, err := ioutil.ReadAll(r)
+		if err != nil {
+			return errors.Wrap(err, "swift failed to read uploaded file")
 		}
-	}
-	if authOpts.Scope != nil {
-		switch {
-		case sc.ProjectName != "":
-			authOpts.Scope.ProjectName = sc.ProjectName
-		case sc.ProjectID != "":
-			authOpts.Scope.ProjectID = sc.ProjectID
+		if _, err = file.Write(buf); err != nil {
+			return errors.Wrap(err, "swift failed to write uploaded file")
 		}
 	}
-	return authOpts
-}
-
-func (c *Container) createContainer(name string) error {
-	return containers.Create(c.client, name, nil).Err
-}
-
-func (c *Container) deleteContainer(name string) error {
-	return containers.Delete(c.client, name).Err
+	return nil
 }
 
-func configFromEnv() SwiftConfig {
-	c := SwiftConfig{
-		AuthUrl:           os.Getenv("OS_AUTH_URL"),
-		Username:          os.Getenv("OS_USERNAME"),
-		Password:          os.Getenv("OS_PASSWORD"),
-		RegionName:        os.Getenv("OS_REGION_NAME"),
-		ContainerName:     os.Getenv("OS_CONTAINER_NAME"),
-		ProjectID:         os.Getenv("OS_PROJECT_ID"),
-		ProjectName:       os.Getenv("OS_PROJECT_NAME"),
-		UserDomainID:      os.Getenv("OS_USER_DOMAIN_ID"),
-		UserDomainName:    os.Getenv("OS_USER_DOMAIN_NAME"),
-		ProjectDomainID:   os.Getenv("OS_PROJECT_DOMAIN_ID"),
-		ProjectDomainName: os.Getenv("OS_PROJECT_DOMAIN_NAME"),
-	}
-
-	return c
+// Delete removes the object with the given name.
+func (c *Container) Delete(ctx context.Context, name string) error {
+	return errors.Wrap(c.connection.LargeObjectDelete(c.name, name), "swift delete object")
 }
 
-// validateForTests checks to see the config options for tests are set.
-func validateForTests(conf SwiftConfig) error {
-	if conf.AuthUrl == "" ||
-		conf.Username == "" ||
-		conf.Password == "" ||
-		(conf.ProjectName == "" && conf.ProjectID == "") ||
-		conf.RegionName == "" {
-		return errors.New("insufficient swift test configuration information")
-	}
+func (*Container) Close() error {
+	// Nothing to close.
 	return nil
 }
 
 // NewTestContainer creates test objStore client that before returning creates temporary container.
 // In a close function it empties and deletes the container.
 func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
-	config := configFromEnv()
-	if err := validateForTests(config); err != nil {
-		return nil, nil, err
-	}
-	containerConfig, err := yaml.Marshal(config)
-	if err != nil {
-		return nil, nil, err
-	}
-
-	c, err := NewContainer(log.NewNopLogger(), containerConfig)
+	config, err := configFromEnv()
 	if err != nil {
-		return nil, nil, err
+		return nil, nil, errors.Wrap(err, "swift loading config from ENV")
 	}
-
 	if config.ContainerName != "" {
 		if os.Getenv("THANOS_ALLOW_EXISTING_BUCKET_USE") == "" {
 			return nil, nil, errors.New("OS_CONTAINER_NAME is defined. Normally this tests will create temporary container " +
@@ -299,30 +346,34 @@ func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
 				"needs to be manually cleared. This means that it is only useful to run one test in a time. This is due " +
 				"to safety (accidentally pointing prod container for test) as well as swift not being fully strong consistent.")
 		}
-
+		c, err := NewContainerFromConfig(log.NewNopLogger(), config, false)
+		if err != nil {
+			return nil, nil, errors.Wrap(err, "swift initializing new container")
+		}
 		if err := c.Iter(context.Background(), "", func(f string) error {
-			return errors.Errorf("container %s is not empty", config.ContainerName)
+			return errors.Errorf("container %s is not empty", c.Name())
 		}); err != nil {
-			return nil, nil, errors.Wrapf(err, "swift check container %s", config.ContainerName)
+			return nil, nil, errors.Wrapf(err, "swift check container %s", c.Name())
 		}
-
-		t.Log("WARNING. Reusing", config.ContainerName, "container for Swift tests. Manual cleanup afterwards is required")
+		t.Log("WARNING. Reusing", c.Name(), "container for Swift tests. Manual cleanup afterwards is required")
 		return c, func() {}, nil
 	}
-
-	tmpContainerName := objstore.CreateTemporaryTestBucketName(t)
-
-	if err := c.createContainer(tmpContainerName); err != nil {
-		return nil, nil, err
+	config.ContainerName = objstore.CreateTemporaryTestBucketName(t)
+	config.SegmentContainerName = config.ContainerName
+	c, err := NewContainerFromConfig(log.NewNopLogger(), config, true)
+	if err != nil {
+		return nil, nil, errors.Wrap(err, "swift initializing new container")
 	}
-
-	c.name = tmpContainerName
-	t.Log("created temporary container for swift tests with name", tmpContainerName)
+	t.Log("created temporary container for swift tests with name", c.Name())
 
 	return c, func() {
 		objstore.EmptyBucket(t, context.Background(), c)
-		if err := c.deleteContainer(tmpContainerName); err != nil {
-			t.Logf("deleting container %s failed: %s", tmpContainerName, err)
+		time.Sleep(time.Second)
+		if err := c.connection.ContainerDelete(c.name); err != nil {
+			t.Logf("deleting container %s failed: %s", c.Name(), err)
+		}
+		if err := c.connection.ContainerDelete(c.segmentsContainer); err != nil {
+			t.Logf("deleting segments container %s failed: %s", c.segmentsContainer, err)
 		}
 	}, nil
 }
diff --git a/pkg/objstore/swift/swift_test.go b/pkg/objstore/swift/swift_test.go
index ecbf640abf..c4aac41860 100644
--- a/pkg/objstore/swift/swift_test.go
+++ b/pkg/objstore/swift/swift_test.go
@@ -34,20 +34,3 @@ tenant_name: something`)
 	// Must result in unmarshal error as there's no `tenant_name` in SwiftConfig.
 	testutil.NotOk(t, err)
 }
-
-func TestAuthOptsFromConfig(t *testing.T) {
-	input := &SwiftConfig{
-		AuthUrl:           "http://identity.something.com/v3",
-		Username:          "thanos",
-		UserDomainName:    "userDomain",
-		ProjectName:       "thanosProject",
-		ProjectDomainName: "projectDomain",
-	}
-
-	authOpts := authOptsFromConfig(input)
-	testutil.Equals(t, "http://identity.something.com/v3", authOpts.IdentityEndpoint)
-	testutil.Equals(t, "thanos", authOpts.Username)
-	testutil.Equals(t, "userDomain", authOpts.DomainName)
-	testutil.Equals(t, "projectDomain", authOpts.Scope.DomainName)
-	testutil.Equals(t, "thanosProject", authOpts.Scope.ProjectName)
-}
diff --git a/scripts/cfggen/main.go b/scripts/cfggen/main.go
index dc7b506975..055a0d1c65 100644
--- a/scripts/cfggen/main.go
+++ b/scripts/cfggen/main.go
@@ -43,7 +43,7 @@ var (
 		client.AZURE:      azure.Config{},
 		client.GCS:        gcs.Config{},
 		client.S3:         s3.DefaultConfig,
-		client.SWIFT:      swift.SwiftConfig{},
+		client.SWIFT:      swift.DefaultConfig,
 		client.COS:        cos.Config{},
 		client.ALIYUNOSS:  oss.Config{},
 		client.FILESYSTEM: filesystem.Config{},

From fceec97a2922451b281d63afa5981049335c415f Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Mon, 8 Jun 2020 10:51:22 +0200
Subject: [PATCH 02/20] Update pkg/objstore/swift/swift.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 0e91163663..fd7c4be027 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -182,8 +182,7 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 	}
 	if sc.SegmentContainerName == "" {
 		sc.SegmentContainerName = sc.ContainerName
-	} else {
-		if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
+	} else if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
 			return nil, err
 		}
 	}

From 44122281559c2df0c0774c5ea2d4852c15bf3a93 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Mon, 8 Jun 2020 11:00:56 +0200
Subject: [PATCH 03/20] fix swift: CR minor fixes

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 CHANGELOG.md                |  2 +-
 pkg/objstore/swift/swift.go | 26 +++++++++++++-------------
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 653ad73afe..233fb430d0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -196,7 +196,7 @@ sse_config:
 - [#2739](https://github.com/thanos-io/thanos/pull/2739) Changed `bucket tool bucket verify` `--id-whitelist` flag to `--id`.
 - [#2748](https://github.com/thanos-io/thanos/pull/2748) Upgrade Prometheus to [@66dfb951c4ca](https://github.com/prometheus/prometheus/commit/66dfb951c4ca2c1dd3f266172a48a925403b13a5) which is after v2.19.0.
   - PromQL now allow us to executed concurrent selects.
-- [#TBA](https://github.com/thanos-io/thanos/pull/TBA) Swift: Switched to a new library [ncw/swift]() providing large objects support.
+- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift]() providing large objects support.
    By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
    To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index fd7c4be027..933bea0e2e 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -183,18 +183,18 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 	if sc.SegmentContainerName == "" {
 		sc.SegmentContainerName = sc.ContainerName
 	} else if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
-			return nil, err
-		}
+		return nil, err
 	}
+}
 
-	container := Container{
-		logger:            logger,
-		name:              sc.ContainerName,
-		connection:        connection,
-		chunkSize:         sc.ChunkSize,
-		segmentsContainer: sc.SegmentContainerName,
-	}
-	return &container, nil
+container := Container{
+logger:            logger,
+name:              sc.ContainerName,
+connection:        connection,
+chunkSize:         sc.ChunkSize,
+segmentsContainer: sc.SegmentContainerName,
+}
+return &container, nil
 }
 
 // Name returns the container name for swift.
@@ -265,10 +265,10 @@ func (c *Container) Attributes(ctx context.Context, name string) (objstore.Objec
 // Exists checks if the given object exists.
 func (c *Container) Exists(ctx context.Context, name string) (bool, error) {
 	_, _, err := c.connection.Object(c.name, name)
+	if c.IsObjNotFoundErr(err) {
+		err = nil
+	}
 	if err != nil {
-		if c.IsObjNotFoundErr(err) {
-			err = nil
-		}
 		return false, errors.Wrap(err, "swift check if file exists")
 	}
 	return true, nil

From d95f13bc31d2be423783e172771b8cd5dad1c07e Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Mon, 8 Jun 2020 12:01:34 +0200
Subject: [PATCH 04/20] fix swift: fix broken syntax

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 933bea0e2e..0c223cd440 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -185,16 +185,15 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 	} else if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
 		return nil, err
 	}
-}
 
-container := Container{
-logger:            logger,
-name:              sc.ContainerName,
-connection:        connection,
-chunkSize:         sc.ChunkSize,
-segmentsContainer: sc.SegmentContainerName,
-}
-return &container, nil
+	container := Container{
+		logger:            logger,
+		name:              sc.ContainerName,
+		connection:        connection,
+		chunkSize:         sc.ChunkSize,
+		segmentsContainer: sc.SegmentContainerName,
+	}
+	return &container, nil
 }
 
 // Name returns the container name for swift.

From 9c7c970802cd60e3edf9ccf37d5341f96a9e809a Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Thu, 2 Jul 2020 23:09:09 +0200
Subject: [PATCH 05/20] Update docs/storage.md
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Co-authored-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 docs/storage.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/storage.md b/docs/storage.md
index 2827337d6b..8e90e7461a 100644
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -340,7 +340,7 @@ Various examples for OpenStack authentication can be found in the [official docu
 By default, OpenStack Swift has a limit for maximum file size of 5 GiB. Thanos index files are often larger than that.
 To resolve this issue, Thanos uses [Static Large Objects (SLO)](https://docs.openstack.org/swift/latest/overview_large_objects.html)
 which are uploaded as segments. These are by default put into the `segments` directory of the same container.
-Default limit for using SLO is 1 GiB which is also maximal size of the segment.
+The default limit for using SLO is 1 GiB which is also the maximum size of the segment.
 If you don't want to use the same container for the segments
 (best practise is to use `<container_name>_segments` to avoid polluting listing of the container objects)
 you can use the `large_file_segments_container_name` option to override the default and put the segments to other container.

From f412066701e0facf9dee62532f98e2b1a62f48df Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Thu, 2 Jul 2020 23:21:42 +0200
Subject: [PATCH 06/20] swift: ad default autVersion

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 CHANGELOG.md                | 3 ++-
 pkg/objstore/swift/swift.go | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 233fb430d0..04f661f8a7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -196,10 +196,11 @@ sse_config:
 - [#2739](https://github.com/thanos-io/thanos/pull/2739) Changed `bucket tool bucket verify` `--id-whitelist` flag to `--id`.
 - [#2748](https://github.com/thanos-io/thanos/pull/2748) Upgrade Prometheus to [@66dfb951c4ca](https://github.com/prometheus/prometheus/commit/66dfb951c4ca2c1dd3f266172a48a925403b13a5) which is after v2.19.0.
   - PromQL now allow us to executed concurrent selects.
-- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift]() providing large objects support.
+- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
    By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
    To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
+
 ### Added
 
 - [#2671](https://github.com/thanos-io/thanos/pull/2671) Tools: Bucket replicate now allows passing repeated `--compaction` and `--resolution` flags.
diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 0c223cd440..0af0022c31 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -31,6 +31,7 @@ import (
 const DirDelim = '/'
 
 var DefaultConfig = Config{
+	AuthVersion:    0, // Means autodetect of the auth API version by the library
 	ChunkSize:      1024 * 1024 * 1024,
 	Retries:        3,
 	ConnectTimeout: "10s",

From dda255cfac8d1ea002b4b9a4229726b9a32ab11a Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Thu, 2 Jul 2020 23:25:35 +0200
Subject: [PATCH 07/20] fix: fixed trailing comma of comment

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 0af0022c31..e2ce0d8929 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -31,7 +31,7 @@ import (
 const DirDelim = '/'
 
 var DefaultConfig = Config{
-	AuthVersion:    0, // Means autodetect of the auth API version by the library
+	AuthVersion:    0, // Means autodetect of the auth API version by the library.
 	ChunkSize:      1024 * 1024 * 1024,
 	Retries:        3,
 	ConnectTimeout: "10s",

From 13c3a5d46ba5486393fa4c339b9dd6351d328cb2 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 14 Jul 2020 20:17:16 +0200
Subject: [PATCH 08/20] fix: drop unused return string

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index e2ce0d8929..860d05a37d 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -151,21 +151,20 @@ func NewContainer(logger log.Logger, conf []byte) (*Container, error) {
 	return NewContainerFromConfig(logger, sc, false)
 }
 
-func ensureContainer(connection *swift.Connection, name string, createIfNotExist bool) (string, error) {
+func ensureContainer(connection *swift.Connection, name string, createIfNotExist bool) error {
 	if _, _, err := connection.Container(name); err != nil {
 		if err != swift.ContainerNotFound {
-			return "", errors.Wrapf(err, "swift verify container %s", name)
+			return errors.Wrapf(err, "swift verify container %s", name)
 		}
 		if !createIfNotExist {
-			return "", fmt.Errorf("unable to find the expected container %s", name)
+			return fmt.Errorf("unable to find the expected container %s", name)
 		}
-		var newContainer swift.Container
 		if err = connection.ContainerCreate(name, swift.Headers{}); err != nil {
-			return "", errors.Wrapf(err, "create container %s", name)
+			return errors.Wrapf(err, "create container %s", name)
 		}
-		return newContainer.Name, nil
+		return nil
 	}
-	return "", nil
+	return nil
 }
 
 func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool) (*Container, error) {
@@ -178,12 +177,12 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 		return nil, errors.Wrap(err, "swift authentication")
 	}
 
-	if _, err := ensureContainer(connection, sc.ContainerName, createContainer); err != nil {
+	if err := ensureContainer(connection, sc.ContainerName, createContainer); err != nil {
 		return nil, err
 	}
 	if sc.SegmentContainerName == "" {
 		sc.SegmentContainerName = sc.ContainerName
-	} else if _, err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
+	} else if err := ensureContainer(connection, sc.SegmentContainerName, createContainer); err != nil {
 		return nil, err
 	}
 

From 6cf34f5d99e31047e51b8020d111a3d0de5f0379 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 14 Jul 2020 20:35:25 +0200
Subject: [PATCH 09/20] fix: renme unused params

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 860d05a37d..5d36d2f19e 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -203,7 +203,7 @@ func (c *Container) Name() string {
 
 // Iter calls f for each entry in the given directory. The argument to f is the full
 // object name including the prefix of the inspected directory.
-func (c *Container) Iter(ctx context.Context, dir string, f func(string) error) error {
+func (c *Container) Iter(_ context.Context, dir string, f func(string) error) error {
 	if dir != "" {
 		dir = strings.TrimSuffix(dir, string(DirDelim)) + string(DirDelim)
 	}
@@ -233,11 +233,11 @@ func (c *Container) get(name string, headers swift.Headers, checkHash bool) (io.
 }
 
 // Get returns a reader for the given object name.
-func (c *Container) Get(ctx context.Context, name string) (io.ReadCloser, error) {
+func (c *Container) Get(_ context.Context, name string) (io.ReadCloser, error) {
 	return c.get(name, swift.Headers{}, true)
 }
 
-func (c *Container) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error) {
+func (c *Container) GetRange(_ context.Context, name string, off, length int64) (io.ReadCloser, error) {
 	// Set Range HTTP header, see the docs https://docs.openstack.org/api-ref/object-store/?expanded=show-container-details-and-list-objects-detail,get-object-content-and-metadata-detail#id76.
 	bytesRange := fmt.Sprintf("bytes=%d-", off)
 	if length != -1 {
@@ -247,7 +247,7 @@ func (c *Container) GetRange(ctx context.Context, name string, off, length int64
 }
 
 // Attributes returns information about the specified object.
-func (c *Container) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error) {
+func (c *Container) Attributes(_ context.Context, name string) (objstore.ObjectAttributes, error) {
 	if name == "" {
 		return objstore.ObjectAttributes{}, errors.New("Object name cannot be empty")
 	}
@@ -262,7 +262,7 @@ func (c *Container) Attributes(ctx context.Context, name string) (objstore.Objec
 }
 
 // Exists checks if the given object exists.
-func (c *Container) Exists(ctx context.Context, name string) (bool, error) {
+func (c *Container) Exists(_ context.Context, name string) (bool, error) {
 	_, _, err := c.connection.Object(c.name, name)
 	if c.IsObjNotFoundErr(err) {
 		err = nil
@@ -279,7 +279,7 @@ func (c *Container) IsObjNotFoundErr(err error) bool {
 }
 
 // Upload writes the contents of the reader as an object into the container.
-func (c *Container) Upload(ctx context.Context, name string, r io.Reader) error {
+func (c *Container) Upload(_ context.Context, name string, r io.Reader) error {
 	size, err := objstore.TryToGetSize(r)
 	if err != nil {
 		level.Warn(c.logger).Log("msg", "could not guess file size, using large object to avoid issues if the file is larger than limit", "name", name, "err", err)
@@ -320,7 +320,7 @@ func (c *Container) Upload(ctx context.Context, name string, r io.Reader) error
 }
 
 // Delete removes the object with the given name.
-func (c *Container) Delete(ctx context.Context, name string) error {
+func (c *Container) Delete(_ context.Context, name string) error {
 	return errors.Wrap(c.connection.LargeObjectDelete(c.name, name), "swift delete object")
 }
 

From d1ca3889e0d61be3820e992533ae0bd9267fe44e Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 14 Jul 2020 22:11:03 +0200
Subject: [PATCH 10/20] fix: fixed tests

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 5d36d2f19e..6b3115043a 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -28,7 +28,10 @@ import (
 )
 
 // DirDelim is the delimiter used to model a directory structure in an object store bucket.
-const DirDelim = '/'
+const (
+	DirDelim    = '/'
+	SegmentsDir = "segments/"
+)
 
 var DefaultConfig = Config{
 	AuthVersion:    0, // Means autodetect of the auth API version by the library.
@@ -213,6 +216,9 @@ func (c *Container) Iter(_ context.Context, dir string, f func(string) error) er
 			return objects, errors.Wrap(err, "swift list object names")
 		}
 		for _, object := range objects {
+			if object == SegmentsDir {
+				continue
+			}
 			if err := f(object); err != nil {
 				return objects, errors.Wrap(err, "swift iteration over objects")
 			}
@@ -263,14 +269,13 @@ func (c *Container) Attributes(_ context.Context, name string) (objstore.ObjectA
 
 // Exists checks if the given object exists.
 func (c *Container) Exists(_ context.Context, name string) (bool, error) {
+	found := true
 	_, _, err := c.connection.Object(c.name, name)
 	if c.IsObjNotFoundErr(err) {
 		err = nil
+		found = false
 	}
-	if err != nil {
-		return false, errors.Wrap(err, "swift check if file exists")
-	}
-	return true, nil
+	return found, err
 }
 
 // IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

From 06bbd06103163d562bd91197532b8313cf71c2a9 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Wed, 12 Aug 2020 16:02:01 +0200
Subject: [PATCH 11/20] rebase

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 CHANGELOG.md | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 04f661f8a7..2136a5337e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -68,6 +68,9 @@ We use _breaking :warning:_ to mark changes that are not backward compatible (re
   - `thanos_replicate_origin_meta_loads_total` can be replaced by `blocks_meta_synced{state="loaded"}`.
   - `thanos_replicate_origin_partial_meta_reads_total` can be replaced by `blocks_meta_synced{state="failed"}`.
 - [#3309](https://github.com/thanos-io/thanos/pull/3309) Compact: _breaking :warning:_ Rename metrics to match naming convention. This includes metrics starting with `thanos_compactor` to `thanos_compact`, `thanos_querier` to `thanos_query` and `thanos_ruler` to `thanos_rule`.
+- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
+   By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
+   To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
 ## [v0.16.0](https://github.com/thanos-io/thanos/releases) - 2020.10.26
 
@@ -196,10 +199,6 @@ sse_config:
 - [#2739](https://github.com/thanos-io/thanos/pull/2739) Changed `bucket tool bucket verify` `--id-whitelist` flag to `--id`.
 - [#2748](https://github.com/thanos-io/thanos/pull/2748) Upgrade Prometheus to [@66dfb951c4ca](https://github.com/prometheus/prometheus/commit/66dfb951c4ca2c1dd3f266172a48a925403b13a5) which is after v2.19.0.
   - PromQL now allow us to executed concurrent selects.
-- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
-   By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
-   To change the defaults see [the docs](./docs/storage.md#openstack-swift).
-
 
 ### Added
 

From 5922d18f60e13bcbb506642d6390af7ca982530c Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 8 Sep 2020 00:29:22 +0200
Subject: [PATCH 12/20] cr: minor CR fixes

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 71 ++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 41 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 6b3115043a..d6d935e300 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -15,6 +15,8 @@ import (
 	"testing"
 	"time"
 
+	"github.com/prometheus/common/model"
+
 	"github.com/go-kit/kit/log/level"
 
 	"github.com/thanos-io/thanos/pkg/runutil"
@@ -37,31 +39,31 @@ var DefaultConfig = Config{
 	AuthVersion:    0, // Means autodetect of the auth API version by the library.
 	ChunkSize:      1024 * 1024 * 1024,
 	Retries:        3,
-	ConnectTimeout: "10s",
-	Timeout:        "5m",
+	ConnectTimeout: model.Duration(10 * time.Second),
+	Timeout:        model.Duration(5 * time.Minute),
 }
 
 type Config struct {
-	AuthVersion          int    `yaml:"auth_version"`
-	AuthUrl              string `yaml:"auth_url"`
-	Username             string `yaml:"username"`
-	UserDomainName       string `yaml:"user_domain_name"`
-	UserDomainID         string `yaml:"user_domain_id"`
-	UserId               string `yaml:"user_id"`
-	Password             string `yaml:"password"`
-	DomainId             string `yaml:"domain_id"`
-	DomainName           string `yaml:"domain_name"`
-	ProjectID            string `yaml:"project_id"`
-	ProjectName          string `yaml:"project_name"`
-	ProjectDomainID      string `yaml:"project_domain_id"`
-	ProjectDomainName    string `yaml:"project_domain_name"`
-	RegionName           string `yaml:"region_name"`
-	ContainerName        string `yaml:"container_name"`
-	ChunkSize            int64  `yaml:"large_object_chunk_size"`
-	SegmentContainerName string `yaml:"large_object_segments_container_name"`
-	Retries              int    `yaml:"retries"`
-	ConnectTimeout       string `yaml:"connect_timeout"`
-	Timeout              string `yaml:"timeout"`
+	AuthVersion          int            `yaml:"auth_version"`
+	AuthUrl              string         `yaml:"auth_url"`
+	Username             string         `yaml:"username"`
+	UserDomainName       string         `yaml:"user_domain_name"`
+	UserDomainID         string         `yaml:"user_domain_id"`
+	UserId               string         `yaml:"user_id"`
+	Password             string         `yaml:"password"`
+	DomainId             string         `yaml:"domain_id"`
+	DomainName           string         `yaml:"domain_name"`
+	ProjectID            string         `yaml:"project_id"`
+	ProjectName          string         `yaml:"project_name"`
+	ProjectDomainID      string         `yaml:"project_domain_id"`
+	ProjectDomainName    string         `yaml:"project_domain_name"`
+	RegionName           string         `yaml:"region_name"`
+	ContainerName        string         `yaml:"container_name"`
+	ChunkSize            int64          `yaml:"large_object_chunk_size"`
+	SegmentContainerName string         `yaml:"large_object_segments_container_name"`
+	Retries              int            `yaml:"retries"`
+	ConnectTimeout       model.Duration `yaml:"connect_timeout"`
+	Timeout              model.Duration `yaml:"timeout"`
 }
 
 func parseConfig(conf []byte) (*Config, error) {
@@ -92,8 +94,8 @@ func configFromEnv() (*Config, error) {
 		ContainerName:        os.Getenv("OS_CONTAINER_NAME"),
 		SegmentContainerName: os.Getenv("SWIFT_SEGMENTS_CONTAINER_NAME"),
 		Retries:              c.Retries,
-		ConnectTimeout:       c.ConnectTimeout.String(),
-		Timeout:              c.Timeout.String(),
+		ConnectTimeout:       model.Duration(c.ConnectTimeout),
+		Timeout:              model.Duration(c.Timeout),
 	}
 	if os.Getenv("SWIFT_CHUNK_SIZE") != "" {
 		var err error
@@ -120,20 +122,8 @@ func connectionFromConfig(sc *Config) (*swift.Connection, error) {
 		TenantId:       sc.ProjectID,
 		TenantDomain:   sc.ProjectDomainName,
 		TenantDomainId: sc.ProjectDomainID,
-	}
-	if sc.ConnectTimeout != "" {
-		connectTimeout, err := time.ParseDuration(sc.ConnectTimeout)
-		if err != nil {
-			return nil, errors.Wrap(err, "swift parsing connectionTimeout")
-		}
-		connection.ConnectTimeout = connectTimeout
-	}
-	if sc.Timeout != "" {
-		timeout, err := time.ParseDuration(sc.Timeout)
-		if err != nil {
-			return nil, errors.Wrap(err, "swift parsing timeout")
-		}
-		connection.Timeout = timeout
+		ConnectTimeout: time.Duration(sc.ConnectTimeout),
+		Timeout:        time.Duration(sc.Timeout),
 	}
 	return &connection, nil
 }
@@ -189,14 +179,13 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 		return nil, err
 	}
 
-	container := Container{
+	return &Container{
 		logger:            logger,
 		name:              sc.ContainerName,
 		connection:        connection,
 		chunkSize:         sc.ChunkSize,
 		segmentsContainer: sc.SegmentContainerName,
-	}
-	return &container, nil
+	}, nil
 }
 
 // Name returns the container name for swift.

From a322aa594edc9f40ecc121749f47618d3f995187 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 8 Sep 2020 00:43:39 +0200
Subject: [PATCH 13/20] fix: use io.copy for writing swift uploaded files

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 16 ++--------------
 1 file changed, 2 insertions(+), 14 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index d6d935e300..681a16fe8b 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -8,7 +8,6 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"io/ioutil"
 	"os"
 	"strconv"
 	"strings"
@@ -296,19 +295,8 @@ func (c *Container) Upload(_ context.Context, name string, r io.Reader) error {
 		return errors.Wrap(err, "swift failed to create file")
 	}
 	defer runutil.CloseWithLogOnErr(c.logger, file, "swift upload obj close")
-	switch f := file.(type) {
-	case io.ReaderFrom:
-		if _, err := f.ReadFrom(r); err != nil {
-			return errors.Wrap(err, "swift failed to write uploaded file")
-		}
-	default:
-		buf, err := ioutil.ReadAll(r)
-		if err != nil {
-			return errors.Wrap(err, "swift failed to read uploaded file")
-		}
-		if _, err = file.Write(buf); err != nil {
-			return errors.Wrap(err, "swift failed to write uploaded file")
-		}
+	if _, err := io.Copy(file, r); err != nil {
+		return errors.Wrap(err, "failed to write uploaded file to swift")
 	}
 	return nil
 }

From 09d59cbbba0d77c01c182868df4428915eecfd8f Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Tue, 8 Sep 2020 00:47:53 +0200
Subject: [PATCH 14/20] fix: drop unneeded error returned by swift
 connectionFromConfig

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 681a16fe8b..b50e466d63 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -106,7 +106,7 @@ func configFromEnv() (*Config, error) {
 	return &config, nil
 }
 
-func connectionFromConfig(sc *Config) (*swift.Connection, error) {
+func connectionFromConfig(sc *Config) *swift.Connection {
 	connection := swift.Connection{
 		Domain:         sc.DomainName,
 		DomainId:       sc.DomainId,
@@ -124,7 +124,7 @@ func connectionFromConfig(sc *Config) (*swift.Connection, error) {
 		ConnectTimeout: time.Duration(sc.ConnectTimeout),
 		Timeout:        time.Duration(sc.Timeout),
 	}
-	return &connection, nil
+	return &connection
 }
 
 type Container struct {
@@ -160,11 +160,7 @@ func ensureContainer(connection *swift.Connection, name string, createIfNotExist
 }
 
 func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool) (*Container, error) {
-	connection, err := connectionFromConfig(sc)
-	if err != nil {
-		return nil, errors.Wrap(err, "swift load config")
-	}
-
+	connection := connectionFromConfig(sc)
 	if err := connection.Authenticate(); err != nil {
 		return nil, errors.Wrap(err, "swift authentication")
 	}

From 6876b7cc722226340b470d24ed57479f64d268de Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Wed, 4 Nov 2020 22:21:11 +0100
Subject: [PATCH 15/20] feat swift: add support for DLO

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 docs/storage.md             |   3 +
 pkg/objstore/swift/swift.go | 120 ++++++++++++++++++++----------------
 2 files changed, 71 insertions(+), 52 deletions(-)

diff --git a/docs/storage.md b/docs/storage.md
index 8e90e7461a..7379b51ee5 100644
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -344,6 +344,8 @@ The default limit for using SLO is 1 GiB which is also the maximum size of the
 If you don't want to use the same container for the segments
 (best practise is to use `<container_name>_segments` to avoid polluting listing of the container objects)
 you can use the `large_file_segments_container_name` option to override the default and put the segments to other container.
+_In rare cases you can switch to [Dynamic Large Objects (DLO)](https://docs.openstack.org/swift/latest/overview_large_objects.html)
+by setting the `use_dynamic_large_objects` to true, but use it with caution since it even more relies on eventual consistency._
 
 [embedmd]:# (flags/config_bucket_swift.txt yaml)
 ```yaml
@@ -369,6 +371,7 @@ config:
   retries: 3
   connect_timeout: 10s
   timeout: 5m
+  use_dynamic_large_objects: false
 ```
 
 #### Tencent COS
diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index b50e466d63..90f1e8a58e 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -42,27 +42,31 @@ var DefaultConfig = Config{
 	Timeout:        model.Duration(5 * time.Minute),
 }
 
+// TODO(FUSAKLA): Added to avoid breaking dependency of Cortex which uses the original struct name SwiftConfig.
+type SwiftConfig Config
+
 type Config struct {
-	AuthVersion          int            `yaml:"auth_version"`
-	AuthUrl              string         `yaml:"auth_url"`
-	Username             string         `yaml:"username"`
-	UserDomainName       string         `yaml:"user_domain_name"`
-	UserDomainID         string         `yaml:"user_domain_id"`
-	UserId               string         `yaml:"user_id"`
-	Password             string         `yaml:"password"`
-	DomainId             string         `yaml:"domain_id"`
-	DomainName           string         `yaml:"domain_name"`
-	ProjectID            string         `yaml:"project_id"`
-	ProjectName          string         `yaml:"project_name"`
-	ProjectDomainID      string         `yaml:"project_domain_id"`
-	ProjectDomainName    string         `yaml:"project_domain_name"`
-	RegionName           string         `yaml:"region_name"`
-	ContainerName        string         `yaml:"container_name"`
-	ChunkSize            int64          `yaml:"large_object_chunk_size"`
-	SegmentContainerName string         `yaml:"large_object_segments_container_name"`
-	Retries              int            `yaml:"retries"`
-	ConnectTimeout       model.Duration `yaml:"connect_timeout"`
-	Timeout              model.Duration `yaml:"timeout"`
+	AuthVersion            int            `yaml:"auth_version"`
+	AuthUrl                string         `yaml:"auth_url"`
+	Username               string         `yaml:"username"`
+	UserDomainName         string         `yaml:"user_domain_name"`
+	UserDomainID           string         `yaml:"user_domain_id"`
+	UserId                 string         `yaml:"user_id"`
+	Password               string         `yaml:"password"`
+	DomainId               string         `yaml:"domain_id"`
+	DomainName             string         `yaml:"domain_name"`
+	ProjectID              string         `yaml:"project_id"`
+	ProjectName            string         `yaml:"project_name"`
+	ProjectDomainID        string         `yaml:"project_domain_id"`
+	ProjectDomainName      string         `yaml:"project_domain_name"`
+	RegionName             string         `yaml:"region_name"`
+	ContainerName          string         `yaml:"container_name"`
+	ChunkSize              int64          `yaml:"large_object_chunk_size"`
+	SegmentContainerName   string         `yaml:"large_object_segments_container_name"`
+	Retries                int            `yaml:"retries"`
+	ConnectTimeout         model.Duration `yaml:"connect_timeout"`
+	Timeout                model.Duration `yaml:"timeout"`
+	UseDynamicLargeObjects bool           `yaml:"use_dynamic_large_objects"`
 }
 
 func parseConfig(conf []byte) (*Config, error) {
@@ -78,23 +82,25 @@ func configFromEnv() (*Config, error) {
 	}
 
 	config := Config{
-		AuthVersion:          c.AuthVersion,
-		AuthUrl:              c.AuthUrl,
-		Password:             c.ApiKey,
-		Username:             c.UserName,
-		UserId:               c.UserId,
-		DomainId:             c.DomainId,
-		DomainName:           c.Domain,
-		ProjectID:            c.TenantId,
-		ProjectName:          c.Tenant,
-		ProjectDomainID:      c.TenantDomainId,
-		ProjectDomainName:    c.TenantDomain,
-		RegionName:           c.Region,
-		ContainerName:        os.Getenv("OS_CONTAINER_NAME"),
-		SegmentContainerName: os.Getenv("SWIFT_SEGMENTS_CONTAINER_NAME"),
-		Retries:              c.Retries,
-		ConnectTimeout:       model.Duration(c.ConnectTimeout),
-		Timeout:              model.Duration(c.Timeout),
+		AuthVersion:            c.AuthVersion,
+		AuthUrl:                c.AuthUrl,
+		Password:               c.ApiKey,
+		Username:               c.UserName,
+		UserId:                 c.UserId,
+		DomainId:               c.DomainId,
+		DomainName:             c.Domain,
+		ProjectID:              c.TenantId,
+		ProjectName:            c.Tenant,
+		ProjectDomainID:        c.TenantDomainId,
+		ProjectDomainName:      c.TenantDomain,
+		RegionName:             c.Region,
+		ContainerName:          os.Getenv("OS_CONTAINER_NAME"),
+		ChunkSize:              DefaultConfig.ChunkSize,
+		SegmentContainerName:   os.Getenv("SWIFT_SEGMENTS_CONTAINER_NAME"),
+		Retries:                c.Retries,
+		ConnectTimeout:         model.Duration(c.ConnectTimeout),
+		Timeout:                model.Duration(c.Timeout),
+		UseDynamicLargeObjects: false,
 	}
 	if os.Getenv("SWIFT_CHUNK_SIZE") != "" {
 		var err error
@@ -103,6 +109,9 @@ func configFromEnv() (*Config, error) {
 			return nil, errors.Wrap(err, "swift parsing chunk size")
 		}
 	}
+	if strings.ToLower(os.Getenv("SWIFT_USE_DYNAMIC_LARGE_OBJECTS")) == "true" {
+		config.UseDynamicLargeObjects = true
+	}
 	return &config, nil
 }
 
@@ -128,17 +137,18 @@ func connectionFromConfig(sc *Config) *swift.Connection {
 }
 
 type Container struct {
-	logger            log.Logger
-	name              string
-	connection        *swift.Connection
-	chunkSize         int64
-	segmentsContainer string
+	logger                 log.Logger
+	name                   string
+	connection             *swift.Connection
+	chunkSize              int64
+	useDynamicLargeObjects bool
+	segmentsContainer      string
 }
 
 func NewContainer(logger log.Logger, conf []byte) (*Container, error) {
 	sc, err := parseConfig(conf)
 	if err != nil {
-		return nil, errors.Wrap(err, "swift parse configs")
+		return nil, errors.Wrap(err, "swift parse config")
 	}
 	return NewContainerFromConfig(logger, sc, false)
 }
@@ -175,11 +185,12 @@ func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool)
 	}
 
 	return &Container{
-		logger:            logger,
-		name:              sc.ContainerName,
-		connection:        connection,
-		chunkSize:         sc.ChunkSize,
-		segmentsContainer: sc.SegmentContainerName,
+		logger:                 logger,
+		name:                   sc.ContainerName,
+		connection:             connection,
+		chunkSize:              sc.ChunkSize,
+		useDynamicLargeObjects: sc.UseDynamicLargeObjects,
+		segmentsContainer:      sc.SegmentContainerName,
 	}, nil
 }
 
@@ -213,7 +224,7 @@ func (c *Container) Iter(_ context.Context, dir string, f func(string) error) er
 
 func (c *Container) get(name string, headers swift.Headers, checkHash bool) (io.ReadCloser, error) {
 	if name == "" {
-		return nil, errors.New("Object name cannot be empty")
+		return nil, errors.New("object name cannot be empty")
 	}
 	file, _, err := c.connection.ObjectOpen(c.name, name, checkHash, headers)
 	if err != nil {
@@ -239,7 +250,7 @@ func (c *Container) GetRange(_ context.Context, name string, off, length int64)
 // Attributes returns information about the specified object.
 func (c *Container) Attributes(_ context.Context, name string) (objstore.ObjectAttributes, error) {
 	if name == "" {
-		return objstore.ObjectAttributes{}, errors.New("Object name cannot be empty")
+		return objstore.ObjectAttributes{}, errors.New("object name cannot be empty")
 	}
 	info, _, err := c.connection.Object(c.name, name)
 	if err != nil {
@@ -277,13 +288,18 @@ func (c *Container) Upload(_ context.Context, name string, r io.Reader) error {
 	}
 	var file io.WriteCloser
 	if size >= c.chunkSize {
-		file, err = c.connection.StaticLargeObjectCreateFile(&swift.LargeObjectOpts{
+		opts := swift.LargeObjectOpts{
 			Container:        c.name,
 			ObjectName:       name,
 			ChunkSize:        c.chunkSize,
 			SegmentContainer: c.segmentsContainer,
 			CheckHash:        true,
-		})
+		}
+		if c.useDynamicLargeObjects {
+			file, err = c.connection.DynamicLargeObjectCreateFile(&opts)
+		} else {
+			file, err = c.connection.StaticLargeObjectCreateFile(&opts)
+		}
 	} else {
 		file, err = c.connection.ObjectCreate(c.name, name, true, "", "", swift.Headers{})
 	}

From 87165558ed9781fca545b87e68943749316569ec Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Wed, 4 Nov 2020 23:49:33 +0100
Subject: [PATCH 16/20] fix swift: drop un-needed sleep

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 1 -
 1 file changed, 1 deletion(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 90f1e8a58e..1a9e3591f3 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -360,7 +360,6 @@ func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
 
 	return c, func() {
 		objstore.EmptyBucket(t, context.Background(), c)
-		time.Sleep(time.Second)
 		if err := c.connection.ContainerDelete(c.name); err != nil {
 			t.Logf("deleting container %s failed: %s", c.Name(), err)
 		}

From 33d6af27bbc28a919b45917731b7d190116ed837 Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Thu, 12 Nov 2020 22:22:42 +0100
Subject: [PATCH 17/20] fix swift: removed swift reference from errror wrapping

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 49 +++++++++++++++++--------------------
 1 file changed, 23 insertions(+), 26 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 1a9e3591f3..7fd8f86256 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -14,23 +14,20 @@ import (
 	"testing"
 	"time"
 
-	"github.com/prometheus/common/model"
-
-	"github.com/go-kit/kit/log/level"
-
-	"github.com/thanos-io/thanos/pkg/runutil"
-
 	"github.com/go-kit/kit/log"
+	"github.com/go-kit/kit/log/level"
 	"github.com/ncw/swift"
 	"github.com/pkg/errors"
-	"gopkg.in/yaml.v2"
-
+	"github.com/prometheus/common/model"
 	"github.com/thanos-io/thanos/pkg/objstore"
+	"github.com/thanos-io/thanos/pkg/runutil"
+	"gopkg.in/yaml.v2"
 )
 
-// DirDelim is the delimiter used to model a directory structure in an object store bucket.
 const (
-	DirDelim    = '/'
+	// DirDelim is the delimiter used to model a directory structure in an object store bucket.
+	DirDelim = '/'
+	// Name of the directory in bucket, where to store file parts of SLO and DLO.
 	SegmentsDir = "segments/"
 )
 
@@ -43,7 +40,7 @@ var DefaultConfig = Config{
 }
 
 // TODO(FUSAKLA): Added to avoid breaking dependency of Cortex which uses the original struct name SwiftConfig.
-type SwiftConfig Config
+type SwiftConfig = Config
 
 type Config struct {
 	AuthVersion            int            `yaml:"auth_version"`
@@ -106,7 +103,7 @@ func configFromEnv() (*Config, error) {
 		var err error
 		config.ChunkSize, err = strconv.ParseInt(os.Getenv("SWIFT_CHUNK_SIZE"), 10, 64)
 		if err != nil {
-			return nil, errors.Wrap(err, "swift parsing chunk size")
+			return nil, errors.Wrap(err, "parsing chunk size")
 		}
 	}
 	if strings.ToLower(os.Getenv("SWIFT_USE_DYNAMIC_LARGE_OBJECTS")) == "true" {
@@ -148,7 +145,7 @@ type Container struct {
 func NewContainer(logger log.Logger, conf []byte) (*Container, error) {
 	sc, err := parseConfig(conf)
 	if err != nil {
-		return nil, errors.Wrap(err, "swift parse config")
+		return nil, errors.Wrap(err, "parse config")
 	}
 	return NewContainerFromConfig(logger, sc, false)
 }
@@ -156,7 +153,7 @@ func NewContainer(logger log.Logger, conf []byte) (*Container, error) {
 func ensureContainer(connection *swift.Connection, name string, createIfNotExist bool) error {
 	if _, _, err := connection.Container(name); err != nil {
 		if err != swift.ContainerNotFound {
-			return errors.Wrapf(err, "swift verify container %s", name)
+			return errors.Wrapf(err, "verify container %s", name)
 		}
 		if !createIfNotExist {
 			return fmt.Errorf("unable to find the expected container %s", name)
@@ -172,7 +169,7 @@ func ensureContainer(connection *swift.Connection, name string, createIfNotExist
 func NewContainerFromConfig(logger log.Logger, sc *Config, createContainer bool) (*Container, error) {
 	connection := connectionFromConfig(sc)
 	if err := connection.Authenticate(); err != nil {
-		return nil, errors.Wrap(err, "swift authentication")
+		return nil, errors.Wrap(err, "authentication")
 	}
 
 	if err := ensureContainer(connection, sc.ContainerName, createContainer); err != nil {
@@ -208,14 +205,14 @@ func (c *Container) Iter(_ context.Context, dir string, f func(string) error) er
 	return c.connection.ObjectsWalk(c.name, &swift.ObjectsOpts{Prefix: dir, Delimiter: DirDelim}, func(opts *swift.ObjectsOpts) (interface{}, error) {
 		objects, err := c.connection.ObjectNames(c.name, opts)
 		if err != nil {
-			return objects, errors.Wrap(err, "swift list object names")
+			return objects, errors.Wrap(err, "list object names")
 		}
 		for _, object := range objects {
 			if object == SegmentsDir {
 				continue
 			}
 			if err := f(object); err != nil {
-				return objects, errors.Wrap(err, "swift iteration over objects")
+				return objects, errors.Wrap(err, "iteration over objects")
 			}
 		}
 		return objects, nil
@@ -228,7 +225,7 @@ func (c *Container) get(name string, headers swift.Headers, checkHash bool) (io.
 	}
 	file, _, err := c.connection.ObjectOpen(c.name, name, checkHash, headers)
 	if err != nil {
-		return nil, errors.Wrap(err, "swift open object")
+		return nil, errors.Wrap(err, "open object")
 	}
 	return file, err
 }
@@ -254,7 +251,7 @@ func (c *Container) Attributes(_ context.Context, name string) (objstore.ObjectA
 	}
 	info, _, err := c.connection.Object(c.name, name)
 	if err != nil {
-		return objstore.ObjectAttributes{}, errors.Wrap(err, "swift get object attributes")
+		return objstore.ObjectAttributes{}, errors.Wrap(err, "get object attributes")
 	}
 	return objstore.ObjectAttributes{
 		Size:         info.Bytes,
@@ -306,16 +303,16 @@ func (c *Container) Upload(_ context.Context, name string, r io.Reader) error {
 	if err != nil {
 		return errors.Wrap(err, "swift failed to create file")
 	}
-	defer runutil.CloseWithLogOnErr(c.logger, file, "swift upload obj close")
+	defer runutil.CloseWithLogOnErr(c.logger, file, "upload object close")
 	if _, err := io.Copy(file, r); err != nil {
-		return errors.Wrap(err, "failed to write uploaded file to swift")
+		return errors.Wrap(err, "uploading object")
 	}
 	return nil
 }
 
 // Delete removes the object with the given name.
 func (c *Container) Delete(_ context.Context, name string) error {
-	return errors.Wrap(c.connection.LargeObjectDelete(c.name, name), "swift delete object")
+	return errors.Wrap(c.connection.LargeObjectDelete(c.name, name), "delete object")
 }
 
 func (*Container) Close() error {
@@ -328,7 +325,7 @@ func (*Container) Close() error {
 func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
 	config, err := configFromEnv()
 	if err != nil {
-		return nil, nil, errors.Wrap(err, "swift loading config from ENV")
+		return nil, nil, errors.Wrap(err, "loading config from ENV")
 	}
 	if config.ContainerName != "" {
 		if os.Getenv("THANOS_ALLOW_EXISTING_BUCKET_USE") == "" {
@@ -340,12 +337,12 @@ func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
 		}
 		c, err := NewContainerFromConfig(log.NewNopLogger(), config, false)
 		if err != nil {
-			return nil, nil, errors.Wrap(err, "swift initializing new container")
+			return nil, nil, errors.Wrap(err, "initializing new container")
 		}
 		if err := c.Iter(context.Background(), "", func(f string) error {
 			return errors.Errorf("container %s is not empty", c.Name())
 		}); err != nil {
-			return nil, nil, errors.Wrapf(err, "swift check container %s", c.Name())
+			return nil, nil, errors.Wrapf(err, "check container %s", c.Name())
 		}
 		t.Log("WARNING. Reusing", c.Name(), "container for Swift tests. Manual cleanup afterwards is required")
 		return c, func() {}, nil
@@ -354,7 +351,7 @@ func NewTestContainer(t testing.TB) (objstore.Bucket, func(), error) {
 	config.SegmentContainerName = config.ContainerName
 	c, err := NewContainerFromConfig(log.NewNopLogger(), config, true)
 	if err != nil {
-		return nil, nil, errors.Wrap(err, "swift initializing new container")
+		return nil, nil, errors.Wrap(err, "initializing new container")
 	}
 	t.Log("created temporary container for swift tests with name", c.Name())
 

From 5d8c6bae5368b6f05ce97aab4b015fc89e65ef6a Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Thu, 12 Nov 2020 22:37:48 +0100
Subject: [PATCH 18/20] fix swift: make create file errors more descriptive

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 pkg/objstore/swift/swift.go | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/pkg/objstore/swift/swift.go b/pkg/objstore/swift/swift.go
index 7fd8f86256..9372382923 100644
--- a/pkg/objstore/swift/swift.go
+++ b/pkg/objstore/swift/swift.go
@@ -293,15 +293,18 @@ func (c *Container) Upload(_ context.Context, name string, r io.Reader) error {
 			CheckHash:        true,
 		}
 		if c.useDynamicLargeObjects {
-			file, err = c.connection.DynamicLargeObjectCreateFile(&opts)
+			if file, err = c.connection.DynamicLargeObjectCreateFile(&opts); err != nil {
+				return errors.Wrap(err, "create DLO file")
+			}
 		} else {
-			file, err = c.connection.StaticLargeObjectCreateFile(&opts)
+			if file, err = c.connection.StaticLargeObjectCreateFile(&opts); err != nil {
+				return errors.Wrap(err, "create SLO file")
+			}
 		}
 	} else {
-		file, err = c.connection.ObjectCreate(c.name, name, true, "", "", swift.Headers{})
-	}
-	if err != nil {
-		return errors.Wrap(err, "swift failed to create file")
+		if file, err = c.connection.ObjectCreate(c.name, name, true, "", "", swift.Headers{}); err != nil {
+			return errors.Wrap(err, "create file")
+		}
 	}
 	defer runutil.CloseWithLogOnErr(c.logger, file, "upload object close")
 	if _, err := io.Copy(file, r); err != nil {

From cb577fe0bfc9a209eab0f433600a84dccafdb2eb Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Mon, 23 Nov 2020 16:23:36 +0100
Subject: [PATCH 19/20] fix: fix go deps

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 go.mod | 1 -
 go.sum | 2 ++
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/go.mod b/go.mod
index 25b930d8cd..8d1e9027d7 100644
--- a/go.mod
+++ b/go.mod
@@ -25,7 +25,6 @@ require (
 	github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
 	github.com/golang/snappy v0.0.2
 	github.com/googleapis/gax-go v2.0.2+incompatible
-	github.com/gophercloud/gophercloud v0.14.0
 	github.com/grpc-ecosystem/go-grpc-middleware v1.1.0
 	github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
 	github.com/hashicorp/golang-lru v0.5.4
diff --git a/go.sum b/go.sum
index 8116b49e55..d6c74adbe2 100644
--- a/go.sum
+++ b/go.sum
@@ -893,6 +893,8 @@ github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxzi
 github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
 github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
 github.com/ncw/swift v1.0.50/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
+github.com/ncw/swift v1.0.52 h1:ACF3JufDGgeKp/9mrDgQlEgS8kRYC4XKcuzj/8EJjQU=
+github.com/ncw/swift v1.0.52/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM=
 github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
 github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
 github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=

From 2695266503f21f11e724e56e4be0974578173a2f Mon Sep 17 00:00:00 2001
From: Martin Chodur <m.chodur@seznam.cz>
Date: Fri, 4 Dec 2020 15:01:35 +0100
Subject: [PATCH 20/20] docs: fix changelog

Signed-off-by: Martin Chodur <m.chodur@seznam.cz>
---
 CHANGELOG.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2136a5337e..eb8430e645 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -24,6 +24,9 @@ We use _breaking :warning:_ to mark changes that are not backward compatible (re
 ### Changed
 
 - [#3496](https://github.com/thanos-io/thanos/pull/3496) s3: Respect SignatureV2 flag for all credential providers.
+- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
+   By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
+   To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
 ## [v0.17.0](https://github.com/thanos-io/thanos/releases/tag/v0.17.0) - 2020.11.18
 
@@ -68,9 +71,6 @@ We use _breaking :warning:_ to mark changes that are not backward compatible (re
   - `thanos_replicate_origin_meta_loads_total` can be replaced by `blocks_meta_synced{state="loaded"}`.
   - `thanos_replicate_origin_partial_meta_reads_total` can be replaced by `blocks_meta_synced{state="failed"}`.
 - [#3309](https://github.com/thanos-io/thanos/pull/3309) Compact: _breaking :warning:_ Rename metrics to match naming convention. This includes metrics starting with `thanos_compactor` to `thanos_compact`, `thanos_querier` to `thanos_query` and `thanos_ruler` to `thanos_rule`.
-- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
-   By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
-   To change the defaults see [the docs](./docs/storage.md#openstack-swift).
 
 ## [v0.16.0](https://github.com/thanos-io/thanos/releases) - 2020.10.26