github.com/tiredofit/s3ql

About

This will build a Docker Image for S3QL), A deduplicating, compressing virtual filesystem that works on S3 compatible buckets.

Maintainer

• Dave Conroy

Table of Contents

• About
• Maintainer
• Table of Contents
• Prerequisites and Assumptions
• Installation
  • Build from Source
  • Prebuilt Images
    • Multi Architecture
• Configuration
  • Quick Start
  • Persistent Storage
  • Environment Variables
    • Base Images used
    • Container Options
    • S3QL Options
• Maintenance
  • File System Maintenance
  • Transport endpoint not connected
  • Shell Access
• Support
  • Usage
  • Bugfixes
  • Feature Requests
  • Updates
• License
• References

Prerequisites and Assumptions

• Allow container to have SYS_ADMIN capabilities

Installation

Build from Source

Clone this repository and build the image with docker build -t (imagename) .

Prebuilt Images

Builds of the image are available on Docker Hub

docker pull docker.io/tiredofdit/s3ql:(imagetag)

Builds of the image are also available on the Github Container Registry

docker pull ghcr.io/tiredofit/docker-s3ql:(imagetag)

The following image tags are available along with their tagged release based on what's written in the Changelog:

Container OS	Tag
Alpine	:latest

Multi Architecture

Images are built primarily for amd64 architecture, and may also include builds for arm/v7, arm64 and others. These variants are all unsupported. Consider sponsoring my work so that I can work with various hardware. To see if this image supports multiple architecures, type docker manifest (image):(tag)

Configuration

Quick Start

• The quickest way to get started is using docker-compose. See the examples folder for a working compose.yml that can be modified for development or production use.

• Set various environment variables to understand the capabilities of this image.

• Map persistent storage for access to configuration and data files for backup.

• Set SYS_ADMIN capabilities

Persistent Storage

The following directories are used for configuration and can be mapped for persistent storage.

Directory	Description
/cache/	(Optional) Cache Files
/data/	Mount directory for S3QL Filesysten
/config/	(Optional) Configuration file directory
/logs/	Log files

---

Environment Variables

Base Images used

This image relies on an Alpine Linux base image that relies on an init system for added capabilities. Outgoing SMTP capabilities are handlded via msmtp. Individual container performance monitoring is performed by zabbix-agent. Additional tools include: bash,curl,less,logrotate,nano,vim.

Be sure to view the following repositories to understand all the customizable options:

Image	Description
OS Base	Customized Image based on Alpine Linux

Container Options

Variable	Description	Default
CACHE_PATH	Cache Directory Path	/cache/
CONFIG_FILE	Configuration File with credentials	s3ql.conf
CONFIG_PATH	Configuration Path	/config/
DATA_PATH	Path to mount S3QL File System	/data/
LOG_PATH	Path for Log Files	/logs/
LOG_TYPE	CONSOLE or FILE	FILE
SETUP_MODE	Automatically create configuration files on container startup from environment variables	AUTO

S3QL Options

Variable	Description	Default
CACHE_SIZE	Cache size in KiB - eg 9765625 for 10GB or auto	auto
COMPRESSION	Compresion type none bzip lzma zlib and compression level 0-9	lzma-6
ENABLE_CACHE	Enable Cache on File system	TRUE
ENABLE_PERSISTENT_CACHE	Enable Cache even after filesystem is not mounted	TRUE
FSCK_ARGS	Arguments to pass to fsck process on container start	``
MKFS_ARGS	Arguments to pass to mkfs process when making filesystem	``
MOUNT_ARGS	Arguments to pass to mount process when mounting filesystem	``
S3_KEY_ID	S3 Key ID	``
S3_KEY_SECRET	S3 Key Secret	``
S3_URI	URI of S3 Bucket eg s3c://s3.ca-central-1.wasabisys.com:443/bucket_name	``
S3QL_PASS	(Optional) Encrypted password for S3QL Filesystem

Maintenance

File System Maintenance

• If you would like to perform file system maintenance, first make sure the file system is dismounted by executing service_down 10-s3ql and then execute fsck-now . Make sure to mount the filesystem again by executing service_up 10-s3ql

Transport endpoint not connected

• If at some time you experience the issue of not being able to unmount your filesystem, try entering into the container and executing force-dismount which should allow the filesystem to be dismounted.

Shell Access

For debugging and maintenance purposes you may want access the containers shell.

docker exec -it (whatever your container name is) bash

Support

These images were built to serve a specific need in a production environment and gradually have had more functionality added based on requests from the community.

Usage

• The Discussions board is a great place for working with the community on tips and tricks of using this image.
• Consider sponsoring me for personalized support

Bugfixes

• Please, submit a Bug Report if something isn't working as expected. I'll do my best to issue a fix in short order.

Feature Requests

• Feel free to submit a feature request, however there is no guarantee that it will be added, or at what timeline.
• Consider sponsoring me regarding development of features.

Updates

• Best effort to track upstream changes, More priority if I am actively using the image in a production environment.
• Consider sponsoring me for up to date releases.

License

MIT. See LICENSE for more details.

References

• http://www.rath.org/s3ql-docs