Skip to content

Commit

Permalink
resolve conflicts
Browse files Browse the repository at this point in the history
  • Loading branch information
unknown authored and unknown committed Sep 10, 2020
2 parents 502308f + e20a5eb commit 564029f
Show file tree
Hide file tree
Showing 13 changed files with 269 additions and 32 deletions.
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,20 @@
[![LICENSE](https://img.shields.io/github/license/sodafoundation/documentation.svg?style=flat-square)](https://github.com/sodafoundation/documentation/blob/master/LICENSE)
[![Netlify Status](https://api.netlify.com/api/v1/badges/906dc6ab-8f72-4d7e-ae9f-93326086589b/deploy-status)](https://app.netlify.com/sites/soda-docs/deploys)

<img src="https://sodafoundation.io/wp-content/uploads/2020/01/SODA_logo_outline_color_800x800.png" width="200" height="200">
<img src="https://sodafoundation.io/wp-content/uploads/2020/01/SODA_logo_outline_color_800x800.png" width="200" height="200">

## Table of Contents

* [Introduction](#introduction)
* [Documentation](#documentation)
* [Quick Start - For Users](#quick-start---to-useexperience)
* [Quick Start - For Developers](#quick-start---to-develop)
* [Latest Releases](#latest-releases)
* [Support and Issues](#support-and-issues)
* [Project Community](#project-community)
* [How to Contribute to this Project](#how-to-contribute-to-this-project)
* [Project Roadmap](#project-roadmap)
* [Join SODA Foundation](#join-soda-foundation)

## Introduction

Expand Down
2 changes: 1 addition & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ defaultContentLanguageInSubdir= true
author = "SODA Authors"
showVisitedLinks = false
disableBreadcrumb = false
disableNextPrev = false
disableNextPrev = true
# Change default color scheme with a variant one. Can be "red", "blue", "green".
themeVariant = "opensds"
disableLanguageSwitchingButton = false
Expand Down
2 changes: 1 addition & 1 deletion content/architecture/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ disableToc: false
---

## SODA Framework
[SODA Foundation](https://sodafoundation.io) is a home for multiple projects for building a unified data framework to support various data and storage solutions. SODA Framework provides a single data framework with infinite possibilites. It brings all the scattered data and storage solutions under single framework ecosystem of projects.
[SODA Foundation](https://sodafoundation.io) is a home for multiple projects for building a unified data framework to support various data and storage solutions. SODA Framework provides a single data framework with infinite possibilites. It brings all the scattered data and storage solutions under a single framework ecosystem of projects.

## SODA Architecture
The high level SODA Architecture and various project constituting this architecture is available at:
Expand Down
66 changes: 66 additions & 0 deletions content/guides/developer-guides/delfin/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: Delfin
description: "Delfin Driver Development Guide"
weight: 10
disableToc: false
tags: ["developer guide", "delfin"]
---

# Driver Development Guide


Delfin is an Infrastructure Management framework under SODA foundation projects for Heterogeneous storage backends (cloud-enabled, offline-storage, etc.) for management, status collection, telemetry and alerting.

## Goals

- Enable third parties to integrate a new backend to SODA Delfin framework
- Collect the steps needed to integrate a new backend to SODA Delfin framework
- Details the interfaces that new backend driver needs to be implemented

## Motivation and background

Delfin is an Infrastructure Management framework developed in Python programming language. It provides a Python plugin interface for adding third party drivers thereby supporting third party backends. The third party driver needs to implement the interfaces defined in the Driver python class, for the framework to use the third party driver. Once drivers implement the interfaces with the details of backend storage, delfin can manage the backend.

## Non-Goals

- Support backend specific implementation details
- Explain internal or higher level framework specific details

## Third party driver integration

SODA Delfin project already contains some [drivers](https://github.com/sodafoundation/delfin/tree/master/delfin/drivers), which can be used as reference by the new third party driver developers.

Existing Delfin Drivers for reference:
- fake_driver - This is a dummy/sample driver used for testing purpose
- Huawei - This driver implements Huawei's OeanStor backend
- VMAX - This driver implements Dell EMC's VMAX storage backend

### Code changes needed

* Add driver plugin 'entry points' to the file 'setup.py'.

* Create driver source code folder under <delfin path>/delfin/drivers/

* Extend base class StorageDriver defined in <delfin path>/delfin/drivers/driver.py, to implement a new driver.

* Implement all the interfaces defined in <delfin path>/delfin/drivers/driver.py, in the new driver.

* Ensure create storages API call from Delfin, can load the driver successfully.

* Ensure APIs of list_*() and alert*() works as expected.

* Raise PR with test reports to Delfin repository.

### How it works

Third party drivers are located and loaded into Delfin, when the create storage API (POST request) is invoked for that backend.

Create Storages POST API contains a request body model namely access_info, which contains fields ‘vendor’ & ‘model’. These fields are used to match and identify the driver from the ‘entry_points’ registered by the driver. Delfin loads this matched driver and registers the driver for the created backend. Any further API calls on this backend will use this newly loaded driver.

Delfin class Driver Manager internally utilizes the Python module ‘stevedore’ for the loading of driver plugins.

## Conclusion

Pluggable design of Driver Manager makes it easy to add third party drivers expanding infrastructure management capabilities of SODA Delfin to multiple storage backends.


91 changes: 79 additions & 12 deletions content/introduction/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,28 +14,95 @@ SODA is SODA Open Data Autonomy. It is an open source unified autonomous data fr
<img src="https://sodafoundation.io/wp-content/uploads/2020/04/sodaautonomy1960-1536x1138.jpg">

## Focus Areas
SODA Foundation focuses to build unified frameworks, APIs and solutions in the areas of Data Mobility, Data Protection, Data Lifecycle, Unified Storage Platform, Cloud Native Storage, Data Governance, Data Orchestration, Data Energy and more. It envisions to provide data autonomy through its open source solutions and standards.

Find more details [here](https://sodafoundation.io/)
SODA Foundation focuses to build unified frameworks, APIs and solutions in the areas of

- Data Mobility
- Data Protection
- Data Lifecycle
- Unified Storage Platform
- Cloud Native Storage
- Data Governance
- Data Orchestration
- Data Energy and more.
It envisions to provide data autonomy through its open source solutions and standards.

More details [here](https://sodafoundation.io/)

## SODA Projects
SODA Foundation is a home of all the projects for data. It hosts many projects and also extends the ecosystem through partners and third party projects which can help to build unified data solutions for various use cases.
SODA Foundation is a home of all the projects for storage and data. It hosts many projects and also extends the ecosystem through partners and third party projects which can help to build unified data solutions for various use cases.


## SODA Architecture
<!--- TODO : Brief description and architecture snapshot needed here...-->
SODA (SODA Open Data Autonomy) Architecture is getting evolved to realize a challenging goal of building a unified framework for data and storage management. It connects the application platforms and solutions to the backend storages seamlessly, be it on prem or cloud through unified API layer. This enables the application platforms to focus to build more valuable use cases rather than worrying about managing the underlying storage backends and data management.

The key architecture tenets are:

- Application Platform agnostic
- Unified API for Data and Storage Management, which are scalable and can evolve
- The overall platform is microservice based, so as to build data solutions for different use cases and technologies
- Future ready-Unified Distributed Data Store
- Seamless vendor agnostic storage backends

As mentioned, SODA Architecture is getting refined and optimized for different application technologies, use cases, platforms and storage backends. If you have any comments and inputs, please join us [here](https://github.com/sodafoundation/).

<img src="https://sodafoundation.io/wp-content/uploads/2020/04/[email protected]">


## SODA Source Code Repository
The Public Source Code repository is located [here](https://github.com/sodafoundation/). Please explore and join the ones that interest you.

The project details can be found:
## SODA Ecosystem
The SODA Ecosystem has many projects under its umbrella, which work in unison to solve the various data and storage challenges.
Some of the important ones are :

- **SODA Dashboard**
SODA Dashboard provides a front end UI which integrates with the different APIs provided by SODA API. This dashboard can be used to test basic SODA functionality.
More details [here](https://sodafoundation.io/projects/soda-dashboard/)

- **SODA API**
The key external interface to platforms, which can do a seamless integration with heterogeneous storage backends. Provides the standardization for Data / Storage Management APIs. More details [here](https://sodafoundation.io/projects/soda-api/)

- **SODA Controller**
In the API flow, controller plays a critical role for all the API flow management and tracking to handle all the state machine and metadata management requirements.
More details [here](https://sodafoundation.io/projects/soda-controller/)

- **SODA Dock**
It is a docking station for heterogeneous storage backends! This is where all the different storage vendors’ drivers for various backends get attached.
More details [here](https://sodafoundation.io/projects/soda-dock/)

- **SODA Plugin**
SODA North-Bound Plugin Project focuses to extend all the industry platforms and application solutions to interface with SODA API or be compliant with it.
More details [here](https://sodafoundation.io/projects/soda-plugins/)

- **SODA Multicloud**
SODA Multicloud project provides a cloud vendor agnostic data management for hybrid cloud, crosscloud or incloud. It can be hosted on prem or cloud native.
More details [here](https://sodafoundation.io/projects/soda-multicloud-dds/)

- **SODA Telemetry**
Provides monitoring via standard interfaces to pull the statistical and health data from the infrastructure (including storage, host, ports) and a highly configurable alert management system
More details [here](https://sodafoundation.io/projects/soda-experiments/)

- **SODA Orchestration**
The Orchestration framework provides flexibility to use existing workflows or define customized workflows to get the simplified execution of tasks.
More details [here](https://sodafoundation.io/projects/soda-experiments/)

- **SODA Intelligence**
It is a reference implementation of intelligent monitoring which can utilize SODA Telemetry, exporter with Kafka (or similar) and a standard ML algorithm.
More details [here](https://sodafoundation.io/projects/soda-experiments/)

- [SODA Project Ecosystem and Brief Information at Project section of SODA Website](https://sodafoundation.io/projects/)
- [SODA Framework Archicture and Project Brief](https://github.com/sodafoundation/design-specs/blob/master/SODAFrameworkAndSODAProjects.md)
- [Project Source Code Repository and Readme](https://github.com/sodafoundation/)

## Official Charter
Please find the official charter for SODA Foundation under Linux Foundation can be found [here](https://sodafoundation.io/the-foundation/charter/)
The official charter for SODA Foundation under Linux Foundation can be found [here](https://sodafoundation.io/the-foundation/charter/)

## Want to be part of this initiative?
**Want to be part of this initiative?**
Please join us to collaborate, research, develop and lead for unified data framework and standards.
Contact us through any of the following channels:
- [SODA Website](https://sodafoundation.io/the-foundation/join/)

- [SODA Website](https://sodafoundation.io/the-foundation/join/)
- [SODA Slack](https://sodafoundation.io/slack)
- [SODA Twitter](https://twitter.com/sodafoundation)
- [SODA Twitter](https://twitter.com/sodafoundation)
- [SODA Mailing List](https://lists.sodafoundation.io)

## More details : [https://sodafoundation.io/](https://sodafoundation.io/)
**For more details, please refer to the [SODA Foundation](https://sodafoundation.io/) website**.
Loading

0 comments on commit 564029f

Please sign in to comment.