[PLANNING] open-graph project planning

[YANG-DB]

Open-Search Graph Planning

General

The purpose of this document is to create a high level plan for a graph engine on-top of the open-search engine which will be used as a store and indexing layer.

This plan will mainly be about relocating & refactoring YangDB from its current open-source location to a new repo location under open-search-projects and adding basic features such as catalog API support and security capabilities.

This plan will quantify the effort needed for this transition and split the work into 4 initial steps which will allow to decouple as much work as possible.

Each step will be either a technical step or a feature step and these steps types will be used in an alternating manner

Setup Stage

Create an open-graph repository which will contain the graph engine capability, question: should the project remain maven driven or move to gradle (currently suggesting to keep as a maven project)

• change WIKI documents and all documentation to reflect open-graph terminology
• change tutorial to reflect usage of open-search
• change license headers to match open-search
• change root pom information to reflect open-search

---

Stage 1

This is a technical debt stage (non-feature oriented step)

The purpose of this step will be moving the project and change package naming includes:

• ASG
  • translator
  • validation
  • strategy

---

• Core
  • client
  • dispatcher
    • asg
    • convertion
    • cursor
    • decorators
    • descriptors
    • driver
    • epb
    • gta
    • interception
    • logging
    • modules
    • ontology
    • profile
    • provision
    • query
    • resource
    • urlSupplier
    • utils
    • validation

---

• Model
  • asgQuery
  • date
  • descriptors
  • plan
  • log
  • logical
  • ontology
  • profile
  • projection
  • query
  • resourceInfo
  • results
  • schema
  • transport
  • validation

---

• Services
  • logging
  • rendering
  • services
    • appRegistrars
    • controllers
    • embedded
    • modules
    • rendering
    • suppliers

---

• data-virtualization
  • virtual-asg
  • virtual-core
  • virtual-epb
  • virtual-gta
  • virtual-stat
  • virtual-unipop

---

• unipop-core
  • common
  • configuration
  • process
  • query
  • schema
  • structure
  • virtual

---

In addition:

• remove domain specific modules and remain with only basic core modules

• verify unit-tests work and enable integration-tests which are non domain-depended

• update all terms in the project's code to reflect open-search

• update all terms in the project's documentation to reflect open-search

This step will result in:

• refactor YangDB into open-graph (naming & terminology)
• maven install results in a list of compiled artifacts
• maven test will pass successfully

---

Stage 2

This is a feature stage (non-technical step)

The purpose of this step will be

• add the catalog domain module (including ontology) to reflect the metadata-catalog issue 685
• Support ontology based DSL (auto generated implementation) API auto-translate to internal query language
• add domain related integration tests in the specific manner custom to open-search

This step will result in - catalog use case :

• add new catalog domain module
• generate (using dedicated table-creation API) indices based on catalog ontology
• Auto generate Client/Server API implementation for DSL
• Call client API to query the catalog registry

---

Stage 3

This is a technical debt stage (non-feature step)

The purpose of this step will be

• reduce to minimal the referential dependencies (keep only mandatory dependencies)
• fixing and expanding the unit-tests and integration test coverage of the modules

This step will result in -

• reduce the project's dependency graph
• validate only eligible license 3rd party libraries are included
• get a real coverage KPI and improve it incrementally each stage

---

Stage 4

This is a feature stage (non-technical step)

The purpose of this step will be to focus on adding security capabilities to open-graph

• add security mechanism for the client API using open-search existing security features
• add Access Level control based on open-search existing access level control features

[YANG-DB]

new repository opened for graph layer