Feature: init CacheRuntime and CacheRuntimeClass api #4478
Conversation
Signed-off-by: jiuyu <[email protected]>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Skipping CI for Draft Pull Request. |
|
1. Background
Fluid is committed to providing Kubernetes users with a simple and efficient solution for cloud data access and cache acceleration. To achieve this, Fluid currently extends compatibility with various distributed data engines in an in-tree manner, including Alluxio, Juicefs, Vineyard, and JindoFS. This enables users to leverage these caching systems to establish seamless data access links with underlying heterogeneous storage systems within the Kubernetes cluster, while also achieving data access cache acceleration.
As Fluid continues to develop, more and more cache engines, in addition to those already integrated, hope to be connected to Fluid in the future to meet more users' requirements for data cache control and lifecycle management. Examples include Dragonfly and cubeFS. However, the existing runtime development integration model requires cache engine developers to have a substantial foundation in Kubernetes development and an in-depth understanding of the full lifecycle management of Fluid Runtime. This undoubtedly increases the difficulty and time costs of integration for developers outside the Kubernetes domain. Therefore, Fluid aims to simplify the development integration model of data cache engines in a cloud-native environment, achieve better division of responsibilities, and allow developers to focus on their areas of expertise as much as possible.
2. The goal of this PR
The Fluid community plans to introduce a common CacheRuntime mechanism to keep the complexity of interaction with Kubernetes and lifecycle management within the community.
In this way, cache engine developers only need to focus on system development and a small amount of adaptation work to connect the engine to Kubernetes, and is compatible with the data manipulation capability of the Fluid. This greatly reduces the access complexity and optimizes the user experience in Kubernetes.
3. Key concepts
There are two new CRD CacheRuntime and CacheRuntimeClass will be introduced, Users can create various types of caching systems through a CacheRuntime CRD by specifying a CacheRuntimeClass, which selects a cache system deployment template provided by the vendor. Fluid constructs the corresponding cache system based on the cache system configuration in CacheRuntime and the RuntimeClass deployment template used. It then integrates, accesses, and caches data according to the data source definitions in the user’s Dataset.