Meta: Schema registry

[tillrohrmann]

In order to store discovered service information (e.g. service descriptors) and to make this information accessible to the Restate cluster, we need a service registry that manages this information.

Tasks

Preview Give feedback

1. Design the interfaces
2. Investigate whether Arc<ArcSwap> is the right approach for data sharing (see Investigate whether Arc<ArcSwap> for accessing shuffle senders in network is more efficient than Arc<Mutex> #136)
3. Implement the interfaces
4. Let the meta own the write side of the registry, and pass around the interfaces for the read side of the registry

[slinkydeveloper]

I moved this back to Todo because we have a primitive version of the Schema registry, but we need to reiterate on it to support all the meta functionalities. Also see #227 (comment).

One of the main concerns with the current approach is keeping consistency across different data structures, in particular with #226 in mind. It would be nice to have a single point where we update these data structures, making easy to guarantee consistency when reading metadata.

One idea I have in mind to achieve that is to define interfaces for all the read access patterns every component needs, something like:

// --- Key extractor and key expander

#[cfg(feature = "key_extractor")]
pub trait KeyExtractorResolver {
    fn resolve_key_extractor(
        &self,
        service_name: impl AsRef<str>,
        method_name: impl AsRef<str>
    ) -> Option<MethodKeyExtractor<'_>>;
}

#[cfg(feature = "key_expander")]
pub trait KeyExpanderResolver {
    fn resolve_key_expander(
        &self,
        service_name: impl AsRef<str>,
    ) -> Option<&ServiceKeyExpander>;
}

// --- Json mapper

#[cfg(feature = "json_mapper")]
pub trait JsonMapperResolver {
    fn resolve_json_mapper(
        &self,
        service_name: impl AsRef<str>,
        method_name: impl AsRef<str>
    ) -> Option<(/* request */ JsonMapper, /* response */ JsonMapper)>;
}

// --- Endpoint resolver

#[cfg(feature = "endpoint_meta")]
pub trait EndpointMetadataResolver {
    fn resolve_latest_endpoint(
        &self,
        service_name: impl AsRef<str>,
    ) -> Option<EndpointMetadata>;

    fn resolve_endpoint(
        &self,
        endpoint_id: &EndpointId,
    ) -> Option<EndpointMetadata>;
}

// --- Reflections resolver

#[cfg(feature = "proto_symbol")]
pub trait ProtoSymbolResolver {
    fn list_services(&self) -> &[&str];

    fn get_file_descriptor_by_symbol_name(&self, symbol: &str) -> Option<FileDescriptor>;

    fn get_file_descriptor(&self, file_name: &str) -> Option<FileDescriptor>;
}

Keep in mind that some of these interfaces already exists, in more or less a similar shape. Once we have those in place, for the first version of Restate we just need a struct that contains all the implementations of these interfaces:

struct MetadataRegistry {
  // Single tree of metadata behind an ArcSwap
}

#[cfg(feature = "key_extractor")]
impl KeyExtractorResolver for MetadataRegistry {}
// etc...

By feature-gating interfaces and implementations, we make sure that individual crates will consume only the part of the schema registry they need. This allows us in future to easily split the MetadataRegistry into separate registries, where individual components living in different processes will just consume what they need of the available metadata, retaining at the same time the ability, in the future worker NodeService, to consistently update all the implementations of these interfaces at once.

Because in every process there is only one Meta (for workers, in future, NodeService) that writes metadata in this struct, we don't need to expose the methods to write this data structure.

[slinkydeveloper]

Additional requirement:

• Flag a service as internal only (that is, only other services can access it) and external only (that is, only accessible from the ingress)