Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support resolving AppSync data sources #3061

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion samtranslator/intrinsics/resource_refs.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@ def __init__(self) -> None:
# { "LogicalId": {"Property": "Value"} }
self._refs: Dict[str, Dict[str, Any]] = {}

def add(self, logical_id, property_name, value): # type: ignore[no-untyped-def]
def add(self, logical_id: str, property_name: str, value: str) -> None:
"""
Add the information that resource with given `logical_id` supports the given `property`, and that a reference
to `logical_id.property` resolves to given `value.

Example:

"MyApi.Deployment" -> "MyApiDeployment1234567890"
"SuperCoolAPI.DataSources:MyDataSource" -> "SuperCoolAPIMyDataSource"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How necessary is this?

The . notation is useful for things that cannot be easily determined, such as the example above where the deployment includes a hash. For everything else, we document the generated logical IDs, so I'm not sure whether it's worth adding a new concept to refer to resources.

Copy link
Contributor

@hoffa hoffa Mar 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it's DataSources, but in the properties it's currently DynamoDBDataSources.

What happens when we add something else than DynamoDBDataSources and customer has same name for two data sources (the other is under a new category, not DynamoDBDataSources)? If that's not allowed, perhaps we can have a single DataSources property instead?


:param logical_id: Logical ID of the resource (Ex: MyLambdaFunction)
:param property_name: Property on the resource that can be referenced (Ex: Alias)
Expand Down
5 changes: 4 additions & 1 deletion samtranslator/model/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from pydantic import BaseModel
from pydantic.error_wrappers import ValidationError

from samtranslator.intrinsics.resource_refs import SupportedResourceReferences
from samtranslator.model.exceptions import (
ExpectedType,
InvalidResourceException,
Expand Down Expand Up @@ -489,7 +490,9 @@ class SamResourceMacro(ResourceMacro, metaclass=ABCMeta):
# Aggregate list of all reserved tags
_RESERVED_TAGS = [_SAM_KEY, _SAR_APP_KEY, _SAR_SEMVER_KEY]

def get_resource_references(self, generated_cfn_resources, supported_resource_refs): # type: ignore[no-untyped-def]
def get_resource_references(
self, generated_cfn_resources: List[Resource], supported_resource_refs: SupportedResourceReferences
) -> SupportedResourceReferences:
"""
Constructs the list of supported resource references by going through the list of CFN resources generated
by to_cloudformation() on this SAM resource. Each SAM resource must provide a map of properties that it
Expand Down
94 changes: 89 additions & 5 deletions samtranslator/model/sam_resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
from samtranslator.internal.types import GetManagedPolicyMap
from samtranslator.internal.utils.utils import passthrough_value, remove_none_values
from samtranslator.intrinsics.resolver import IntrinsicsResolver
from samtranslator.intrinsics.resource_refs import SupportedResourceReferences
from samtranslator.metrics.method_decorator import cw_timer
from samtranslator.model import (
PassThroughProperty,
Expand Down Expand Up @@ -2157,6 +2158,69 @@ class SamGraphQLApi(SamResourceMacro):
ResolverCodeSettings: Optional[Dict[str, Any]]
Functions: Optional[Dict[str, Dict[str, Any]]]

validate_setattr = False

referable_properties = {
"DataSources": DataSource.resource_type,
}

def __init__(
self,
logical_id: Optional[Any],
relative_id: Optional[str] = None,
depends_on: Optional[List[str]] = None,
attributes: Optional[Dict[str, Any]] = None,
):
super().__init__(logical_id, relative_id=relative_id, depends_on=depends_on, attributes=attributes)
# map generated CFN resources ids to internal logical (relative) ids for referable_multiresource_properies
self._cfn_id_to_relative_id_map: Dict[str, str] = {}

def get_resource_references(
self, generated_cfn_resources: List[Resource], supported_resource_refs: SupportedResourceReferences
) -> SupportedResourceReferences:
"""Construct the list of supported resource references.

Override SamResourceMacro implementation of this method.

Base implementation of this method maintains SupportedResourceReference dict like this
"MyApi.Deployment" -> "MyApiDeployment1234567890"
because all what is necessary is to reference a single resource.

For GraphQLApi, ability to reference __one of__ resources, like a data source, is required.
"SuperCoolAPI.DataSources.MyDataSource" -> "SuperCoolAPIMyDataSource"
And at the same time, attributes like ApiId still has 1:1 mapping:
"SuperCoolAPI.ApiId" -> "SuperCoolAPI1234567"

Parameters
----------
generated_cfn_resources
List of CloudFormation resources generated by this SAM resource.
supported_resource_refs
Object holding the mapping between property names and LogicalId of the generated CFN resource it maps to.

Returns
-------
Updated supported_resource_refs.
"""
# Create a map of {ResourceType: [CfnLogicalId, ]} for quick access
resource_id_by_type: Dict[str, List[str]] = {}
for resource in generated_cfn_resources:
if resource.resource_type in resource_id_by_type:
resource_id_by_type[resource.resource_type].append(resource.logical_id)
else:
resource_id_by_type[resource.resource_type] = [resource.logical_id]

for property_name, cfn_type in self.referable_properties.items():
if cfn_type in resource_id_by_type:
if len(resource_id_by_type[cfn_type]) == 1: # only a single resource has been generated like Api
supported_resource_refs.add(self.logical_id, property_name, resource_id_by_type[cfn_type][0])
else:
for cfn_resource_id in resource_id_by_type[cfn_type]:
composite_name = f"{property_name}:{self._cfn_id_to_relative_id_map[cfn_resource_id]}"
supported_resource_refs.add(self.logical_id, composite_name, cfn_resource_id)

return supported_resource_refs

@cw_timer
def to_cloudformation(self, **kwargs: Any) -> List[Resource]:
model = self.validate_properties_and_return_model(aws_serverless_graphqlapi.Properties)
Expand Down Expand Up @@ -2286,10 +2350,7 @@ def _construct_ddb_datasources(
resources: List[Resource] = []

for relative_id, ddb_datasource in ddb_datasources.items():
datasource_logical_id = self.logical_id + relative_id
cfn_datasource = DataSource(
logical_id=datasource_logical_id, depends_on=self.depends_on, attributes=self.resource_attributes
)
cfn_datasource = self._create_data_source(relative_id)

# Datasource "Name" property must be unique from all other datasources.
cfn_datasource.Name = ddb_datasource.Name or relative_id
Expand All @@ -2302,7 +2363,7 @@ def _construct_ddb_datasources(
cfn_datasource.DynamoDBConfig = self._parse_ddb_config(ddb_datasource)

cfn_datasource.ServiceRoleArn, permissions_resources = self._parse_datasource_role(
ddb_datasource, cfn_datasource.get_runtime_attr("arn"), relative_id, datasource_logical_id, kwargs
ddb_datasource, cfn_datasource.get_runtime_attr("arn"), relative_id, cfn_datasource.logical_id, kwargs
)

resources.extend([cfn_datasource, *permissions_resources])
Expand Down Expand Up @@ -2592,3 +2653,26 @@ def make_runtime_dict(r: aws_serverless_graphqlapi.Runtime) -> AppSyncRuntimeTyp
raise InvalidResourceException(
relative_id, "'Runtime' must be defined as a property here or in 'ResolverCodeSettings.'"
)

def _create_data_source(self, relative_id: str) -> DataSource:
"""Create CFN resource.

Factory method to generate CFN data source id, create CFN resource,
and map internal logical (relative) id to CFN resource id.

Parameters
----------
relative_id
internal data source logical id

Returns
-------
CFN data source resource
"""
data_source_logical_id = self.logical_id + relative_id
# update map of generated CFN resources ids
# to internal logical (relative) ids since DataSource is in referable_properties
self._cfn_id_to_relative_id_map[data_source_logical_id] = relative_id
return DataSource(
logical_id=data_source_logical_id, depends_on=self.depends_on, attributes=self.resource_attributes
)
54 changes: 54 additions & 0 deletions tests/translator/input/graphqlapi_ddb_datasource_reference.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
Transform: AWS::Serverless-2016-10-31
Resources:
SuperCoolAPI:
Type: AWS::Serverless::GraphQLApi
Properties:
Name: SomeApi
SchemaInline: |
type Todo {
id: ID!
name: String
description: String
priority: Int
}
type Mutation {
addTodo(id: ID!, name: String, description: String, priority: Int): Todo
}
type Query {
getNote(id: ID!): Todo
}
schema {
query: Query
mutation: Mutation
}
XrayEnabled: true
Auth:
Type: AWS_IAM
Tags:
key1: value1
key2: value2
DynamoDBDataSources:
MyDataSource:
TableName: some-table
TableArn: big-arn
AnotherDataSource:
TableName: cool-table
TableArn: table-arn

GetTodoResolver:
Description: Don't do anything, just reference
Type: AWS::AppSync::Resolver
Properties:
ApiId: !GetAtt SuperCoolAPI.ApiId
DataSourceName: !GetAtt SuperCoolAPI.DataSources:AnotherDataSource.Name
TypeName: Query
FieldName: getNote
RequestMappingTemplate: |
{
"version": "2017-02-28",
"operation": "GetItem",
"key": {
"NoteId": $util.dynamodb.toDynamoDBJson($context.arguments.id)
}
}
ResponseMappingTemplate: $util.toJson($context.result)
Loading