@DynamoDbSecondaryPartitionKey and @DynamoDbSecondarySortKey annotations have no effect on table.createTable()

[acouvreur]

Describe the bug

Given the following class with two simple enums (Status and CommandType):

@DynamoDbBean
@Getter
@Setter
public class Command {

    public static final String COMMANDS_BY_STATUS = "commands_by_status";
    public static final String COMMANDS_BY_TYPE = "commands_by_type";

    private UUID id;
    private Status status;
    private CommandType type;

    @DynamoDbPartitionKey
    public UUID getId() {
        return id;
    }

    @DynamoDbSecondaryPartitionKey(indexNames = COMMANDS_BY_STATUS)
    public Status getStatus() {
        return this.status;
    }

    @DynamoDbSortKey
    @DynamoDbSecondarySortKey(indexNames = {COMMANDS_BY_STATUS, COMMANDS_BY_TYPE})
    public CommandType getType() {
        return this.type;
    }

And another service in charge of the initialization of this specific bean:

@Service
public class DynamoDbService {
    private static final TableSchema<Command> tableSchema = TableSchema.fromBean(Command.class);
    private final DynamoDbTable<Command> table;
    private final DynamoDbClient client;
    private final String tableName;

    @Autowired
    public DynamoDbService(DynamoDbEnhancedClient enhancedClient, DynamoDbClient client, DynamoDbTableNameResolver resolver) {
        this.client = client;
        this.tableName = resolver.resolve(Command.class);
        this.table = enhancedClient.table(tableName, tableSchema);
    }

    @PostConstruct
    public void createTableIfNotExists() {
        if (!this.tableExists(tableName)) {
            table.createTable()
        }
    }
}

The call to TableSchema.fromBean(Command.class); produces the following table metadata with the index by name map like:

Right here I can see the the schema correctly scans the annotations.

---

When calling table.createTable(), the following thing happens:

It makes a calls to

aws-sdk-java-v2/services-custom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/internal/client/DefaultDynamoDbTable.java

Lines 116 to 119 in 50c3fb1

	@Override
	public void createTable() {
	createTable(CreateTableEnhancedRequest.builder().build());
	}

Which then goes all the way down with the default empty request to software.amazon.awssdk.enhanced.dynamodb.internal.operations.CreateTableOperation.java.

In the generateRequest you have the following piece of code:

aws-sdk-java-v2/services-custom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/internal/operations/CreateTableOperation.java

Lines 69 to 93 in 50c3fb1

	String primaryPartitionKey = tableSchema.tableMetadata().primaryPartitionKey();
	Optional<String> primarySortKey = tableSchema.tableMetadata().primarySortKey();
	Set<String> dedupedIndexKeys = new HashSet<>();
	dedupedIndexKeys.add(primaryPartitionKey);
	primarySortKey.ifPresent(dedupedIndexKeys::add);
	List<software.amazon.awssdk.services.dynamodb.model.GlobalSecondaryIndex> sdkGlobalSecondaryIndices = null;
	List<software.amazon.awssdk.services.dynamodb.model.LocalSecondaryIndex> sdkLocalSecondaryIndices = null;
	if (this.request.globalSecondaryIndices() != null) {
	sdkGlobalSecondaryIndices =
	this.request.globalSecondaryIndices().stream().map(gsi -> {
	String indexPartitionKey = tableSchema.tableMetadata().indexPartitionKey(gsi.indexName());
	Optional<String> indexSortKey = tableSchema.tableMetadata().indexSortKey(gsi.indexName());
	dedupedIndexKeys.add(indexPartitionKey);
	indexSortKey.ifPresent(dedupedIndexKeys::add);
	return software.amazon.awssdk.services.dynamodb.model.GlobalSecondaryIndex
	.builder()
	.indexName(gsi.indexName())
	.keySchema(generateKeySchema(indexPartitionKey, indexSortKey.orElse(null)))
	.projection(gsi.projection())
	.provisionedThroughput(gsi.provisionedThroughput())
	.build();
	}).collect(Collectors.toList());
	}

As you can see:

• sdkGlobalSecondaryIndices is initialized to null
• sdkLocalSecondaryIndices is initialized to null

And the following call

if (this.request.globalSecondaryIndices() != null) {

Directly checks the empty default request.

It does not check for the table schema metadata on secondary indices.

Expected Behavior

I'd expect the @DynamoDbSecondaryPartitionKey and @DynamoDbSecondarySortKey annotations to be used when calling table.createTable() from the table schema of the same bean.

Current Behavior

The current call to table.createTable() produces the following schema:

{
  "AttributeDefinitions": [
    {
      "AttributeName": "id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "type",
      "AttributeType": "S"
    }
  ],
  "TableName": "local_command",
  "KeySchema": [
    {
      "AttributeName": "id",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "type",
      "KeyType": "RANGE"
    }
  ],
  "TableStatus": "ACTIVE",
  "CreationDateTime": "2023-04-20T13:40:20.610Z",
  "ProvisionedThroughput": {
    "LastIncreaseDateTime": "1970-01-01T00:00:00.000Z",
    "LastDecreaseDateTime": "1970-01-01T00:00:00.000Z",
    "NumberOfDecreasesToday": 0,
    "ReadCapacityUnits": 0,
    "WriteCapacityUnits": 0
  },
  "TableSizeBytes": 0,
  "ItemCount": 0,
  "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/local_command",
  "BillingModeSummary": {
    "BillingMode": "PAY_PER_REQUEST",
    "LastUpdateToPayPerRequestDateTime": "2023-04-20T13:40:20.610Z"
  }
}

If you look closely at my description of the issue, I explain how I searched for the payload crafting.

Reproduction Steps

Create the @DynamoDbBean from my example and try to create the table from the schema.

Possible Solution

The fix is counter intuitive because of how I expected the annotations to work.

I just used the createTable method with the request argument to produce the following:

if (!this.tableExists(tableName)) {
        Projection all = Projection.builder()
                .projectionType(ProjectionType.ALL)
                .build();

        table.createTable(r -> {
                r.globalSecondaryIndices(
                EnhancedGlobalSecondaryIndex.builder()
                        .indexName(COMMANDS_BY_STATUS)
                        .projection(all)
                        .build(),
                EnhancedGlobalSecondaryIndex.builder()
                        .indexName(COMMANDS_BY_TYPE)
                        .projection(all)
                        .build()
                );
        });
}

Which produces the final and expected schema on DynamoD:

{
  "AttributeDefinitions": [
    {
      "AttributeName": "id",
      "AttributeType": "S"
    },
    {
      "AttributeName": "type",
      "AttributeType": "S"
    },
    {
      "AttributeName": "status",
      "AttributeType": "S"
    }
  ],
  "TableName": "local_command",
  "KeySchema": [
    {
      "AttributeName": "id",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "type",
      "KeyType": "RANGE"
    }
  ],
  "TableStatus": "ACTIVE",
  "CreationDateTime": "2023-04-20T13:46:23.072Z",
  "ProvisionedThroughput": {
    "LastIncreaseDateTime": "1970-01-01T00:00:00.000Z",
    "LastDecreaseDateTime": "1970-01-01T00:00:00.000Z",
    "NumberOfDecreasesToday": 0,
    "ReadCapacityUnits": 0,
    "WriteCapacityUnits": 0
  },
  "TableSizeBytes": 0,
  "ItemCount": 0,
  "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/local_command",
  "BillingModeSummary": {
    "BillingMode": "PAY_PER_REQUEST",
    "LastUpdateToPayPerRequestDateTime": "2023-04-20T13:46:23.072Z"
  },
  "GlobalSecondaryIndexes": [
    {
      "IndexName": "commands_by_status",
      "KeySchema": [
        {
          "AttributeName": "status",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "type",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL",
      },
      "IndexStatus": "ACTIVE",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 0,
        "WriteCapacityUnits": 0
      },
      "IndexSizeBytes": 0,
      "ItemCount": 0,
      "IndexArn": "arn:aws:dynamodb:ddblocal:000000000000:table/local_command/index/commands_by_status"
    },
    {
      "IndexName": "commands_by_type",
      "KeySchema": [
        {
          "AttributeName": "id",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "type",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL",
      },
      "IndexStatus": "ACTIVE",
      "ProvisionedThroughput": {
        "ReadCapacityUnits": 0,
        "WriteCapacityUnits": 0
      },
      "IndexSizeBytes": 0,
      "ItemCount": 0,
      "IndexArn": "arn:aws:dynamodb:ddblocal:000000000000:table/local_command/index/commands_by_type"
    }
  ]
}

Additional Information/Context

No response

AWS Java SDK version used

2.20.30

JDK version used

18.0.2

Operating System and version

MacOS M1 (13.3.1)

[debora-ito]

@acouvreur I see how this is counter intuitive. Thank you for reaching out.

[breader124]

Hi, is it open for contributions? If so, could you please assign it to me? I'd like to help

[acouvreur]

Hi, is it open for contributions? If so, could you please assign it to me? I'd like to help

The fix seems straightforward, but the annotation still lacks configuration on the projection though.

[debora-ito]

@breader124 yes, assigned you to the issue.

[debora-ito]

Feature implemented and released in SDK version 2.20.86.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[sraji]

I am running into an issue with creating secondary index. If anyone is familiar with the fix, can I get some help please.

Error:
Attempt to execute an operation that requires a secondary index without defining the index attributes in the table metadata. Index name: ByChannelContextId

UserBotDao.kt

@DynamoDbBean
data class UserBotDao(
    @get:DynamoDbPartitionKey
    @get:DynamoDbAttribute("userId")
    var userId: String? = null,

    @get:DynamoDbSortKey
    @get:DynamoDbAttribute("providerType")
    var providerType: String? = null,

    @get:DynamoDbSecondaryPartitionKey(indexNames = ["ByChannelContextId"])
    @get:DynamoDbAttribute("channelContextId")
    var channelContextId: String? = null,

    @get:DynamoDbUpdateBehavior(UpdateBehavior.WRITE_IF_NOT_EXISTS)
    @get:DynamoDbAttribute("create_at")
    var createdAt: Long = Instant.now().epochSecond,

    @get:DynamoDbUpdateBehavior(UpdateBehavior.WRITE_ALWAYS)
    @get:DynamoDbAttribute("update_at")
    var updatedAt: Long = Instant.now().epochSecond,

    @get:DynamoDbAttribute("created_by")
    var createdBy: String? = null,

    @get:DynamoDbAttribute("updated_by")
    var updatedBy: String? = null
)

val all = Projection.builder()
            .projectionType(ProjectionType.ALL)
            .build()

        // Define the table schema
        val tableSchema = TableSchema.fromBean(clazz)
        logger.info("Creating table $tableName with schema $tableSchema")
        val table = client.table(tableName, tableSchema)

        table.createTable { r: CreateTableEnhancedRequest.Builder ->
            r.globalSecondaryIndices(
                EnhancedGlobalSecondaryIndex.builder()
                    .indexName("ByChannelContextId")
                    .projection(all)
                    .build()
            )
        }
        ```