Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table Manager - UnknownOperationException error syncing tables #1484

Closed
Serrvosky opened this issue Jul 1, 2019 · 8 comments · Fixed by #1486
Closed

Table Manager - UnknownOperationException error syncing tables #1484

Serrvosky opened this issue Jul 1, 2019 · 8 comments · Fixed by #1486

Comments

@Serrvosky
Copy link

Hello everyone. I'm having this problem with Table Manager. What can I do to resolve it?

level=info ts=2019-07-01T09:46:23.407230048Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2489
level=info ts=2019-07-01T09:46:23.910604647Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2566
level=info ts=2019-07-01T09:46:24.419691089Z caller=table_manager.go:349 msg="creating table" table=cortex
level=info ts=2019-07-01T09:46:24.918574976Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2508
level=info ts=2019-07-01T09:46:25.410324388Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2498
level=info ts=2019-07-01T09:46:25.912701971Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=247 expected_tables=247
level=error ts=2019-07-01T09:46:27.359170097Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: f6d1f11e-0833-4842-8080-6edb1adb2798"
level=info ts=2019-07-01T09:48:21.915757892Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=247 expected_tables=247
level=error ts=2019-07-01T09:48:22.921463444Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 2b38dc64-2b8a-4ab2-b040-9f722f77f586"
level=info ts=2019-07-01T09:50:21.917929145Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=247 expected_tables=247
level=error ts=2019-07-01T09:50:22.929712054Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 6c650b7e-ce0a-43e5-a002-1c99f4a153d9"
@bboreham bboreham changed the title Table Manager - error syncing tables Table Manager - UnknownOperationException error syncing tables Jul 1, 2019
@bboreham
Copy link
Contributor

bboreham commented Jul 1, 2019

I don't know what is causing "UnknownOperationException", however you should adjust your configuration to stop it creating hundreds of tables - whatever from date it is starting at is some years in the past.

DynamoDB has a limit of 255 tables per user by default, also it is a waste of time and resources to create tables you don't need.

If you still get the error, please post your full set of command-line arguments and any yaml files supplied to the table-manager.

@Serrvosky
Copy link
Author

This is my table manager deployment logs file

level=info ts=2019-07-01T12:06:11.484421951Z caller=cortex.go:196 msg=initialising module=server
level=info ts=2019-07-01T12:06:11.485288993Z caller=server.go:116 http=[::]:80 grpc=[::]:9095 msg="server listening on addresses"
level=info ts=2019-07-01T12:06:11.48568748Z caller=cortex.go:196 msg=initialising module=table-manager
level=info ts=2019-07-01T12:06:11.485962066Z caller=main.go:57 msg="Starting Cortex" version="(version=, branch=, revision=)"
level=info ts=2019-07-01T12:06:11.486463683Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:06:11.492244339Z caller=table_manager.go:165 msg="error syncing tables" err="RequestError: send request failed\ncaused by: Post http://dynamodb.default.svc.cluster.local:8000/: dial tcp 10.245.161.171:8000: connect: connection refused"
level=info ts=2019-07-01T12:08:11.487880106Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=info ts=2019-07-01T12:08:13.770392673Z caller=table_manager.go:349 msg="creating table" table=cortex
level=info ts=2019-07-01T12:08:22.287106151Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2579
level=info ts=2019-07-01T12:08:22.882947377Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2580
level=info ts=2019-07-01T12:08:24.072760076Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2581
level=info ts=2019-07-01T12:08:24.579750242Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2578
level=info ts=2019-07-01T12:08:26.639282297Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2579
level=info ts=2019-07-01T12:08:27.456794983Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2580
level=info ts=2019-07-01T12:08:28.083572585Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2581
level=info ts=2019-07-01T12:08:28.710209071Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2582
level=info ts=2019-07-01T12:08:29.880927152Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2578
level=info ts=2019-07-01T12:08:36.037863033Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2582
level=info ts=2019-07-01T12:10:11.4866617Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:10:13.995867902Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 81999c1d-b63a-4382-8e82-ff04ce810f13"
level=info ts=2019-07-01T12:12:11.48668331Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:12:13.403495666Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 587335d1-ae9d-4341-8005-eb9881c0e6f9"
level=info ts=2019-07-01T12:14:11.486635892Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:14:13.964100273Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: bcb1457d-1126-42f1-bf72-7491d60817d1"
level=info ts=2019-07-01T12:16:11.486622785Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:16:28.961550494Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: cb120106-96c8-462e-a016-697eebf0775b"
level=info ts=2019-07-01T12:18:11.48697065Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:18:21.228486519Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 77a519bb-1c5c-45d1-a3ed-1166202de793"
level=info ts=2019-07-01T12:20:11.486666909Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:20:14.661868394Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 70d8a4da-7ae8-40c5-a7eb-bdcfa1795f3a"
level=info ts=2019-07-01T12:22:11.486940531Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:22:15.198515498Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 2bb14b79-d501-4a50-9c4f-d32b01c20c42"
level=info ts=2019-07-01T12:24:11.48911715Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:24:44.415902201Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 42fa6baa-2928-4817-aade-dcb046186ddd"
level=info ts=2019-07-01T12:26:11.486645234Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:26:58.801132521Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: acb1e096-83e4-49d2-a104-c8efa419d478"
level=info ts=2019-07-01T12:28:11.486725341Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:29:56.343050542Z caller=table_manager.go:174 msg="error syncing tables" err="RequestError: send request failed\ncaused by: Post http://dynamodb.default.svc.cluster.local:8000/: read tcp 10.244.0.88:59486->10.245.161.171:8000: read: connection reset by peer"
level=info ts=2019-07-01T12:30:11.489476948Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=info ts=2019-07-01T12:30:13.229406497Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2579
level=info ts=2019-07-01T12:30:18.393159563Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2581
level=info ts=2019-07-01T12:30:19.415447404Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2578
level=info ts=2019-07-01T12:30:19.634924337Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2580
level=info ts=2019-07-01T12:30:21.188526302Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2581
level=info ts=2019-07-01T12:30:21.572584832Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2582
level=info ts=2019-07-01T12:30:21.923232365Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2582
level=info ts=2019-07-01T12:30:22.214740384Z caller=table_manager.go:349 msg="creating table" table=cortex
level=info ts=2019-07-01T12:30:25.819926634Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2579
level=info ts=2019-07-01T12:30:29.607695613Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2578
level=info ts=2019-07-01T12:30:35.156803138Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2580
level=info ts=2019-07-01T12:32:11.489133931Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:32:12.499279155Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 0081e9dd-b6d0-45ed-97d6-c0d2e548298b"
level=info ts=2019-07-01T12:34:11.486626875Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:34:12.493215817Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: e5196fcb-edec-443d-a8a2-95819e97d6a6"
level=info ts=2019-07-01T12:36:11.486787371Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:36:12.495770899Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 9be95ae5-f2d6-4270-b746-c3cfe1d166c4"
level=info ts=2019-07-01T12:38:11.486777628Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T12:38:12.493044467Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: An unknown operation was requested.\n\tstatus code: 400, request id: 9bf318ee-8e36-42ff-825a-cff414767cbf"

To create table manager deployment, I'm using the file that we can find in this repo. I just change all the dates.

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: table-manager
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: table-manager
    spec:
      containers:
      - name: table-manager
        image: quay.io/cortexproject/cortex:master-e1ab5495
        imagePullPolicy: IfNotPresent
        args:
        - -target=table-manager
        - -server.http-listen-port=80
        - -dynamodb.original-table-name=cortex
        - -dynamodb.url=dynamodb://user:[email protected]:8000
        - -dynamodb.periodic-table.prefix=cortex_weekly_
        - -dynamodb.periodic-table.from=2019-06-01
        - -dynamodb.chunk-table.from=2019-06-01
        ports:
        - containerPort: 80

@bboreham
Copy link
Contributor

bboreham commented Jul 1, 2019

Looks like you are using dynamodb-local, and since the errors only appear on second sync I guess that component is failing the operation to read tags.
Similar report: hashicorp/terraform-provider-aws#529

@Serrvosky
Copy link
Author

Serrvosky commented Jul 1, 2019

If I use the image that are in k8s dynamodb's file:

deangiberson/aws-dynamodb-local

That give the output in the comment above (my last comment -> #1484 (comment)).

Now i change to AWS Local DynamoDB image:

amazon/dynamodb-local

and this is my new log's file:

level=info ts=2019-07-01T15:31:06.697495411Z caller=main.go:57 msg="Starting Cortex" version="(version=, branch=, revision=)"
level=info ts=2019-07-01T15:31:06.703350655Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=info ts=2019-07-01T15:31:06.763178087Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2578
level=info ts=2019-07-01T15:31:07.326440732Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2579
level=info ts=2019-07-01T15:31:07.730825941Z caller=table_manager.go:349 msg="creating table" table=cortex
level=info ts=2019-07-01T15:31:08.228638458Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2579
level=info ts=2019-07-01T15:31:08.716420889Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2580
level=info ts=2019-07-01T15:31:09.245988263Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2582
level=info ts=2019-07-01T15:31:09.738137864Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2582
level=info ts=2019-07-01T15:31:10.23479765Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2578
level=info ts=2019-07-01T15:31:10.732909299Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2581
level=info ts=2019-07-01T15:31:11.299694036Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2580
level=info ts=2019-07-01T15:31:11.734739419Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2581
level=info ts=2019-07-01T15:33:10.234589628Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T15:33:11.414046595Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 730f55a7-6ea3-4d96-8b25-2f5b5ffceacd"
level=info ts=2019-07-01T15:35:06.701859279Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T15:35:07.712020065Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 01974609-9163-4d2b-ab94-5a5e03c2349f"
level=info ts=2019-07-01T15:37:06.701861969Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T15:37:07.710052471Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 8ecffaaf-0212-4ccc-ad90-161880189c00"
level=info ts=2019-07-01T15:39:06.707055982Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=11 expected_tables=11
level=error ts=2019-07-01T15:39:07.724937962Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: f8b58f17-308a-4248-ac40-863ee4180994

And only now I have some errors about tagging
There are any K8s example to put this working with another DB? Cassandra for example?

@bboreham
Copy link
Contributor

bboreham commented Jul 1, 2019

I made a couple of PRs intended to improve matters. If you could try out the config one that'd be great.

(You can ignore the table-manager sync errors until midnight (UTC) Wednesday)

@Serrvosky
Copy link
Author

@bboreham, Thanks. I will "follow" your PR then.

@Serrvosky
Copy link
Author

Serrvosky commented Jul 3, 2019

Hello @bboreham,

I already tryed deploy cortex project with your most recent changes, however and I'm still having problems with table manager deployment.

Table manager logs:

...
level=error ts=2019-07-04T08:31:10.586292135Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 9638fe7d-d580-41c9-a88e-88d6e88a0b5b"
level=info ts=2019-07-04T08:33:09.484880643Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:33:10.490126193Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: f5546e44-b0f4-4be5-8eef-516f96d0b09e"
level=info ts=2019-07-04T08:35:09.484855998Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:35:10.491479839Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 9ece48fe-6c2d-44d6-be8a-e74815e6199a"
level=info ts=2019-07-04T08:37:09.485363641Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:37:39.486455314Z caller=table_manager.go:174 msg="error syncing tables" err="RequestError: send request failed\ncaused by: Post http://dynamodb.default.svc.cluster.local:8000/: dial tcp 10.245.179.147:8000: i/o timeout"
level=info ts=2019-07-04T08:39:09.484901566Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:39:39.48572384Z caller=table_manager.go:174 msg="error syncing tables" err="RequestError: send request failed\ncaused by: Post http://dynamodb.default.svc.cluster.local:8000/: dial tcp 10.245.179.147:8000: i/o timeout"
level=info ts=2019-07-04T08:41:09.484874619Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:41:39.485498621Z caller=table_manager.go:174 msg="error syncing tables" err="RequestError: send request failed\ncaused by: Post http://dynamodb.default.svc.cluster.local:8000/: dial tcp 10.245.179.147:8000: i/o timeout"
level=info ts=2019-07-04T08:43:09.484866391Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=info ts=2019-07-04T08:43:09.511841659Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2583
level=info ts=2019-07-04T08:43:10.163481546Z caller=table_manager.go:349 msg="creating table" table=cortex
level=info ts=2019-07-04T08:43:10.502265873Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2582
level=info ts=2019-07-04T08:43:10.998365552Z caller=table_manager.go:349 msg="creating table" table=cortex_chunks_2583
level=info ts=2019-07-04T08:43:11.506415559Z caller=table_manager.go:349 msg="creating table" table=cortex_weekly_2582
level=info ts=2019-07-04T08:45:09.484892394Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:45:10.491608787Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: c2196702-42e6-4a31-afbc-03aa2f013060"
level=info ts=2019-07-04T08:47:09.484837396Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:47:10.491468346Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: bed590ff-70e0-469d-9485-08f9a7da6c59"
level=info ts=2019-07-04T08:49:09.484910657Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:49:10.491936919Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 4248e824-11a7-4667-9247-e9d0fe1c283b"
level=info ts=2019-07-04T08:51:09.484861868Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:51:10.492890091Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: beab03d4-ea4a-45a5-a0a5-63113bafa5fd"
level=info ts=2019-07-04T08:53:09.484848774Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:53:10.494355217Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 3c3c29e5-c440-4846-9fc0-f7e6843756b3"
level=info ts=2019-07-04T08:55:09.484893542Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:55:10.49172487Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: a19f147d-a6e7-4022-90d1-c63061bd13b0"
level=info ts=2019-07-04T08:57:09.484837203Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:57:10.49157828Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 75497e78-4d0f-4dd8-a0a7-e0f378482c4e"
level=info ts=2019-07-04T08:59:09.484950183Z caller=table_manager.go:206 msg="synching tables" num_expected_tables=5 expected_tables=5
level=error ts=2019-07-04T08:59:10.491603853Z caller=table_manager.go:174 msg="error syncing tables" err="UnknownOperationException: Tagging is not currently supported in DynamoDB Local.\n\tstatus code: 400, request id: 909f52ba-68d2-42c3-bee1-d7f0a4f25648"
...

It's getting a huge headache deploying Cortex in K8s.

I think I didn't show yet all the scenario that I want to implement. I will leave here a scheme to show what I want to implement. If it's possible, please I would like to ear your opinion about the possibility of implement it.

image

@Serrvosky
Copy link
Author

Serrvosky commented Jul 4, 2019

Hello again @bboreham.

I tried again with AWS DynamoDB Free Tier and it works perfectly. I'm not sure, but probably, AWS just turn off some features of that docker image that are important to run Cortex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants