Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Kamelet "aws-sqs-source" is invalid #2334

Closed
syunull opened this issue Jan 5, 2025 · 1 comment · Fixed by #2336
Closed

The Kamelet "aws-sqs-source" is invalid #2334

syunull opened this issue Jan 5, 2025 · 1 comment · Fixed by #2336
Assignees
Milestone

Comments

@syunull
Copy link

syunull commented Jan 5, 2025

Hello,

I'm trying to setup a function in knative to consume s3:ObjectCreated:* events from an sqs queue.

I was trying to follow this article https://knative.dev/blog/articles/consuming_s3_data_with_knative/ but it seems the aws-s3-source also sends the file body in the message.

I was trying to use the following Pipe.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: aws-sqs-source-pipe
  # annotations:
  #   trait.camel.apache.org/mount.config: "secret:aws-s3-credentials"
spec:
  # integration:
  #   dependencies:
  #   - "camel:cloudevents"
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: aws-sqs-source
    properties:
      queueNameOrArn: arn:aws:sqs:us-west-2:11111111111:queue
      accessKey: accessKey
      secretKey: secretKey
      region: us-east-1
    dataTypes:
      out:
        scheme: aws2-s3
        format: application-cloudevents
  sink:
    dataTypes:
      in:
        scheme: http
        format: application-cloudevents
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: default

The operator had the log of

{"level":"info","ts":"2025-01-05T03:01:38Z","logger":"camel-k.controller.integration","msg":"Integration error","request-namespace":"camel-k","request-name":"aws-sqs-source-pipe","api-version":"camel.apache.org/v1","kind":"Integration","ns":"camel-k","name":"aws-sqs-source-pipe","reason":"InitializationFailed","error-message":"error during trait customization: kamelets trait execution failed: kamelets [aws-sqs-source] not found in (Kubernetes[namespace=camel-k], Empty[]) repositories"}
k -n camel-k get kamelets
NAME                                               TYPE     PROVIDER                     BUNDLED   CAMEL VERSION
avro-deserialize-action                            action   Apache Software Foundation   true      4.8.0
avro-serialize-action                              action   Apache Software Foundation   true      4.8.0
aws-bedrock-agent-runtime-sink                     sink     Apache Software Foundation   true      4.8.0
aws-bedrock-text-sink                              sink     Apache Software Foundation   true      4.8.0
aws-cloudtrail-source                              source   Apache Software Foundation   true      4.8.0
aws-cloudwatch-sink                                sink     Apache Software Foundation   true      4.8.0
aws-ddb-sink                                       sink     Apache Software Foundation   true      4.8.0
aws-ddb-streams-source                             source   Apache Software Foundation   true      4.8.0
aws-ec2-sink                                       sink     Apache Software Foundation   true      4.8.0
aws-eventbridge-sink                               sink     Apache Software Foundation   true      4.8.0
aws-kinesis-firehose-sink                          sink     Apache Software Foundation   true      4.8.0
aws-kinesis-sink                                   sink     Apache Software Foundation   true      4.8.0
aws-kinesis-source                                 source   Apache Software Foundation   true      4.8.0
aws-lambda-sink                                    sink     Apache Software Foundation   true      4.8.0
aws-redshift-sink                                  sink     Apache Software Foundation   true      4.8.0
aws-redshift-source                                source   Apache Software Foundation   true      4.8.0
aws-s3-event-based-source                          source   Apache Software Foundation   true      4.8.0
aws-s3-sink                                        sink     Apache Software Foundation   true      4.8.0
aws-s3-source                                      source   Apache Software Foundation   true      4.8.0
aws-s3-streaming-upload-sink                       sink     Apache Software Foundation   true      4.8.0
aws-secrets-manager-sink                           sink     Apache Software Foundation   true      4.8.0
aws-ses-sink                                       sink     Apache Software Foundation   true      4.8.0
aws-sns-fifo-sink                                  sink     Apache Software Foundation   true      4.8.0
aws-sns-sink                                       sink     Apache Software Foundation   true      4.8.0
aws-sqs-batch-sink                                 sink     Apache Software Foundation   true      4.8.0
aws-sqs-fifo-sink                                  sink     Apache Software Foundation   true      4.8.0
aws-sqs-sink                                       sink     Apache Software Foundation   true      4.8.0
aws-sts-assume-role-action                         action   Apache Software Foundation   true      4.8.0
aws-timestream-query-sink                          sink     Apache Software Foundation   true      4.8.0
aws-translate-action                               action   Apache Software Foundation   true      4.8.0
azure-cosmosdb-sink                                sink     Apache Software Foundation   true      4.8.0
azure-cosmosdb-source                              source   Apache Software Foundation   true      4.8.0
azure-eventhubs-sink                               sink     Apache Software Foundation   true      4.8.0
azure-eventhubs-source                             source   Apache Software Foundation   true      4.8.0
azure-functions-sink                               sink     Apache Software Foundation   true      4.8.0
azure-servicebus-sink                              sink     Apache Software Foundation   true      4.8.0
azure-servicebus-source                            source   Apache Software Foundation   true      4.8.0
azure-storage-blob-append-sink                     sink     Apache Software Foundation   true      4.8.0
azure-storage-blob-changefeed-source               source   Apache Software Foundation   true      4.8.0
azure-storage-blob-event-based-source              source   Apache Software Foundation   true      4.8.0
azure-storage-blob-sink                            sink     Apache Software Foundation   true      4.8.0
azure-storage-blob-source                          source   Apache Software Foundation   true      4.8.0
azure-storage-datalake-sink                        sink     Apache Software Foundation   true      4.8.0
azure-storage-datalake-source                      source   Apache Software Foundation   true      4.8.0
azure-storage-files-sink                           sink     Apache Software Foundation   true      4.8.0
azure-storage-files-source                         source   Apache Software Foundation   true      4.8.0
azure-storage-queue-sink                           sink     Apache Software Foundation   true      4.8.0
azure-storage-queue-source                         source   Apache Software Foundation   true      4.8.0
beer-source                                        source   Apache Software Foundation   true      4.8.0
bitcoin-source                                     source   Apache Software Foundation   true      4.8.0
caffeine-action                                    action   Apache Software Foundation   true      4.8.0
cassandra-sink                                     sink     Apache Software Foundation   true      4.8.0
cassandra-source                                   source   Apache Software Foundation   true      4.8.0
ceph-sink                                          sink     Apache Software Foundation   true      4.8.0
ceph-source                                        source   Apache Software Foundation   true      4.8.0
chuck-norris-source                                source   Apache Software Foundation   true      4.8.0
chunk-template-action                              action   Apache Software Foundation   true      4.8.0
coffee-source                                      source   Apache Software Foundation   true      4.8.0
couchbase-sink                                     sink     Apache Software Foundation   true      4.8.0
counter-source                                     source   Apache Software Foundation   true      4.8.0
cron-source                                        source   Apache Software Foundation   true      4.8.0
data-type-action                                   action   Apache Software Foundation   true      4.8.0
delay-action                                       action   Apache Software Foundation   true      4.8.0
djl-image-to-text-action                           action   Apache Software Foundation   true      4.8.0
dns-dig-action                                     action   Apache Software Foundation   true      4.8.0
dns-ip-action                                      action   Apache Software Foundation   true      4.8.0
dns-lookup-action                                  action   Apache Software Foundation   true      4.8.0
drop-field-action                                  action   Apache Software Foundation   true      4.8.0
drop-header-action                                 action   Apache Software Foundation   true      4.8.0
drop-headers-action                                action   Apache Software Foundation   true      4.8.0
dropbox-sink                                       sink     Apache Software Foundation   true      4.8.0
dropbox-source                                     source   Apache Software Foundation   true      4.8.0
earthquake-source                                  source   Apache Software Foundation   true      4.8.0
elasticsearch-index-sink                           sink     Apache Software Foundation   true      4.8.0
elasticsearch-search-source                        source   Apache Software Foundation   true      4.8.0
exec-sink                                          sink     Apache Software Foundation   true      4.8.0
extract-field-action                               action   Apache Software Foundation   true      4.8.0
fhir-sink                                          sink     Apache Software Foundation   true      4.8.0
fhir-source                                        source   Apache Software Foundation   true      4.8.0
file-watch-source                                  source   Apache Software Foundation   true      4.8.0
freemarker-template-action                         action   Apache Software Foundation   true      4.8.0
ftp-sink                                           sink     Apache Software Foundation   true      4.8.0
ftp-source                                         source   Apache Software Foundation   true      4.8.0
ftps-sink                                          sink     Apache Software Foundation   true      4.8.0
ftps-source                                        source   Apache Software Foundation   true      4.8.0
github-commit-source                               source   Apache Software Foundation   true      4.8.0
github-event-source                                source   Apache Software Foundation   true      4.8.0
github-pullrequest-comment-source                  source   Apache Software Foundation   true      4.8.0
github-pullrequest-source                          source   Apache Software Foundation   true      4.8.0
github-tag-source                                  source   Apache Software Foundation   true      4.8.0
google-bigquery-sink                               sink     Apache Software Foundation   true      4.8.0
google-calendar-source                             source   Apache Software Foundation   true      4.8.0
google-functions-sink                              sink     Apache Software Foundation   true      4.8.0
google-mail-source                                 source   Apache Software Foundation   true      4.8.0
google-pubsub-sink                                 sink     Apache Software Foundation   true      4.8.0
google-pubsub-source                               source   Apache Software Foundation   true      4.8.0
google-sheets-sink                                 sink     Apache Software Foundation   true      4.8.0
google-sheets-source                               source   Apache Software Foundation   true      4.8.0
google-storage-event-based-source                  source   Apache Software Foundation   true      4.8.0
google-storage-sink                                sink     Apache Software Foundation   true      4.8.0
google-storage-source                              source   Apache Software Foundation   true      4.8.0
graphql-sink                                       sink     Apache Software Foundation   true      4.8.0
has-header-filter-action                           action   Apache Software Foundation   true      4.8.0
header-matches-filter-action                       action   Apache Software Foundation   true      4.8.0
hoist-field-action                                 action   Apache Software Foundation   true      4.8.0
http-secured-sink                                  sink     Apache Software Foundation   true      4.8.0
http-secured-source                                source   Apache Software Foundation   true      4.8.0
http-sink                                          sink     Apache Software Foundation   true      4.8.0
http-source                                        source   Apache Software Foundation   true      4.8.0
infinispan-sink                                    sink     Apache Software Foundation   true      4.8.0
infinispan-source                                  source   Apache Software Foundation   true      4.8.0
insert-field-action                                action   Apache Software Foundation   true      4.8.0
insert-header-action                               action   Apache Software Foundation   true      4.8.0
is-tombstone-filter-action                         action   Apache Software Foundation   true      4.8.0
jira-add-comment-sink                              sink     Apache Software Foundation   true      4.8.0
jira-add-issue-sink                                sink     Apache Software Foundation   true      4.8.0
jira-oauth-source                                  source   Apache Software Foundation   true      4.8.0
jira-source                                        source   Apache Software Foundation   true      4.8.0
jira-transition-issue-sink                         sink     Apache Software Foundation   true      4.8.0
jira-update-issue-sink                             sink     Apache Software Foundation   true      4.8.0
jms-amqp-10-sink                                   sink     Apache Software Foundation   true      4.8.0
jms-amqp-10-source                                 source   Apache Software Foundation   true      4.8.0
jms-apache-artemis-sink                            sink     Apache Software Foundation   true      4.8.0
jms-apache-artemis-source                          source   Apache Software Foundation   true      4.8.0
jms-ibm-mq-sink                                    sink     Apache Software Foundation   true      4.8.0
jms-ibm-mq-source                                  source   Apache Software Foundation   true      4.8.0
jms-pooled-apache-artemis-sink                     sink     Apache Software Foundation   true      4.8.0
jms-pooled-apache-artemis-source                   source   Apache Software Foundation   true      4.8.0
jolt-transformation-action                         action   Apache Software Foundation   true      4.8.0
jslt-action                                        action   Apache Software Foundation   true      4.8.0
json-deserialize-action                            action   Apache Software Foundation   true      4.8.0
json-patch-action                                  action   Apache Software Foundation   true      4.8.0
json-schema-validator-action                       action   Apache Software Foundation   true      4.8.0
json-serialize-action                              action   Apache Software Foundation   true      4.8.0
jsonata-action                                     action   Apache Software Foundation   true      4.8.0
kafka-apicurio-registry-not-secured-sink           sink     Apache Software Foundation   true      4.8.0
kafka-apicurio-registry-not-secured-source         source   Apache Software Foundation   true      4.8.0
kafka-azure-schema-registry-sink                   sink     Apache Software Foundation   true      4.8.0
kafka-azure-schema-registry-source                 source   Apache Software Foundation   true      4.8.0
kafka-batch-apicurio-registry-not-secured-source   source   Apache Software Foundation   true      4.8.0
kafka-batch-apicurio-registry-source               source   Apache Software Foundation   true      4.8.0
kafka-batch-azure-schema-registry-source           source   Apache Software Foundation   true      4.8.0
kafka-batch-manual-commit-action                   action   Apache Software Foundation   true      4.8.0
kafka-batch-not-secured-source                     source   Apache Software Foundation   true      4.8.0
kafka-batch-scram-source                           source   Apache Software Foundation   true      4.8.0
kafka-batch-source                                 source   Apache Software Foundation   true      4.8.0
kafka-batch-ssl-source                             source   Apache Software Foundation   true      4.8.0
kafka-manual-commit-action                         action   Apache Software Foundation   true      4.8.0
kafka-not-secured-apicurio-registry-json-source    source   Apache Software Foundation   true      4.8.0
kafka-not-secured-apicurio-registry-sink           sink     Apache Software Foundation   true      4.8.0
kafka-not-secured-apicurio-registry-source         source   Apache Software Foundation   true      4.8.0
kafka-not-secured-sink                             sink     Apache Software Foundation   true      4.8.0
kafka-not-secured-source                           source   Apache Software Foundation   true      4.8.0
kafka-scram-sink                                   sink     Apache Software Foundation   true      4.8.0
kafka-scram-source                                 source   Apache Software Foundation   true      4.8.0
kafka-sink                                         sink     Apache Software Foundation   true      4.8.0
kafka-source                                       source   Apache Software Foundation   true      4.8.0
kafka-ssl-sink                                     sink     Apache Software Foundation   true      4.8.0
kafka-ssl-source                                   source   Apache Software Foundation   true      4.8.0
kubernetes-namespaces-source                       source   Apache Software Foundation   true      4.8.0
kubernetes-nodes-source                            source   Apache Software Foundation   true      4.8.0
kubernetes-pods-source                             source   Apache Software Foundation   true      4.8.0
log-action                                         action   Apache Software Foundation   true      4.8.0
log-sink                                           sink     Apache Software Foundation   true      4.8.0
mail-imap-source                                   source   Apache Software Foundation   true      4.8.0
mail-sink                                          sink     Apache Software Foundation   true      4.8.0
mariadb-sink                                       sink     Apache Software Foundation   true      4.8.0
mariadb-source                                     source   Apache Software Foundation   true      4.8.0
mask-field-action                                  action   Apache Software Foundation   true      4.8.0
message-timestamp-router-action                    action   Apache Software Foundation   true      4.8.0
minio-sink                                         sink     Apache Software Foundation   true      4.8.0
minio-source                                       source   Apache Software Foundation   true      4.8.0
mongodb-changes-stream-source                      source   Apache Software Foundation   true      4.8.0
mongodb-sink                                       sink     Apache Software Foundation   true      4.8.0
mongodb-source                                     source   Apache Software Foundation   true      4.8.0
mqtt-sink                                          sink     Apache Software Foundation   true      4.8.0
mqtt-source                                        source   Apache Software Foundation   true      4.8.0
mqtt5-sink                                         sink     Apache Software Foundation   true      4.8.0
mqtt5-source                                       source   Apache Software Foundation   true      4.8.0
ms-exchange-online-imap-oauth-source               source   Apache Software Foundation   true      4.8.0
mustache-template-action                           action   Apache Software Foundation   true      4.8.0
mvel-template-action                               action   Apache Software Foundation   true      4.8.0
mysql-sink                                         sink     Apache Software Foundation   true      4.8.0
mysql-source                                       source   Apache Software Foundation   true      4.8.0
nats-sink                                          sink     Apache Software Foundation   true      4.8.0
nats-source                                        source   Apache Software Foundation   true      4.8.0
nominatim-geocode-action                           action   Apache Software Foundation   true      4.8.0
ogcapi-features-action                             action   Apache Software Foundation   true      4.8.0
opensearch-index-sink                              sink     Apache Software Foundation   true      4.8.0
opensearch-search-source                           source   Apache Software Foundation   true      4.8.0
oracle-database-sink                               sink     Apache Software Foundation   true      4.8.0
oracle-database-source                             source   Apache Software Foundation   true      4.8.0
pdf-action                                         action   Apache Software Foundation   true      4.8.0
postgresql-sink                                    sink     Apache Software Foundation   true      4.8.0
postgresql-source                                  source   Apache Software Foundation   true      4.8.0
predicate-filter-action                            action   Apache Software Foundation   true      4.8.0
protobuf-deserialize-action                        action   Apache Software Foundation   true      4.8.0
protobuf-serialize-action                          action   Apache Software Foundation   true      4.8.0
pulsar-sink                                        sink     Apache Software Foundation   true      4.8.0
pulsar-source                                      source   Apache Software Foundation   true      4.8.0
redis-sink                                         sink     Apache Software Foundation   true      4.8.0
redis-source                                       source   Apache Software Foundation   true      4.8.0
regex-router-action                                action   Apache Software Foundation   true      4.8.0
replace-field-action                               action   Apache Software Foundation   true      4.8.0
resolve-pojo-schema-action                         action   Apache Software Foundation   true      4.8.0
rest-openapi-sink                                  sink     Apache Software Foundation   true      4.8.0
salesforce-composite-upsert-sink                   sink     Apache Software Foundation   true      4.8.0
salesforce-create-sink                             sink     Apache Software Foundation   true      4.8.0
salesforce-delete-sink                             sink     Apache Software Foundation   true      4.8.0
salesforce-source                                  source   Apache Software Foundation   true      4.8.0
salesforce-update-sink                             sink     Apache Software Foundation   true      4.8.0
scp-sink                                           sink     Apache Software Foundation   true      4.8.0
set-body-action                                    action   Apache Software Foundation   true      4.8.0
set-kafka-key-action                               action   Apache Software Foundation   true      4.8.0
sftp-sink                                          sink     Apache Software Foundation   true      4.8.0
sftp-source                                        source   Apache Software Foundation   true      4.8.0
simple-filter-action                               action   Apache Software Foundation   true      4.8.0
slack-sink                                         sink     Apache Software Foundation   true      4.8.0
slack-source                                       source   Apache Software Foundation   true      4.8.0
snowflake-sink                                     sink     Apache Software Foundation   true      4.8.0
snowflake-source                                   source   Apache Software Foundation   true      4.8.0
splunk-hec-sink                                    sink     Apache Software Foundation   true      4.8.0
splunk-sink                                        sink     Apache Software Foundation   true      4.8.0
splunk-source                                      source   Apache Software Foundation   true      4.8.0
spring-rabbitmq-sink                               sink     Apache Software Foundation   true      4.8.0
spring-rabbitmq-source                             source   Apache Software Foundation   true      4.8.0
sqlserver-sink                                     sink     Apache Software Foundation   true      4.8.0
sqlserver-source                                   source   Apache Software Foundation   true      4.8.0
ssh-sink                                           sink     Apache Software Foundation   true      4.8.0
ssh-source                                         source   Apache Software Foundation   true      4.8.0
string-template-action                             action   Apache Software Foundation   true      4.8.0
telegram-sink                                      sink     Apache Software Foundation   true      4.8.0
telegram-source                                    source   Apache Software Foundation   true      4.8.0
throttle-action                                    action   Apache Software Foundation   true      4.8.0
timer-source                                       source   Apache Software Foundation   true      4.8.0
timestamp-router-action                            action   Apache Software Foundation   true      4.8.0
topic-name-matches-filter-action                   action   Apache Software Foundation   true      4.8.0
twitter-directmessage-source                       source   Apache Software Foundation   true      4.8.0
twitter-search-source                              source   Apache Software Foundation   true      4.8.0
twitter-timeline-source                            source   Apache Software Foundation   true      4.8.0
value-to-key-action                                action   Apache Software Foundation   true      4.8.0
velocity-template-action                           action   Apache Software Foundation   true      4.8.0
webhook-source                                     source   Apache Software Foundation   true      4.8.0
wttrin-source                                      source   Apache Software Foundation   true      4.8.0
xj-identity-action                                 action   Apache Software Foundation   true      4.8.0
xj-template-action                                 action   Apache Software Foundation   true      4.8.0
k -n camel-k apply -f https://raw.githubusercontent.com/apache/camel-kamelets/refs/heads/4.8.x/kamelets/aws-sqs-source.kamelet.yaml
The Kamelet "aws-sqs-source" is invalid:
* spec.definition.properties.waitTimeSeconds.minimum: Invalid value: "integer": spec.definition.properties.waitTimeSeconds.minimum in body must be of type string: "integer"
* spec.definition.properties.maxMessagesPerPoll.maximum: Invalid value: "integer": spec.definition.properties.maxMessagesPerPoll.maximum in body must be of type string: "integer"
* spec.definition.properties.maxMessagesPerPoll.minimum: Invalid value: "integer": spec.definition.properties.maxMessagesPerPoll.minimum in body must be of type string: "integer"
* spec.definition.properties.visibilityTimeout.minimum: Invalid value: "integer": spec.definition.properties.visibilityTimeout.minimum in body must be of type string: "integer"
*

Client Version: v1.30.5
Server Version: v1.31.3-eks-56e63d8

@oscerd oscerd self-assigned this Jan 6, 2025
@oscerd oscerd modified the milestones: 4.10.0, 4.8.3 Jan 6, 2025
@oscerd
Copy link
Contributor

oscerd commented Jan 7, 2025

It should be fixed on main and 4.8.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants