Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example Request: CamelJmsToFileExample.java #1079

Closed
RnD-John opened this issue Nov 22, 2019 · 34 comments
Closed

Example Request: CamelJmsToFileExample.java #1079

RnD-John opened this issue Nov 22, 2019 · 34 comments

Comments

@RnD-John
Copy link

Please provide a JMS to File example that runs on camel-K.

The regular camel example is here: https://github.com/apache/camel/tree/master/examples/camel-example-jms-file

Thank you,
-John

@davesargrad
Copy link

davesargrad commented Nov 25, 2019

Hi @RnD-John

I've tried to create an example myself (hopefully one that will also help you), since I have been trying to understand how to handle dependencies on packages such as org.apache.camel.component.jms. Typically these would be declared in pom.xml for automatic download by maven

The sample I've tried is just to take the camel-k Sample.java and to begin to import packages that relate to your JMS interest.

Using the following code, I see the sample pod fail to run properly

`import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.component.jms.JmsComponent;

public class Sample extends RouteBuilder {
@OverRide
public void configure() throws Exception {
from("timer:tick")
.log("Hello Camel K!");
}
}
`

The error I see in the POD log follows

image

So clearly camel-k is complaining about the following dependency:

<dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jms</artifactId> </dependency>

Camel-K team, what am I missing here? How do I make use of a component such as camel-jms?

@nicolaferraro do you have some guidance for us?

@nicolaferraro nicolaferraro added this to the 1.0.0-M5 milestone Nov 25, 2019
@RnD-John
Copy link
Author

@davesargrad - When attempting to run a similar test to what you describe, I received the same dependency errors.

I also tried to run the camel based example without explicitly requesting the packages (assuming that camel-k may automatically include them for me). That results in unknown symbol errors.

I thought my approach might be fundamentally flawed so I just posted it as a feature request. Thank you for the help.

-John

@davsclaus
Copy link
Contributor

Camel K only detects components in the route DSL, eg like to("jms:xxx") etc. Java import statements are "just noise", and if you add a import for a component that are unused like you do, then you get this error, unless you tell Camel K to add camel-jms as dependecy from the command line arguments.

@davesargrad
Copy link

davesargrad commented Nov 25, 2019

Thanks @davsclaus
How would I add camel-jms as a dependency from the command line?

Assume for example, that I need the following:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-jms</artifactId>
    <version>2.24.1</version>
</dependency>

How would I invoke kamel to build an integrationplatform that contains the proper dependency?

A more complete example includes the use of something such as the ActiveMQConnectionFactory

import org.apache.activemq.ActiveMQConnectionFactory;

 ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://mali.corp.sensis.com:61616");
        
        // Note we can explicit name the component
        context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));
        // end::e2[]
        // Add some configuration by hand ...
        // tag::e3[]
        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("test-jms:queue:sargradtest.queue").to("file://test");
            }
        });
<dependency>
    <groupId>org.apache.activemq</groupId>
    <artifactId>activemq-all</artifactId>
    <version>5.15.5</version>
</dependency>

@davesargrad
Copy link

davesargrad commented Nov 25, 2019

The following code works when running only inside camel... I'm trying to reduce this to a camel-k footprint


package com.sensis.example;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.camel.CamelContext;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.impl.DefaultCamelContext;


public final class CamelJmsToFileExample {

    private CamelJmsToFileExample() {        
    }
    
    public static void main(String args[]) throws Exception {

        CamelContext context = new DefaultCamelContext();

        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://mali.corp.sensis.com:61616");
        
        context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

        context.addRoutes(new RouteBuilder() {
            public void configure() {
                from("test-jms:queue:sargradtest.queue").to("file://test");
            }
        });

        context.start();

   
        context.stop();
    }
}

Note: also replace .to("file://test") with .log(msg)

@davsclaus
Copy link
Contributor

@davesargrad
Copy link

@davsclaus

This is what I have tried:

kamel run -d mvn:org.apache.activemq.activemq-core:5.5.1 -d mvn:org.apache.activemq.activemq-camel:5.5.1 Sample.java

The error that shows up in the camel-k-kit builder is:

image

@lburgazzoli
Copy link
Contributor

Looks liek the GAV is not in the right format, try with:

mvn:org.apache.activemq:activemq-core:5.5.1

note the : between groupId and artifactId

@davesargrad
Copy link

Looks liek the GAV is not in the right format, try with:

mvn:org.apache.activemq:activemq-core:5.5.1

note the : between groupId and artifactId

Wow!! That worked!

@davesargrad
Copy link

davesargrad commented Nov 25, 2019

The only problem with the running route is access to the camel context.

The code I am trying to use:

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.ProducerTemplate;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;

public class Sample extends RouteBuilder {
  @Override
  public void configure() throws Exception {

        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://mali.corp.sensis.com:61616");

        // Note we can explicit name the component
        context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

          from("test-jms:queue:sargradtest.queue")
                .log("Got a message !");
  }
}

The Error:
image

@lburgazzoli
Copy link
Contributor

I think it is getContext()

@davesargrad
Copy link

@lburgazzoli
You are right yet again.

Incrementally closer.

image

@lburgazzoli
Copy link
Contributor

looks like it lacks one of the spring classes, thus I do not know which artifact provides it, likely to be spring-core

@davsclaus
Copy link
Contributor

Yeah you need all of the JARs for the activemq-client, as its using the JMS client, so you should have that as dependency, and also maybe one of the connection pools.

You can use the activemq component instead of jms as its for ActiveMQ 5.x. Where you can say

-d camel-activemq to tell it to add this component which should come with the JMS Client and connection pool et all.

@davesargrad
Copy link

@lburgazzoli and @davsclaus

Thanks guys.
I'll see if I can do what you are suggesting..

Great to have your feedback.

@hguerrero
Copy link
Contributor

@lburgazzoli my PR #1067 with AMQP examples cover the usage of JMS client with Qpid, but the manage of ConnectionFactory apply to any other component using that type of configuration.

@nicolaferraro nicolaferraro removed this from the 1.0.0-M5 milestone Dec 3, 2019
@davesargrad
Copy link

davesargrad commented Dec 3, 2019

@davsclaus and @lburgazzoli

You guys gave me what I needed to get this working.

The following command line successfully runs the JMS integration

kamel run -d mvn:org.apache.activemq:activemq-all:5.5.1 -d mvn:org.apache.activemq:activemq-camel:5.5.1 Sample.java

Sample.java

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.ProducerTemplate;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;

public class Sample extends RouteBuilder {
  @Override
  public void configure() throws Exception {

        ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://activemq.svc.ip:61616");

        CamelContext context = getContext();
        // Note we can explicit name the component
        context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

          from("test-jms:queue:sargradtest.queue")
                .log("Got a message !");
  }
}

I was able to receive data from a real ActiveMQ service.

image

@davsclaus
Copy link
Contributor

Yeah and we are also working on fixing so camel-activemq works nicer OOTB
https://issues.apache.org/jira/browse/CAMEL-14220

So all you need is to just set
camel.component.activemq.brokerURL=tcp:xxxx

Then it comes with all dependencies included as well

@RnD-John
Copy link
Author

RnD-John commented Dec 3, 2019

@davesargrad et all,
Thank you for working through this. I will give this a shot.
-John

@RnD-John RnD-John closed this as completed Dec 3, 2019
@RnD-John
Copy link
Author

RnD-John commented Dec 5, 2019

Question:
How do I pass my connection factory URI as an argument?

Background:
I was able to connect to an ActiveMQ broker using the code posted here. Upon trying to parameterize the code so that I can pass in the queue name and the URI from the command line, I found that I can only use the --property command to set sub-strings within the Camel-K DSL portion.

using the command:
kamel run -d mvn:org.apache.activemq:activemq-all:5.5.1 -d mvn:org.apache.activemq:activemq-camel:5.5.1 -p my.message="It WORKED" -p my.message2="STILL" -p my.queue="camelktest" -p my.uri="" --name "ParameterizedBAMQ" BasicAMQ.java

on the file:
BasicAMQ.java:
`import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.ProducerTemplate;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;

public class BasicAMQ extends RouteBuilder {
@OverRide
public void configure() throws Exception {

    ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("{{my.uri}}");

    CamelContext context = getContext();
    // Note we can explicit name the component
    context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

      from("test-jms:queue:{{my.queue}}.queue")
            .log("{{my.message}} {{my.message2}} ! ${body} ");

}
}`

I get the error:
Exception in thread "main" java.lang.IllegalArgumentException: Invalid broker URI: {{my.uri}}

How do I pass my connection factory URI as an argument?

@RnD-John RnD-John reopened this Dec 5, 2019
@davesargrad
Copy link

Hi @RnD-John
Did you figure out the answer to your last question?

@nicolaferraro perhaps you have some insights on how a parameter can be passed into the "route code". The "my.uri" above is a perfect example of something that should not be hard-coded. I'd think its easy to pass this in on the "kamel" command line.

@RnD-John
Copy link
Author

@davesargrad I was out of the office for a few days; however, I still don't have a great solution for this. The best solution I have come up with is to utilize file io to read the desired values from a from a file on the disk. In general, I'd still like to see what @nicolaferraro or others recommend in terms of potential 'native' support.

@nicolaferraro
Copy link
Member

Native support will come with properties when we switch to Camel 3.0.1 (next version is expected to support Camel 3.0.0).

For the time being a possible solution is to use env variables:

ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(System.getenv("MY_URI"));

And:

kamel run --env MY_URI=xxx TheIntegration.java

It's more a workaround waiting for property-based configuration, but it should work.

@lburgazzoli
Copy link
Contributor

You can also use

public class BasicAMQ extends RouteBuilder {
    @PropertyInject("my.uri") 
    String uri;

     ....
}

Then you should be able to provide my.uti value either from a configmap or env var.

@davesargrad
Copy link

You can also use

public class BasicAMQ extends RouteBuilder {
    @PropertyInject("my.uri") 
    String uri;

     ....
}

Then you should be able to provide my.uti value either from a configmap or env var.

Hi @lburgazzoli
Oddly, I see the annotation PropertyInject in older versions of camel.

Its not in the newest javadoc.

@lburgazzoli
Copy link
Contributor

camel-core as been modularized in camel 3, the javadoc for the annotation is here: https://www.javadoc.io/doc/org.apache.camel/camel-api/latest/index.html

@davesargrad
Copy link

camel-core as been modularized in camel 3, the javadoc for the annotation is here: https://www.javadoc.io/doc/org.apache.camel/camel-api/latest/index.html

Thanks @lburgazzoli !! Good to know

@RnD-John
Copy link
Author

@lburgazzoli, using the @PropertyInject Method is throwing the following error:

Exception in thread "main" org.apache.camel.RuntimeCamelException: org.joor.ReflectException: java.lang.reflect.InvocationTargetException
at org.apache.camel.RuntimeCamelException.wrapRuntimeCamelException(RuntimeCamelException.java:52)
at org.apache.camel.k.listener.RoutesConfigurer.load(RoutesConfigurer.java:69)
at org.apache.camel.k.listener.RoutesConfigurer.accept(RoutesConfigurer.java:51)
at org.apache.camel.k.listener.AbstractPhaseListener.accept(AbstractPhaseListener.java:32)
at org.apache.camel.k.main.ApplicationRuntime$MainListenerAdapter.lambda$invokeListeners$0(ApplicationRuntime.java:152)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.apache.camel.k.main.ApplicationRuntime$MainListenerAdapter.invokeListeners(ApplicationRuntime.java:151)
at org.apache.camel.k.main.ApplicationRuntime$MainListenerAdapter.beforeStart(ApplicationRuntime.java:125)
at org.apache.camel.main.MainSupport.beforeStart(MainSupport.java:114)
at org.apache.camel.main.MainSupport.run(MainSupport.java:76)
at org.apache.camel.k.main.ApplicationRuntime.run(ApplicationRuntime.java:70)
at org.apache.camel.k.main.Application.main(Application.java:42)
Caused by: org.joor.ReflectException: java.lang.reflect.InvocationTargetException
at org.joor.Reflect.on(Reflect.java:876)
at org.joor.Reflect.create(Reflect.java:698)
at org.joor.Reflect.create(Reflect.java:662)
at org.apache.camel.k.loader.java.JavaSourceRoutesLoader.load(JavaSourceRoutesLoader.java:47)
at org.apache.camel.k.listener.RoutesConfigurer.load(RoutesConfigurer.java:67)
... 17 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.joor.Reflect.on(Reflect.java:873)
... 21 more
Caused by: java.lang.NullPointerException
at java.net.URI$Parser.parse(URI.java:3042)
at java.net.URI.(URI.java:588)
at org.apache.activemq.ActiveMQConnectionFactory.createURI(ActiveMQConnectionFactory.java:168)
at org.apache.activemq.ActiveMQConnectionFactory.(ActiveMQConnectionFactory.java:131)
at BasicAMQ.(BasicAMQ.java:17)
... 26 more
<

I updated my script as follows:
`import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.ProducerTemplate;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;

public class BasicAMQ extends RouteBuilder
{
//standard java, need to inject properties to use them//
@org.apache.camel.PropertyInject("{{my.uri}}")
private String uri;
@org.apache.camel.PropertyInject("{{my.component}}")
private String component;
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(uri);

@Override
public void configure() throws Exception
{
    CamelContext context = getContext();
    // Note we can explicit name the component
    context.addComponent(component, JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

    //DSL starts here, can directly use {{}} syntax for properties//
    from(component+":queue:{{my.queue}}.queue")
    .log("{{my.prefix}} ${body} ");
}

}
`

I run with the command:
kamel run -d mvn:org.apache.activemq:activemq-all:5.5.1 -d mvn:org.apache.activemq:activemq-camel:5.5.1 -p my.uri="<MY_URI>" -p my.component="test-jms" -p my.queue="camelktestParametric" -p my.prefix="Got a message: " --name "ParameterizedBAMQ" BasicAMQ.java

@lburgazzoli
Copy link
Contributor

property inject do not require {{}}

@RnD-John
Copy link
Author

using the values as keys directly also throws the error...
@org.apache.camel.PropertyInject("my.uri")
private String uri;
@org.apache.camel.PropertyInject("my.component")
private String component;

@RnD-John
Copy link
Author

@lburgazzoli, The PropertyInject method is desirable because everything can come from command line properties or from a config map. I'd still like to get this method to work if possible.

@nicolaferraro, In the mean time, I tested the environment variable method and that worked as expected.

Native support will come with properties when we switch to Camel 3.0.1 (next version is expected to support Camel 3.0.0).

For the time being a possible solution is to use env variables:

ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(System.getenv("MY_URI"));

And:

kamel run --env MY_URI=xxx TheIntegration.java

It's more a workaround waiting for property-based configuration, but it should work.

@lburgazzoli
Copy link
Contributor

lburgazzoli commented Dec 12, 2019

So I have a route like:

import org.apache.camel.PropertyInject;
import org.apache.camel.builder.RouteBuilder;

public class Sample extends RouteBuilder {
  @PropertyInject("my.url")
  String url;

  @Override
  public void configure() throws Exception {
    from("timer:tick")
      .setBody().constant(url)
      .log("${body}");
  }
}

The I do run it with:

kamel run --dev -p my.url=test examples/Sample.java

And what I see in my toute is:

[2] 2019-12-12 21:19:31.752 INFO  [main] ApplicationRuntime - Add listener: org.apache.camel.k.listener.ContextConfigurer@3c9d0b9d
[2] 2019-12-12 21:19:31.760 INFO  [main] ApplicationRuntime - Add listener: org.apache.camel.k.listener.RoutesConfigurer@7ee955a8
[2] 2019-12-12 21:19:31.763 INFO  [main] ApplicationRuntime - Add listener: org.apache.camel.k.listener.RoutesDumper@6ac13091
[2] 2019-12-12 21:19:31.793 INFO  [main] RuntimeSupport - Looking up loader for language: java
[2] 2019-12-12 21:19:31.805 INFO  [main] RuntimeSupport - Found loader org.apache.camel.k.loader.java.JavaSourceRoutesLoader@6f96c77 for language java from service definition
[2] 2019-12-12 21:19:32.536 INFO  [main] RoutesConfigurer - Loading routes from: file:/etc/camel/sources/i-source-000/Sample.java?language=java
[2] 2019-12-12 21:19:32.537 INFO  [main] ApplicationRuntime - Listener org.apache.camel.k.listener.RoutesConfigurer@7ee955a8 executed in phase ConfigureRoutes
[2] 2019-12-12 21:19:32.555 INFO  [main] BaseMainSupport - Using properties from: file:/etc/camel/conf/application.properties
[2] 2019-12-12 21:19:33.023 INFO  [main] ApplicationRuntime - Listener org.apache.camel.k.listener.ContextConfigurer@3c9d0b9d executed in phase ConfigureContext
[2] 2019-12-12 21:19:33.024 INFO  [main] DefaultCamelContext - Apache Camel 3.0.0-RC3 (CamelContext: camel-k) is starting
[2] 2019-12-12 21:19:33.028 INFO  [main] DefaultManagementStrategy - JMX is disabled
[2] 2019-12-12 21:19:33.180 INFO  [main] DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[2] 2019-12-12 21:19:33.192 INFO  [main] DefaultCamelContext - Route: route1 started and consuming from: timer://tick
[2] 2019-12-12 21:19:33.196 INFO  [main] DefaultCamelContext - Total 1 routes, of which 1 are started
[2] 2019-12-12 21:19:33.199 INFO  [main] DefaultCamelContext - Apache Camel 3.0.0-RC3 (CamelContext: camel-k) started in 0.173 seconds
[2] 2019-12-12 21:19:33.200 INFO  [main] ApplicationRuntime - Listener org.apache.camel.k.listener.RoutesDumper@6ac13091 executed in phase Started
[2] 2019-12-12 21:19:34.223 INFO  [Camel (camel-k) thread #1 - timer://tick] route1 - test
[2] 2019-12-12 21:19:35.196 INFO  [Camel (camel-k) thread #1 - timer://tick] route1 - test

So the properties binding works. I think your issue is that the connection factory is instantiated at class constructions, you should move it to the configure method as the injection happens after the class has been constructed, before that the url is null

@davesargrad
Copy link

davesargrad commented Dec 13, 2019

@lburgazzoli

Good clean example... Ty

@RnD-John
Copy link
Author

per @lburgazzoli

I think your issue is that the connection factory is instantiated at class constructions, you should move it to the configure method as the injection happens after the class has been constructed, before that the url is null

This fixes the issue. Thank you for the assistance. For those interested, the new code is as follows:

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.CamelContext;
import org.apache.camel.component.jms.JmsComponent;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.PropertyInject;

import javax.jms.ConnectionFactory;

import org.apache.activemq.ActiveMQConnectionFactory;

public class BasicAMQ extends RouteBuilder
{
//standard java, need to inject properties to use them//
@PropertyInject("my.uri")
private String uri;
@PropertyInject("my.component")
private String component;

@Override
public void configure() throws Exception
{
    ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(uri);

    CamelContext context = getContext();
    context.addComponent(component, JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));

    //DSL starts here, can directly use {{}} syntax for properties//
    from("{{my.component}}:queue:{{my.queue}}.queue")
    .log("{{my.prefix}} ${body} ");
}

}

Run via:

kamel run -d mvn:org.apache.activemq:activemq-all:5.5.1 -d mvn:org.apache.activemq:activemq-camel:5.5.1 -p my.uri="<MY_URI>" -p my.component="test-jms" -p my.queue="camelktestParametric" -p my.prefix="Got a message: " --name "ParameterizedBAMQ" BasicAMQ.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants