Skip to content

Commit

Permalink
Infinispan extension upgrades
Browse files Browse the repository at this point in the history 
* Upgrade to 12.1.5.Final
* ISPN-13146 Improve the documentation and defaults
* ISPN-12780 Protobuf schema registration property
* ISPN-13137 Avoid creating the Remote cache without configuration
  • Loading branch information
@karesti
karesti committed Jul 2, 2021
1 parent 82793e9 commit cecc97d
Show file tree
Hide file tree
Showing 8 changed files with 205 additions and 93 deletions.
2 changes: 1 addition & 1 deletion bom/application/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@
<junit.jupiter.version>5.7.2</junit.jupiter.version>
<testng.version>6.14.2</testng.version>
<assertj.version>3.20.2</assertj.version>
<infinispan.version>12.1.4.Final</infinispan.version>
<infinispan.version>12.1.5.Final</infinispan.version>
<infinispan.protostream.version>4.4.1.Final</infinispan.protostream.version>
<caffeine.version>2.9.1</caffeine.version>
<netty.version>4.1.65.Final</netty.version>
Expand Down
140 changes: 103 additions & 37 deletions docs/src/main/asciidoc/infinispan-client.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ provides functionality to allow the client that can connect to said server when
Check out the https://infinispan.org/documentation[Infinispan documentation] to find out more about the Infinispan project,
in particular the https://infinispan.org/docs/stable/titles/hotrod_java/hotrod_java.html[Hot Rod Java client guide].


== Solution

We recommend that you follow the instructions in the next sections and create the application step by step.
Expand Down Expand Up @@ -43,15 +44,70 @@ This will add the following to your pom.xml
----

The Infinispan client is configurable in the `application.properties` file that can be
provided in the `src/main/resources` directory. These are the properties that
can be configured in this file:

include::{generated-dir}/config/quarkus-infinispan-client.adoc[opts=optional, leveloffset=+1]
provided in the `src/main/resources` directory.

It is also possible to configure a `hotrod-client.properties` as described in the Infinispan user guide. Note that
the `hotrod-client.properties` values overwrite any matching property from the other configuration values (eg. near cache).
This properties file is build time only and if it is changed, requires a full rebuild.

== Running and connecting to Infinispan
If you are new to Infinispan, check the 5 minutes https://infinispan.org/get-started/[Getting stated with Infinispan] tutorial to
learn how to run a server locally.

From Infinispan 12, authentication and authorization are enabled implicitly. Create a user named `admin` to implicitly have a user with the admin role.

If you are running a container, passing the USER="admin" and PASS="password" will make the trick.

If you are running a downloaded distribution, use the Command Line Tool from the downloaded server folder.
[source,bash]
----
./bin/cli.sh user create admin -p password
----

Once your Infinispan Server is running, connect to the Infinispan Server witIh these properties.

[source,properties]
----
# Your configuration properties
quarkus.infinispan-client.server-list=localhost:11222
# Authentication
quarkus.infinispan-client.auth-username=admin
quarkus.infinispan-client.auth-password=password
## Docker 4 Mac workaround
quarkus.infinispan-client.client-intelligence=BASIC
----

== Authentication

This chart illustrates what mechanisms have been verified to be working properly with
the Quarkus Infinispan Client extension.

.Mechanisms
|===
| Name | Verified

| DIGEST-MD5
| [green]*Y*

| PLAIN
| [green]*Y*

| SCRAM
| [red]*N*

| EXTERNAL
| [green]*Y*

| GSSAPI
| [red]*N*

| Custom
| [red]*N*

|===

== Serialization (Key Value types support)

By default the client will support keys and values of the following types: byte[],
Expand Down Expand Up @@ -94,12 +150,19 @@ public class Book {
}
----

Serialization of user types uses a library based on protobuf, called Protostream.
Serialization of user types uses a library based on protobuf,
called https://github.com/infinispan/protostream[Protostream].

[TIP]
Infinispan caches can store keys and values in different encodings, but the recommended way
is https://developers.google.com/protocol-buffers[Protobuf].
The https://infinispan.org/docs/stable/titles/encoding/encoding.html[Marshalling and Encoding] guide
describes how Infinispan encodes data and explains how to use Protobuf and Protostream in detail.

=== Annotation based Serialization

This can be done automatically by adding protostream annotations to your user classes.
In addition a single Initializer annotated interface is required which controls how
In addition, a single Initializer annotated interface is required which controls how
the supporting classes are generated.

Here is an example of how the preceding classes should be changed:
Expand Down Expand Up @@ -159,21 +222,25 @@ Here is an example of how the preceding classes should be changed:
If your classes have only mutable fields, then the `ProtoFactory` annotation
is not required, assuming your class has a no arg constructor.

Then all that is required is a very simple `SerializationContextInitializer` interface with an annotation
Then all that is required is a very simple `GeneratedSchema` interface with an annotation
on it to specify configuration settings

.BookContextInitializer.java
.BooksSchema.java
[source,java]
----
import org.infinispan.protostream.SerializationContextInitializer;
import org.infinispan.protostream.GeneratedSchema;
import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder;
import org.infinispan.protostream.types.java.math.BigDecimalAdapter;
@AutoProtoSchemaBuilder(includeClasses = { Book.class, Author.class, BigDecimalAdapter.class }, schemaPackageName = "book_sample")
interface BookContextInitializer extends SerializationContextInitializer {
interface BookStoreSchema extends GeneratedSchema {
}
----

[TIP]
Protostream provides default Protobuf mappers for commonly used types as `BigDecimal`,
included under the `org.infinispan.protostream.types` package.

So in this case we will automatically generate the marshaller and schemas for the included classes and
place them in the schema package automatically. The package does not have to be provided, but if you
utilize querying, you must know the generated package.
Expand Down Expand Up @@ -357,6 +424,27 @@ annotation is not required and if it is not supplied, the default cache will be

NOTE: Other types may be supported for injection, please see other sections for more information

=== Registering Protobuf Schemas with Infinispan Server

One the serialization on the client guide is done, you must register Protobuf schemas
with Infinispan Server if you want to perform queries or convert from `Protobuf` to other media types
such as `json`.

[TIP]
If you log to the Infinispan Console for a local running server in `http://localhost:11222`,
you can check the schemas that exist under the `Schemas` tab.

The `quarkus.infinispan-client.use-schema-registration` property is `true` by default, so the extension
will take care of the schema upload to Infinispan. This is very useful for development mode.
However, if you need to deal yourself with the schema evolution over time,
turn the property to 'false' for production environments.
Use the https://infinispan.org/docs/infinispan-operator/master/operator.html[Infinispan Operator]
for Kubernetes deployments, or the Infinispan Web Console, the
https://infinispan.org/docs/stable/titles/rest/rest.html#rest_v2_protobuf_schemas[REST API] or the
https://infinispan.org/docs/stable/titles/encoding/encoding.html#registering-sci-remote-caches_marshalling[Hot Rod Java Client]
for any other uses cases.


== Querying

The Infinispan client supports both indexed and non indexed querying as long as the
Expand Down Expand Up @@ -407,37 +495,15 @@ and/or keystore. This is further detailed https://infinispan.org/docs/stable/tit
The Infinispan Client extension enables SSL by default. You can read more about this
at link:native-and-ssl[Using SSL With Native Executables].

== Authentication

This chart illustrates what mechanisms have been verified to be working properly with
the Quarkus Infinispan Client extension.

.Mechanisms
|===
| Name | Verified

| DIGEST-MD5
| [green]*Y*

| PLAIN
| [green]*Y*

| EXTERNAL
| [green]*Y*

| GSSAPI
| [red]*N*

| Custom
| [red]*N*

|===

The guide for configuring these can be found https://infinispan.org/docs/stable/titles/hotrod_java/hotrod_java.html#hotrod_authentication[here].
However you need to configure these through the `hotrod-client.properties` file if using Dependency Injection.
However, you need to configure these through the `hotrod-client.properties` file if using Dependency Injection.

== Additional Features

The Infinispan Client has additional features that were not mentioned here. This means this
feature was not tested in a Quarkus environment and they may or may not work. Please let us
know if you need these added!

== Configuration Reference

include::{generated-dir}/config/quarkus-infinispan-client.adoc[opts=optional, leveloffset=+1]
12 changes: 5 additions & 7 deletions ...ment/src/main/java/io/quarkus/infinispan/client/deployment/InfinispanClientProcessor.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,7 @@
import org.infinispan.client.hotrod.logging.LogFactory;
import org.infinispan.commons.marshall.ProtoStreamMarshaller;
import org.infinispan.commons.util.Util;
import org.infinispan.protostream.BaseMarshaller;
import org.infinispan.protostream.EnumMarshaller;
import org.infinispan.protostream.FileDescriptorSource;
import org.infinispan.protostream.MessageMarshaller;
import org.infinispan.protostream.RawProtobufMarshaller;
import org.infinispan.protostream.SerializationContextInitializer;
import org.infinispan.protostream.*;
import org.jboss.jandex.AnnotationInstance;
import org.jboss.jandex.AnnotationTarget;
import org.jboss.jandex.ClassInfo;
Expand Down Expand Up @@ -148,7 +143,10 @@ InfinispanPropertiesBuildItem setup(ApplicationArchivesBuildItem applicationArch

InfinispanClientProducer.handleProtoStreamRequirements(properties);
Collection<ClassInfo> initializerClasses = index.getAllKnownImplementors(DotName.createSimple(
SerializationContextInitializer.class.getName()));
GeneratedSchema.class.getName()));
initializerClasses
.addAll(index.getAllKnownImplementors(DotName.createSimple(GeneratedSchema.class.getName())));

Set<SerializationContextInitializer> initializers = new HashSet<>(initializerClasses.size());
for (ClassInfo ci : initializerClasses) {
Class<?> initializerClass = Thread.currentThread().getContextClassLoader().loadClass(ci.toString());
Expand Down
61 changes: 34 additions & 27 deletions .../runtime/src/main/java/io/quarkus/infinispan/client/runtime/InfinispanClientProducer.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
public class InfinispanClientProducer {
private static final Log log = LogFactory.getLog(InfinispanClientProducer.class);

public static final String USE_SCHEMA_REGISTRATION = "useSchemaRegistration";
public static final String PROTOBUF_FILE_PREFIX = "infinispan.client.hotrod.protofile.";
public static final String PROTOBUF_INITIALIZERS = "infinispan.client.hotrod.proto-initializers";

Expand All @@ -56,41 +57,39 @@ public class InfinispanClientProducer {

private void initialize() {
log.debug("Initializing CacheManager");
Configuration conf;
if (properties == null) {
// We already loaded and it wasn't present - so use an empty config
conf = new ConfigurationBuilder().build();
} else {
conf = builderFromProperties(properties).build();
// We already loaded and it wasn't present - so don't initialize the cache manager
return;
}
cacheManager = new RemoteCacheManager(conf);

// TODO: do we want to automatically register all the proto file definitions?
RemoteCache<String, String> protobufMetadataCache = null;

Set<SerializationContextInitializer> initializers = (Set) properties.remove(PROTOBUF_INITIALIZERS);
if (initializers != null) {
for (SerializationContextInitializer initializer : initializers) {
if (protobufMetadataCache == null) {
protobufMetadataCache = cacheManager.getCache(
ProtobufMetadataManagerConstants.PROTOBUF_METADATA_CACHE_NAME);
}
protobufMetadataCache.put(initializer.getProtoFileName(), initializer.getProtoFile());
}
}
Configuration conf = builderFromProperties(properties).build();
cacheManager = new RemoteCacheManager(conf);

for (Map.Entry<Object, Object> property : properties.entrySet()) {
Object key = property.getKey();
if (key instanceof String) {
String keyString = (String) key;
if (keyString.startsWith(InfinispanClientProducer.PROTOBUF_FILE_PREFIX)) {
String fileName = keyString.substring(InfinispanClientProducer.PROTOBUF_FILE_PREFIX.length());
String fileContents = (String) property.getValue();
if (properties.containsKey(USE_SCHEMA_REGISTRATION) && (Boolean) properties.get(USE_SCHEMA_REGISTRATION)) {
RemoteCache<String, String> protobufMetadataCache = null;
Set<SerializationContextInitializer> initializers = (Set) properties.remove(PROTOBUF_INITIALIZERS);
if (initializers != null) {
for (SerializationContextInitializer initializer : initializers) {
if (protobufMetadataCache == null) {
protobufMetadataCache = cacheManager.getCache(
ProtobufMetadataManagerConstants.PROTOBUF_METADATA_CACHE_NAME);
}
protobufMetadataCache.put(fileName, fileContents);
protobufMetadataCache.put(initializer.getProtoFileName(), initializer.getProtoFile());
}
}
for (Map.Entry<Object, Object> property : properties.entrySet()) {
Object key = property.getKey();
if (key instanceof String) {
String keyString = (String) key;
if (keyString.startsWith(InfinispanClientProducer.PROTOBUF_FILE_PREFIX)) {
String fileName = keyString.substring(InfinispanClientProducer.PROTOBUF_FILE_PREFIX.length());
String fileContents = (String) property.getValue();
if (protobufMetadataCache == null) {
protobufMetadataCache = cacheManager.getCache(
ProtobufMetadataManagerConstants.PROTOBUF_METADATA_CACHE_NAME);
}
protobufMetadataCache.put(fileName, fileContents);
}
}
}
}
Expand Down Expand Up @@ -210,6 +209,14 @@ private ConfigurationBuilder builderFromProperties(Properties properties) {
properties.put(ConfigurationProperties.TRUST_STORE_TYPE, infinispanClientRuntimeConfig.trustStoreType.get());
}

if (infinispanClientRuntimeConfig.serverList.isPresent()) {
properties.put(ConfigurationProperties.SERVER_LIST, infinispanClientRuntimeConfig.serverList.get());
}

if (infinispanClientRuntimeConfig.useSchemaRegistration.isPresent()) {
properties.put(USE_SCHEMA_REGISTRATION, infinispanClientRuntimeConfig.useSchemaRegistration.get());
}

builder.withProperties(properties);

return builder;
Expand Down
Loading

0 comments on commit cecc97d

Please sign in to comment.