Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document infinispan storage solution for Caching service #2167

Merged
merged 10 commits into from
May 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/extend/extend-apiml/api-mediation-caching-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ The Caching service is available only for internal Zowe applications, and is not
- [Storage methods](#storage-methods)
- [VSAM](#vsam)
- [Redis](#redis)
- [Infinispan](#infinispan)
- [InMemory](#inmemory)
- [How to start the service](#how-to-start-the-service)
- [Methods to use the Caching service API](#methods-to-use-the-caching-service-api)
Expand Down Expand Up @@ -45,6 +46,11 @@ For more information about the VSAM storage access method, see [Using VSAM as a
Redis is a common storage solution that runs outside of the z/OS platform. It can store data structures in key-value pairs, has high-availability support, and is highly performant.

For more information about the Redis storage access method, see [Using Redis as a storage solution through the Caching service](./api-mediation-redis.md).
### Infinispan

Infinispan is a storage solution that can also run on the z/OS platform. It can store data structures in key-value pairs, has high-availability support, and is highly performant.

For more information about the Infinispan storage access method, see [Using Infinispan as a storage solution through the Caching service](./api-mediation-infinispan.md).
### InMemory
TODO (add in memory procedure)
The InMemory storage method is a method suitable for testing and integration verification. Be sure not to use InMemory storage in production.
Expand Down
53 changes: 53 additions & 0 deletions docs/extend/extend-apiml/api-mediation-infinispan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Using Infinispan as a storage solution through the Caching service

As an API developer, you can configure Infinispan as a storage solution through the Caching service. This article describes how to configure your storage solution for Infinispan.
You can configure Infinispan for high availability as well as to replicate data to provide data durability and availability.

- [Understanding Infinispan](#understanding-infinispan)
- [Infinispan configuration](#infinispan-configuration)

## Understanding Infinispan

Infinispan is a storage solution that stores data structures in key-value pairs. The API Caching service uses hash sets, where each
service storing data via the Caching service has its own hash, and each data entry is a key-value entry within that service's Infinispan hash set.

For more information on Infinispan, see the [official Infinispan documentation](https://infinispan.org/documentation/).

### Infinispan replica instances

Infinispan can be used with both on standalone instance and High Availability mode. In case of multiple Caching Service instances,
you will have to specify all the cluster nodes (members). Each Infinispan node is bound to specific Caching Service instance and runs on a different port and host, which can be configured. See the [nfinispan configuration](#infinispan-configuration) to know how to configure multiple Infinispan nodes.

For more information on Infinispan replication and how to configure a replica instance, see the [official Infinispan Cross-site Replication documentation](https://infinispan.org/docs/stable/titles/xsite/xsite.html).
taban03 marked this conversation as resolved.
Show resolved Hide resolved

## Infinispan configuration

Configure Infinispan as a storage solution through the Caching service by setting the following environment variables. Environment variables can be set by adding them to `instance.env`.

* **`CACHING_STORAGE_INFINISPAN_INITIALHOSTS`**

This property specifies the list of cluster nodes (members). In case of multiple instances, the value for each Caching Service instance can be
either a list of all the members, separated by a comma, or just the replica. The format is `${JGROUPS_BIND_ADDRESS}[${JGROUPS_BIND_PORT}]`.
taban03 marked this conversation as resolved.
Show resolved Hide resolved

**Example:**
`CACHING_STORAGE_INFINISPAN_INITIALHOST=caching-service[7600]`


* **`CACHING_STORAGE_INFINISPAN_PERSISTENCE_DATALOCATION`**

The path where the Soft-Index store will keep its data files for the Infinispan Soft-Index Cache Store.
The default value is `data`. If you run the Caching Service in HA and the instances use the same filesystem,
you have to specify a different value of the `CACHING_STORAGE_INFINISPAN_PERSISTENCE_DATALOCATION` property for each
instance. For more information, see [Soft-Index File Store](https://infinispan.org/blog/2014/10/31/soft-index-file-store).


* **`JGROUPS_BIND_PORT`**

The Jgroups port used by Infinispan.


* **`JGROUPS_BIND_ADDRESS`**

The Jgroups address used by Infinispan.


30 changes: 22 additions & 8 deletions docs/user-guide/configure-caching-service-ha.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuring the Caching Service for HA

Zowe uses the Caching Service to centralize the state data persistent in high availability (HA) mode. If you are runnning the caching service on z/OS there are two storage methods: `inMemory` or `VSAM`. If you are running the caching service off platform, such as a linux or windows container image, it is also possible to specify `redis`.
Zowe uses the Caching Service to centralize the state data persistent in high availability (HA) mode. If you are runnning the caching service on z/OS there are three storage methods: `inMemory`, `infinispan` or `VSAM`. If you are running the caching service off platform, such as a linux or windows container image, it is also possible to specify `redis` or `infinispan`.

To learn more about Caching Service, see [Using the Caching Service](../extend/extend-apiml/api-mediation-caching-service.md).

Expand All @@ -14,7 +14,7 @@ For users

```
zowe
components
components:
caching-service:
enabled: true
port: 7555
Expand All @@ -34,7 +34,7 @@ For users

```
zowe
components
components:
caching-service:
enabled: true
port: 7555
Expand All @@ -54,26 +54,40 @@ For users


```
zowe
components
zowe:
components:
caching-service:
enabled: true
port: 7555
storage:
size: 10000
evictionStrategy: reject
mode: redis
redis
redis:
masterNodeUri:
timeout: 60
sentinel
sentinel:
masterInstance
nodes
ssl
ssl:
enabled: true
keystore:
keystorePassword:
trustStore:
trustStorePassword
```

- **infinispan**

Infinispan is designed to be run mainly on z/OS since it offers good performance. To enable this method, set the value of `zowe.components.caching-service.storage.mode` to `infinispan` in the `zowe.yaml` configuration file.
Infinispan environment variables are not currently following the v2 naming convention, so they must be defined into `zowe.environments` section. For more information on these properties and their values see [Infinispan configuration](../extend/extend-apiml/api-mediation-infinispan.md#infinispan-configuration).


```
zowe
environments:
JGROUPS_BIND_PORT:
JGROUPS_BIND_ADDRESS:
CACHING_STORAGE_INFINISPAN_INITIALHOSTS:
CACHING_STORAGE_INFINISPAN_PERSISTENCE_DATALOCATION:
```
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ The following parameters can be set to customize the configuration of the Gatewa
The following values are used to customize the configuration of Caching Service.

- `ZWE_CACHING_SERVICE_PORT=7555`: The port the Caching Service will use.
- `ZWE_CACHING_SERVICE_PERSISTENT=`: This field sets the storage type used to persist data in the Caching Service. Valid options are `REDIS` or `VSAM`. `REDIS` is currently only supported as an off-Z storage solution. `VSAM` is only supported on Z.
- `ZWE_CACHING_SERVICE_PERSISTENT=`: This field sets the storage type used to persist data in the Caching Service. Valid options are `REDIS`, `INFINISPAN` or `VSAM`. `REDIS` is currently only supported as an off-Z storage solution. `VSAM` is only supported on Z. `INFINISPAN` is both an off-Z and Z solution, and it is mainly intended for z/OS as it offers good performance.
- `ZWE_CACHING_SERVICE_VSAM_DATASET`: This field sets the `VSAM` dataset name to be used to store Caching Service data. This field is required if `ZWE_CACHING_SERVICE_PERSISTENT` is set to `VSAM`, otherwise this field is not needed.

Refer to detailed section about [API Gateway configuration](api-mediation/api-gateway-configuration.md)
Expand Down Expand Up @@ -534,6 +534,10 @@ These configurations can be used under the `components.caching-service` section:
- `storage.redis.ssl.keystorePassword`: Specifies the password used to unlock the keystore. This is equivalent to the `CACHING_STORAGE_REDIS_SSL_KEYSTOREPASSWORD` variable in `instance.env`.
- `storage.redis.ssl.truststore`: Specifies the truststore file used to keep other parties public keys and certificates. This is equivalent to the `CACHING_STORAGE_REDIS_SSL_TRUSTSTORE` variable in `instance.env`.
- `storage.redis.ssl.truststorePassword`: Specifies the password used to unlock the truststore. This is equivalent to the `CACHING_STORAGE_REDIS_SSL_TRUSTSTOREPASSWORD` variable in `instance.env`.
- `storage.infinispan.initialHosts`: Specifies the list of cluster nodes (members) used in Cross-Site Replication. This is equivalent to the `CACHING_STORAGE_INFINISPAN_INITIALHOSTS` variable in `instance.env`.
- `storage.infinispan.persistence.dataLocation`: Specifies he path where the Soft-Index store will keep its data files for the Infinispan Soft-Index Cache Store. This is equivalent to the `CACHING_STORAGE_INFINISPAN_PERSISTENCE_DATALOCATION` variable in `instance.env`.
- `jgroups.bind.port`: Specifies the Jgroups port for used by Infinispan. This is equivalent to the `JGROUPS_BIND_PORT` variable in `instance.env`.
- `jgroups.bind.address`: Specifies the Jgroups address for used by Infinispan. This is equivalent to the `JGROUPS_BIND_ADDRESS` variable in `instance.env`.
- `environment.preferIpAddress`: Set this parameter to `true` to advertise a service IP address instead of its hostname. **Note:** this configuration is deprecated. Zowe start script will ignore this value and always set it to `false`. This is equivalent to the `APIML_PREFER_IP_ADDRESS` variable in `instance.env`.
- `apiml.security.ssl.verifySslCertificatesOfServices`: Specifies whether APIML should verify certificates of services in strict mode. Set to `true` will enable `strict` mode that APIML will validate both if the certificate is trusted in turststore, and also if the certificate Common Name or Subject Alternate Name (SAN) match the service hostname. This is equivalent to the `VERIFY_CERTIFICATES` variable defined in `<keystore-dir>/zowe-certificates.env`.
- `apiml.security.ssl.nonStrictVerifySslCertificatesOfServices`: Defines whether APIML should verify certificates of services in non-strict mode. Setting to `true` will enable `non-strict` mode where APIML will validate if the certificate is trusted in turststore, but ignore the certificate Common Name or Subject Alternate Name (SAN) check. Zowe will ignore this configuration if strict mode is enabled with `apiml.security.ssl.verifySslCertificatesOfServices`. This is equivalent to the `NONSTRICT_VERIFY_CERTIFICATES` variable defined in `<keystore-dir>/zowe-certificates.env`.
Expand Down