Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[refactor][doc] Refactor the information architecture of Schema topics #18242

Merged
merged 11 commits into from
Nov 5, 2022
605 changes: 604 additions & 1 deletion site2/docs/admin-api-schemas.md

Large diffs are not rendered by default.

78 changes: 1 addition & 77 deletions site2/docs/client-libraries-cpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,80 +412,4 @@ For complete examples, refer to [C++ client examples](https://github.com/apache/

## Schema

This section describes some examples about schema. For more information about schema, see [Pulsar schema](schema-get-started.md).

### Avro schema

- The following example shows how to create a producer with an Avro schema.

```cpp
static const std::string exampleSchema =
"{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\","
"\"fields\":[{\"name\":\"a\",\"type\":\"int\"},{\"name\":\"b\",\"type\":\"int\"}]}";
Producer producer;
ProducerConfiguration producerConf;
producerConf.setSchema(SchemaInfo(AVRO, "Avro", exampleSchema));
client.createProducer("topic-avro", producerConf, producer);
```

- The following example shows how to create a consumer with an Avro schema.

```cpp
static const std::string exampleSchema =
"{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\","
"\"fields\":[{\"name\":\"a\",\"type\":\"int\"},{\"name\":\"b\",\"type\":\"int\"}]}";
ConsumerConfiguration consumerConf;
Consumer consumer;
consumerConf.setSchema(SchemaInfo(AVRO, "Avro", exampleSchema));
client.subscribe("topic-avro", "sub-2", consumerConf, consumer)
```

### ProtobufNative schema

The following example shows how to create a producer and a consumer with a ProtobufNative schema.

1. Generate the `User` class using Protobuf3 or later versions.

```protobuf
syntax = "proto3";

message User {
string name = 1;
int32 age = 2;
}
```

2. Include the `ProtobufNativeSchema.h` in your source code. Ensure the Protobuf dependency has been added to your project.

```cpp
#include <pulsar/ProtobufNativeSchema.h>
```

3. Create a producer to send a `User` instance.

```cpp
ProducerConfiguration producerConf;
producerConf.setSchema(createProtobufNativeSchema(User::GetDescriptor()));
Producer producer;
client.createProducer("topic-protobuf", producerConf, producer);
User user;
user.set_name("my-name");
user.set_age(10);
std::string content;
user.SerializeToString(&content);
producer.send(MessageBuilder().setContent(content).build());
```

4. Create a consumer to receive a `User` instance.

```cpp
ConsumerConfiguration consumerConf;
consumerConf.setSchema(createProtobufNativeSchema(User::GetDescriptor()));
consumerConf.setSubscriptionInitialPosition(InitialPositionEarliest);
Consumer consumer;
client.subscribe("topic-protobuf", "my-sub", consumerConf, consumer);
Message msg;
consumer.receive(msg);
User user2;
user2.ParseFromArray(msg.getData(), msg.getLength());
```
To work with [Pulsar schema](schema-overview.md) using C++ clients, see [Schema - Get started](schema-get-started.md). For specific schema types that C++ clients support, see [code](https://github.com/apache/pulsar-client-cpp/blob/main/include/pulsar/Schema.h#L51-L132).
momo-jun marked this conversation as resolved.
Show resolved Hide resolved
91 changes: 2 additions & 89 deletions site2/docs/client-libraries-java.md
Original file line number Diff line number Diff line change
Expand Up @@ -1158,103 +1158,16 @@ tv.forEach((key, value) -> /*operations on all existing messages*/)

## Schema

In Pulsar, all message data consists of byte arrays "under the hood." [Message schemas](schema-get-started.md) enable you to use other types of data when constructing and handling messages (from simple types like strings to more complex, application-specific types). If you construct, say, a [producer](#producer) without specifying a schema, then the producer can only produce messages of type `byte[]`. The following is an example.
In Pulsar, all message data consists of byte arrays "under the hood." [Message schemas](schema-overview.md) enable you to use other types of data when constructing and handling messages (from simple types like strings to more complex, application-specific types). If you construct, say, a [producer](#producer) without specifying a schema, then the producer can only produce messages of type `byte[]`. The following is an example.

```java
Producer<byte[]> producer = client.newProducer()
.topic(topic)
.create();
```

The producer above is equivalent to a `Producer<byte[]>` (in fact, you should *always* explicitly specify the type). If you'd like to use a producer for a different type of data, you'll need to specify a **schema** that informs Pulsar which data type will be transmitted over the [topic](reference-terminology.md#topic).
The producer above is equivalent to a `Producer<byte[]>` (in fact, you should *always* explicitly specify the type). If you'd like to use a producer for a different type of data, you need to specify a **schema** that informs Pulsar which data type will be transmitted over the topic. For more examples, see [Schema - Get started](schema-get-started.md).

### AvroBaseStructSchema example

Let's say that you have a `SensorReading` class that you'd like to transmit over a Pulsar topic:

```java
public class SensorReading {
public float temperature;

public SensorReading(float temperature) {
this.temperature = temperature;
}

// A no-arg constructor is required
public SensorReading() {
}

public float getTemperature() {
return temperature;
}

public void setTemperature(float temperature) {
this.temperature = temperature;
}
}
```

You could then create a `Producer<SensorReading>` (or `Consumer<SensorReading>`) like this:

```java
Producer<SensorReading> producer = client.newProducer(JSONSchema.of(SensorReading.class))
.topic("sensor-readings")
.create();
```

The following schema formats are currently available for Java:

* No schema or the byte array schema (which can be applied using `Schema.BYTES`):

```java
Producer<byte[]> bytesProducer = client.newProducer(Schema.BYTES)
.topic("some-raw-bytes-topic")
.create();
```

Or, equivalently:

```java
Producer<byte[]> bytesProducer = client.newProducer()
.topic("some-raw-bytes-topic")
.create();
```

* `String` for normal UTF-8-encoded string data. Apply the schema using `Schema.STRING`:

```java
Producer<String> stringProducer = client.newProducer(Schema.STRING)
.topic("some-string-topic")
.create();
```

* Create JSON schemas for POJOs using `Schema.JSON`. The following is an example.

```java
Producer<MyPojo> pojoProducer = client.newProducer(Schema.JSON(MyPojo.class))
.topic("some-pojo-topic")
.create();
```

* Generate Protobuf schemas using `Schema.PROTOBUF`. The following example shows how to create the Protobuf schema and use it to instantiate a new producer:

```java
Producer<MyProtobuf> protobufProducer = client.newProducer(Schema.PROTOBUF(MyProtobuf.class))
.topic("some-protobuf-topic")
.create();
```

* Define Avro schemas with `Schema.AVRO`. The following code snippet demonstrates how to create and use Avro schema.

```java
Producer<MyAvro> avroProducer = client.newProducer(Schema.AVRO(MyAvro.class))
.topic("some-avro-topic")
.create();
```

### ProtobufNativeSchema example

For examples of ProtobufNativeSchema, see [`SchemaDefinition` in `Complex type`](schema-understand.md#complex-type).

## Authentication

Expand Down
Loading