Added 2.10.0 release note #14398
Merged
Added 2.10.0 release note #14398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…che#13376) Master Issue: apache#11269 ### Motivation In order to increase the overall security of our Pulsar docker images, they should default to run as the non-root user. While updating these permissions, I make sure to comply with the OpenShift spec so the docker image can run on that platform out of the box. Once we finalize these changes, we will need to update the Apache Pulsar Helm chart to make sure that deployments take advantage of this feature. We'll use the `fsGroup` to make sure that k8s sets the appropriate file system permissions for the zookeeper, bookkeeper, and function pods. ### Modifications * Default to run as UID 10000. As noted in the `Dockerfile`, this UID is arbitrary. No logic should rely on this id. * Update filesystem permissions so that the group user has sufficient write permission. The group user is 0 (root). * Remove unnecessary write access. * The `/pulsar/{conf,data,logs}` directories and their members must be writable by the root group. I don't know of any other directories that need to be written to. Note that the `bin/pulsar-admin` too creates a log file in the `/pulsar/logs` directory. Please let me know if there are any additional * Note also that the executable file permissions are already set in our git repo. Those permissions are inherited by the docker image when we run the `COPY` directive in the `Dockerfile`. * There are no changes to the function worker in the k8s runtime. We do not need them because we already merged apache@04b5da0. * Add note to `conf/bkenv.sh`, as it is a `.sh` script that is not executable (and doesn't need to be). * Update test docker image and `supervisord` configuration. Note: it's unclear to me how the OpenShift spec handles restarts. I know that the UID is arbitrary. It's possible that the umask needs to be switched from `022` to `002`. Setting the umask in the docker image does not persist for consumers of the image, so this would need to be set in a helm chart. ### Verifying this change You can access a test image built with these changes here: `michaelmarshall/pulsar:2.10.0-SNAPSHOT`. I have already run some manual tests like `bin/pulsar standalone` in the container. I still need to deploy an actual cluster to verify that all of the unique components work correctly. Because we already merged apache@04b5da0, the upgrade scenarios are already simplified. If this change is in 2.10.0, that means 2.8 and 2.9 will be compatible for certain function worker upgrade scenarios. I wrote test criteria in apache#11269. I'll need to follow up on that criteria using my newly build image. I should be able to look closer at this tomorrow. We'll also need tests to pass, as I modified some tests with this PR. ### References The following links were useful in understanding how to make these changes: * https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html * https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids ### Does this pull request potentially affect one of the following parts: This PR updates our Docker images in a breaking way. It could result in bookkeepers, zookeepers, or functions with insufficient permissions. We will mitigate these permissions by updating the helm chart. These changes are easily overridden by extending the docker image. In k8s, you can use the pod's `securityContext` to override the user or group. (cherry picked from commit f7f8619)
codelipenghui
added
the
doc
Anonymitaet
reviewed
Feb 21, 2022
site2/website/release-notes.md
Outdated
| ### 2.10.0 | ||
| #### 2022-02-28 | ||
|
|
||
| ### Update notice |
There was a problem hiding this comment.
Suggested change
| ### Update notice | |
| ### Important notice |
site2/website/release-notes.md
Outdated
|
|
||
| ### Update notice | ||
| Remove -XX:-ResizePLAB JVM option which degrades performance on JDK11 [#12940](https://github.com/apache/pulsar/pull/12940) | ||
| Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) |
There was a problem hiding this comment.
Suggested change
| Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) | |
| - Enable TCP keepAlive flag on the sockets [#12982](https://github.com/apache/pulsar/pull/12982) |
As shown in the preview, the layout is incorrect. Suggest adding - before each item
site2/website/release-notes.md
Outdated
| [PIP 120] Enabled client memory limit controller by default [#13344](https://github.com/apache/pulsar/pull/13344) | ||
| Make Docker images non-root, by default, and OpenShift compliant [#13376](https://github.com/apache/pulsar/pull/13376) | ||
| [PIP 122] Change loadBalancer default loadSheddingStrategy to ThresholdShedder [#13733](https://github.com/apache/pulsar/pull/13733) | ||
| Fixed netcat returning early for probe [#14088](https://github.com/apache/pulsar/pull/14088) |
There was a problem hiding this comment.
Some items are in the past tense while others are in the present tense. Please keep the same tense for all items.
|
@DaveDuggins could you please help review? Thanks |
site2/website/release-notes.md
Outdated
| - [PIP 79] Added lazy-loading feature to PartitionedProducer [#10279](https://github.com/apache/pulsar/pull/10279) | ||
| - [PIP 84] Pulsar client: Redeliver command add epoch [#10478](https://github.com/apache/pulsar/pull/10478) | ||
| - [PIP 86] Pulsar Functions: Preload and release external resources [#13205](https://github.com/apache/pulsar/pull/13205) | ||
| - [PIP 97] Update Authentication Interfaces to Include Async Authentication Methods [#12104](https://github.com/apache/pulsar/pull/12104) |
There was a problem hiding this comment.
This PIP is not completed for 2.10. I reverted the main changes #14424. We can remove the line here.
Suggested change
| - [PIP 97] Update Authentication Interfaces to Include Async Authentication Methods [#12104](https://github.com/apache/pulsar/pull/12104) |
momo-jun
reviewed
Feb 23, 2022
| ### 2.10.0 | ||
| #### 2022-02-28 | ||
|
|
||
| ### Important notice |
There was a problem hiding this comment.
IMHO, should it be "Highlights" to introduce key enhancements? The "Importance Notice" section is more likely talking about notes that require attention, compatibility issues, etc.
There was a problem hiding this comment.
The items under this section are introduced default configuration changes or deployment changes, so it should be "Importance Notice"
There was a problem hiding this comment.
Thanks for clarifying this. I suggest we add an introductory sentence like "This release introduces the following default configuration changes and deployment changes that require your attention."
dave2wave
reviewed
Feb 24, 2022
site2/website/release-notes.md
Outdated
| - Add deleteSubscriptionDispatchRate method for v1 namespace [#13711](https://github.com/apache/pulsar/pull/13711) | ||
| - Fix NPE of checkReplication [#13720](https://github.com/apache/pulsar/pull/13720) | ||
| - Optimize deduplicationSnapshotIntervalSeconds with HierarchyTopicPolicies [#13769](https://github.com/apache/pulsar/pull/13769) | ||
| - Fix call sync method in async rest api for `internalDeletePartitionedTopic` [#13805](https://github.com/apache/pulsar/pull/13805) |
There was a problem hiding this comment.
I think it makes sense to group all of the "Fix call sync method in async rest api for ..." fixes into sub bullet points.
- Fix call sync method in async rest api for
- for ...
- for ...
site2/website/release-notes.md
Outdated
| - [PIP 135] Include MetadataStore backend for Etcd [#13225](https://github.com/apache/pulsar/pull/13225) | ||
|
|
||
| ### Broker | ||
| - [PIP 45] Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) |
There was a problem hiding this comment.
I think it makes sense to group all of the "[PIP 45]" fixes into sub bullet points.
- [PIP 45]
- ...
- ...
|
The pr had no activity for 30 days, mark with Stale label. |
|
@Anonymitaet @dave2wave @michaeljmarshall @momo-jun I have updated the release note, please help review, thanks. |
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
momo-jun
reviewed
Apr 12, 2022
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Co-authored-by: momo-jun <[email protected]>
Anonymitaet
reviewed
Apr 13, 2022
site2/website/release-notes.md
Outdated
| - [PIP 45] Pluggable metadata interface | ||
| - Added BookKeeper metadata adapter based on MetadataStore [#12770](https://github.com/apache/pulsar/pull/12770) | ||
| - Add Rocksdb metadata store [#12776](https://github.com/apache/pulsar/pull/12776) | ||
| - Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) |
There was a problem hiding this comment.
Suggested change
| - Converted BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) | |
| - Convert BookieRackAffinityMapping to use MetadataStore [#12841](https://github.com/apache/pulsar/pull/12841) |
please keep verb tense consistent and check all occurrences
There was a problem hiding this comment.
@Anonymitaet Fixed.
Anonymitaet
approved these changes
Apr 18, 2022
aparajita89
pushed a commit
to aparajita89/pulsar
that referenced
this pull request
Apr 18, 2022
Nicklee007
pushed a commit
to Nicklee007/pulsar
that referenced
this pull request
Apr 20, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.