docs: update plugin docs for secrets/auth multiplexing (#16923)

* docs: update plugin docs for secrets/auth multiplexing

* update index

* update plugin development

* fix spacing in code snippet

* update links to multiplexing resources

* add note on sdk version and update db example text

* Update website/content/docs/plugins/plugin-architecture.mdx

Co-authored-by: Loann Le <[email protected]>

* reword index intro

* Update website/content/docs/plugins/plugin-development.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/plugins/plugin-development.mdx

Co-authored-by: Loann Le <[email protected]>

* remove word and fix code format

Co-authored-by: Loann Le <[email protected]>

website/content/docs/plugins/index.mdx

@@ -7,12 +7,11 @@ description: Learn about Vault's plugin system.
 
 # Plugin System
 
-All Vault auth methods and secrets engines are considered plugins. This simple concept
-allows both built-in and external plugins to be treated like Legos. Any plugin
-can exist at multiple different locations. Different versions of a plugin may
-be at each location, with each version differing from Vault's version.
-
-@include 'plugin-file-permissions-check.mdx'
+All Vault auth methods and secrets engines are considered plugins. This concept
+allows both built-in and external plugins to be treated like building blocks.
+Any plugin can exist at multiple different mount paths. Different versions of a
+plugin may be at each location, with each version differing from Vault's
+version.
 
 ## Built-In Plugins
 
@@ -35,7 +34,7 @@ or they can be [built from source](/docs/plugins/plugin-development#building-a-p
 
 Vault's external plugins are completely separate, standalone applications that
 Vault executes and communicates with over RPC. Each time a Vault secret engine
-or auth method is mounted, a new process is spawned. However, database plugins
-can be made to implement [plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
-which allows a single plugin process to be used for multiple database
-connections.
+or auth method is mounted, a new process is spawned. However, plugins can be
+made to implement [plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
+to improve performance. Plugin multiplexing allows plugin processes to be
+reused across all mounts of a given type.

website/content/docs/plugins/plugin-architecture.mdx

@@ -9,7 +9,7 @@ description: Learn about Vault's plugin architecture.
 Vault's external plugins are completely separate, standalone applications that Vault
 executes and communicates with over RPC. This means the plugin process does not
 share the same memory space as Vault and therefore can only access the
-interfaces and arguments given to it. This also means a crash in a plugin can not
+interfaces and arguments given to it. This also means a crash in a plugin cannot
 crash the entirety of Vault.
 
 It is possible to enable a custom plugin with a name that's identical to a
@@ -36,16 +36,16 @@ limited to:
 
 The lifecycle of plugin processes are managed automatically by Vault.
 Termination of these processes are typical in certain scenarios, such as the
-ones listed above. Vault will start plugin processes when needed, typically by
-lazily loading the plugin when a request that requires the plugin is received by
-Vault. A plugin process may be started or terminated through other internal
-processes within Vault as well. Since Vault manages and tracks the lifecycle of
-its plugins, these processes should not be terminated by anything other than
-Vault.
+ones listed above. Vault will start plugin processes when they are enabled. A
+plugin process may be started or terminated through other internal processes
+within Vault as well. Since Vault manages and tracks the lifecycle of its
+plugins, these processes should not be terminated by anything other than Vault.
+If a plugin process is shutdown out-of-band, the plugin process will be lazily
+loaded when a request that requires the plugin is received by Vault.
 
 ### External Plugin Scaling Characteristics
 
-External plugins are able to leverage [Performance Standbys](/docs/enterprise/performance-standby)
+External plugins can leverage [Performance Standbys](/docs/enterprise/performance-standby)
 without any explicit action by a plugin author. The default behavior of Vault
 Enterprise is to attempt to handle all requests, including requests to plugins,
 on performance standbys. If the plugin request makes any attempt to modify
@@ -56,15 +56,11 @@ without any effort on the plugin author's part.
 
 ## Plugin Communication
 
-Vault creates a mutually authenticated TLS connection for communication with
-the plugin's RPC server. Database secrets engines make use of the AutoMTLS
-feature of [go-plugin](https://www.github.com/hashicorp/go-plugin) which will
-automatically negotiate mTLS for transport authentication. For all other
-plugins, Vault passes a [wrapping token](/docs/concepts/response-wrapping) to
-the plugin process' environment. This token is single use and has a short TTL.
-Once unwrapped, it provides the plugin with a uniquely generated TLS
-certificate and private key for it to use to talk to the original Vault
-process.
+Vault communicates with external plugins over RPC. To secure this
+communication, Vault creates a mutually authenticated TLS connection with the
+plugin's RPC server. Plugins make use of the AutoMTLS feature of
+[go-plugin](https://www.github.com/hashicorp/go-plugin) which will
+automatically negotiate mutual TLS for transport authentication.
 
 The [`api_addr`](/docs/configuration#api_addr) must be set in order for the
 plugin process to establish communication with the Vault server during mount
@@ -147,19 +143,22 @@ on the upgrade procedure can be found in
 
 ## Plugin Multiplexing
 
-Database plugins can be made to implement plugin multiplexing,
-allowing a single plugin process to be used for multiple database
-connections. This single process, per database plugin, will be multiplexed
-across all Vault namespaces for mounts of this type. Multiplexing a plugin
-does not affect the current behavior of existing plugins.
+To avoid spawning multiple plugin processes for mounts of the same type,
+plugins can implement plugin multiplexing. This allows a single
+plugin process to be used for multiple mounts of a given type. This single
+process will be multiplexed across all Vault namespaces for mounts of this
+type. Multiplexing a plugin does not affect the current behavior of existing
+plugins.
 
 To enable multiplexing, the plugin must be compiled with the `ServeMultiplex`
-function call from Vault's `dbplugin` package. At this time, there is no
-opt-out capability for plugins that implement multiplexing. To use a
-non-multiplexed plugin, run an older version of the plugin, i.e., the
-plugin calls the `dbplugin.Serve` function. More details
-on implementing plugin multiplexing can be found in
-[Serving a Multiplexed Plugin](/docs/secrets/databases/custom#serving-a-plugin-with-multiplexing).
+function call from Vault's respective `plugin` or `dbplugin` SDK packages. At
+this time, there is no opt-out capability for plugins that implement
+multiplexing. To use a non-multiplexed plugin, run an older version of the
+plugin, i.e., the plugin calls the `Serve` function.
+
+More resources on implementing plugin multiplexing:
+* [Database secrets engines](/docs/secrets/databases/custom#serving-a-plugin-with-multiplexing)
+* [Secrets engines and auth methods](/docs/plugins/plugin-development)
 
 ## Troubleshooting
 

website/content/docs/plugins/plugin-development.mdx

@@ -26,11 +26,20 @@ a plugin is basic command-line skills and basic knowledge of the
 
 Your plugin implementation needs to satisfy the interface for the plugin
 type you want to build. You can find these definitions in the docs for the
-backend running the plugin. 
+backend running the plugin.
 
-~> Note: Plugins should be prepared to handle multiple concurrent requests 
+~> Note: Plugins should be prepared to handle multiple concurrent requests
 from Vault.
 
+## Serving A Plugin
+
+### Serving A Plugin with Multiplexing
+
+~> Plugin multiplexing requires `github.com/hashicorp/vault/sdk v0.5.4` or above.
+
+The following code exhibits an example main package for a Vault plugin using
+the Vault SDK for a secrets engine or auth method:
+
 ```go
 package main
 
@@ -50,7 +59,7 @@ func main() {
 	tlsConfig := apiClientMeta.GetTLSConfig()
 	tlsProviderFunc := api.VaultPluginTLSProvider(tlsConfig)
 
-	err := plugin.Serve(&plugin.ServeOpts{
+	err := plugin.ServeMultiplex(&plugin.ServeOpts{
 		BackendFactoryFunc: myPlugin.Factory,
 		TLSProviderFunc:    tlsProviderFunc,
 	})
@@ -66,6 +75,29 @@ func main() {
 And that's basically it! You would just need to change `myPlugin` to your actual
 plugin.
 
+## Plugin Backwards Compatibility with Vault
+
+Let's take a closer look at a snippet from the above main package.
+
+```go
+	err := plugin.ServeMultiplex(&plugin.ServeOpts{
+		BackendFactoryFunc: myPlugin.Factory,
+		TLSProviderFunc:    tlsProviderFunc,
+	})
+```
+
+The call to `plugin.ServeMultiplex` ensures that the plugin will use
+Vault's [plugin
+multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing) feature.
+However, this plugin will not be multiplexed if it is run by a version of Vault
+that does not support multiplexing. Vault will simply fall back to a plugin
+version that it can run. Additionally, we set the `TLSProviderFunc` to ensure
+that our plugin is backwards compatible with versions of Vault that do not
+support automatic mutual TLS for secure [plugin
+communication](/docs/plugins/plugin-architecture#plugin-communication). If you
+are certain your plugin does not need backwards compatibility, this field can
+be omitted.
+
 [api_addr]: /docs/configuration#api_addr
 
 ## Building a Plugin from Source

website/content/docs/plugins/plugin-management.mdx

@@ -30,9 +30,9 @@ integrity:
 ```shell-session
 $ vault plugin register -sha256=<SHA256 Hex value of the plugin binary> \
     secret \                  # type
-    my-secrets
+    passthrough-plugin
 
-Success! Registered plugin: myplugin-database-plugin
+Success! Registered plugin: passthrough-plugin
 ```
 
 ## Enabling/Disabling External Plugins

website/content/docs/secrets/databases/custom.mdx

@@ -115,9 +115,7 @@ other functions will need to open a connection.
 
 The plugin runs as a separate binary outside of Vault, so the plugin itself
 will need a `main` function. Use the `ServeMultiplex` function within
-`sdk/database/dbplugin/v5` to serve your multiplexed plugin. You will also need
-to pass some TLS configuration information that Vault uses when initializing
-the plugin.
+`sdk/database/dbplugin/v5` to serve your multiplexed plugin.
 
 Below is an example setup:
 
@@ -184,9 +182,7 @@ Replacing `MyDatabase` with the actual implementation of your database plugin.
 ### Serving A Plugin without Multiplexing
 
 Serving a plugin without multiplexing requires calling the `Serve` function
-from `sdk/database/dbplugin/v5` to serve your plugin. You will also need to
-pass some TLS configuration information that Vault uses when initializing the
-plugin.
+from `sdk/database/dbplugin/v5` to serve your plugin.
 
 The setup is exactly the same as the multiplexed case above, except for the
 `Run` function: