docs: add load balancing module documentation

docs/source/SUMMARY.md

@@ -48,6 +48,7 @@
     - [UDT (User defined type)](data-types/udt.md)
 
 - [Load balancing](load-balancing/load-balancing.md)
+    - [Default policy](load-balancing/default-policy.md)
 
 - [Retry policy configuration](retry-policy/retry-policy.md)
     - [Fallthrough retry policy](retry-policy/fallthrough.md)

docs/source/index.md

@@ -17,7 +17,7 @@ Although optimized for Scylla, the driver is also compatible with [Apache Cassan
 * [Making queries](queries/queries.md) - Making different types of queries (simple, prepared, batch, paged)
 * [Execution profiles](execution-profiles/execution-profiles.md) - Grouping query execution configuration options together and switching them all at once
 * [Data Types](data-types/data-types.md) - How to use various column data types
-* [Load balancing](load-balancing/load-balancing.md) - Load balancing configuration, local datacenters etc.
+* [Load balancing](load-balancing/load-balancing.md) - Load balancing configuration
 * [Retry policy configuration](retry-policy/retry-policy.md) - What to do when a query fails, query idempotence
 * [Driver metrics](metrics/metrics.md) - Statistics about the driver - number of queries, latency etc.
 * [Logging](logging/logging.md) - Viewing and integrating logs produced by the driver

docs/source/load-balancing/default-policy.md

@@ -0,0 +1,160 @@
+# DefaultPolicy
+
+`DefaultPolicy` is the default load balancing policy in Scylla Rust Driver. It
+can be configured to be datacenter-aware and token-aware. Datacenter failover
+for queries with non-local consistency mode is also supported.
+
+## Creating a DefaultPolicy
+
+`DefaultPolicy` can be created only using `DefaultPolicyBuilder`. The
+`builder()` method of `DefaultPolicy` returns a new instance of
+`DefaultPolicyBuilder` with the following default values:
+
+- `preferred_datacenter`: `None`
+- `is_token_aware`: `true`
+- `permit_dc_failover`: `false`
+- `latency_awareness`: `None`
+
+You can use the builder methods to configure the desired settings and create a
+`DefaultPolicy` instance:
+
+```rust
+# extern crate scylla;
+# fn test_if_compiles() {
+use scylla::load_balancing::DefaultPolicy;
+
+let default_policy = DefaultPolicy::builder()
+        .prefer_datacenter("dc1".to_string())
+        .token_aware(true)
+        .permit_dc_failover(true)
+        .build();
+# }
+```
+
+### Semantics of `DefaultPolicy`
+
+#### Preferred Datacenter
+
+The `preferred_datacenter` field in `DefaultPolicy` allows the load balancing
+policy to prioritize nodes based on their location. When a preferred datacenter
+is set, the policy will treat nodes in that datacenter as "local" nodes, and
+nodes in other datacenters as "remote" nodes. This affects the order in which
+nodes are returned by the policy when selecting replicas for read or write
+operations. If no preferred datacenter is specified, the policy will treat all
+nodes as local nodes.
+
+When datacenter failover is disabled (`permit_dc_failover` is set to
+false), the default policy will only include local nodes in load balancing
+plans. Remote nodes will be excluded, even if they are alive and available to
+serve requests.
+
+#### Datacenter Failover
+
+In the event of a datacenter outage or network failure, the nodes in that
+datacenter may become unavailable, and clients may no longer be able to access
+the data stored on those nodes. To address this, the `DefaultPolicy` supports datacenter
+failover, which allows to route requests to nodes in other datacenters if the
+local nodes are unavailable.
+
+Datacenter failover can be enabled in `DefaultPolicy` by `permit_dc_failover`
+setting in the builder. When this flag is set, the policy will prefer to return
+alive remote replicas if datacenter failover is permitted and possible due to
+consistency constraints.
+
+#### Token awareness
+
+Token awareness refers to a mechanism by which the driver is aware of the token
+range assigned to each node in the cluster. Tokens are assigned to nodes to
+partition the data and distribute it across the cluster.
+
+When a user wants to read or write data, the driver can use token awareness to
+route the request to the correct node based on the token range of the data
+being accessed. This can help to minimize network traffic and improve
+performance by ensuring that the data is accessed locally as much as possible.
+
+In the case of `DefaultPolicy`, token awareness is enabled by default, meaning
+that the policy will prefer to return alive local replicas if the token is
+available. This means that if the client is requesting data that falls within
+the token range of a particular node, the policy will try to route the request
+to that node first, assuming it is alive and responsive.
+
+Token awareness can significantly improve the performance and scalability of
+applications built on Scylla. By using token awareness, users can ensure that
+data is accessed locally as much as possible, reducing network overhead and
+improving throughput.
+
+Please note that for token awareness to be applied, a statement must be
+prepared before being executed.
+
+### Latency awareness
+
+Latency awareness is a mechanism that penalises nodes whose measured recent
+average latency classifies it as falling behind the others.
+
+Every `update_rate` the global minimum average latency is computed,
+and all nodes whose average latency is worse than `exclusion_threshold`
+times the global minimum average latency become penalised for
+`retry_period`. Penalisation involves putting those nodes at the very end
+of the query plan. As it is often not truly beneficial to prefer
+faster non-replica than replicas lagging behind the non-replicas,
+this mechanism may as well worsen latencies and/or throughput.
+
+> **Warning**
+>
+> Using latency awareness is **NOT** recommended, unless prior
+>benchmarks prove its beneficial impact on the specific workload's
+>performance. Use with caution.
+
+### Creating a latency aware DefaultPolicy
+
+```rust
+# extern crate scylla;
+# fn example() {
+use scylla::load_balancing::{
+    LatencyAwarenessBuilder, DefaultPolicy
+};
+use std::time::Duration;
+
+let latency_awareness_builder = LatencyAwarenessBuilder::new()
+    .exclusion_threshold(3.)
+    .update_rate(Duration::from_secs(3))
+    .retry_period(Duration::from_secs(30))
+    .minimum_measurements(200);
+
+let policy = DefaultPolicy::builder()
+        // Here further customisation is, of course, possible.
+        // e.g.: .prefer_datacenter(...)
+        .latency_awareness(latency_awareness_builder)
+        .build();
+# }
+```
+
+```rust
+# extern crate scylla;
+# fn test_if_compiles() {
+use scylla::load_balancing::DefaultPolicy;
+
+let default_policy = DefaultPolicy::builder()
+        .prefer_datacenter("dc1".to_string())
+        .token_aware(true)
+        .permit_dc_failover(true)
+        .build();
+# }
+```
+
+### Node order in produced plans
+
+The DefaultPolicy prefers to return nodes in the following order:
+
+1. Alive local replicas (if token is available & token awareness is enabled)
+2. Alive remote replicas (if datacenter failover is permitted & possible due to consistency constraints)
+3. Alive local nodes
+4. Alive remote nodes (if datacenter failover is permitted & possible due to consistency constraints)
+5. Enabled down nodes
+And only if latency awareness is enabled:
+6. Penalised: alive local replicas, alive remote replicas, ... (in order as above).
+
+If no preferred datacenter is specified, all nodes are treated as local ones.
+
+Replicas in the same priority groups are shuffled. Non-replicas are randomly
+rotated (similarly to a round robin with a random index).

docs/source/load-balancing/load-balancing.md

@@ -1 +1,124 @@
 # Load balancing
+
+## Introduction
+
+The driver uses a load balancing policy to determine which node(s) to contact
+when executing a query. Load balancing policies implement the
+`LoadBalancingPolicy` trait, which contains methods to generate a load
+balancing plan based on the query information and the state of the cluster.
+
+Load balancing policies do not influence to which nodes connections are
+being opened. For a node connection blacklist configuration refer to
+`scylla::transport::host_filter::HostFilter`, which can be set session-wide
+using `SessionBuilder::host_filter` method.
+
+## Plan
+
+When a query is prepared to be sent to the database, the load balancing policy
+constructs a load balancing plan. This plan is essentially a list of nodes to
+which the driver will try to send the query. The first elements of the plan are
+the nodes which are the best to contact (e.g. they might be replicas for the
+requested data or have the best latency).
+
+## Policy
+
+The Scylla/Cassandra driver provides a default load balancing policy (see
+[Default Policy](default-policy.md) for details), but you can
+also implement your own custom policies that better suit your specific use
+case. To use a custom policy, you simply need to implement the
+`LoadBalancingPolicy` trait and pass an instance of your custom policy to the
+used execution profile.
+
+Our recommendation is to use [`Default Policy`](default-policy.md) with token-
+awareness enabled and latency-awareness disabled.
+
+## Configuration
+
+Load balancing policies can be configured via execution profiles. In the code
+sample provided, a new execution profile is created using
+`ExecutionProfile::builder()`, and the load balancing policy is set to the
+`DefaultPolicy` using `.load_balancing_policy(policy)`.
+
+The newly created execution profile is then converted to a handle using
+`.into_handle()`, and passed as the default execution profile to the
+`SessionBuilder` using `.default_execution_profile_handle(handle)`.
+
+```rust
+# extern crate scylla;
+# use std::error::Error;
+# async fn check_only_compiles(uri: &str) -> Result<(), Box<dyn Error>> {
+use scylla::SessionBuilder;
+use scylla::load_balancing::DefaultPolicy;
+use scylla::transport::ExecutionProfile;
+use scylla::transport::session::Session;
+use std::sync::Arc;
+
+let policy = Arc::new(DefaultPolicy::default());
+
+let profile = ExecutionProfile::builder()
+    .load_balancing_policy(policy)
+    .build();
+let handle = profile.into_handle();
+
+let session: Session = SessionBuilder::new()
+    .known_node(&uri)
+    .default_execution_profile_handle(handle)
+    .build()
+    .await?;
+# return Ok(())
+# }
+```
+
+In addition to being able to configure load balancing policies through
+execution profiles at the session level, the driver also allow for setting
+execution profile handles on a per-query basis. This means that for each query,
+a specific execution profile can be selected with a customized load balancing
+settings.
+
+## `LoadBalancingPolicy` trait
+
+### `pick` and `fallback`:
+
+Most queries are sent successfully on the first try. In such cases, only the
+first element of the load balancing plan is needed, so it's usually unnecessary
+to compute entire load balancing plan. To optimize this common case, the
+`LoadBalancingPolicy` trait provides two methods: `pick` and `fallback`.
+
+`pick` returns the first node to contact for a given query, which is usually
+the best based on a particular load balancing policy. If `pick` returns `None`,
+then `fallback` will not be called.
+
+`fallback`, returns an iterator that provides the rest of the nodes in the load
+balancing plan. `fallback` is called only when using the initial picked node
+fails (or when executing speculatively).
+
+It's possible for the `fallback` method to include the same node that was
+returned by the `pick` method. In such cases, the query execution layer filters
+out the picked node from the iterator returned by `fallback`.
+
+### `on_query_success` and `on_query_failure`:
+
+The `on_query_success` and `on_query_failure` methods are useful for load
+balancing policies because they provide feedback on the performance and health
+of the nodes in the cluster.
+
+When a query is successfully executed, the `on_query_success` method is called
+and can be used by the load balancing policy to update its internal state. For
+example, a policy might use the latency of the successful query to update its
+latency statistics for each node in the cluster. This information can be used
+to make decisions about which nodes to contact in the future.
+
+On the other hand, when a query fails to execute, the `on_query_failure` method
+is called and provides information about the failure. The error message
+returned by Cassandra can help determine the cause of the failure, such as a
+node being down or overloaded. The load balancing policy can use this
+information to update its internal state and avoid contacting the same node
+again until it's recovered.
+
+```eval_rst
+.. toctree::
+   :hidden:
+   :glob:
+
+   default-policy
+```