Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graph: Make the ES index we log to configurable #5210

Merged
merged 1 commit into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

- The ElasticSearch index to which `graph-node` logs can now be configured
with the `GRAPH_ELASTIC_SEARCH_INDEX` environment variable which defaults
to `subgraph`

## v0.34.0
### What's New

Expand All @@ -23,7 +27,7 @@

- Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block. [(#5112)](https://github.com/graphprotocol/graph-node/pull/5112)
- Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality. [(#5013)](https://github.com/graphprotocol/graph-node/pull/5013)
- Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error `assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block()`. Should that happen, the [migration script](https://github.com/graphprotocol/graph-node/blob/master/store/postgres/migrations/2024-01-05-170000_ds_corruption_fix_up/up.sql) should be rerun against the affected shard. [(#5083)](https://github.com/graphprotocol/graph-node/pull/5083)
- Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error `assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block()`. Should that happen, the [migration script](https://github.com/graphprotocol/graph-node/blob/master/store/postgres/migrations/2024-01-05-170000_ds_corruption_fix_up/up.sql) should be rerun against the affected shard. [(#5083)](https://github.com/graphprotocol/graph-node/pull/5083)
- Increased the base backoff time for RPC, enhancing stability and reliability under load. [(#4984)](https://github.com/graphprotocol/graph-node/pull/4984)
- Resolved an issue related to spawning offchain data sources from existing offchain data source mappings. [(#5051)](https://github.com/graphprotocol/graph-node/pull/5051)[(#5092)](https://github.com/graphprotocol/graph-node/pull/5092)
- Resolved an issue where eth-call results for reverted calls were being cached in call cache. [(#4879)](https://github.com/graphprotocol/graph-node/pull/4879)
Expand Down
6 changes: 6 additions & 0 deletions graph/src/env/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ pub struct EnvVars {
/// Set by the environment variable
/// `GRAPH_ELASTIC_SEARCH_MAX_RETRIES`. The default value is 5.
pub elastic_search_max_retries: usize,
/// The name of the index in ElasticSearch to which we should log. Set
/// by `GRAPH_ELASTIC_SEARCH_INDEX`. The default is `subgraph`.
pub elastic_search_index: String,
/// If an instrumented lock is contended for longer than the specified
/// duration, a warning will be logged.
///
Expand Down Expand Up @@ -224,6 +227,7 @@ impl EnvVars {
inner.elastic_search_flush_interval_in_secs,
),
elastic_search_max_retries: inner.elastic_search_max_retries,
elastic_search_index: inner.elastic_search_index,
lock_contention_log_threshold: Duration::from_millis(
inner.lock_contention_log_threshold_in_ms,
),
Expand Down Expand Up @@ -326,6 +330,8 @@ struct Inner {
elastic_search_flush_interval_in_secs: u64,
#[envconfig(from = "GRAPH_ELASTIC_SEARCH_MAX_RETRIES", default = "5")]
elastic_search_max_retries: usize,
#[envconfig(from = "GRAPH_ELASTIC_SEARCH_INDEX", default = "subgraph")]
elastic_search_index: String,
#[envconfig(from = "GRAPH_LOCK_CONTENTION_LOG_THRESHOLD_MS", default = "100")]
lock_contention_log_threshold_in_ms: u64,

Expand Down
2 changes: 1 addition & 1 deletion graph/src/log/factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ impl LoggerFactory {
elastic_logger(
ElasticDrainConfig {
general: elastic_config,
index: String::from("subgraph-logs"),
index: ENV_VARS.elastic_search_index.clone(),
custom_id_key: String::from("subgraphId"),
custom_id_value: loc.hash.to_string(),
flush_interval: ENV_VARS.elastic_search_flush_interval,
Expand Down
Loading