Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid long heavy tasks in the GraphQL service #2340

Merged
merged 37 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2fe7bb6
Prepare the GraphQL service for the switching to `async` methods
xgreenx Oct 11, 2024
ab5e940
Updated CHANGELOG.md
xgreenx Oct 11, 2024
1782684
Updated all pagination queries to work with the `Stream` instead of `…
xgreenx Oct 12, 2024
5192c95
Making change non-breaking for now
xgreenx Oct 12, 2024
feeb816
Updated CHANGELOG.md
xgreenx Oct 12, 2024
4237feb
Avoid long heavy tasks in the GraphQL service
xgreenx Oct 13, 2024
142cd1d
Use correct naming
xgreenx Oct 13, 2024
671e80e
Updated CHANGELOG.md
xgreenx Oct 13, 2024
2354d10
Make closure more readable
xgreenx Oct 13, 2024
d982fbc
Apply comment from parent PR
xgreenx Oct 14, 2024
c355cd1
Use more clear naming for the logic
xgreenx Oct 14, 2024
7846db4
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
5a752bc
Fix flakiness
xgreenx Oct 14, 2024
a8c1942
Apply suggestions from code review
xgreenx Oct 14, 2024
df80801
Apply comments from PR
xgreenx Oct 14, 2024
e414046
Merge branch 'master' into feature/prepare-graphql-for-async
xgreenx Oct 14, 2024
3d1e2d8
Merge branch 'refs/heads/feature/async-pagination-queries' into featu…
xgreenx Oct 14, 2024
8cb24b0
Add link to issue for TODO
xgreenx Oct 14, 2024
74bf964
Merge branch 'master' into feature/prepare-graphql-for-async
xgreenx Oct 14, 2024
3868d45
Merge branch 'feature/prepare-graphql-for-async' into feature/async-p…
xgreenx Oct 14, 2024
678e0be
Use `futures::TryStreamExt`
xgreenx Oct 14, 2024
314cd37
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
727b2da
Implemented `yield_each` for the stream allowing other tasks to proceed
xgreenx Oct 14, 2024
d5d7934
Merge branch 'refs/heads/master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
73adec0
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
5e0ec92
Make CI happy
xgreenx Oct 14, 2024
981b690
Merge branch 'master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
3521a72
Merge with `master`
xgreenx Oct 14, 2024
946f46c
Merge branch 'refs/heads/feature/async-pagination-queries' into featu…
xgreenx Oct 14, 2024
c4ad916
Merge branch 'master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
2866da1
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
d8e159c
Merge branch 'master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
c61436e
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
71d3692
Merge branch 'master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
35c7184
Merge branch 'master' into feature/async-pagination-queries
xgreenx Oct 14, 2024
d8c8c5e
Merge branch 'feature/async-pagination-queries' into feature/avoid-lo…
xgreenx Oct 14, 2024
866f538
Merge branch 'refs/heads/master' into feature/avoid-long-heavy-tasks
xgreenx Oct 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

### Changed

- [2334](https://github.com/FuelLabs/fuel-core/pull/2334): Prepare the GraphQL service for the switching to `async` methods.
- [2337](https://github.com/FuelLabs/fuel-core/pull/2337): Updated all pagination queries to work with the async stream instead of the sync iterator.
- [2340](https://github.com/FuelLabs/fuel-core/pull/2340): Avoid long heavy tasks in the GraphQL service by splitting work into batches.

#### Breaking
- [2337](https://github.com/FuelLabs/fuel-core/pull/2337): The maximum number of processed coins from the `coins_to_spend` query is limited to `max_inputs`.

## [Version 0.39.0]

### Added
Expand Down
1 change: 1 addition & 0 deletions benches/benches/transaction_throughput.rs
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ where
test_builder.trigger = Trigger::Never;
test_builder.utxo_validation = true;
test_builder.gas_limit = Some(10_000_000_000);
test_builder.block_size_limit = Some(1_000_000_000_000);

// spin up node
let transactions: Vec<Transaction> =
Expand Down
1 change: 1 addition & 0 deletions bin/fuel-core/src/cli/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,7 @@ impl Command {
let config = Config {
graphql_config: GraphQLConfig {
addr,
database_batch_size: graphql.database_batch_size,
max_queries_depth: graphql.graphql_max_depth,
max_queries_complexity: graphql.graphql_max_complexity,
max_queries_recursive_depth: graphql.graphql_max_recursive_depth,
Expand Down
4 changes: 4 additions & 0 deletions bin/fuel-core/src/cli/run/graphql.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ pub struct GraphQLArgs {
#[clap(long = "port", default_value = "4000", env)]
pub port: u16,

/// The size of the batch fetched from the database by GraphQL service.
#[clap(long = "graphql-database-batch-size", default_value = "100", env)]
pub database_batch_size: usize,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this have to be configurable? can we just set it to a global now to reduce the config surface? we have too many options rn :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to make it configurable for now to be able to adjust our performance on the fly. Later we can remove it when we have better ideas about the best value


/// The max depth of GraphQL queries.
#[clap(long = "graphql-max-depth", default_value = "16", env)]
pub graphql_max_depth: usize,
Expand Down
2 changes: 1 addition & 1 deletion crates/client/assets/schema.sdl
Original file line number Diff line number Diff line change
Expand Up @@ -938,7 +938,7 @@ type Query {
"""
owner: Address!,
"""
The list of requested assets` coins with asset ids, `target` amount the user wants to reach, and the `max` number of coins in the selection. Several entries with the same asset id are not allowed.
The list of requested assets` coins with asset ids, `target` amount the user wants to reach, and the `max` number of coins in the selection. Several entries with the same asset id are not allowed. The result can't contain more coins than `max_inputs`.
"""
queryPerAsset: [SpendQueryElementInput!]!,
"""
Expand Down
Loading
Loading