Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transaction pool #299

Merged
merged 46 commits into from
Jun 27, 2024
Merged

Transaction pool #299

merged 46 commits into from
Jun 27, 2024

Conversation

sideninja
Copy link
Contributor

@sideninja sideninja commented Jun 12, 2024

Closes: #293

Description

This PR implements the first version of a transaction pool. Current functionality of the pool is to make sure transactions that are not yet sealed are kept in the pool (will be used to inspect the pool through the API - part of another PR), and to make sure the status is correctly reported to the client in case of invalid transactions as well as valid transactions.

This will also allow for pending transaction stream to be connected to the pool events but that requires some changes in the patterns of subscribers and publishers used from flow-go.


For contributor use:

  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the standards mentioned here.
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

Summary by CodeRabbit

  • New Features

    • Introduced TxPool for improved transaction management and error handling.
    • Added a new error type ErrInvalidEVMTransaction to handle invalid Ethereum transactions.
    • Updated transaction handling logic in the system for better efficiency.
  • Bug Fixes

    • Enhanced error handling for gas price issues and invalid EVM transactions.
  • Improvements

    • Updated Go version to 1.22 and dependencies to newer versions for better performance and security.
    • Improved handling of concurrent transaction submissions.
  • Tests

    • Added new test cases for handling Ethereum transaction failures and improved e2e tests.

@sideninja sideninja self-assigned this Jun 12, 2024
@sideninja
Copy link
Contributor Author

Waiting for flow-go tools update

Copy link
Contributor

coderabbitai bot commented Jun 12, 2024

Walkthrough

The updates encompass improved error handling for gas prices and invalid EVM transactions, the introduction of a transaction pool to manage pending transactions, and updated dependencies and test cases. Notable adjustments include renaming ErrGasPriceTooLow to GasPriceTooLowError, adding the ErrInvalidEVMTransaction error, and implementing a TxPool for efficient transaction management. Enhancements span files like api.go, errors.go, requester/pool.go, and requester/requester.go.

Changes

Files/Paths Change Summary
api/api.go Enhanced SendRawTransaction to handle specific gas price and invalid EVM transaction errors.
api/errors/errors.go, models/errors.go Renamed ErrGasPriceTooLow to GasPriceTooLowError; added ErrInvalidEVMTransaction error.
go.mod, tests/go.mod Updated Go version to 1.22 and various dependencies to newer versions.
requester/pool.go Introduced TxPool struct for handling and tracking transaction statuses and pooling strategies.
requester/requester.go Added txPool to EVM struct and SendRawTransaction method. Introduced buildTransaction function.
tests/e2e_web3js_test.go Added a new test case for Ethereum transaction failure handling.
tests/web3js/eth_failure_handling_test.js New test file to handle and simulate Ethereum transfer failures.
tests/integration_test.go, tests/helpers.go Improved concurrent transaction handling and added debug prints for raw transactions.

Sequence Diagrams

Transaction Submission Flow

sequenceDiagram
    participant Client
    participant API
    participant TxPool
    participant Blockchain

    Client->>API: SendRawTransaction(data)
    API->>TxPool: Add Transaction to Pool
    TxPool->>Blockchain: Send Transaction
    Blockchain-->>TxPool: Execution Result
    TxPool-->>API: Transaction Status
    API-->>Client: Response with Transaction Hash
Loading

Error Handling Flow

sequenceDiagram
    participant Client
    participant API
    participant TxPool
    participant Blockchain

    Client->>API: SendRawTransaction(data)
    API->>TxPool: Add Transaction to Pool
    TxPool->>Blockchain: Send Transaction
    Blockchain-->>TxPool: Error (e.g., GasPriceTooLowError)
    TxPool-->>API: Error Response
    API-->>Client: Error Details
Loading

Assessment against linked issues

Objective Addressed Explanation
Implement transaction pool for pending transactions (#293)
Handle invalid transaction results correctly (#293)
Expose transaction pool inspection APIs (#293) No specific mention of inspection API exposure in changes.

Poem

In the land where bytecodes hop,
Transactions never halt nor stop.
A pool was built to hold the queue,
With careful checks, it maneuvers through.
Errors met with grace and care,
Ensuring that the flow is fair.
Progress made in every line,
Oh, how the bits and bytes entwine. 😊

📝 Happy coding and may your errors be ever few! 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Outside diff range and nitpick comments (1)
api/api.go (1)

Line range hint 688-688: Consider refactoring the nested if blocks in the getCadenceHeight method to reduce complexity.

- if number, ok := blockNumberOrHash.Number(); ok {
-     if number < 0 {
-         // negative values are special values and we only support latest height
-         return height, nil
-     }
-     height, err := b.blocks.GetCadenceHeight(uint64(number.Int64()))
-     if err != nil {
-         b.logger.Error().Err(err).Msg("failed to get cadence height")
-         return 0, err
-     }
-     return height, nil
- } else if hash, ok := blockNumberOrHash.Hash(); ok {
-     evmHeight, err := b.blocks.GetHeightByID(hash)
-     if err != nil {
-         b.logger.Error().Err(err).Msg("failed to get block by hash")
-         return 0, err
-     }
-     height, err = b.blocks.GetCadenceHeight(evmHeight)
-     if err != nil {
-         b.logger.Error().Err(err).Msg("failed to get cadence height")
-         return 0, err
-     }
-     return height, nil
- }
+ if number, ok := blockNumberOrHash.Number(); ok && number >= 0 {
+     return b.handlePositiveNumber(number)
+ } else if hash, ok := blockNumberOrHash.Hash(); ok {
+     return b.handleHash(hash)
+ }
+ return 0, fmt.Errorf("invalid arguments; neither block nor hash specified")

Extracting the logic into separate methods handlePositiveNumber and handleHash can significantly simplify the main flow in getCadenceHeight, making it easier to understand and maintain.

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 573c763 and 73fe6e8.

Files ignored due to path filters (2)
  • go.sum is excluded by !**/*.sum
  • tests/go.sum is excluded by !**/*.sum
Files selected for processing (11)
  • api/api.go (1 hunks)
  • api/errors/errors.go (1 hunks)
  • go.mod (2 hunks)
  • models/errors.go (1 hunks)
  • services/requester/cadence/run.cdc (1 hunks)
  • services/requester/pool.go (1 hunks)
  • services/requester/pool_test.go (1 hunks)
  • services/requester/requester.go (7 hunks)
  • tests/e2e_web3js_test.go (1 hunks)
  • tests/go.mod (2 hunks)
  • tests/web3js/eth_failure_handling_test.js (1 hunks)
Files skipped from review due to trivial changes (4)
  • api/errors/errors.go
  • go.mod
  • services/requester/cadence/run.cdc
  • tests/e2e_web3js_test.go
Additional context used
golangci-lint
api/api.go

688-688: if ok has complex nested blocks (complexity: 5) (nestif)

Additional comments not posted (9)
services/requester/pool_test.go (1)

10-14: The test function Test_RunTemplateDivider correctly checks for the presence of the evmErrorDivider in the runTxScript. Good use of string containment to validate the script's content.

tests/web3js/eth_failure_handling_test.js (1)

5-20: The test function transfer failure due to incorrect nonce effectively simulates a transfer with an incorrect nonce and checks for error handling. This is crucial for ensuring robustness in transaction management.

models/errors.go (1)

9-10: The addition of ErrInvalidEVMTransaction is a good enhancement to the error handling capabilities, allowing for more specific error management in EVM transactions.

services/requester/pool.go (1)

43-115: The Send method in TxPool effectively manages the transaction lifecycle, from submission to sealing, with robust error handling. The use of a ticker and timeout for polling transaction status is a good approach. Ensure thorough testing of the error handling paths to maintain reliability.

tests/go.mod (2)

3-3: Updated Go version to 1.22.

This aligns with the latest stable release and ensures compatibility with new language features and improvements.


9-9: Updated dependencies to newer versions.

This is crucial for maintaining the security and performance of the application by leveraging the latest fixes and features from these libraries.

Also applies to: 13-13

services/requester/requester.go (3)

94-94: Added txPool to the EVM struct.

This is essential for managing transactions efficiently, aligning with the PR's goal to implement a transaction pool.


131-131: Initialization of txPool in the NewEVM constructor.

Proper initialization is crucial for ensuring that the transaction pool is ready to be used when the EVM instance is created.


183-184: Error handling in transaction sending.

Proper error handling ensures that issues during the transaction sending process are caught and logged, maintaining robustness.

@@ -149,20 +156,32 @@ func (e *EVM) SendRawTransaction(ctx context.Context, data []byte) (common.Hash,
return common.Hash{}, err
}

// todo do further validation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

The transaction validation logic is not yet implemented in the SendRawTransaction method.

  • Location: services/requester/requester.go at line 159
Analysis chain

Placeholder for transaction validation logic.

Ensure that the validation logic is implemented before merging, as this is critical for the security and integrity of transactions.

Scripts executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Check if transaction validation logic is implemented.

# Test: Search for the validation logic in the EVM implementation.
ast-grep --lang go --pattern $'func (e *EVM) SendRawTransaction($_, $_) $_ {
  $$$
  // todo do further validation
  $$$
}'

Length of output: 3379

Comment on lines 22 to 33
// todo this is a simple implementation of the transaction pool that is mostly used
// to track the status of submitted transaction, but transactions will always be submitted
// right away, future improvements can make it so the transactions are collected in the pool
// and after submitted based on different strategies.

type TxPool struct {
logger zerolog.Logger
client *CrossSporkClient
pool map[gethCommon.Hash]*gethTypes.Transaction
// todo add a broadcaster for pending transaction streaming
// todo add methods to inspect transaction pool state
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initial implementation of TxPool is simple and functional. Consider addressing the TODOs regarding the addition of a broadcaster for pending transaction streaming and methods to inspect the transaction pool state in future iterations.

Would you like me to help with implementing the broadcaster or inspection methods in the future?

api/api.go Outdated
Comment on lines 138 to 147

// handle specific typed errors
var errGasPriceTooLow *errs.GasPriceTooLowError
if errors.As(err, &errGasPriceTooLow) {
return common.Hash{}, errGasPriceTooLow
}
if errors.Is(err, models.ErrInvalidEVMTransaction) {
return common.Hash{}, err
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refine error handling in SendRawTransaction to improve readability and maintainability.

- var errGasPriceTooLow *errs.GasPriceTooLowError
- if errors.As(err, &errGasPriceTooLow) {
-     return common.Hash{}, errGasPriceTooLow
- }
- if errors.Is(err, models.ErrInvalidEVMTransaction) {
-     return common.Hash{}, err
- }
+ switch {
+ case errors.As(err, &errGasPriceTooLow):
+     return common.Hash{}, errGasPriceTooLow
+ case errors.Is(err, models.ErrInvalidEVMTransaction):
+     return common.Hash{}, err
+ }

This change uses a switch statement to handle different types of errors, which is more scalable and easier to read, especially if more error types are to be handled in the future.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// handle specific typed errors
var errGasPriceTooLow *errs.GasPriceTooLowError
if errors.As(err, &errGasPriceTooLow) {
return common.Hash{}, errGasPriceTooLow
}
if errors.Is(err, models.ErrInvalidEVMTransaction) {
return common.Hash{}, err
}
// handle specific typed errors
var errGasPriceTooLow *errs.GasPriceTooLowError
switch {
case errors.As(err, &errGasPriceTooLow):
return common.Hash{}, errGasPriceTooLow
case errors.Is(err, models.ErrInvalidEVMTransaction):
return common.Hash{}, err
}

# Conflicts:
#	go.mod
#	go.sum
#	services/requester/requester.go
#	tests/e2e_web3js_test.go
#	tests/go.mod
#	tests/go.sum
// the run.cdc script panics with the evm specific error as the message which we
// extract and return to the client. Any error returned that is evm specific
// is a validation error due to assert statement in the run.cdc script.
func parseInvalidError(errorMessage string) (error, bool) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we can do without this parseInvalidError, if we make the change from this comment. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't completely because the error will contain Cadence error as well

for {
select {
case <-ticker.C:
res, err := t.client.GetTransactionResult(ctx, flowTx.ID())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's very nice 👏 This will allow us to better handle the keys from the rotation signers.

Copy link
Collaborator

@m-Peter m-Peter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall 👏
Just a comment about how we handle the evm error from EVM.run.

# Conflicts:
#	services/requester/requester.go
#	tests/e2e_web3js_test.go
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Outside diff range and nitpick comments (2)
tests/helpers.go (2)

Line range hint 51-51: Potential Security Issue: Exposed Private Key

The servicePrivateKey is hardcoded, which is a security risk. It should be retrieved securely, such as from an environment variable or a secure vault.

- servicePrivateKey = "61ceacbdce419e25ee8e7c2beceee170a05c9cab1e725a955b15ba94dcd747d2"
+ servicePrivateKey = os.Getenv("SERVICE_PRIVATE_KEY")

Line range hint 54-54: Potential Security Issue: Exposed Private Key

Similar to the previous comment, eoaTestPrivateKey is also hardcoded. This should also be retrieved in a secure manner.

- eoaTestPrivateKey = "f6d5333177711e562cabf1f311916196ee6ffc2a07966d9d4628094073bd5442"
+ eoaTestPrivateKey = os.Getenv("EOA_TEST_PRIVATE_KEY")
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1dd8a2f and 25d502f.

Files selected for processing (4)
  • api/api.go (1 hunks)
  • services/requester/requester.go (7 hunks)
  • tests/e2e_web3js_test.go (1 hunks)
  • tests/helpers.go (1 hunks)
Files skipped from review due to trivial changes (1)
  • tests/e2e_web3js_test.go
Additional context used
Gitleaks
tests/helpers.go

51-51: Detected a Generic API Key, potentially exposing access to various services and sensitive operations. (generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations. (generic-api-key)

Additional comments not posted (3)
services/requester/requester.go (2)

93-93: Initialization of txPool

The addition of txPool to the EVM struct is consistent with the PR objectives to handle transaction pooling. Ensure that the TxPool is properly integrated and tested.


133-133: Initialization of txPool in Constructor

Good integration of the txPool in the NewEVM constructor, aligning with the system design to manage transactions effectively.

api/api.go (1)

184-193: Refactor error handling in SendRawTransaction for improved readability and maintainability.

The current implementation uses a switch statement for error handling which is a good practice for scalability. However, the previous comments suggest refining this further. Since the comments are duplicate and still valid, no additional changes are needed here.

tests/helpers.go Outdated
@@ -409,6 +409,7 @@ func (r *rpcTest) getReceipt(hash string) (*types.Receipt, error) {

func (r *rpcTest) sendRawTx(signed []byte) (common.Hash, error) {
rpcRes, err := r.request("eth_sendRawTransaction", fmt.Sprintf(`["0x%x"]`, signed))
fmt.Println("###", string(rpcRes), "err", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sensitive Information Logging

The print statement logs potentially sensitive information which could be a security risk. Consider removing or sanitizing these logs.

- fmt.Println("###", string(rpcRes), "err", err)
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
fmt.Println("###", string(rpcRes), "err", err)

Comment on lines 139 to 149
tx, err := evm.buildTransaction(
context.Background(),
evm.replaceAddresses(createCOAScript),
cadence.UFix64(coaFundingBalance),
)
logger.Warn().Err(err).Str("id", id.String()).Msg("COA resource auto-creation status")
if err != nil {
logger.Warn().Err(err).Msg("COA resource auto-creation failure")
}
if err := evm.client.SendTransaction(context.Background(), *tx); err != nil {
logger.Warn().Err(err).Msg("COA resource auto-creation failure")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handling COA Resource Auto-Creation

This section attempts to handle the creation of a COA resource but ignores errors. It's crucial to handle these errors explicitly to avoid silent failures.

-		// we ignore errors for now since creation of already existing COA resource will fail, which is fine for now
+		if err != nil && !errors.IsAlreadyExists(err) {
+			return nil, fmt.Errorf("failed to auto-create COA resource: %w", err)
+		}

Committable suggestion was skipped due to low confidence.

Comment on lines +161 to +166
// todo do further validation

from, err := types.Sender(types.LatestSignerForChainID(tx.ChainId()), tx)
if err != nil {
return common.Hash{}, fmt.Errorf("failed to derive the sender: %w", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Transaction Validation Placeholder

The TODO comment indicates that further validation is needed. Ensure this validation is implemented to enhance the security and integrity of transactions before merging this PR.

  • The TODO comment is still present in the SendRawTransaction method at line 161.
  • Although there are some validation checks, such as gas price validation, additional comprehensive validation logic is necessary.
Analysis chain

Transaction Validation Placeholder

The TODO comment indicates that further validation is needed. Ensure this validation is implemented to enhance the security and integrity of transactions before merging this PR.

Scripts executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Check if transaction validation logic is implemented.

# Test: Search for the validation logic in the EVM implementation.
ast-grep --lang go --pattern $'func (e *EVM) SendRawTransaction($_, $_) $_ {
  $$$
  // todo do further validation
  $$$
}'

Length of output: 3257

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 25d502f and 5d866aa.

Files selected for processing (4)
  • api/api.go (1 hunks)
  • services/requester/cadence/run.cdc (1 hunks)
  • services/requester/pool.go (1 hunks)
  • tests/e2e_web3js_test.go (1 hunks)
Files skipped from review as they are similar to previous changes (2)
  • api/api.go
  • tests/e2e_web3js_test.go

@@ -16,7 +16,7 @@ transaction(hexEncodedTx: String) {
)
assert(
txResult.status == EVM.Status.failed || txResult.status == EVM.Status.successful,
message: "failed to execute evm transaction: ".concat(txResult.errorCode.toString())
message: "evm_error=".concat(txResult.errorMessage).concat("\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a more human-friendly error message field.

As suggested by m-Peter in the previous comments, using a more human-friendly errorMessage field could improve the user experience, especially when errors occur. This change was agreed upon by sideninja and should be implemented before merging.

Would you like me to help implement this change?

Comment on lines +19 to +22
// todo this is a simple implementation of the transaction pool that is mostly used
// to track the status of submitted transaction, but transactions will always be submitted
// right away, future improvements can make it so the transactions are collected in the pool
// and after submitted based on different strategies.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document future improvements for transaction collection strategies.

The TODO comments suggest future improvements for collecting transactions in the pool based on different strategies. It's crucial to document these strategies in more detail or create issues to track their implementation.

Would you like me to help with documenting these strategies or implementing them in a future iteration?

Comment on lines +24 to +30
type TxPool struct {
logger zerolog.Logger
client *CrossSporkClient
pool *sync.Map
// todo add a broadcaster for pending transaction streaming
// todo add methods to inspect transaction pool state
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider implementing the broadcaster and state inspection methods soon.

The TODO comments mention the need for a broadcaster for pending transaction streaming and methods to inspect the transaction pool state. These features are essential for full functionality and should be prioritized in future updates.

Would you like me to help with implementing these features?

Comment on lines +102 to +114
// this will extract the evm specific error from the Flow transaction error message
// the run.cdc script panics with the evm specific error as the message which we
// extract and return to the client. Any error returned that is evm specific
// is a validation error due to assert statement in the run.cdc script.
func parseInvalidError(err error) (error, bool) {
r := regexp.MustCompile(evmErrorRegex)
matches := r.FindStringSubmatch(err.Error())
if len(matches) != 2 {
return nil, false
}

return fmt.Errorf("%w: %s", models.ErrInvalidEVMTransaction, matches[1]), true
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optimize error parsing function by pre-compiling regex.

The parseInvalidError function compiles the regex pattern every time it is called. This can be optimized by compiling the regex pattern once at a global level and reusing it across function calls.

- r := regexp.MustCompile(evmErrorRegex)
+ var compiledRegex = regexp.MustCompile(evmErrorRegex)
+ r := compiledRegex

Committable suggestion was skipped due to low confidence.

Comment on lines 40 to 100
// Send flow transaction that executes EVM run function which takes in the encoded EVM transaction.
// The flow transaction status is awaited and an error is returned in case of a failure in submission,
// or an EVM validation error.
// Until the flow transaction is sealed the transaction will stay in the transaction pool marked as pending.
func (t *TxPool) Send(
ctx context.Context,
flowTx *flow.Transaction,
evmTx *gethTypes.Transaction,
) error {
if err := t.client.SendTransaction(ctx, *flowTx); err != nil {
return err
}

// add to pool
t.pool.Store(evmTx.Hash(), evmTx)

const fetchInterval = time.Millisecond * 500
const fetchTimeout = time.Minute * 3
ticker := time.NewTicker(fetchInterval)
timeout := time.NewTimer(fetchTimeout)

defer func() {
t.pool.Delete(evmTx.Hash())
timeout.Stop()
ticker.Stop()
}()

for {
select {
case <-ticker.C:
res, err := t.client.GetTransactionResult(ctx, flowTx.ID())
if err != nil {
return fmt.Errorf("failed to retrieve flow transaction result %s: %w", flowTx.ID(), err)
}
// wait until transaction is sealed
if res.Status < flow.TransactionStatusSealed {
continue
}

if res.Error != nil {
t.logger.Error().Err(res.Error).
Str("flow-id", flowTx.ID().String()).
Str("evm-id", evmTx.Hash().Hex()).
Msg("flow transaction error")

if err, ok := parseInvalidError(res.Error); ok {
return err
}

// hide specific cause since it's an implementation issue
return fmt.Errorf("failed to submit flow evm transaction %s", evmTx.Hash())
}

return nil
case <-timeout.C:
err := fmt.Errorf("failed to retrieve evm transaction result: %s, flow id: %s", evmTx.Hash(), flowTx.ID())
t.logger.Error().Err(err).Msg("failed to get result")
return err
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review the use of regex for error parsing in high-frequency methods.

The implementation of Send uses regex to parse errors, which might not be the most efficient method in a function that could be called frequently. Consider optimizing this by compiling the regex pattern once and reusing it.

- r := regexp.MustCompile(evmErrorRegex)
+ var compiledRegex = regexp.MustCompile(evmErrorRegex)
+ r := compiledRegex

Committable suggestion was skipped due to low confidence.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (2)
tests/helpers.go (2)

Line range hint 51-51: Potential Exposure of Sensitive Information

The servicePrivateKey variable might contain sensitive information and should not be hardcoded. Consider using environment variables or secure vault solutions to manage private keys.

- servicePrivateKey = "61ceacbdce419e25ee8e7c2beceee170a05c9cab1e725a955b15ba94dcd747d2"
+ servicePrivateKey = os.Getenv("SERVICE_PRIVATE_KEY")

Line range hint 54-54: Potential Exposure of Sensitive Information

The eoaTestPrivateKey variable might contain sensitive information and should not be hardcoded. Consider using environment variables or secure vault solutions to manage private keys.

- eoaTestPrivateKey = "f6d5333177711e562cabf1f311916196ee6ffc2a07966d9d4628094073bd5442"
+ eoaTestPrivateKey = os.Getenv("EOA_TEST_PRIVATE_KEY")
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5d866aa and 2575792.

Files selected for processing (1)
  • tests/helpers.go (1 hunks)
Additional context used
Gitleaks
tests/helpers.go

51-51: Detected a Generic API Key, potentially exposing access to various services and sensitive operations. (generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations. (generic-api-key)

logger.Warn().Err(err).Msg("COA resource auto-creation failure")
}
if err := evm.client.SendTransaction(context.Background(), *tx); err != nil {
logger.Warn().Err(err).Msg("COA resource auto-creation failure")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.Warn().Err(err).Msg("COA resource auto-creation failure")
logger.Warn().Err(err).Msg("error while send transaction to the network")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to keep context in which transaction failed: COA creation, because otherwise we won't know in logs what failed. I can update the message to be more clear it was sending COA transaction tho.

Copy link
Collaborator

@m-Peter m-Peter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 💯 Just a minor suggestion on an error message

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2575792 and bddcb6b.

Files selected for processing (5)
  • api/api.go (1 hunks)
  • services/requester/pool.go (1 hunks)
  • services/requester/requester.go (7 hunks)
  • tests/e2e_web3js_test.go (1 hunks)
  • tests/integration_test.go (1 hunks)
Files skipped from review as they are similar to previous changes (4)
  • api/api.go
  • services/requester/pool.go
  • services/requester/requester.go
  • tests/e2e_web3js_test.go
Additional comments not posted (1)
tests/integration_test.go (1)

120-129: Ensure transaction execution verification is robust.

The use of assert.Eventually to verify that all transactions have been executed correctly is appropriate. However, the loop inside the assertion could be optimized by breaking early if any transaction fails, rather than checking all transactions before returning a result. This would make the test fail faster in case of an error.
[REFACTOR_SUGGESTion]

-    for _, h := range hashes {
-        rcp, err := rpcTester.getReceipt(h.String())
-        if err != nil || rcp == nil || uint64(1) != rcp.Status {
-            return false
-        }
-    }
-    return true
+    for _, h := range hashes {
+        rcp, err := rpcTester.getReceipt(h.String())
+        if err != nil || rcp == nil || uint64(1) != rcp.Status {
+            return false
+        }
+    }
+    return true

Comment on lines +105 to +112
// send raw transaction waits for transaction result and blocks, but since we don't have the result
// available until block is committed bellow we must continue without waiting, we will get all the
// transaction receipts later to confirm transactions have been successful, we only add a bit of delay
// to ensure order of transactions was correct, because there is no other way to proceed once transaction
// is submitted over network
go rpcTester.sendRawTx(signed)
time.Sleep(50 * time.Millisecond)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarification and potential optimization in transaction handling.

The added comments clarify the non-blocking nature of transaction submissions in the test, which is essential for understanding the test's concurrency model. However, consider using a more controlled approach for managing transaction order and delays, such as using a semaphore or a waitgroup, to ensure more deterministic behavior in concurrent environments.

@sideninja sideninja merged commit d69a16c into main Jun 27, 2024
2 checks passed
@m-Peter m-Peter deleted the gregor/tx-sender-pool branch July 29, 2024 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

Transaction pool
2 participants