Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][Connector-V2][ElasticSearch] Improve es bulk sink retriable mechanism #3148

Merged
merged 1 commit into from
Nov 21, 2022

Conversation

harveyyue
Copy link
Contributor

@harveyyue harveyyue commented Oct 20, 2022

These code change as bleow:

  1. Improve the retriable mechanism of es bulk sink
  2. Introduce a method computeRetryWaitTimeMillis to compute the wait time
  3. Fix the attempt times bug within method retryWithException

Purpose of this pull request

Check list

public class RetryUtils {
private static final Logger LOG = LoggerFactory.getLogger(RetryUtils.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

return null;
}, retryMaterial);
} catch (Exception e) {
throw new RuntimeException("ElasticSearch execute batch statement error", e);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use SeaTunnelException and we have an issue to discuss the unified Exception define. If you interested in it, you can see #3045.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@Hisoka-X
Copy link
Member

Hisoka-X commented Nov 2, 2022

Please resolve ci problem, thanks.

/**
* An arbitrary absolute maximum practical retry time.
*/
public static final long MAX_RETRY_TIME_MS = TimeUnit.HOURS.toMillis(1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The max retry is 1 hours, it's to long. 1 minutes will be better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree.

@Hisoka-X
Copy link
Member

The other code is fine.

harveyyue added a commit to harveyyue/seatunnel that referenced this pull request Nov 21, 2022
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@EricJoy2048 EricJoy2048 merged commit 02ef38e into apache:dev Nov 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants