Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove timeout behavior of translation engine rule tests #694

Merged
merged 3 commits into from
Jun 4, 2021

Conversation

daxhaw
Copy link
Contributor

@daxhaw daxhaw commented Jun 3, 2021

This change will cause the translation engine ruleset tests to hang if the rules are not called at all. This "wait_for_rule" function now just waits for global variables to be set inside the rule so they are no longer dependent upon timing. TeamCity itself will flag long-running builds and warn us that they are hanging as part of build alerts.

These changes fixed if_stmt2 as well which I was also seeing failing locally.

There is still a chance that a rule is called when it shouldn't be. Verifying that a rule is not called is tougher than verifying that a rule is called. My only choice here was to move the verification that a rule was called after verifying that all the rules that were supposed to be called were called. I'll watch teamcity to see if this leads to other failures.

In the tag_define_rules test we actually had another race condition where the rule was finishing and setting the flag but then we were immediately checking that another's table value was changed (the rule sums course hours on insert of a registration record). My guess is that the global was set but the rules_engine hadn't committed the rule transaction so those values were not changed in time for the test to verify them. The fix for this was to add another rule that fires when the table's column in question (course.hours) was updated. Waiting on this rule guarantees that the rule that fired on insert had committed.

I kicked off Production_gdev_18_04 and ProductionGaiaRelease_gdev builds off my branch to see if they pass with these changes as well.

@senderista
Copy link
Contributor

So, here's a somewhat higher-level question prompted by the race you mentioned: for transactions that mutate non-transactional global state, shouldn't we provide a way to make that mutation also transactional, i.e. conditional on the transaction committing? I guess this would really just mean exposing triggers directly to the client? Isn't this something that our customers will actually need to do, for side-effects within a transaction that can't be rolled back or shouldn't be visible unless and until the transaction commits? The programming model I imagine here is basically execute_in_transaction(std::function<bool()> txn, std::function<void()> on_commit, std::function<void()> on_abort). This would transparently retry on conflicts and would execute the on_commit() and on_abort() callbacks on respective success or failure (where the latter would presumably be a user-defined failure condition rather than a conflict abort, unless we wanted to allow a user-specified action for retry exhaustion). These callbacks could be executed either on a thread pool or directly on the session thread (the latter of course would be necessary if the callbacks referenced thread-local state).

Copy link
Contributor

@senderista senderista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like the right approach for a test that isn't really verifying perf objectives.

{
bool wait_for_rule(bool& rule_called);
bool wait_for_rule(std::atomic<int32_t>& count_times, int32_t expected_count);
} // namespace rule_test_helpers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newline at end of file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh. Fixed.

@daxhaw
Copy link
Contributor Author

daxhaw commented Jun 3, 2021

@senderista - I think it would be useful to expose transaction events that users could bind to. Just brainstorming here but ideally one could subscribe to the specific commit event for a given rule as the programming model. We currently have no way to identify a specific rule, however. Something like:

OnUpdate(S:Student) : id(my_student_rule)
{
...
}

OnCommit(my_student_rule)
{
...
}

OnAbort(my_student_rule)
{
...
}

@senderista
Copy link
Contributor

@senderista - I think it would be useful to expose transaction events that users could bind to.

So these event handlers would be defined in the ruleset files themselves, i.e. they would be part of the declarative API?

@daxhaw daxhaw force-pushed the dax/remove_rule_test_timeouts branch from 5316f35 to a35163b Compare June 4, 2021 00:14
@daxhaw daxhaw merged commit 56ac592 into master Jun 4, 2021
@daxhaw daxhaw deleted the dax/remove_rule_test_timeouts branch June 4, 2021 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants