Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[native]Send completed get data result if velox task has finished
The following race condition can cause table writer hang when scale writer is enabled T1 coordinator schedule a new task writer T2 one or any source task has produced all the data and all have been consumed. It triggers task to complete T3 the new task writer tries to fetch data from the closed source, and find the source task is not running, then wait for timeout and return an empty result to retry from the writer task T4 the new task writer keeps retrying and after the presto task has been reclaimed, the new data fetch will triggers a new presto task creation and continue retrying with empty result. The short-term fix for this is to return completion result in T3 if the velox task is in finished state instead of returning empty result. A completed task will stay in one-minute which is sufficient long in practice to handle this race condition. Have verified in Meta internal testing.
- Loading branch information