Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add killswitch for tasks #140

Merged
merged 1 commit into from
Oct 21, 2016
Merged

Add killswitch for tasks #140

merged 1 commit into from
Oct 21, 2016

Conversation

alde
Copy link
Contributor

@alde alde commented Sep 21, 2016

Add possibility to kill tasks

Use either the REST API or hermit to kill a task you no longer wish to
run.

POST /task/{taskID}/kill
hermit kill {taskID}

Add button in GUI for killing tasks

@@ -70,6 +70,12 @@ func routes(h Handler, conf *config.Config) Routes {
Handler: h.GetFromSandbox("stderr"),
},
Route{
Name: "Kill",
Method: "POST",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't DELETE /task/{taskId} make more sense here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of it first, but it's not actually deleting the task, it'll still be viewable through the API, so semantically DELETE felt wrong

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DELETE should imho only be used when you're deleting the resource identified by the url. However in this case the task will still be in the database because we want to be able to track that it was killed with later GETs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe DELETE can be used even if you're doing a soft delete on the resource, still being available for inspection by an admin (i.e. I could still be in the database). Although I think you're right about that in that case it shouldn't be available on GET /task/{taskId}. I'm fine with this.

@@ -13,7 +13,8 @@ const (
TaskState_TASK_LOST TaskState = "TASK_LOST"
TaskState_TASK_ERROR TaskState = "TASK_ERROR"
// Custom eremetic states
TaskState_TASK_QUEUED TaskState = "TASK_QUEUED"
TaskState_TASK_QUEUED TaskState = "TASK_QUEUED"
TaskState_TASK_TERMINATING TaskState = "TASK_TERMINATING"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for having two concepts for the same thing, e.g. kill vs terminate? This signals to me that there might be other reasons for the task to terminate than manually killing it.

Will the task be marked as finished when successfully terminated? I think it would be more helpful to the developer if you could see which tasks that were forcibly stopped.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When it's successfully killed it ends up in TASK_LOST
TASK_KILLED is also used by mesos as a final state, and I intended this state to more log the state change that the kill command was received, but it is not yet terminated

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that we should keep the naming consistent.

I believe it will become TASK_KILLED but we should test that.

return nil
}

_, err = s.driver.KillTask(&mesos.TaskID{Value: proto.String(taskId)})
Copy link
Collaborator

@keis keis Sep 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would try to kill tasks even if they are not yet running (and likely fail) and it would then later become running when we get to it in the queue.

So it's not actually giving us a way to clear tasks from the queue that we are not able to launch due to bad resource specs like we wanted 🍶

@@ -13,7 +13,8 @@ const (
TaskState_TASK_LOST TaskState = "TASK_LOST"
TaskState_TASK_ERROR TaskState = "TASK_ERROR"
// Custom eremetic states
TaskState_TASK_QUEUED TaskState = "TASK_QUEUED"
TaskState_TASK_QUEUED TaskState = "TASK_QUEUED"
TaskState_TASK_TERMINATING TaskState = "TASK_TERMINATING"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that we should keep the naming consistent.

I believe it will become TASK_KILLED but we should test that.

@alde alde force-pushed the add-killswitch-for-tasks branch 2 times, most recently from 4a118ca to 847dd42 Compare September 26, 2016 07:21
@alde
Copy link
Contributor Author

alde commented Sep 26, 2016

Rebased on master

@keis
Copy link
Collaborator

keis commented Oct 4, 2016

needs rebase 😅

@alde alde force-pushed the add-killswitch-for-tasks branch from 847dd42 to 736dd86 Compare October 4, 2016 07:27
Use either the REST API or hermit to kill a task you no longer wish to
run.

POST /task/{taskID}/kill
hermit kill {taskID}

Add button in the GUI to kill a task
@alde alde force-pushed the add-killswitch-for-tasks branch from 736dd86 to 113c07b Compare October 5, 2016 07:29
@alde
Copy link
Contributor Author

alde commented Oct 5, 2016

rebased again

@alde alde merged commit c35bc29 into master Oct 21, 2016
@alde alde deleted the add-killswitch-for-tasks branch October 21, 2016 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants