Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of num failed tasks for a worker. #1543

Closed
wants to merge 1 commit into from
Closed

Keep track of num failed tasks for a worker. #1543

wants to merge 1 commit into from

Conversation

dhruvg
Copy link
Contributor

@dhruvg dhruvg commented Feb 10, 2016

Add bookkeeping for number of failed tasks when getting work from the scheduler and workers.

@dhruvg dhruvg closed this Feb 10, 2016
@dhruvg dhruvg reopened this Feb 10, 2016
@dhruvg
Copy link
Contributor Author

dhruvg commented Feb 10, 2016

@erikbern Any idea what is up with the builds? Doesn't seem to be a code issue.

Last two builds have error-ed because of this: https://travis-ci.org/spotify/luigi/jobs/108323647

@dhruvg dhruvg closed this Feb 10, 2016
@dhruvg dhruvg reopened this Feb 10, 2016
@dhruvg
Copy link
Contributor Author

dhruvg commented Feb 11, 2016

@erikbern My hunch is that it is impacting the performance of that test case since it is called get_work thousands of times and we are now calculating num failed tasks on each request. But locally on my machine, that test takes < 10 seconds.

@dhruvg
Copy link
Contributor Author

dhruvg commented Feb 16, 2016

@erikbern bump?

@dhruvg dhruvg closed this Feb 17, 2016
@dhruvg dhruvg reopened this Feb 17, 2016
@dhruvg dhruvg closed this Oct 11, 2016
@erikbern
Copy link
Contributor

erikbern commented Oct 12, 2016

any reason this was closed? i guess it's been sitting here for a long time, not sure why

@Tarrasch
Copy link
Contributor

After a quick glance of the code I don't understand it. It just threads along the variable but it never seems to print or use it for anything. Anyway still looks interesting. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants