Adding Different Queuing Strategies #4

emsa23 · 2013-10-06T08:11:36Z

The changes moves to Priority Sorter to 2.0-SNAPSHOT

"Legacy" is still supported and the Plugin will in "Legacy" mode still behave like 1.x versions.
Converting to "Advanced" mode will preserve relative priorities.

In "Advanced" mode:

Configurable number of priorities
Configurable Default priority
Set priority on Jobs based on View and/or job-name filter (new Action/Page on main page)
- For each view a list a strategies for settings priorities can be set
- Priority can be set based on way the jobs is started
- Priority can be set by using Job Parameter
Option to allow or prevent setting Priority directly on Jobs
Option to select priority-sorting algorithm between:
- "Absolute" (what Priority Sorter does today)
- "Fair Queuing" (equally distribute resources over a number of "buckets")
- "Weighted Fair Queuing" (weighted distribution of resources over a number of "buckets")*

buildhive · 2013-10-06T08:14:52Z

Jenkins » priority-sorter-plugin #5 SUCCESS
This pull request looks good
(what's this?)

jenkinsadmin · 2013-10-06T12:02:09Z

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

oleg-nenashev · 2013-10-06T15:23:19Z

src/main/java/jenkins/advancedqueue/AdvancedQueueSorter.java

+			});
+			return;
+		}		
+		// If the queue is empty reset the internal priority counters


What about two other strategies?
Anyway, it would be better to have a class for each strategy, which stores sorting methods and configuration stuff. Then, it will be much easier to convert these classes to extensions at some point.

The two strategies above this line doesn't use the counter but the two below do.

Yes it would be better and I do plan in the next dev phase treat strategies as Extensions but as of now I would like to keep this hard-coded and make sure that the overall concept and implementation is valid before going there.

buildhive · 2013-10-06T18:07:28Z

Jenkins » priority-sorter-plugin #6 SUCCESS
This pull request looks good
(what's this?)

buildhive · 2013-10-06T18:20:15Z

Jenkins » priority-sorter-plugin #7 SUCCESS
This pull request looks good
(what's this?)

buildhive · 2013-10-06T18:22:41Z

Jenkins » priority-sorter-plugin #8 SUCCESS
This pull request looks good
(what's this?)

buildhive · 2013-10-06T18:30:57Z

Jenkins » priority-sorter-plugin #9 SUCCESS
This pull request looks good
(what's this?)

job-priority.

buildhive · 2013-10-06T20:00:57Z

Jenkins » priority-sorter-plugin #10 SUCCESS
This pull request looks good
(what's this?)

cloudbees-pull-request-builder · 2013-10-16T16:20:35Z

plugins » priority-sorter-plugin #1 SUCCESS
This pull request looks good

Caching actual/calculated priority value on the Job to avoid recalculation when rendering the priority column.

cloudbees-pull-request-builder · 2013-10-16T17:55:44Z

plugins » priority-sorter-plugin #2 SUCCESS
This pull request looks good

cloudbees-pull-request-builder · 2013-10-31T19:52:06Z

plugins » priority-sorter-plugin #20 SUCCESS
This pull request looks good

cloudbees-pull-request-builder · 2013-10-31T19:53:14Z

plugins » priority-sorter-plugin #21 SUCCESS
This pull request looks good

oleg-nenashev · 2013-11-01T05:18:36Z

What about cache-alike approach? You can just recalculate the queue only once after the restart.

In my installation:

The build queue sometimes reaches 10000 items
Users like launching of 1-second multi-configuration jobs, so Jenkins' throughput reaches about 100 build completions per second
According to profiling, saving of queue is a main bottleneck. Additional saving in priority-sorter may significantly decrease throughput (needs to be evaluated)

emsa23 · 2013-11-01T06:24:31Z

Ok, I understand the problem and will implement another solution.

Do you have any other consideration on the setup; any other major things need fixing?

cloudbees-pull-request-builder · 2013-11-01T07:03:23Z

plugins » priority-sorter-plugin #22 FAILURE
Looks like there's a problem with this pull request

cloudbees-pull-request-builder · 2013-11-01T07:15:29Z

plugins » priority-sorter-plugin #23 SUCCESS
This pull request looks good

oleg-nenashev · 2013-11-02T09:33:02Z

PriorityConfiguration::get() returns null on start in my case. Saving of global configuration fails, many other methods may suffer as well

emsa23 · 2013-11-03T09:04:28Z

Isn't that strange? I have seen that happen and I do not really understand how that can even happen ... ?

oleg-nenashev · 2013-11-03T14:35:19Z

It may happen when the plugin goes to legacy mode, but somehow continues operating in AdvancedQueue. BTW, I cannot reproduce the issue after cleanup of "work" directory, so the issue may be related to switching between dev. versions

…plemented as describable items, but the config.jelly has not been implemented yet. Signed-off-by: Oleg Nenashev <[email protected]>

emsa23 · 2013-11-03T15:53:44Z

But even in Legacy mode all extension should be loaded I cannot see how this can happen.

I guess it has something to do with dev versions and incomplete deploys.

Signed-off-by: Oleg Nenashev <[email protected]>

…buckets number definitions Signed-off-by: Oleg Nenashev <[email protected]>

Signed-off-by: Oleg Nenashev <[email protected]>

Describable strategies.

cloudbees-pull-request-builder · 2013-11-10T15:33:07Z

plugins » priority-sorter-plugin #24 SUCCESS
This pull request looks good

cloudbees-pull-request-builder · 2013-11-10T16:13:55Z

plugins » priority-sorter-plugin #25 SUCCESS
This pull request looks good

Adding Different Queue Sorting Strategies and Strategies to assign Priorities to Jobs.

jglick · 2018-01-11T23:10:00Z

src/main/java/jenkins/advancedqueue/sorter/SorterStrategy.java

+
+import jenkins.model.Jenkins;
+
+import org.apache.tools.ant.ExtensionPoint;


emsa23 added 4 commits September 22, 2013 18:50

Clean-up, updated developer info, new baseline is version 1.509.1

4374e24

Adding Advanced Queue Sorting options and upgrade path.

8a5a854

Added documentation, a new icon and did some cleanup.

856f028

Using View.getItem() instead if View.getItems()

7ddae54

oleg-nenashev reviewed Oct 6, 2013
View reviewed changes

Assigning values to instance variables.

1a91ee3

emsa23 added 2 commits October 6, 2013 20:17

Using logger not streams.

e0718a3

Calling save() after changes to Project.

e00aaf5

getUrlName() now returns null in legacy-mode

c8cb9e2

Validating regexp-patterns and fixed issue with calculating

a5a791a

job-priority.

emsa23 added 2 commits October 16, 2013 18:06

Updated self-getters to do direct lookup.

6f4689e

Checking valid "numberOfPriorities" on submit.

ec20558

Caching actual/calculated priority value for Jobs.

ffbb936

Caching actual/calculated priority value on the Job to avoid recalculation when rendering the priority column.

Invalidate cached priority when saving Job.

d5cd1e8

Slightly better layout.

31a66db

Recalculates the Queue on startup rather than saving it to disk.

9f310a5

Removed java 7 method.

d1508bd

Draft implementation of describable strategies. Now all strategies im…

334328e

…plemented as describable items, but the config.jelly has not been implemented yet. Signed-off-by: Oleg Nenashev <[email protected]>

oleg-nenashev and others added 9 commits November 3, 2013 22:35

Fixed errors, added contributors to license headers

bbdecf1

Signed-off-by: Oleg Nenashev <[email protected]>

Re-factored previous implementation of strategy UI / storage

c8c5256

Signed-off-by: Oleg Nenashev <[email protected]>

Fixed a bug with switching to Advanced mode (haven't worked).

6bbe4bd

Signed-off-by: Oleg Nenashev <[email protected]>

Renamed POM name of the plugin (just for appearance in IDEs)

8f2ae30

Signed-off-by: Oleg Nenashev <[email protected]>

[NOT READY] A draft implementation of SorterStrategies with built-in …

b56ad31

…buckets number definitions Signed-off-by: Oleg Nenashev <[email protected]>

Finished strategy-bundled priority buckets management

bcb83e8

Signed-off-by: Oleg Nenashev <[email protected]>

Cleanup of strategies code (including codestyle change)

f3ee1e3

Signed-off-by: Oleg Nenashev <[email protected]>

Refactoring: switched JobGroup creation method to camelCase

81adaf7

Signed-off-by: Oleg Nenashev <[email protected]>

Merge pull request #3 from oleg-nenashev/master

76c88a7

Describable strategies.

Made value transient to reset at restart.

48296a8

emsa23 added a commit that referenced this pull request Nov 11, 2013

Merge pull request #4 from emsa23/master

9394253

Adding Different Queue Sorting Strategies and Strategies to assign Priorities to Jobs.

emsa23 merged commit 9394253 into jenkinsci:master Nov 11, 2013

jglick mentioned this pull request Jan 11, 2018

[JEP-200] Oops #42

Merged

jglick reviewed Jan 11, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Different Queuing Strategies #4

Adding Different Queuing Strategies #4

emsa23 commented Oct 6, 2013

buildhive commented Oct 6, 2013

jenkinsadmin commented Oct 6, 2013

oleg-nenashev Oct 6, 2013

emsa23 Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

cloudbees-pull-request-builder commented Oct 16, 2013

cloudbees-pull-request-builder commented Oct 16, 2013

cloudbees-pull-request-builder commented Oct 31, 2013

cloudbees-pull-request-builder commented Oct 31, 2013

oleg-nenashev commented Nov 1, 2013

emsa23 commented Nov 1, 2013

cloudbees-pull-request-builder commented Nov 1, 2013

cloudbees-pull-request-builder commented Nov 1, 2013

oleg-nenashev commented Nov 2, 2013

emsa23 commented Nov 3, 2013

oleg-nenashev commented Nov 3, 2013

emsa23 commented Nov 3, 2013

cloudbees-pull-request-builder commented Nov 10, 2013

cloudbees-pull-request-builder commented Nov 10, 2013

jglick Jan 11, 2018


		import jenkins.model.Jenkins;

		import org.apache.tools.ant.ExtensionPoint;

Adding Different Queuing Strategies #4

Adding Different Queuing Strategies #4

Conversation

emsa23 commented Oct 6, 2013

buildhive commented Oct 6, 2013

jenkinsadmin commented Oct 6, 2013

oleg-nenashev Oct 6, 2013

Choose a reason for hiding this comment

emsa23 Oct 6, 2013

Choose a reason for hiding this comment

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

buildhive commented Oct 6, 2013

cloudbees-pull-request-builder commented Oct 16, 2013

cloudbees-pull-request-builder commented Oct 16, 2013

cloudbees-pull-request-builder commented Oct 31, 2013

cloudbees-pull-request-builder commented Oct 31, 2013

oleg-nenashev commented Nov 1, 2013

emsa23 commented Nov 1, 2013

cloudbees-pull-request-builder commented Nov 1, 2013

cloudbees-pull-request-builder commented Nov 1, 2013

oleg-nenashev commented Nov 2, 2013

emsa23 commented Nov 3, 2013

oleg-nenashev commented Nov 3, 2013

emsa23 commented Nov 3, 2013

cloudbees-pull-request-builder commented Nov 10, 2013

cloudbees-pull-request-builder commented Nov 10, 2013

jglick Jan 11, 2018

Choose a reason for hiding this comment