Refactor uris/fetch

[sheepkiller]

Move merging of uris/fetch from launch task to create tasks by
replacing URIs/Fetch element from EremeticTask to a single one, FetchURIs,
loosely based on mesos protobuf.
This change breaks the 1:1 mapping on request to task.

[sheepkiller]

Hi,

I was unhappy with my patch, so I propose to merge URIs and Fetch sub element when the task is created instead of the moment the task is effectively launched.
Please note that this patch is also about where backward compatibility is supposed to happen. IMHO, putting stuff in scheduler package is the best way to slow down migration to mesos-go NG

[keis]

Agreed that keeping stuff out of scheduler/ when possible is a good call because it's also the biggest by far submodule. It's always bit worrying to break backwards compatibility even if reading back the uris is a very minor usecase.

It would be possible to reconstruct the request if we add some metadata to URIs but I don't think it's worth it. But at the very least we should keep the recently added "fetch" except that it now would contain both "uris" and "fetch" from the request.

[sheepkiller]

Hi @keis,

My intent is not to break backward compatibility at user layer, but to use NewEremeticTask as "a translator", to map request to task. uris has features which fetch doesn't support (i.e. auto extract of archives files), and this behavior is done via mergeURIs().
Eremetic has a nice software design and I don't want to taint it.
Futhermore, at one point, some elements of the request (slave_constraints for example which should be rename to agent_constraints, or could be improved with operators) or the task (mesos is slowly moving away from docker runtime engine) will change and the original internal mapping request -> task will be hard to maintain.
My PR is mainly here to take some time to think where and how this mapping should be done.

thanks for your time

[keis]

Was thinking this one should still be added to the json as "fetch"

[sheepkiller]

will do, thanks for the review

[keis]

Sure thing mapping the internal structure is perfectly fine. But it is desirable that the information POSTed is retrievable later on, and ideally in a similar format. We're still in 0.x so moving around and deprecating things is still pretty easy and if going this route it would only be natural to eventually remove "uris".

We have some issues tracking the other things you mention (#64 and #59) but it's nothing we're actively working on.

Thanks for helping out and making eremetic better :)

[alde]

We should consider removing URIs and having only the new Fetch feature before 1.0, as it is more or less a duplication, with the latter having more features. As @keis said we are still in "unstable" mode which means the API can quite easily change.

Being able to get back what you sent in with a POST (with the exception of masked environment variables) is something I'd like to keep as far as possible.

[sheepkiller]

Thanks for the clarifications. I neglected the convenience to get back a valid request from the /task endpoint.
I may be a tool, but I have to clarify one point. What is the expected behavior when you call /task/{id} endpoint

• you want receive back a JSON of the task, based on your original request
• you want receive back a JSON of the task, usable as a request but formated as expected by the latest version of eremetic
  (in case of this PR, if you post uris, you got back a request with fetch)

[keis]

I don't think it has to exactly match any of those but the rule of least surprise tells us it should look familiar. The main use is being able to answer stuff like

• what is eremetic running right now (GET /task/)?
• what was this thing that just failed or misbehaved (GET /task/{id})?

Nudging people over to the new style is probably a good thing

[keis]

Anyway LGTM and if you rebase on latest master I think coveralls will shut up as well :)

[sheepkiller]

please don't merge it for now, I've just noticed an upleasant side effect of omitempty in types.URI definition

[alde]

Sure, let us know when you feel it's ready to merge.

[sheepkiller]

you can merge it, thanks. I'd prefer to not squash last commit, because it fixes types.URI struct

[keis]

Thanks 😸