Skip to content
This repository has been archived by the owner on Aug 29, 2023. It is now read-only.

Arithmetics operation update #167

Closed
JanisGailis opened this issue Mar 23, 2017 · 1 comment
Closed

Arithmetics operation update #167

JanisGailis opened this issue Mar 23, 2017 · 1 comment

Comments

@JanisGailis
Copy link
Member

JanisGailis commented Mar 23, 2017

@forman: I like the diff op, but I still disagree with the arithmetic op. We should really move to an op, where users can specify a normal expression which we either

  • eval without globals but with a safe locals environment, see cate.util.safe_eval()
  • and/or use numexpr for boosting performance.
@JanisGailis JanisGailis added this to the IPM5 milestone Mar 23, 2017
@JanisGailis JanisGailis self-assigned this Mar 23, 2017
@forman forman added the feature label Jun 30, 2017
@forman forman modified the milestones: IPM6, IPM5 Jul 3, 2017
@forman
Copy link
Member

forman commented Jul 21, 2017

Idea: use new new_expression_op() function and register it as operation compute with expression as only input.

Problems:

This leads to a workflow step

  • with no other step inputs than expression, therefore it appears as it wouldn't have any dependencies to other steps. But actually, the expression may refer to any other step output by referencing their generated resources.
  • with a single return value of unknown data type, because an expression may evaluate to any Python data type.

The latter problem may be avoided by offering dedicated compute_<type> operations.
The first may be avoided by treating steps containing a compute operation (or one containing general Python expressions evaluated in the context of the workflow) as being dependent on all other steps. This way they would be re-invoked if any other step input changes.

@forman forman modified the milestones: IPM6, IRM7 Sep 21, 2017
@forman forman self-assigned this Dec 5, 2017
@forman forman closed this as completed in ae7ed7b Jan 9, 2018
forman added a commit that referenced this issue Jan 9, 2018
added new compute(ds, expression) operation (resolves #167)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants