Expressing dependencies between parameters and code

[jlstevens]

I'm pulling some suggestions out of #160 to file as a separate issue. As it turns out there are different things we might want to have param express:

1. Relationships between parameters i.e update parameter settings based on changes to another parameter. There may be some easy way to express such relationship or there might be some code supplied by the user to link parameters (e.g using the set_hook idea)
2. Relationships between parameters and regular code in parameterized classes that might consume parameter values and/or set them. This is what this issue is about and the idea is to express that a method might make use of a parameter and might set other parameters. If the method has output, you may make the assumption that the output will have changed when the parameters that are read by that method have changed.
3. General specifications of input/output types of methods, functions etc. This would be very useful for some things but currently out of the scope of what we want to achieve in the short term.

Here are my proposals to address 2. made in issue #160:

class Example(param.Parameterized):

    A = param.Number(default=10)

    B = param.Number(default=5)

    @param.dependencies([A],[B])
    def link_a_to_b(self, **parameters):
        self.B = parameters['A']/2.

This expresses the parameters read ([A]) and the parameters written to ([B]) using a decorator. Alternatively, you could have more granular decorators:

class Example(param.Parameterized):

    A = param.Number(default=10)

    B = param.Number(default=5)

    @param.reads([A])
    @param.writes([B])
    def link_a_to_b(self, **parameters):
        self.B = parameters['A']/2.

The information captured by the decorator needs to be stored somewhere. Presumably we don't want either 1) a global mapping of parameterized classes and their parameter dependencies as this would be fragile 2) distributing this information among the parameters themselves (would be very space inefficient if each parameter holds such information). This means the natural place to keep track of this is in the parameterized classes, presumably somewhere in param.Parameterized.

The limitation of this is 1) it wouldn't be customizable per instance 2) it would be information defined once when the class is defined. Neither of these seems to be a real problem as we don't often shove/mutate methods on classes after they are defined.

As for API, I really would like #154 tackled (putting the API onto a sub-object) first but that seems unlikely to happen. Here is one simple idea of how the information could be accessed:

example = Example()
reads, writes = example.param_dependencies('link_a_to_b') # OR
reads, writes = Example.param_dependencies('link_a_to_b') 

This just regurgitates what was supplied to the @param.dependencies decorator where the reads and writes might be empty lists. I'm not sure whether these lists should just contain parameter names or the parameter objects themselves. Maybe this could be an optional argument to the param_dependencies method where I would favor names by default.

One thing worth mentioning is by making certain assumptions you could follow a chain of changes without executing user code. For instance, if you assume that changing parameter a read by method1 changes parameter b that it writes to, you might then assume that the output of method2 is changed as that method reads parameter b.

Note that this proposal simply aims to capture the which parameters are read/written to by methods and doesn't aim to do anything with this information. I have no particular proposals for this just yet but I can imagine param offering utilities to trace the potential dependency graph of parameters and perhaps execute various sorts of callback when parameters change.

[jbednar]

@param.reads([A]), @param.writes([B])

Seems like those could be @param.reads(A1,A2), @param.writes(B1,B2), i.e., no list notation needed? Parameter names should always be valid attribute names, right?

They could also be called @param.depends and @param.modifies. On balance I think that having the modifies be very visible will help people understand the code; people expect methods to depend on parameters, but they don't expect parameters to be modified often by methods.

We should consider having the method docstring call out the docstrings for the parameters it depends on, which is a tension I have often found--We encourage parameters to be documented, but when studying a method I often don't know which parameters are expected to affect it, and what they can be set to. Being able to express this relationship can be very useful in general.

This means the natural place to keep track of this is in the parameterized classes, presumably somewhere in param.Parameterized.

I agree; this seems like class-level information.

As for API, I really would like #154 tackled (putting the API onto a sub-object) first but that seems unlikely to happen.

If you have a proposed possible name for the subobject, we can certainly start trying out that idea here, by adding new functionality onto a subobject. It might be cleaner to do it all at once later, though.

I'm not sure whether these lists should just contain parameter names or the parameter objects themselves.

I'm not sure what's better; I guess people are likely to be keeping things indexed by parameter name anyway, so maybe names are easier to work with.

[jlstevens]

Both @param.depends and @param.modifies seem like good names to me and those don't need the parameters to be in lists. I think I would still like a decorator that does the function of both (that will need to use lists to separate the two types of parameter. I suggested @param.dependencies but maybe you have another, ideally shorter, suggestion?

To be very explicit you could have:

@param.dependencies(depends=[A],modifies=[B])

If we find a shorter name than dependencies then I think I would still prefer this form that using two decorators on top of each other.

[jbednar]

Personally, I think that modifies is nearly always going to be empty, so I prefer the two-decorator version. Or it could be @param.depends(A1,A2,modifies=[B1,B2]), which explicitly encodes the assumption that modifies is a rarer case.

[jlstevens]

That last proposal seems good to me!

[jbednar]

One downside: it would actually be @param.depends(A1,A2,modifies=['B1','B2']), which is a bit awkward.

[jbednar]

Anyway, what I'm interested in leading up to is something like the nyc_taxi-paramnb notebook, but with the crucial bits expressed at the param level, and with the ability to have finer-grained bits of code executed if appropriate. I'm trying to see what more we need beyond expressing those dependencies, but am coming up short.

Following the above suggestions, I'm trying to imagine we have a class defined like:

class Options(param.Parameterized):
    alpha       = param.Magnitude(default=0.75)
    colormap    = param.ObjectSelector(default=cm["fire"], objects=cm.values())
    location    = param.ObjectSelector(default="pickup",   objects=["pickup","dropoff"])
    passengers  = param.ObjectSelector(default=1,          objects=passenger_counts)

    tiles = param.Parameter(gv.WMTS('https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg')))
    trips = param.Parameter()
    plot = param.Parameter()

    @param.depends(alpha,modifies=["tiles"])
    def update_tiles(self):
        self.tiles = self.tiles.opts(style=dict(alpha=self.alpha)) 

    @param.depends(colormap,location,passengers,modifies=["trips"])
    def update_trips(self):
        df_filt = df[df.passenger_count==self.passengers]
        points = hv.Points(gv.Dataset(df_filt, kdims=[self.location+'_x', self.location+'_y'], vdims=[]))
        self.trips = datashade(points, width=800, height=475, x_sampling=1, y_sampling=1, 
                               cmap=self.colormap, element_type=gv.Image,
                               dynamic=False, x_range=x_range, y_range=y_range)
                
    @param.depends(tiles,trips,modifies=['plot'])
    def update_plot(self):
        self.plot = self.tiles * self.trips

Here I didn't want to create tiles or trips or plot parameters, but don't see how else to express the notion that some of the code is going to produce things needed by other bits of the code. I also am not sure how update_trips is going to get the x_range and y_range (and width and height?) generated by the Bokeh plot, nor how someone would know when looking at this that somewhere there will be a Bokeh/HoloViews plot to show. @philippjfr, any idea where to go from here? @jlstevens, do your ideas of declaring outputs help any? @ceball, any ideas?

[jlstevens]

... it would actually be @param.depends(A1,A2,modifies=['B1','B2']), which is a bit awkward.

Why do the modifies elements have to be strings? Won't those parameters be declared and available to reference just like A1 and A2?

... but don't see how else to express the notion that some of the code is going to produce things needed by other bits of the code.

As long as you define your methods in a suitable order, why not allow method names in depends instead of defining dummy parameters? Couldn't the last method be:

@param.depends(update_tiles, update_trips, modifies=['plot'])
    def update_plot(self):
        self.plot = self.tiles * self.trips

Both self.tiles and self.trips could then be normal python attributes.

[jbednar]

Makes sense. Ok, then, proposal v0.2:

class Options(param.Parameterized):
    alpha       = param.Magnitude(default=0.75)
    colormap    = param.ObjectSelector(default=cm["fire"], objects=cm.values())
    location    = param.ObjectSelector(default="pickup",   objects=["pickup","dropoff"])
    passengers  = param.ObjectSelector(default=1,          objects=passenger_counts)
    tile_url = param.String("https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg", precedence=-1)
    plot = param.View()

    @param.depends(alpha)
    def update_tiles(self):
        self.tiles = gv.WMTS(self.tile_url)).opts(style=dict(alpha=self.alpha))

    @param.depends(colormap,location,passengers)
    def update_trips(self):
        df_filt = df[df.passenger_count==self.passengers]
        points = hv.Points(gv.Dataset(df_filt, kdims=[self.location+'_x', self.location+'_y'], vdims=[]))
        self.trips = datashade(points, width=800, height=475, x_sampling=1, y_sampling=1, 
                               cmap=self.colormap, element_type=gv.Image,
                               dynamic=False, x_range=x_range, y_range=y_range)
                
    @param.depends(update_tiles,update_trips,modifies=[plot])
    def update_plot(self):
        self.plot = self.tiles * self.trips

Seems like that would let you express dependencies between the methods, yes. I've kept "self.plot" and declared it as a View parameter (akin to the one in paramnb) to denote that there will be a plot to show, and to give the Widgets class somewhere to start when figuring out what to call to get output (find all View parameters, find all methods that modify them (hopefully just 1, to avoid ambiguity!), find all dependencies of those methods, and recursively follow that until something doesn't have dependencies on methods, then run that and unroll running the rest until the output is safely in self.plot (and any other View parameters). I may be missing some steps, but it seems like that would work.

Still not sure how to pass the x_range, y_range, width, and height needed for this to be the callback for a DynamicMap generating a Bokeh plot.

[philippjfr]

Not to be a buzzkill but those examples don't really make much sense to me at all in their current form. In the end the objects need to be wrapped in a DynamicMap so that the whole thing works without having to rerender the whole plot each time a parameter changes and for the RangeXY stream to be added. If you were to define self.update_plot as the callback to the DynamicMap you would once again have lost all the granularity that you were trying to get by defining the dependencies between the methods.

Based on your example here's what I think might make sense but is pretty ugly and probably bears little relation to what you want:

class Options(param.Parameterized):
    alpha       = param.Magnitude(default=0.75)
    colormap    = param.ObjectSelector(default=cm["fire"], objects=cm.values())
    location    = param.ObjectSelector(default="pickup",   objects=["pickup","dropoff"])
    passengers  = param.ObjectSelector(default=1,          objects=passenger_counts)
    datashade = param.Boolean(default=True)
    tiles = param.Parameter(gv.WMTS('https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg')))
    trips = param.Parameter()
    plot = param.Parameter()

    def __init__(self, **params):
        super(Options, self).__init__(**params)
        self.update_trips()
        self.plot = self.view()

    def view(self):
        tiles = hv.DynamicMap(lambda **params: self.tiles, streams=[ParamValues(self, depends=self.update_tiles)])
        trips = hv.DynamicMap(lambda **params: self.trips, streams=[ParamValues(self, depends=self.update_tiles)])
        if self.datashade:
            trips = datashade(trips, width=800, height=475, x_sampling=1, y_sampling=1, 
                      element_type=gv.Image, streams=[ParamValues(self, rename={'colormap': 'cmap'}, RangeXY])
        return tiles * trips

    @param.depends(alpha)
    def update_tiles(self):
        self.tiles = self.tiles.opts(style=dict(alpha=self.alpha)) 

    @param.depends(location, passengers)
    def update_trips(self):
        df_filt = df[df.passenger_count==self.passengers]
        self.trips = hv.Points(gv.Dataset(df_filt, kdims=[self.location+'_x', self.location+'_y'], vdims=[]))

    @param.depends(datashade)
    def update_plot(self):
        self.plot = self.view()

The param.depends information basically ends up being used as additional information by the ParamValues stream to decide when to trigger an update and which parameters to memoize on. Since chaining of DynamicMaps already defines the dependencies between outputs I don't quite see where the modifies keyword would fit in this scheme. I've also added a datashade parameter to this to demonstrate how you could trigger a full rerender using this approach. I realize this is probably not at all what you had in mind, but I think we all need to do some brainstorming to determine how we can make this proposal interact nicely with DynamicMaps.

There might be a way to use a class like this as a special type of Callable for a DynamicMap to accept but I'd have to think more about what that might look like.

[philippjfr]

After discussing this for a while I've come up with a related but slightly different proposal (let's call it v0.3). Instead of assigning to parameters and composing those in a method we declare individual methods which declare their dependencies and return an object. The original suggestion of assigning to parameters and declaring the modifications is still valid but most of the time you would use methods that return objects when working with a DynamicMap. Here's a relatively complex example that demonstrates both parameter dependencies and dependencies between methods:

class Options(param.Parameterized):
    alpha       = param.Magnitude(default=0.75)
    colormap    = param.ObjectSelector(default=cm["fire"], objects=cm.values())
    location    = param.ObjectSelector(default="pickup",   objects=["pickup","dropoff"])
    passengers  = param.ObjectSelector(default=1,          objects=passenger_counts)
    tile_url = param.String(default='https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg')
    datashade = param.Boolean(default=True)

    @param.depends(alpha, tile_url)
    def update_tiles(self):
        return gv.WMTS(self.tile_url).opts(style=dict(alpha=self.alpha))
    
    @param.depends(location, passengers)
    def update_trips(self):
        df_filt = df[df.passenger_count==self.passengers]
        self.points = hv.Points(gv.Dataset(df_filt, kdims=[self.location+'_x', self.location+'_y'], vdims=[]))
        return self.points
    
    @param.depends(colormap, update_trips)
    def update_datashaded(self, x_range=None, y_range=None):
        return datashade(self.points, cmap=self.colormap, x_range=x_range, y_range=y_range, dynamic=False)

    @param.viewable
    @param.depends(datashade)
    def viewable(self):
        trips = dynamic(self.update_datashaded, streams=[RangeXY]) if self.datashade else dynamic(self.update_trips)
        return dynamic(self.update_tiles) * trips

The dynamic function in this example takes a method decorated with dependencies and returns a DynamicMap with streams appropriately set up. As you can see the dynamic method will also accept additional streams letting you inject other parameters such as x_range and y_range easily.

The param.viewable decorator in this proposal simply tells the widget framework which methods return viewable objects it can render. The widget frameworks will let you register handlers for displaying different viewable types making this approach very general. By declaring a viewable method with a dependency you are declaring that the plot should be re-rendered completely every time the dependency changes (in this case datashade).

The main difficulty in implementing this will be implementing the dependency resolution. Since methods that simply return an object shouldn't be eagerly evaluated whenever a dependency changes (because that would be pointless), we need an implementation that lazily evaluates methods that return objects when you call them. For example imagine calling update_datashaded after location has changed, if update_trips isn't eagerly evaluated then the return value won't respect that new location so we'll have to keep track of the fact that update_trips is dirty and resolve that part of the graph before calling update_datashaded. That way the DynamicMaps that have been created can be triggered after the location is changed, and param will automatically resolve the dependencies of update_datashaded by calling update_trips first.

On the other hand methods that have side-effects, i.e. modify some parameter, should be eagerly called whenever their dependency changes ensuring that the modification is respected. My suggestion would be that methods that are decorated with a modifies argument are eagerly evaluated when their dependencies change, otherwise the dependencies resolve lazily as needed, i.e. when you call the method.

(Note that in practice update_datashade would just call update_trips directly, the (anti-)pattern demonstrated here is just for illustrative purposes).

[jlstevens]

This last proposal appeals to me the most: I always want my methods to have return values as I hate calling code for their side-effects. As we discussed, figuring out how to generate the DynamicMaps properly will be the tricky bit here.

[jbednar]

Looks good to me!

It's a little tricky that update_trips both returns a value and sets self.points. I guess update_trips needs to return a value for it to be usable in in the DynamicMap in viewable(). And then it needs to set self.points so that update_datashaded can avoid calling update_trips in the case where the filtering hasn't changed but we turned on datashading? It might be simpler to avoid setting self.trips in this case and just have update_datashaded call update_trips explicitly, but I guess having it this way is necessary to show how dependency on a method would be done.

Would height and width be handled the same as x_range and y_range?

By declaring a viewable method with a dependency you are declaring that the plot should be re-rendered completely every time the dependency changes (in this case datashade).

As the simplest case, the behavior should also be reasonable, if inefficient, when not declaring any dependencies. Probably safest is to always re-run the viewable when any parameter changes?

[philippjfr]

It might be simpler to avoid setting self.trips in this case and just have update_datashaded call update_trips explicitly, but I guess having it this way is necessary to show how dependency on a method would be done.

That's correct, I added a note that this was only for illustrative purposes, calling update_trips inside update_datashaded is the correct thing to do.

Would height and width be handled the same as x_range and y_range?

Yes, that's right.

As the simplest case, the behavior should also be reasonable, if inefficient, when not declaring any dependencies. Probably safest is to always re-run the viewable when any parameter changes?

Absolutely not, if there are no dependencies it should never be re-executed automatically (as that will be the most common case). I don't mind having some simple syntax that expresses a dependency on all parameters though.

[jbednar]

that will be the most common case

What exactly will be the most common case? I would think that if someone is writing a class like this, the most common case will be that they have defined some parameters, they have one method that depends on some combination of those parameters, and that any of those parameters could potentially affect the result of that method because we have no further information from them about which ones actually do. If they go ahead and specifically declare that only certain parameters are dependencies, great, but otherwise don't we have to assume that any of them could be?

[jbednar]

Basically, I think that declaring such dependencies should be required only when they want to have fine-grained control over bits of the computation; in cases where all of the computations are relatively cheap, there should be no need to have to collect a list of all the parameters involved; just re-run everything!

[philippjfr]

What exactly will be the most common case? I would think that if someone is writing a class like this, the most common case will be that they have defined some parameters, they have one method that depends on some combination of those parameters, and that any of those parameters could potentially affect the result of that method because we have no further information from them about which ones actually do.

I agree that might be the most common case, although I don't recommend it. Regardless they don't want to trigger a complete rerender on every parameter change, that's what the DynamicMap is for.

[jbednar]

I guess I disagree then; I am very happy to recommend it to them as as reasonable starting point. I don't think people should do any more work they they have to, to achieve their goals. If the first, simplest expression of parameters+method is sufficient, great! Done! If they need to pull out certain bits because others are very expensive, then there should be a reasonably easy way to do that. But I wouldn't recommend that all people typically do it the most expressive way; people should declare just enough to get their jobs done, and move on!

[philippjfr]

Sure, introductory tutorials can show that approach, but any examples should split it up in the recommended way. Recomputing a datashaded image because the tile alpha has changed is terrible!

[jlstevens]

I see the arguments both ways but I do think I agree with Philipp more on this one - if the simple thing is easy to declare but performs poorly (because every parameter changes causes an update) then no one will be happy. Explicit is better than implicit!

Absolutely not, if there are no dependencies it should never be re-executed automatically

Not declaring depends at all should result in the assumption that no parameter is being read/written. I say it is better to assume nice methods follow a nice functional, programming style by default, not maximally intertwined stateful programming! Besides, I don't see why you wouldn't sometimes want a parameterized class with a method returning a static viewable.

I don't mind having some simple syntax that expresses a dependency on all parameters though.

Again, I don't see why the viewable decorator can't have arguments. This could be one such argument where you explicitly say that the viewable is to be refreshed on any parameter change.

[jbednar]

but any examples should split it up in the recommended way. Recomputing a datashaded image because the tile alpha has changed is terrible!

Sure, with a datashaded layer, it's almost certainly going to be worth separating out that bit, making anything else fast. But most people aren't using datashader, yet most people do need dashboards. And so we shouldn't make it any harder than it needs to be in the many, many cases where the data isn't dauntingly big and people don't need to squeeze every bit of optimization out of it.

you explicitly say that the viewable is to be refreshed on any parameter change.

Seems fine; we could present "always regenerate when any parameter changes" as the first, simplest pass when explaining this, then show people when and why they would want to go beyond that. Python sure needs an All to complement its None, though!

[philippjfr]

I think this has gotten a bit confused as we are talking about slightly different things. When I'm talking about a rerender I mean redrawing a plot and, in case of bokeh, completely replacing all associated models. That's distinct from completely regenerating the HoloViews object in a single method. A change in a viewable decorated method should always trigger a full re-render, so you only declare parameters depending on it when you need to, i.e. when switching out a plot completely (e.g. when switching from a datashaded to a non-datashaded plot).

When you're using the dynamic utility I have no problem if a completely undecorated method means that you want to depend on all parameters, e.g. you could do something like this:

def monolithic_update(self, x_range, y_range, width, height):
    ...
    return tiles * plot

@param.viewable
def viewable(self):
    return dynamic(self.monolithic_update)

This way your monolithic update method can easily do all the stuff you've described but it's still different from this pattern, which completely rerenders on every parameter change:

@param.viewable
@param.depends('all')
def monolithic_update(self):
    ...
    return tiles * plot

[jbednar]

Right; I agree -- the most common should not be to assume we need to do something like swap out a datashaded version of a plot. The default is that something about the data or options is going to change, and the existing plot needs updating to reflect that, but not that the entire type of plot will change. So our first example shouldn't include "datashaded", and anything like that should be explicitly forced and we need to explain why in that case.

[jbednar]

Ok, it sounds like we may have a workable plan? Is there anything stopping us from going ahead with this? It seems like a much more workable approach than what we have now, to me, and something that will let us build complex, maintainable GUIs in the browser, while also supporting a lot of workflows that we probably don't even know yet.

So,

1. Should we go ahead with this?
2. Who should implement which bits?
3. How soon could this be feasible?

[jlstevens]

I think we need a prototype first where the dynamic utility is implemented (in parambokeh?) and we pretend we have access to the information that would be made available to param via the decorators. It will be easier to discuss the individual components of the solution once we have a proof-of-concept, even if it is hacked together for now.

[philippjfr]

Right agreed, prototyping a basic param.depends implementation and the dynamic utility which will make use of that information shouldn't be too difficult. I think the main difficulty will come in resolving the dependencies appropriately, which isn't necessarily required for a proof of concept.

[jbednar]

We agreed a few weeks back that (1) we should go ahead with prototyping this, (2) that @ceball (assisted by the rest of us as needed) will do the main work, but didn't really schedule it. As part of another project we have now scheduled this to start as of November 15.

[jbednar]

Unfortunately, no work was done in November or December either, due to other priorities that came up. :-(

We do still plan to do this, when we have time. To bring this issue up to date, some notes from what I remember:

1. We plan to start with proposal 0.3 above, with modifications as below.

2. If we can merge Allow dynamically adding new subplots holoviews#2174, we should be able to change element types, numbers of objects, etc. dynamically within a Bokeh figure, obviating the need for completely replacing plots as worried above. We should thus be able to present a single approach that covers more different needs, which will make this work simpler. @philippjfr can add details, including information on what's needed to be able to merge 2174.

3. Where it says "param.viewable" in the 0.3 code above, instead I believe we decided to have a general "param.output" or "param.returns" decorator that specifies the output type, and we can have parambokeh, etc. look for types that it knows how to display so that it can generate output.

@jlstevens, @ceball, and @philippjfr, please edit this comment or add explanations below if I mischaracterized something or you think we should have a different plan.

[philippjfr]

If we can merge holoviz/holoviews#2174, we should be able to change element types, numbers of objects, etc. dynamically within a Bokeh figure, obviating the need for completely replacing plots as worried above.

This is not quite true, HoloViews will probably never provide a construct that let's you add new plots to a layout. That said the proposal above does have a clear way to replace/redraw plots entirely so I'm not sure why we need to abandon a mechanism to do a full rerender at the parambokeh level, it's always going to be a useful thing to do. As for the PR, I don't think it will take too much to get it merged (it's basically working already) but it's a fundamental change and will require a fair amount of testing.

Otherwise that discussion and the plan make sense to me.

[jbednar]

Thanks. Just to be clear, I didn't mean we should "abandon" the proposal for full rerendering, just that I think it's no longer going to be required very often, and it's not desirable when it's not required.

[ceball]

We've implemented the bits of this we think we need. New issues should be opened for any required functionality that didn't happen, if not already covered by #262 or other existing issues (e.g. #140, #196).