Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support injecting via generics #6

Merged
merged 43 commits into from
Jul 13, 2023
Merged

Support injecting via generics #6

merged 43 commits into from
Jul 13, 2023

Conversation

SimonHeybrock
Copy link
Member

This replaces #4.

Details in Container are a proof of concepts and likely still wrong in some cases. Will refactor and add tests after initial discussion. complex_workflow_test.py provides an example of how actual client code might look. Note the slightly verbose creation of parametrized/scoped domain types, but the simple/trivial way of using functions with generics in their type hints with Container.

@SimonHeybrock
Copy link
Member Author

I pushed a major update, the mechanism is now more or less at a point where I wanted to get. This is ready for a more detailed review.

return wrapper
def _bind_free_typevars(tp: type, bound: Dict[TypeVar, type]) -> type:
if isinstance(tp, TypeVar):
return bound[tp]
Copy link
Member

@YooSunYoung YooSunYoung Jul 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it also be bound.get(tp, tp) in case it's not bound to anything, or is that not possible at all?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want it to fail in this case, so __getitem__ works (to get KeyError). I could use what you suggest, to get an UnsatisfiedRequirement elsewhere. Not sure which is better? Or maybe we should have another specific exception like UnboundTypeVar?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think raising UnboundTypeVar makes more sense because that's what you'll get from static type checker also.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I think it shouldn't allow TypeVar as a return annotation at all, since it will not be injected to anyone, right....?

args = get_args(key)
if args in subproviders:
raise ValueError(f'Provider for {key} already exists')
subproviders[args] = provider
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we forbid this kind of nested generic arguments...?

VT = TypeVar("VT")
Bike = NewType("Bike", object)
Car = NewType("Car", object)

class Time(sl.Scope[VT], float): ...

def provide_bike_time() -> Time[Bike]:
  return Time(1)
def provide_car_time() -> Time[Car]:
  return Time(0.1)

def provide_list_of_time(bike_time: Time[VT]) -> List[Time[VT]]
  return [bike_time]

It should be

TimeT = TypeVar("TimeT")
def provide_list_of_time(bike_time: TimeT) -> List[TimeT]
  return [bike_time]

And then we can compute nested generic type like; List[Time[Car]] or List[Time[Bike]].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good point, will have to think about whether this will be required. I opened #9.

@YooSunYoung
Copy link
Member

YooSunYoung commented Jul 13, 2023

If there is a provider that loads file and if the file is replaced, we need to rebuild the dask delayed again.
In that case, should we just rebuild the Container ...?
Or do you want to have an interface that clears the Container._cache??

@SimonHeybrock
Copy link
Member Author

If there is a provider that loads file and if the file is replaced, we need to rebuild the dask delayed again. In that case, should we just rebuild the Container ...? Or do you want to have an interface that clears the Container._cache??

I don't think we need to do anything? Container._cache is not caching results, it is caching tasks. If you run the pipeline again, it should load the file again.

@YooSunYoung
Copy link
Member

Now I remembered that once the delayed.compute() is done, it will no long using the previous intermediate results...
Then it should be fine indeed.
It will still use the same function even if we replace any existing provider,
but it's not allowed anyways so it is not an issue either I guess...?

@SimonHeybrock
Copy link
Member Author

It will still use the same function even if we replace any existing provider, but it's not allowed anyways so it is not an issue either I guess...?

Yes, I am indeed not allowing this currently, to avoid such complications.

@SimonHeybrock SimonHeybrock merged commit a9380f1 into main Jul 13, 2023
@SimonHeybrock SimonHeybrock deleted the child-containers branch July 13, 2023 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants