Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle ImageDataSets on Kedro-viz #1043

Closed
1 task done
rashidakanchwala opened this issue Sep 2, 2022 · 3 comments
Closed
1 task done

Handle ImageDataSets on Kedro-viz #1043

rashidakanchwala opened this issue Sep 2, 2022 · 3 comments
Labels
Issue: Feature Request Quick Win Low/Medium priorities but quick to do

Comments

@rashidakanchwala
Copy link
Contributor

rashidakanchwala commented Sep 2, 2022

Description

We recently started supporting Matplotlib plots on Kedro-viz which are basically static pngs and use dataset MatplotlibWriter. It makes sense to extend this support to other kedro datasets like ImageDataSet which also use png.

Recently, we also implemented a new load method for plotly, matplotlib and tracking datasets. At the moment, if you try to click on an ImageDataSet it gives you the following Warning

kedro.io.core - WARNING - exists()not implemented forImageDataSet. Assuming output does not exist.

Maybe implementing the above will also fix this issue.

To replicate the above error -- run the kedro-viz demo project locally and click on cancellation_policy_grid node.

Context

People who use ImageDataSets can also start seeing their images on Kedro-viz

Checklist

  • Include labels so that we can categorise your feature request
@rashidakanchwala rashidakanchwala added Issue: Feature Request Quick Win Low/Medium priorities but quick to do labels Sep 2, 2022
@rashidakanchwala rashidakanchwala changed the title Handle ImageDataSets Handle ImageDataSets on Kedro-viz Sep 2, 2022
@antonymilne
Copy link
Contributor

I think this is a good idea and could be easily added as a Quick Win, but I'm pretty sure that ImageDataSet is way less popular than the ones we've added the previews for already, so I don't think it is high priority. We could easily implement it following the current scheme, but the code would add another case to the logic in flowchart.DataNode, which is already quite messy. As per #980 and the comments in code that say TODO: improve this scheme, I don't like how we currently have lots of different switches for self.dataset_type == "kedro.extras.datasets.matplotlib.matplotlib_writer.MatplotlibWriter" etc. So maybe we should figure out a better way to refactor all this. Or, even more ambitious, we would try to solve the more general issue here: #907.

So yeah, definitely makes sense to add this. It's just a question of whether we want to add it as a quick fix or figure out a more general solution.

Also note:

  • the WARNING message should no longer be emitted after Remove "exists() not implemented" warning #1044
  • the demo-project uses a custom demo_project.extras.datasets.image_dataset.ImageDataSet which is not the same as kedro's builtin dataset type and crucially doesn't expose an exists method. So if we implement this for ImageDataSet then we'd need to change the dataset type to the builtin one for the image to show up

@rashidakanchwala
Copy link
Contributor Author

rashidakanchwala commented Sep 5, 2022

I think what you said makes a lot of sense. I will also leave this issue to be reviewed for backlog grooming but I also agree ImageDataSet is minor. Maybe we can create a case for this issue again when there's user demand !!

@tynandebold
Copy link
Member

We'll close this for now and will reopen if and when there's user demand for this 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue: Feature Request Quick Win Low/Medium priorities but quick to do
Projects
None yet
Development

No branches or pull requests

3 participants