Adding Figure 4 - Leave One Image Out Analysis #40
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
I have a few questions and comments here:
- For panel A, what is "rank of prediction"? Does 1 mean the best prediction group or something? Also, why is rank 1 have high prediction probability for both correctly predicted phenotype and not? Wouldn't we expect that yes(correct prediction) should be the highest all the across?
- For panel B, what is a good output here? Also, what is the blue dotted line represent? Is each point a feature from each data feature space? Do we want to see more points on the left side of the blue line?
- For panel C, this seems the most clear that with the LOIO that there are more correct prediction than not but I don't know what "strict threshold" means. Can you explain what that is?
- For panel D, the x-axis title looks wrong to me... If the legend is yes or no if the phenotypic class was predicted correctly, then why would the x-axis represent "cell proportions of incorrect phenotype predictions". This seems contradictory to me.
There was a problem hiding this comment.
Sorry for my delay in answering these questions. Thanks for asking them. I will do my best to address below. We will definitely need to clarify these in the future paper.
- For each cell, we receive 15 phenotype predictions (p values per phenotype). The x axis is simply this prediction, rank-ordered by probability. The color means if the prediction was correct or not. For example, a cell could be labeled by apoptosis and we predict interphase with 0.8, apoptosis with 0.15 and all the rest add up to 0.5. This cell is in the plot 15 different times, yellow in each case except it is blue in x-axis rank 2.
- Yep, correct. A good output is everything to the left of the dotted line in the
Shuffled: FALSEmodels. - I added a comment describing what "high_threshold" was in the plot. This should help with interpretation, but we will definitely also add clarity in the manuscript!
- Ah! Thanks for bringing this up. The important distinction here is "Categories". These are the higher-order categories that mitocheck assigned (so groups of individual phenotypes). Your comment here made me change the legend to be more clear! Thanks
|
|
||
| output_fig_loio <- file.path("figures", "main_figure_4_loio.png") | ||
|
|
||
| # Set constants |
There was a problem hiding this comment.
I recommend adding more to this comment, since there is only one constant which is the threshold. What does this threshold represent and why that number.
There was a problem hiding this comment.
Good point, will do in next commits!
|
The new LOIO should be done today - I'll address your comments and update the visualization later! (sorry for the delay!) |
|
New LOIO analysis completed! I will get this merged in and we can discuss at our meeting today |
Note: This figure will change. I am retraining the models (performing a new grid search for each individual image left out) and adding shuffled models. The shuffled models will likely belong in a supplement, not the main figure