ASHLAR/MCQUANT modules #534
Comments
|
Thanks, @MDHowe4. I've been wanting to add support for ASHLAR's filepattern feature for quite a while, so I can work on that next. Can you tell me a little bit more your workflow? Your I'm not sure I follow the MCQUANT issue. Are you suggesting to make |
|
Hello, my name is Emmanuel and I am working with @MDHowe4 on creating a mutichannel immunofluorescence pipeline for Akoya/CODEX images. The issue we have run into is that newer Akoya CODEX software outputs qptiffs which combine all field of views into a single tiff file. This is great to save space but is not compatible with downstream analysis (including MCMICRO I believe) because we need to process each region separately. I was able to take in the output tiles from the CODEX machine using filepattern to recreate the OME.TIFF for each region that we need using ASHLAR but ran into issues with ASHLAR introducing a single pixel gaps between some of the tiles in certain cycles. (see image.sc forum below if interested) I was able to overcome this by copying/renaming all the tiles so that ASHLAR recognizes them as a single cycle with n number of channels. I do this through a python script. This preprocessing is done for all the regions of an experiment. Once all the tiles have been renamed, the bash script above iterates through each of the regions, runs ASHLAR for each region, producing a single ome.tiff for each region. In our use case, it would be nice if MCMICRO would recognize the filepattern so that we could start our pipieline on the renamed tiles as opposed to having to stitch/run ashlar ourselves outside of MCMICRO. Please let me know if anything is unclear. https://forum.image.sc/t/ashlar-how-to-pass-multiple-images-to-be-stitched/49864/51?page=3
|
|
Thanks for providing additional context. My question was more on the output side. After you run ASHLAR, what do you do with the output Do you structure them like this? or like this? |
|
We have been structuring as example number 1 you gave. Sorry, was under the impression you meant on the ASHLAR side. |
|
Hi @MDHowe4 and @emmanuel-contreras, I have a prototype solution in my own fork. Can you try running it on just one of your regions and let me know if it works? Structure your data as follows: and add the following to your (NOTE that I replaced Use the following command to feed the above directory to my fork ( If this works, I will merge the prototype into the main MCMICRO pipeline and work on supporting multiple images in the |
|
Thanks for all the help Artem, I pulled and executed your branch pipeline as you described above and received this error: For posterity I did modify some of the inputs above to make it compatible with our cluster environment. I submitted this to our SLURM queue: And used the custom config that I've been using previously: and the params.yml file: |
|
This is apparently an error that @emmanuel-contreras had seen previously [https://github.com/labsyspharm/ashlar/issues/170] that was solved by bumping ashlar to version: 1.18.0b3. |
|
I went ahead and just did that once I realized I could update the version directly within MCMICRO. I have bumped the version on my side up and used this updated params.yml file: Which resulted in this error: However, within the work directory I was able to find a stitched reg001.ome.tif. Upon inspection this appears to be the same as produced by the container outside MCMICRO. I believe the command is therefore functioning as expected, but there is a likely unrelated error with the PublishDir directive. |
|
Hi @MDHowe4, Yes, this is unrelated. We recently parameterized how the files are "published" from work directories to the project directory. Because you are supplying your config file with to your I'll go ahead and merge my fork into |
|
Hi @ArtemSokolov, Sorry for the delayed reply. I added the parameter you suggested and everything ran smoothly once re-ran. We look forward to any updates on supporting multiple images at once. |
|
Thanks a ton @ArtemSokolov for adding that functionality. I pulled the updated main branch and it ran perfectly on an input directory containing just a single region (reg001). However, when a new test directory containing two input tile directories reg001 and reg002 it fails to complete due to failing to recognize the reg002 pattern in the filenames for the second directory as seen here in the case where we match reg001: This isn't surprising obviously but I am unable to find a pattern that matches for regXXX name cases. I have tried reg001 --> reg{series:3} which did not work in the pattern matching. Any ideas for how to handle our specific case here? |
|
Hi @MDHowe4, This is actually very straightforward to do at the MCMICRO level, now that the pipeline keeps track of multiple samples. Can you try It should now replace Let me know if it's still giving you issues. |
|
@MDHowe4 I hope that this is working as intended now. Going to close this issue, but feel free to reopen if you are still running into problems. |
Hello,
Our group is planning to use MCMICRO and Minerva analysis/gater as a workflow for multiplexed image analysis and had a couple issues and suggestions on these two modules. We have been able to get MCMICRO to work on our codex tiles, but it doesn't seem possible currently to integrate our ASHLAR step (which is currently being run outside MCMICRO) within MCMICRO. Our current ASHLAR command is as follows:
The only function of ASHLAR we are using is the stitching step to produce single ome.tiff images that are compatible with downstream steps in MCMICRO, however, the images coming off the codex machine aren't bio-formats compatible and we have to preprocess the tiles into a structure that ASHLAR recognizes. This has worked well as we are able to produce an equivalent output to the Akoya processor for individual images, but since we are using the undocumented “filepattern” command we have used the separate ASHLAR container to run this step. It doesn't currently seem possible to recreate this within MCMICRO to parse our specific data formats. Any suggestions? It would be ideal to have as many steps handled by MCMICRO, but I understand dealing with all of these non-standard data formats is challenging and more options are probably planned in the nf-core pipeline.
MCQUANT currently only outputs mean_intensity and if specified,
--intensity_propsor--mask_props. This works perfectly when we also want the median_intensity outputs, but the only option is to concatenate this data to the right of mean_intensity within the CSV. This has poor compatibility in Minerva analysis/gater if you plan to gate on the median_intensity as Minerva_analysis/gater will default to the mean_intensity columns. Shuffling all of the channels isn't really feasible either with large numbers of channels. My suggestion would be to allow exact control of the output from MCQUANT for better compatibility. I suppose this could also be handled in a future update by Minerva analysis/gater. This is only a minor obstacle in our workflow that can be easily handled by post-processing, but could maybe be handled within MCMICRO as well.The text was updated successfully, but these errors were encountered: