Conda package for RAiDER

[jlmaurer]

Hi all,

I'm prepping for developing a conda package for RAiDER. This is needed to address a lot of install problems users have been reporting. However, there are a few items that need cleaned up prior to building the recipe. The main one is pip dependencies of RAiDER, as these are not explicitly supported by conda recipes, as noted in this stack overflow issue, although one can include them when building the package. We have three packages that are currently installed using pip: geopandas, boto3, and dem-stitcher. Does anyone know if there is a reason we currently install geopandas and boto3 through pip instead of conda, since they already have conda builds available?

Pinging @dbekaert @cmarshak @sssangha @bbuzz31

[cmarshak]

For dem-stitcher, we used rasterio. It has some issues in 3.10. I have found that PyPI tends to be slightly more up-to-date than the conda-forge channel (this could be simply my lack of experience with conda).

If you put everything in conda-forge just make sure to use 3.3* gdal when using rasterio as there is currently is unexpected behavior in 3.4 (see [1]). Here is an example env that is being tested in dem-stitcher.

Definitely, use as much conda-forge with rasterio when using netcdf4 and rioxarray! It will be a further headache to get these libraries to play nicely. Hope this helps.

[1] conda-forge/rasterio-feedstock#243

[jhkennedy]

With conda-forge/staged-recipes#19470 and #350 merged, RAiDER is very close to ready for conda-forge!

What's left is:

• finish update dem.py with new dem-stitcher api #339 so that RAiDER depend on dem_sticher v2.3.0 or later as that's the only version available on conda-forge:

  $  conda search -c conda-forge dem_stitcher 
  Loading channels: done
  # Name                       Version           Build  Channel             
  dem_stitcher                   2.3.0    pyhd8ed1ab_0  conda-forge

• update version.txt and tag a new (pre-, or beta, or whatever) release as we need a git archive link for the recipe.

Once those are done, I can start the recipe.

[jlmaurer]

@jhkennedy thanks! I was about to issue a PR to use the new conda version of dem-stitcher, but local testing wasn't working for me. I got confused until I realized that I'm on my new mac, and I'm guessing dem-stitcher doesn't yet support the arm processors; have the same issue with isce. Any idea if this is a conda thing that will eventually be fixed or something that the maintainers have to take care of?

[jhkennedy]

@jlmaurer there's a bit of work to do to migrate packages to arm64. Details are here:
https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/

We'll have to migrate dem-sticher (and ISCE) and any other packages that haven't been migrated yet. I recently did it for autoRIFT and it was easier than expected overall, but I'm still not particularly confident in those builds yet.

You can also use rosetta emulation to install x86 conda envs on Mac:
https://towardsdatascience.com/how-to-manage-conda-environments-on-an-apple-silicon-m1-mac-1e29cb3bad12

[jlmaurer]

@jhkennedy I just tagged a new release and updated the version.txt file (merged #357 ). You should be able to start on the recipe now. Thanks!

[jhkennedy]

I've started the PR here:
conda-forge/staged-recipes#20100

@cmarshak @jlmaurer if you could comment on it you're willing to be a maintainer, that'd be great.

[jlmaurer]

hi @dbekaert @jhkennedy @cmarshak @bbuzz31 @sssangha I have successfully installed and run RAiDER from conda-forge! Thanks a ton @jhkennedy for your help getting this one going. @dbekaert I'll be working on a PR to update the front-facing readme.

[jlmaurer]

Closing this as conda channel is open!