diff --git a/changelog/531.doc.rst b/changelog/531.doc.rst new file mode 100644 index 00000000..3650b0cc --- /dev/null +++ b/changelog/531.doc.rst @@ -0,0 +1 @@ +Add a how-to guide explaining how to replace outdated dataset metadata files. diff --git a/docs/howto_guides/asdf_warnings.rst b/docs/howto_guides/asdf_warnings.rst new file mode 100644 index 00000000..6b94c495 --- /dev/null +++ b/docs/howto_guides/asdf_warnings.rst @@ -0,0 +1,29 @@ +.. _dkist:howto-guide:asdf-warnings: + +Understand version warnings when loading ASDFs +============================================== + +When you load a recently-downloaded ASDF file you may see a warning something like this: + +:: + + AsdfPackageVersionWarning: File '' was created with extension URI 'asdf://asdf-format.org/astronomy/gwcs/extensions/gwcs-1.2.0' (from package gwcs==0.24.0), but older package (gwcs==0.22.0) is installed. + + +The extension and package it references will vary. +This warning means that the `dkist` package or one of its dependencies is out of date. +To correct this, you should update your Python tools installation (and its dependencies) with + +.. code-block:: bash + + pip install --upgrade dkist + + +if you installed using `pip` or + +.. code-block:: bash + + conda update dkist + + +if you used `conda`. diff --git a/docs/howto_guides/index.rst b/docs/howto_guides/index.rst index 4e288539..1d6ace5f 100644 --- a/docs/howto_guides/index.rst +++ b/docs/howto_guides/index.rst @@ -11,3 +11,5 @@ If you're starting fresh you might want to check out the :ref:`dkist:tutorial:in :maxdepth: 1 sample_data + replacing_asdfs + asdf_warnings diff --git a/docs/howto_guides/replacing_asdfs.rst b/docs/howto_guides/replacing_asdfs.rst new file mode 100644 index 00000000..350e9dbf --- /dev/null +++ b/docs/howto_guides/replacing_asdfs.rst @@ -0,0 +1,47 @@ +.. _dkist:howto-guide:replacing-asdfs: + +Update Previously Downloaded ASDF Files +======================================= + +The DKIST Data Center will occasionally update the ASDF files for all the datasets it stores, due to changes or corrections to the metadata. +This means that ASDF files you have downloaded previously might become outdated and need to be re-downloaded. + +You should periodically re-download your local ASDF files to keep up to date with changes to the dataset metadata. +To do so, you can use the `overwrite` keyword argument when downloading an ASDF file. + +For example: + +.. code-block:: python + + Fido.fetch(res, path="~/sunpy/data/{instrument}/{dataset_id}", overwrite=True) # doctest: +SKIP + + +where ``res`` is the result of a ``Fido`` search for some dataset you already have saved locally. + +You might also need to force a refresh of the sample data which is included with the Python tools. +Again, we can use the ``overwrite`` keyword for this. + +.. code-block:: python + + from dkist.data.sample import download_all_sample_data + + download_all_sample_data(overwrite=True) # doctest: +REMOTE_DATA + + +In the past, the naming convention for the metadata ASDF files has also changed, meaning that the usual checks to stop you downloading a dataset you already have locally may fail and you will end up with two separate metadata files. +If you try and load a dataset where a file with an old name is present you will see a warning similar to this: + +.. code-block:: python + + >>> from dkist import load_dataset + >>> ds = load_dataset('~/sunpy/data/VISP/AGLKO/') # doctest: +SKIP + + WARNING: DKISTUserWarning: ASDF files with old names (VISP_L1_20221024T212807_AGLKO_user_tools.asdf) were found in this directory and ignored. You may want to delete these files. [dkist.dataset.loader] + + +When this happens the newer ASDF file is loaded so the old one can safely be ignored. +However, to remove the warning the old file can simply be deleted or moved elsewhere. + +Note that this warning was added in dkist version 1.10.0. +In older versions the loader will return a list containing the corresponding dataset for each ASDF file present, which is likely to cause problems. +Deleting the old file will still solve the issue, although you should also update your Python tools installation to v1.10.0 or later.