new pyaps0.3.6 can not download dataset #1321

fukun364202818 · 2025-02-10T11:47:05Z

Description of the problem

The program encountered an error at the "step - correct_troposphere" step.

Full script that generated the error

PASTE CODE HERE

Full error message

******************** step - correct_troposphere ********************
Input data seems to be geocoded. Lookup file not needed.
Atmospheric correction using Weather Re-analysis dataset (PyAPS, Jolivet et al., 2011)
Weather Re-analysis dataset: ERA5

tropo_pyaps3.py -f /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries.h5 --model ERA5 -g /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 -w ${WEATHER_DIR}
weather model: ERA5 - dry (hydrostatic) and wet delay
weather directory: /home/PycharmProjects/hyp3_mintpy/2025_junlian
output tropospheric delay     time-series file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
output corrected displacement time-series file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries_ERA5.h5
read dates/time info from file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries.h5
time of cloest available product: 11:00 UTC
make directory: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5

--------------------------------------------------------------------------------
Download global atmospheric model files...
update mode: ON
output file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
1) output file either do NOT exist or is NOT newer than all GRIB files.
run or skip: run
--------------------------------------------------
downloading weather model data using PyAPS ...
number of grib files to download: 32
--------------------------------------------------
INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-10 19:32:16,309 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-10 19:32:16,309 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 32: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5/ERA5_N20_N40_E100_E110_20240107_11.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2024'], 'month': ['01'], 'day': ['07'], 'time': ['11:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 100, 20, 110]}
WARNING: the 1 attempt to download failed, retry it.

INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-10 19:32:17,285 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-10 19:32:17,285 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 32: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5/ERA5_N20_N40_E100_E110_20240107_11.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2024'], 'month': ['01'], 'day': ['07'], 'time': ['11:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 100, 20, 110]}
WARNING: the 2 attempt to download failed, retry it.

INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-10 19:32:18,258 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-10 19:32:18,258 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 32: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5/ERA5_N20_N40_E100_E110_20240107_11.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2024'], 'month': ['01'], 'day': ['07'], 'time': ['11:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 100, 20, 110]}


**************************************************
WARNING: downloading failed for 3 times, stop trying and continue.
**************************************************



--------------------------------------------------------------------------------
Calculate tropospheric delay and write to HDF5 file...
update mode: ON
output file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
1) output file either do NOT exist or is NOT newer than all GRIB files.
run or skip: run
open geometry file: geometryGeo.h5
reading incidenceAngle  data from file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 ...
reading height          data from file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 ...
UTM coordinates detected, convert UTM into lat/lon
--------------------------------------------------
create HDF5 file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5 with w mode
create dataset  : date       of |S1                       in size of (0,)                 with compression = None
create dataset  : timeseries of <class 'numpy.float32'>   in size of (0, 2837, 3579)      with compression = None
close  HDF5 file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
--------------------------------------------------
calculating absolute delay for each date using PyAPS (Jolivet et al., 2011; 2014) ...
number of grib files used: 0
Traceback (most recent call last):
  File "/root/miniconda3/envs/hyp3-mintpy/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/cli/smallbaselineApp.py", line 209, in main
    run_smallbaselineApp(inps)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/smallbaselineApp.py", line 1155, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/smallbaselineApp.py", line 935, in run
    self.run_tropospheric_delay_correction(sname)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/smallbaselineApp.py", line 682, in run_tropospheric_delay_correction
    mintpy.cli.tropo_pyaps3.main(iargs)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/cli/tropo_pyaps3.py", line 166, in main
    run_tropo_pyaps3(inps)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/tropo_pyaps3.py", line 741, in run_tropo_pyaps3
    calc_delay_timeseries(inps)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/tropo_pyaps3.py", line 671, in calc_delay_timeseries
    prog_bar = ptime.progressBar(maxValue=num_date, print_msg=~inps.verbose)
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/objects/progress.py", line 84, in __init__
    self.reset()
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/objects/progress.py", line 90, in reset
    self.update_amount(0)  # Build progress bar string
  File "/root/miniconda3/envs/hyp3-mintpy/lib/python3.9/site-packages/mintpy/objects/progress.py", line 109, in update_amount
    percentDone = (diffFromMin / float(self.span)) * 100.0
ZeroDivisionError: float division by zero

System information

Operating system:
Python environment:
MintPy version:
InSAR processor/product:

The text was updated successfully, but these errors were encountered:

codeautopilot · 2025-02-10T11:47:44Z

Potential solution

The plan to solve the bug involves addressing two main issues: the failure to download weather model data due to outdated API syntax and the ZeroDivisionError in the progress bar calculation. The solution involves updating the API request syntax to align with the latest CDS API requirements and modifying the progress bar logic to handle cases where there are no dates to process.

What is causing this bug?

The bug is caused by two primary issues:

API Syntax Update: The CDS API syntax has changed, leading to failed download attempts for the weather model data. The script does not handle these changes, resulting in incomplete data retrieval.
ZeroDivisionError in Progress Bar: The error occurs when the progress bar attempts to calculate the percentage done with a zero span, which happens when there are no dates to process (num_date is zero).

Code

To address these issues, the following code modifications are recommended:

Update API Request Syntax: Ensure the API request syntax is up-to-date with the latest CDS API requirements. This involves reviewing the API documentation and adjusting the request parameters accordingly.
Modify Progress Bar Logic: Update the progressBar class to handle zero-span scenarios gracefully.

Example Code Modification for Progress Bar

# In src/mintpy/objects/progress.py

def update_amount(self, newAmount=0, suffix=''):
    ...
    if self.span == 0:
        percentDone = 0
    else:
        diffFromMin = float(self.amount - self.min)
        percentDone = (diffFromMin / float(self.span)) * 100.0
    ...

How to replicate the bug

To replicate the bug, follow these steps:

Set Up Environment: Ensure you have the same environment as described in the system information (Ubuntu 20.04, Conda, MintPy 1.6.1).
Run the Script: Execute the script that performs the "correct_troposphere" step using the PyAPS tool with the ERA5 weather model.
Observe the Error: The script will attempt to download weather model data and fail due to outdated API syntax, leading to a ZeroDivisionError when processing the progress bar.

By implementing the proposed solutions, the script should handle API syntax changes and zero-span scenarios in the progress bar, preventing the bug from occurring.

Click here to create a Pull Request with the proposed solution

Files used for this task:

Changes on src/mintpy/cli/tropo_pyaps3.py

Analysis of `src/mintpy/cli/tropo_pyaps3.py`

The file tropo_pyaps3.py is a command-line interface script for performing tropospheric corrections using weather models via PyAPS. The script is part of the MintPy package and is responsible for parsing command-line arguments, validating inputs, and invoking the main processing function run_tropo_pyaps3.

Key Observations

Argument Parsing: The script uses argparse to handle command-line arguments. It includes options for specifying input files, date lists, weather models, and output files.
Weather Model Selection: The script currently supports the ERA5 weather model for atmospheric corrections. The choice of model is specified using the -m or --model argument.
Weather Directory Handling: The script expands environment variables and user directories for the --weather-dir argument, defaulting to the current directory if not specified.
Error Handling: The script checks for the existence of input files and ensures that either a file or a date list is provided. It raises appropriate errors if these conditions are not met.
Default Values: The script sets default values for several options, such as the tropospheric delay file and the corrected displacement file, based on the provided inputs.

Potential Causes of the Bug

API Syntax Update: The error message indicates that the CDS API syntax has changed, which could lead to failed download attempts. The script does not appear to handle API syntax updates or exceptions related to failed downloads.
Download Failures: The script does not include logic to handle repeated download failures beyond retry attempts. This could result in the process continuing with incomplete data, leading to downstream errors.

Recommendations

Update API Request Syntax: Ensure that the API request syntax used for downloading weather model data is up-to-date with the latest CDS API requirements. This may involve reviewing the API documentation and adjusting the request parameters accordingly.
Exception Handling for Downloads: Implement robust exception handling for download operations. This should include logging detailed error messages and potentially aborting the process if critical data cannot be retrieved.
User Feedback: Provide clear feedback to the user when downloads fail, including potential reasons and steps to resolve the issue (e.g., checking API credentials or network connectivity).

By addressing these areas, the script can be made more resilient to changes in external APIs and improve its handling of download-related errors.

Changes on src/mintpy/objects/progress.py

Bug Analysis Report for `src/mintpy/objects/progress.py`

Issue Summary

The user reported a bug where the program fails at the "correct_troposphere" step due to a ZeroDivisionError. The error occurs when the progress bar is being updated, specifically when calculating the percentage done, which involves dividing by the span (difference between max and min values).

Analysis

The ZeroDivisionError is raised in the update_amount method of the progressBar class. This happens because the span attribute, which is calculated as maxValue - minValue, is zero. This situation arises when maxValue is equal to minValue, which implies that there are no dates to process (i.e., num_date is zero).

Relevant Code Section

def update_amount(self, newAmount=0, suffix=''):
    ...
    diffFromMin = float(self.amount - self.min)
    percentDone = (diffFromMin / float(self.span)) * 100.0
    ...

Potential Cause

The error is likely caused by the num_date being zero, which results in maxValue being set to zero during the initialization of the progressBar object. This leads to a division by zero when calculating percentDone.

Recommendations

To prevent the ZeroDivisionError, the progressBar class should handle cases where maxValue is zero. Here are some potential solutions:

Check for Zero Span: Before performing the division, check if span is zero and handle it appropriately, such as setting percentDone to zero or skipping the progress bar update.
Conditional Initialization: When initializing the progressBar, ensure that maxValue is greater than minValue. If not, skip the progress bar creation or set a default non-zero span.
Graceful Handling: Modify the update_amount method to handle zero-span scenarios gracefully, possibly by displaying a message indicating no progress is needed.

Example Code Modification

def update_amount(self, newAmount=0, suffix=''):
    ...
    if self.span == 0:
        percentDone = 0
    else:
        diffFromMin = float(self.amount - self.min)
        percentDone = (diffFromMin / float(self.span)) * 100.0
    ...

Conclusion

The ZeroDivisionError in the progress bar implementation is due to a zero-span scenario, which occurs when there are no dates to process. Implementing checks and handling for zero-span cases will resolve the issue and prevent the error from occurring.

This comment was generated by AI. Information provided may be incorrect.

Current plan usage: 0%

Have feedback or need help?
Documentation
[email protected]

yunjunz · 2025-02-10T13:40:17Z

Please try to re-run the tropo_pyaps3.py with --debug-mode turned ON, after the above PR #1322 is merged, to show a more detailed error message regarding the ERA5 downloading.

tropo_pyaps3.py -f /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries.h5 --model ERA5 -g /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 -w ${WEATHER_DIR} --debug-mode

fukun364202818 · 2025-02-11T04:53:03Z

command:
tropo_pyaps3.py -f /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries.h5 --model ERA5 -g /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 -w /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5/ --debug-mode
output:

weather model: ERA5 - dry (hydrostatic) and wet delay
weather directory: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5
output tropospheric delay     time-series file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
output corrected displacement time-series file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries_ERA5.h5
read dates/time info from file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/timeseries.h5
time of cloest available product: 11:00 UTC

--------------------------------------------------------------------------------
Download global atmospheric model files...
update mode: ON
output file: /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/ERA5.h5
1) output file exists and is newer than all GRIB files.
2) output file does NOT have the same len/wid as the geometry file /home/PycharmProjects/hyp3_mintpy/2025_junlian/inputs/geometryGeo.h5 or does NOT contain all dates
run or skip: run
--------------------------------------------------
downloading weather model data using PyAPS ...
number of grib files to download: 32
--------------------------------------------------
INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-11 12:44:54,328 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-11 12:44:54,329 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 32: /home/PycharmProjects/hyp3_mintpy/2025_junlian/ERA5/ERA5/ERA5_N20_N40_E100_E110_20240107_11.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2024'], 'month': ['01'], 'day': ['07'], 'time': ['11:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [np.int64(40), np.int64(100), np.int64(20), np.int64(110)]}
Traceback (most recent call last):
  File "/root/tools/miniforge/envs/hyp3-mintpy/bin/tropo_pyaps3.py", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/root/tools/MintPy/src/mintpy/cli/tropo_pyaps3.py", line 172, in main
    run_tropo_pyaps3(inps)
    ~~~~~~~~~~~~~~~~^^^^^^
  File "/root/tools/MintPy/src/mintpy/tropo_pyaps3.py", line 739, in run_tropo_pyaps3
    inps.grib_files = dload_grib_files(
                      ~~~~~~~~~~~~~~~~^
        inps.grib_files,
        ^^^^^^^^^^^^^^^^
    ...<2 lines>...
        debug_mode=inps.debug_mode,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/root/tools/MintPy/src/mintpy/tropo_pyaps3.py", line 506, in dload_grib_files
    pa.ECMWFdload(date_list2dload, hour, grib_dir, **kwargs)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/pyaps3/autoget.py", line 131, in ECMWFdload
    c.retrieve(ds_name, indict, target=fname)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/datapi/legacy_api_client.py", line 169, in retrieve
    submitted = self.client.submit_and_wait_on_results(
        collection_id=name,
        request=request,
    )
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/datapi/api_client.py", line 413, in submit_and_wait_on_results
    return self._retrieve_api.submit(collection_id, request).make_results()
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/datapi/processing.py", line 727, in submit
    return self.get_process(collection_id).submit(request)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/datapi/processing.py", line 319, in submit
    job = Job.from_request(
        "post",
    ...<2 lines>...
        **self._request_kwargs,
    )
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/datapi/processing.py", line 172, in from_request
    response = robust_request(
        method, url, headers=headers, **request_options, **kwargs
    )
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/multiurl/http.py", line 479, in wrapped
    r = call(main_url, *args, **kwargs)
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
    ~~~~~~~~~^
        method=request.method.upper(),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<10 lines>...
        hooks=merge_hooks(request.hooks, self.hooks),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/requests/models.py", line 370, in prepare
    self.prepare_body(data, files, json)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/site-packages/requests/models.py", line 510, in prepare_body
    body = complexjson.dumps(json, allow_nan=False)
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ~~~~~~^^^^^
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/json/encoder.py", line 261, in iterencode
    return _iterencode(o, 0)
  File "/root/tools/miniforge/envs/hyp3-mintpy/lib/python3.13/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
                    f'is not JSON serializable')
TypeError: Object of type int64 is not JSON serializable

fukun364202818 · 2025-02-11T05:25:47Z

#1322
I also tried running python PyAPS/tests/test_calc.py and python PyAPS/tests/test_dload.py, and the results were correct.

command-1:

python PyAPS/tests/test_calc.py

output-1:

------------------------------------------------
import pyaps3 from /root/tools/PyAPS/src/pyaps3/__init__.py
------------------------------------------------
test tropospheric delay calculation from ERA5.
------------------------------------------------
read ISCE geometry files: hgt/los/lat/lon.rdr
calculate tropospheric delay from GRB files...
------------------------------------------------
INFO: INCIDENCE ANGLE AS AN ARRAY
INFO: AREA COVERAGE IN SNWE: (33.85, 30.05, 129.05, 132.45)
PROGRESS: READING GRIB FILE
INFO: USING PRESSURE LEVELS OF ERA-INT OR ERA-5 DATA
INFO: IMAGE DIMENSIONS: 15 LATITUDES AND 13 LONGITUDES
PROGRESS: INTERPOLATING FROM PRESSURE TO HEIGHT LEVELS
PROGRESS: COMPUTING DELAY FUNCTIONS
INFO: INCIDENCE ANGLE AS AN ARRAY
INFO: AREA COVERAGE IN SNWE: (33.85, 30.05, 129.05, 132.45)
PROGRESS: READING GRIB FILE
INFO: USING PRESSURE LEVELS OF ERA-INT OR ERA-5 DATA
INFO: IMAGE DIMENSIONS: 15 LATITUDES AND 13 LONGITUDES
PROGRESS: INTERPOLATING FROM PRESSURE TO HEIGHT LEVELS
PROGRESS: COMPUTING DELAY FUNCTIONS
PROGRESS: FINE INTERPOLATION OF HEIGHT LEVELS
PROGRESS: CREATE THE BILINEAR INTERPOLATION FUNCTION
PROGRESS: MAPPING THE DELAY
[============================================================]      0s /      0s
PROGRESS: FINE INTERPOLATION OF HEIGHT LEVELS
PROGRESS: CREATE THE BILINEAR INTERPOLATION FUNCTION
PROGRESS: MAPPING THE DELAY
[============================================================]      0s /      0s
------------------------------------------------
Passed tropospheric delay calculation from ERA5.

command-2:

python PyAPS/tests/test_dload.py

output-2:

------------------------------------------------
import pyaps3 from /root/tools/PyAPS/src/pyaps3/__init__.py
------------------------------------------------
test ERA5 data download
NOTE: Account setup is required on the Copernicus Climate Data Store (CDS).
      More detailed info can be found on: https://cds.climate.copernicus.eu/how-to-api
      Add your account info to ~/.cdsapirc file.
INFO: You are using the latest ECMWF platform for downloading datasets:  https://cds.climate.copernicus.eu/api
2025-02-11 13:19:23,776 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-11 13:19:23,776 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 1 of 2: /root/tools/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200601_14.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2020'], 'month': ['06'], 'day': ['01'], 'time': ['14:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 120, 30, 140]}
2025-02-11 13:19:24,289 INFO Request ID is 0d37cbe8-5991-4ce5-a6a7-13e20097ab50
2025-02-11 13:19:24,525 INFO status has been updated to accepted
2025-02-11 13:19:33,631 INFO status has been updated to running
2025-02-11 13:19:38,907 INFO status has been updated to successful
2025-02-11 13:19:43,825 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-11 13:19:43,825 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
Downloading 2 of 2: /root/tools/PyAPS/tests/data/ERA5/ERA5_N30_N40_E120_E140_20200901_14.grb
{'product_type': ['reanalysis'], 'variable': ['geopotential', 'temperature', 'specific_humidity'], 'year': ['2020'], 'month': ['09'], 'day': ['01'], 'time': ['14:00'], 'pressure_level': ['1', '2', '3', '5', '7', '10', '20', '30', '50', '70', '100', '125', '150', '175', '200', '225', '250', '300', '350', '400', '450', '500', '550', '600', '650', '700', '750', '775', '800', '825', '850', '875', '900', '925', '950', '975', '1000'], 'data_format': 'grib', 'area': [40, 120, 30, 140]}
2025-02-11 13:19:44,364 INFO Request ID is b4b92036-12b8-40f1-9fac-b0e0eb794da7
2025-02-11 13:19:44,587 INFO status has been updated to accepted
2025-02-11 13:19:58,952 INFO status has been updated to successful
------------------------------------------------
Downloads OK
------------------------------------------------

fukun364202818 · 2025-02-13T11:24:25Z

#1322
Okay, based on your error code hints, I believe I've resolved the issue. In the get_snwe function within tropo_pyaps3.py, the original return (S, N, W, E) was returning S, N, W, and E as NumPy's np.int64 type, while the expected type was standard Python's int. Therefore, I modified the return statement to return (int(S), int(N), int(W), int(E)), which successfully fixed the problem.

yunjunz · 2025-02-13T14:07:48Z

Great! That makes sense. Could you please issue a PR in mintpy for this fix? We could also fix this in pyaps3 repo for more robustness.

fukun364202818 · 2025-02-14T01:42:56Z

Great! That makes sense. Could you please issue a PR in mintpy for this fix? We could also fix this in pyaps3 repo for more robustness.

Okay, I've submitted the PR.

yunjunz mentioned this issue Feb 10, 2025

tropo_pyaps3: add --debug-mode option #1322

Merged

6 tasks

yunjunz mentioned this issue Feb 16, 2025

autoget.ECMWFdload(): bugfix for snwe in numpy.int64 or float insarlab/PyAPS#45

Merged

yunjunz linked a pull request Feb 16, 2025 that will close this issue

tropo_pyaps3.get_snwe(): return native int instead of numpy.int64 #1324

Merged

yunjunz closed this as completed in #1324 Feb 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new pyaps0.3.6 can not download dataset #1321

new pyaps0.3.6 can not download dataset #1321

fukun364202818 commented Feb 10, 2025

codeautopilot bot commented Feb 10, 2025 •

edited

Loading

Analysis of `src/mintpy/cli/tropo_pyaps3.py`

Key Observations

Potential Causes of the Bug

Recommendations

Bug Analysis Report for `src/mintpy/objects/progress.py`

Issue Summary

Analysis

Relevant Code Section

Potential Cause

Recommendations

Example Code Modification

Conclusion

yunjunz commented Feb 10, 2025 •

edited

Loading

fukun364202818 commented Feb 11, 2025

fukun364202818 commented Feb 11, 2025

fukun364202818 commented Feb 13, 2025

yunjunz commented Feb 13, 2025

fukun364202818 commented Feb 14, 2025

new pyaps0.3.6 can not download dataset #1321

new pyaps0.3.6 can not download dataset #1321

Comments

fukun364202818 commented Feb 10, 2025

codeautopilot bot commented Feb 10, 2025 • edited Loading

Potential solution

What is causing this bug?

Code

Example Code Modification for Progress Bar

How to replicate the bug

Analysis of src/mintpy/cli/tropo_pyaps3.py

Key Observations

Potential Causes of the Bug

Recommendations

Bug Analysis Report for src/mintpy/objects/progress.py

Issue Summary

Analysis

Relevant Code Section

Potential Cause

Recommendations

Example Code Modification

Conclusion

yunjunz commented Feb 10, 2025 • edited Loading

fukun364202818 commented Feb 11, 2025

fukun364202818 commented Feb 11, 2025

command-1:

output-1:

command-2:

output-2:

fukun364202818 commented Feb 13, 2025

yunjunz commented Feb 13, 2025

fukun364202818 commented Feb 14, 2025

codeautopilot bot commented Feb 10, 2025 •

edited

Loading

Analysis of `src/mintpy/cli/tropo_pyaps3.py`

Bug Analysis Report for `src/mintpy/objects/progress.py`

yunjunz commented Feb 10, 2025 •

edited

Loading