SDSS spectrum query fails (np.genfromtxt error related)

[banados]

I'm running into a problem when I query the SDSS spectrum of a specific object (other ~3000 were ok) but this might not be the only case where this happens.

I am using:

python 3.5.2
astroquery 0.3.2
astropy 1.2.1
numpy 1.11.1

Here is the example that fails:

from astroquery.sdss import SDSS
s = SDSS.get_spectra_async(plate=5177, mjd=56245, fiberID=114)

    $PATH/numpy/lib/npyio.py in genfromtxt(fname, dtype, comments, delimiter, skip_header,     skip_footer, converters, missing_values, filling_values, usecols, names, excludelist, deletechars,    replace_space, autostrip, case_sensitive, defaultfmt, unpack, usemask, loose, invalid_raise, max_rows)
   1767             # Raise an exception ?
   1768             if invalid_raise:
-> 1769                 raise ValueError(errmsg)
   1770             # Issue a warning ?
   1771             else:

ValueError: Some errors were detected !
    Line #60 (got 4 columns instead of 1)
    Line #123 (got 2 columns instead of 1)
    Line #124 (got 2 columns instead of 1)
    Line #129 (got 4 columns instead of 1)
    Line #130 (got 2 columns instead of 1)
    Line #131 (got 2 columns instead of 1)
    Line #132 (got 2 columns instead of 1)
    Line #134 (got 4 columns instead of 1)
    Line #135 (got 2 columns instead of 1)
    Line #137 (got 2 columns instead of 1)

I could narrow down the problem in astroquery happening here

The io.BytesIO(response.content) returns a web page with an error that cannot be read by np.genfromtxt nor taken care by the if above it.

However, I don't think there should be any error raised for this particular object, as its spectrum does exist:

http://skyserver.sdss.org/dr12/en/tools/explore/summary.aspx?plate=5177&mjd=56245&fiber=114

[MSeifert04]

Could you try using the latest master branch of astroquery?

If I run your script I encounter no errors:

>>> from astroquery.sdss import SDSS
>>> s = SDSS.get_spectra_async(plate=5177, mjd=56245, fiberID=114)
>>> x = s[0]
>>> y = x.get_fits()
>>> y
[<astropy.io.fits.hdu.image.PrimaryHDU at 0x1dbb4c2cc88>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb4c26748>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb4c532b0>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb4c59cf8>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb50d4198>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb50e46d8>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb50f4c18>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb55e8198>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb55f86d8>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb5608c18>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb561d198>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb562c6d8>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb563cc18>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb5650198>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb56606d8>,
 <astropy.io.fits.hdu.table.BinTableHDU at 0x1dbb5670c18>]

Versions:
astroquery - '0.3.3.dev3360'
astropy - '1.3.dev16002'
python - major=3, minor=5, micro=2, releaselevel='final', serial=0
numpy - '1.11.1'

Maybe that was fixed in astropy or astroquery.

[banados]

It's interesting that it does not fail to you. I updated to the latest master branches of astropy and astroquery and I still get the same error.

astroquery '0.3.3.dev3367'
astropy '1.3.dev16105'
Python 3.5.2
numpy '1.11.1'

I'm not sure what causes this difference. Can anyone else confirm if the example above works or not in their installation?

[MSeifert04]

I think this will be easier to verify as soon as #740 is merged.

[bsipocz]

I can't reproduce this error, so I am closing the issue now.

[fmannucci]

Hi, I am having the same problem with:
astroquery 0.4.7.d
python 3.10.9
numpy 1.23.5.

Using (for example) the example in the doc, as any other call:

pos = coords.SkyCoord('0h8m05.63s +14d50m23.3s', frame='icrs')
xid = SDSS.query_region(pos, spectro=True)


Traceback (most recent call last):

  File "/var/folders/m2/32fmbv0s3hg46rj5ccx2r5g40000gn/T/ipykernel_49747/284473386.py", line 2, in <module>
    xid = SDSS.query_region(pos, spectro=True)

  File "/Users/filippo/opt/anaconda3/envs/Py310/lib/python3.10/site-packages/astroquery/utils/class_or_instance.py", line 25, in f
    return self.fn(obj, *args, **kwds)

  File "/Users/filippo/opt/anaconda3/envs/Py310/lib/python3.10/site-packages/astroquery/utils/process_asyncs.py", line 29, in newmethod
    result = self._parse_result(response, verbose=verbose)

  File "/Users/filippo/opt/anaconda3/envs/Py310/lib/python3.10/site-packages/astroquery/sdss/core.py", line 905, in _parse_result
    arr = np.atleast_1d(np.genfromtxt(io.BytesIO(response.content),

  File "/Users/filippo/opt/anaconda3/envs/Py310/lib/python3.10/site-packages/numpy/lib/npyio.py", line 2291, in genfromtxt
    raise ValueError(errmsg)

ValueError: Some errors were detected !
    Line #3 (got 3 columns instead of 1)
    Line #5 (got 3 columns instead of 1)
    Line #6 (got 4 columns instead of 1)
    Line #8 (got 4 columns instead of 1)
    Line #12 (got 4 columns instead of 1)
    Line #15 (got 6 columns instead of 1)
    Line #17 (got 4 columns instead of 1)
    Line #21 (got 3 columns instead of 1)
    Line #31 (got 3 columns instead of 1)

It works fine on another machine running python 3.8 and astroquery 0.4.3

[bsipocz]

@banados - indeed I get a KeyNotFoundError in response.text with your example, however it's due to another issue. Could you open a separate, new bug report for it, please?

[astro-fermi]

I am also getting the same error. The same commands were running fine last week though.

from astroquery.sdss import SDSS
from astropy import coordinates

pos = coordinates.SkyCoord(15.03034u.deg,-0.538486u.deg,frame='fk5')
xid = SDSS.query_region(pos, spectro=True)

Traceback (most recent call last):
File "", line 1, in
File "/opt/miniconda3/lib/python3.8/site-packages/astroquery/utils/class_or_instance.py", line 25, in f
return self.fn(obj, *args, **kwds)
File "/opt/miniconda3/lib/python3.8/site-packages/astroquery/utils/process_asyncs.py", line 29, in newmethod
result = self._parse_result(response, verbose=verbose)
File "/opt/miniconda3/lib/python3.8/site-packages/astroquery/sdss/core.py", line 905, in _parse_result
arr = np.atleast_1d(np.genfromtxt(io.BytesIO(response.content),
File "/opt/miniconda3/lib/python3.8/site-packages/numpy/lib/npyio.py", line 2291, in genfromtxt
raise ValueError(errmsg)
ValueError: Some errors were detected !
Line #3 (got 3 columns instead of 1)
Line #5 (got 3 columns instead of 1)
Line #6 (got 4 columns instead of 1)
Line #8 (got 4 columns instead of 1)
Line #12 (got 4 columns instead of 1)
Line #15 (got 6 columns instead of 1)
Line #17 (got 4 columns instead of 1)
Line #21 (got 3 columns instead of 1)
Line #31 (got 2 columns instead of 1)

[fmannucci]

done, #2644 , thanks!