Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XMatch.query missing cat1 when usign two previously uploaded tables #3115

Closed
masgura opened this issue Oct 4, 2024 · 4 comments · Fixed by #3116
Closed

XMatch.query missing cat1 when usign two previously uploaded tables #3115

masgura opened this issue Oct 4, 2024 · 4 comments · Fixed by #3116

Comments

@masgura
Copy link

masgura commented Oct 4, 2024

Hi, I'm trying to match two previously uploaded catalogue this way:

radec_mid = SkyCoord(a_mid, d_mid, unit=(u.deg, u.deg))
columns = ['Gmag', 'BP-RP', 'RAJ2000', 'DEJ2000', 'ra', 'dec', 'pmra', 'mpdec', 'source_id']
custom_visizer = Vizier(columns=columns)
custom_visizer.ROW_LIMIT = -1

astropy.utils.data.clear_download_cache(hashorurl=None, pkgname='astropy')

stars_1 = custom_visizer.query_region(radec_mid, radius=self.telescope.fov, catalog='I/355/gaiadr3',
                                    **kwargs)
stars_2 = custom_visizer.query_region(radec_mid, radius=self.telescope.fov, catalog='I/350/gaiaedr3',
                                      **kwargs)

stars = XMatch.query(cat1=stars_1[0], cat2=stars_2[0], max_distance=1 * u.arcsec, colRA1='RAJ2000',
                     colDec1='DEJ2000', colRA2='RAJ2000', colDec2='RAJ2000')

I checked that the two tables are correctly uploaded, but the XMatch.query function gives me this error:

Message: Missing parameter "cat1".
                 Trace:
cds.xmatch.exception.CmdParseException: Missing parameter "cat1".
        at cds.xmatch.servlet.SyncXMatch.exec(SyncXMatch.java:656)
        at cds.xmatch.servlet.SyncXMatch.doPost(SyncXMatch.java:476)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
        at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2486)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:844)

But, then I do it this way:

stars = XMatch.query(cat1=stars_1[0], cat2='vizier:I/350/gaiaedr3, max_distance=1 * u.arcsec, 
                    colRA1='RAJ2000', colDec1='DEJ2000' )

Everything works fine

@ManonMarchand
Copy link
Member

ManonMarchand commented Oct 4, 2024

I can reproduce. But why don't you directly crossmatch the online versions of the catalogs? Is there something that does not allow you to use the second version?

Notes (MRE + payload)

MRE:

from astropy.coordinates import SkyCoord
from astroquery.vizier import Vizier
from astroquery.xmatch import XMatch
import astropy.units as u
a_mid = [0, 1, 2]
d_mid = [0, 1, 2]
radec_mid = SkyCoord(a_mid, d_mid, unit=(u.deg, u.deg))
columns = ['Gmag', 'BP-RP', 'RAJ2000', 'DEJ2000', 'ra', 'dec', 'pmra', 'pmdec', 'source_id']
custom_visizer = Vizier(columns=columns)
stars_1 = custom_visizer.query_region(radec_mid, radius=1 * u.arcmin, catalog='I/355/gaiadr3')
stars_2 = custom_visizer.query_region(radec_mid, radius=1 * u.arcmin, catalog='I/350/gaiaedr3')
stars = XMatch.query(cat1=stars_1[0], cat2=stars_2[0], max_distance=1 * u.arcsec, colRA1='RAJ2000',colDec1='DEJ2000', colRA2='RAJ2000', colDec2='DEJ2000', get_query_payload=True)

payload

{'request': 'xmatch', 'distMaxArcsec': 1.0, 'RESPONSEFORMAT': 'votable', 'colRA1': 'RAJ2000', 'colDec1': 'DEJ2000', 'colRA2': 'RAJ2000', 'colDec2': 'RAJ2000', 'area': 'allsky'}, {'files': {'cat2': ('cat1.csv', '_q,Gmag,BP-RP,RAJ2000,DEJ2000,RA_ICRS,DE_ICRS,pmRA,Source\n1,19.143797,2.243973,0.00946094931,-0.00079864601,0.00943691398,-0.00089684879,-5.408,2546034966433885568\n1,15.303568,1.211627,359.98957129186,0.01231070377,359.98956482804,0.01221930842,-1.454,2738188646457069696\n2,17.732948,1.480492,0.99561091719,0.98872594776,0.99561768899,0.98866391680,1.523,2738366075850046848\n2,18.970488,0.706100,1.00424618364,0.99990676310,1.00424775988,0.99986963242,0.355,2738366110209789440\n2,21.569563,1.293295,0.99109555451,1.00262549104,0.99109555451,1.00262549104,,2738369030787551616\n2,19.180887,0.893375,0.99873735124,1.01248301258,0.99873885341,1.01243455415,0.338,2738369065147293312\n2,19.353508,1.927811,0.99153877353,1.01194957128,0.99153566149,1.01189465689,-0.700,2738369133866769664\n3,19.884580,2.667747,2.01246088493,2.00572405014,2.01246672429,2.00574515931,1.313,2738898823593708032\n3,17.465452,1.530697,2.00085284723,2.01526896679,2.00087905030,2.01524154494,5.892,2738910574624232320\n3,21.072521,0.654532,1.99263015486,2.01139934726,1.99263015486,2.01139934726,,2738910608984947200\n')}

@masgura
Copy link
Author

masgura commented Oct 4, 2024

Since I need only some columns, I noted that uploading the catalogue by column names is way faster than using the whole catalogue (7 seconds compared to 1/2 minutes), so I was trying to check if uploading both + matching them is faster than uploading the first and matching it with the online one (which takes about 25-30 seconds). Matching the two online versions is waay slower

@ManonMarchand
Copy link
Member

Were you able to try again? It should be fixed in the development version, see here to see how to install it: Building from source

@masgura
Copy link
Author

masgura commented Oct 21, 2024

Sorry for the late replay, everything works fine now, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants