sample_glue_loader returns 'Something went wrong...' on the front-end

[eponkratova]

Expected Behavior

Glue tables metadata is displayed on the front-end

Current Behavior

On the front-end, I see 'Something went wrong...'

Possible Solution

Steps to Reproduce

I am running Amundsen on my local machine:

1. followed the installation guide at https://github.com/amundsen-io/amundsen/blob/master/docs/installation.md, including the required python libraries.
2. modified sample_glue_loader and added aws session token.
3. when I opened the front-end, the tables from Glue were listed but when I click any, the error displayed 'Something went wrong...' . The example of the URL of one of the tables (iriscsv) located in in the iris database is:
   http://localhost:5000/table_detail/AWSSessionToken=%22/glue/iris/iriscsv.

Screenshots (if appropriate)

Context

I am doing a POC.

Your Environment

[feng-tao]

do you verify the table is ingested into neo4j correctly with neo4j UI?

[eponkratova]

I can see the tables on neo4j, when I switch to the table view, I see the similar content for all tables retrieved:
{
"name": "iriscsv",
"publisher_last_updated_epoch_ms": 1610903614689,
"published_tag": "1610874811",
"key": "glue://sessiontoken=".iris/iriscsv",
"is_view": false
}

[OElesin]

I'm experiencing the same thing as well. Any help here

[eponkratova]

OElesin, check the Athena one if you need to get crawl the s3 data. If you use Python, not Airflow, I slightly modified the original code at https://github.com/eponkratova/articles/blob/master/data_discovery/code/athena_data_loader.py

[OElesin]

@eponkratova, I tried this and got the same error. This is quite surprising .

I executed the API request from the browser console and got the error below:

{
"msg": "Encountered error: Metadata request failed",
"status_code": 500,
"tableData": {}
}

[OElesin]

From the error logs

self.client = get_proxy_client()
--
File "/usr/local/lib/python3.7/site-packages/amundsen_metadata-3.0.1-py3.7.egg/metadata_service/proxy/__init__.py", line 47, in get_proxy_client
client_kwargs=client_kwargs)
TypeError: __init__() got an unexpected keyword argument 'client_kwargs'

[OElesin]

@feng-tao and @eponkratova, the issue is likely due to recent changes in the metadata library. I reverted the image being used to amundsen-metadata:3.0.0 and everything seems to be fixed.

Looks like this is from: https://github.com/amundsen-io/amundsenmetadatalibrary/blob/master/metadata_service/proxy/__init__.py#L47

[eponkratova]

@OElesin , so, did you make the Glue connector working as well?

[OElesin]

@eponkratova, both are working.

[eponkratova]

Aha, thank you, @OElesin

[feng-tao]

cc @mgorsk1

[NiravLangaliya]

I have installed new code and I am also facing the same issue and it complains about

TypeError: __init__() got an unexpected keyword argument 'client_kwargs'

[feng-tao]

@mgorsk1 could you take a look?

[mgorsk1]

sure, looking into it

[mgorsk1]

apologies, that's my blunder. I've submitted fix amundsen-io/amundsenmetadatalibrary#258 any chance you can verify if that helped @OElesin @eponkratova

[feng-tao]

no worry, @mgorsk1 's change (amundsen-io/amundsenmetadatalibrary#258) is just merged. @NiravLangaliya could you pull from master and see if the issue is gone?

[feng-tao]

Feel free to reopen if the issue persists after pulling from master.

[cpollard0]

@feng-tao - I'm getting the same issue now with loading Glue data. I'm on the latest branch from master, my data is in neo4j. When I click on a table in the WebUI I get 404 not found. Any guidance would be appreciated!