After every step, I still have amundsen with zero datasets

[telmofranciscmultivision]

Hi,

I've followed all steps from tutorial, but without success. When opening amundsen, i see no datasets, someone can help me?
(already loaded mysql data, and elastic and neo4j has data.

Amundsen:

Neo4j:

[emreakg]

This is the search page, so a problem here doesn't indicate a problem with the metadata service or neo4j. Could you check some logs from the frontend and search services, and post anything unusual? Like stacktraces or 4xx/5xx http error codes.

[telmofranciscmultivision]

Hi,

Yes indeed I saw an error, but i dont know what it means :

2021-04-04T17:42:20+0000.284 [DEBUG] connectionpool._make_request:461 (15:MainThread) - http://amundsenmetadata:5000 "GET /user/test_user_id/follow/ HTTP/1.1" 200 41

2021-04-04T17:42:20+0000.290 [DEBUG] connectionpool._make_request:461 (14:MainThread) - http://amundsenmetadata:5000 "GET /user/test_user_id HTTP/1.1" 500 141

2021-04-04T17:42:20+0000.292 [ERROR] v0.current_user:38 (14:MainThread) - Encountered error: failed to fetch user with user_id: test_user_id

2021-04-04T17:42:20+0000.292 [ERROR] v0.current_user:47 (14:MainThread) - Encountered exception: Expecting value: line 1 column 1 (char 0)

Traceback (most recent call last):

File "/app/amundsen_application/api/v0.py", line 42, in current_user

'user': dump_user(load_user(response.json()))

File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json

return complexjson.loads(self.text, **kwargs)

File "/usr/local/lib/python3.7/json/init.py", line 348, in loads

return _default_decoder.decode(s)

File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode

raise JSONDecodeError("Expecting value", s, err.value) from None

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

2021-04-04T17:42:20+0000.295 [DEBUG] connectionpool._new_conn:231 (15:MainThread) - Starting new HTTP connection (1): amundsenmetadata:5000

2021-04-04T17:42:20+0000.303 [DEBUG] connectionpool._new_conn:231 (14:MainThread) - Starting new HTTP connection (1): amundsenmetadata:5000

2021-04-04T17:42:20+0000.597 [DEBUG] connectionpool._make_request:461 (15:MainThread) - http://amundsenmetadata:5000 "GET /tags/ HTTP/1.1" 200 25

2021-04-04T17:42:20+0000.599 [DEBUG] connectionpool._make_request:461 (14:MainThread) - http://amundsenmetadata:5000 "GET /latest_updated_ts HTTP/1.1" 204 0

2021-04-04T17:42:20+0000.605 [DEBUG] connectionpool._new_conn:231 (15:MainThread) - Starting new HTTP connection (1): amundsenmetadata:5000

2021-04-04T17:42:20+0000.835 [DEBUG] connectionpool._make_request:461 (15:MainThread) - http://amundsenmetadata:5000 "GET /popular_tables/?limit=4 HTTP/1.1" 200 29

[emreakg]

Isn't there another error/erorrs, either one on your amundsensearch container, or an error similar to this one on amundsenfrontend, but happened because amundsensearch:5000 returned a non 200 code, and not amundsenmetadata.
This error you posted happened because amundsen tried to fetch user information for test_user_id, but there wasn't a user with such an id. It is harmless.

Try ignoring any of the logs before, and use the search box for something, see what errors that generates.

[telmofranciscmultivision]

The thing is that i dont know from where comes that test_user_id.
I just find that error, anything else:
2021-04-02T22:24:48+0000.080 [ERROR] v0.current_user:38 (9:MainThread) - Encountered ERROR: failed to fetch user with user_id: test_user_id

2021-04-02T22:24:48+0000.080 [ERROR] v0.current_user:47 (9:MainThread) - Encountered exception: Expecting value: line 1 column 1 (char 0)

raise JSONDecodeError("Expecting value", s, err.value) from None

[emreakg]

OK, please run these 2 commands from the command line:
curl -X GET "http://localhost:9200/table_search_index/_search"
curl -X GET "http://localhost:5001/search?query_term=test"

The first one searches for any document with the index alias table_search_index. You said that elasticsearch isn't empty, but maybe your index that contains table documents doesn't have the correct alias defined.
The second one is the GET request your frontend service is sending to your search service. If all went well, this should have returned something and populated your 2nd screenshot.

Please upload the outputs of these commands. I've written the commands assuming you used the default docker setup. If you have modified the ports, you will need to change them here as well. port 9200 one is supposed to be the elasticsearch container, port 5001 is supposed to be the amundsensearch container.

[telmofranciscmultivision]

Hi @Fraznist ! First of all thank you very much for the support! The ports are correct, I'm using docker-amundsen.yml to run the services, so they are all by default.
Regarding the curls:

curl -X GET "http://localhost:9200/table_search_index/_search":

{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"tables3dee55f6-a0e5-4a8e-bed9-6abae3ed4cd6","_type":"table","_id":"ii_9nXgB4h2PfVJ6dkpK","_score":1.0,"_source":{"badges":[],"cluster":"def","column_descriptions":["","","","",""],"column_names":["city","address","firstname","lastname","personid"],"database":"mysql","description":"","display_name":"movies_db.persons","key":"mysql://def.movies_db/persons","last_updated_timestamp":null,"name":"persons","programmatic_descriptions":[],"schema":"movies_db","schema_description":null,"tags":[],"total_usage":0,"unique_usage":0}}]}}

curl -X GET "http://localhost:5001/search?query_term=test:

{"total_results": 0, "results": []}

[emreakg]

This is weird. Its ok for your second curl to not return anything because I just copy pasted from my local amundsen. Your table isn't named test. Thats my bad, sorry. Try: curl -X GET "http://localhost:5001/search?query_term=persons

Meanwhile, I indexed your document to my elasticsearch instance, and just by indexing it, I can find the table in my search results.

I can't really reproduce your issue. If the curl to 5001 with persons works, then the culprit needs to be the frontend container. But you have said that there isn't an error on the frontend container, about search. I'm gonna ask you to check the frontend container logs for errors again, specifically after you searched for persons in the frontend.

[telmofranciscmultivision]

Hi once again @Fraznist !! I think I had finaly results :) thank you very much, now i'm going to integrate the superset with it

Best Regards

[laura-sanches-movile]

Hi @telmofranciscmultivision I'm facing the same problem with that test_user_id, how did you fixed it?

[telmofranciscmultivision]

Hi, are you using which extractor? mysql?

[laura-sanches-movile]

I'm using BigQuery extractor

[telmofranciscmultivision]

Hmm in that case I dont know, but could you provide the error youre facing?

[laura-sanches-movile]

After searching for a while I saw that maybe it was a problem that they fixed in the latest version: #939
Thank you very much!

[telmofranciscmultivision]

After searching for a while I saw that maybe it was a problem that they fixed in the latest version: #939
Thank you very much!

Perfect!! any doubts just ask :)