Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import any other than en-US fails #10

Closed
JohnMoor opened this issue Aug 17, 2023 · 3 comments
Closed

Import any other than en-US fails #10

JohnMoor opened this issue Aug 17, 2023 · 3 comments

Comments

@JohnMoor
Copy link
Contributor

When trying to do an open-data import and selecting "nl" or even "de" it fails. This happens both on existing and clean new space.
en-US imports fine

Errors in log show.

172.30.32.1 - - [16/Aug/2023:19:52:09 +0200] "GET /api/group/ HTTP/1.1" 200 72 "https://[mydomain]space-manage/1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" {'selected_version': 'nl', 'selected_datatypes': ['food', 'unit', 'category', 'property', 'store', 'conversion'], 'update_existing': True, 'use_metric': True} Internal Server Error: /api-import-open-data/ Traceback (most recent call last): File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 357, in execute return Database.Cursor.execute(self, query, params) sqlite3.IntegrityError: UNIQUE constraint failed: cookbook_unit.space_id, cookbook_unit.open_data_slug The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/recipes/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner response = get_response(request) File "/opt/recipes/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/recipes/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view return view_func(*args, **kwargs) File "/opt/recipes/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view return self.dispatch(request, *args, **kwargs) File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch response = self.handle_exception(exc) File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception self.raise_uncaught_exception(exc) File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception raise exc File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, *args, **kwargs) File "/opt/recipes/cookbook/views/api.py", line 1479, in post response_obj['unit'] = len(data_importer.import_units()) File "/opt/recipes/cookbook/helper/open_data_importer.py", line 36, in import_units return Unit.objects.bulk_create(insert_list, update_conflicts=True, update_fields=('name', 'plural_name', 'base_unit', 'open_data_slug'), unique_fields=('space', 'name',)) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/models/query.py", line 816, in bulk_create returned_columns = self._batched_insert( File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1825, in _batched_insert self._insert( File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1791, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql cursor.execute(sql, params) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 357, in execute return Database.Cursor.execute(self, query, params) django.db.utils.IntegrityError: UNIQUE constraint failed: cookbook_unit.space_id, cookbook_unit.open_data_slug 172.30.32.1 - - [16/Aug/2023:19:52:18 +0200] "POST /api-import-open-data/ HTTP/1.1" 500 145 "https://[mydomain]space-manage/1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"
@Petitgnoll6
Copy link
Contributor

Petitgnoll6 commented Aug 20, 2023 •
edited
Loading

Hello,
I don't know if it's the same problem but I have an error when trying to import any set of data, including en-US.
It's on a brand new working install (i can create recipes, ingredients, cookbook, etc). I tried to delete the only group and then create a new one to "reset" created resources, but it didn't solve the error.

The log : (from freshly up docker)

Checking` configuration...
Waiting for database to be ready...
Database is ready
Migrating database
Operations to perform:
  Apply all migrations: account, admin, auth, authtoken, contenttypes, cookbook, oauth2_provider, sessions, sites, socialaccount
Running migrations:
  No migrations to apply.
Generating static files
js-reverse file written to /opt/recipes/cookbook/static/django_js_reverse

1 static file copied to '/opt/recipes/staticfiles', 642 unmodified, 1373 post-processed.
Done
[2023-08-20 07:18:21 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2023-08-20 07:18:21 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-08-20 07:18:21 +0000] [1] [INFO] Using worker: gthread
[2023-08-20 07:18:21 +0000] [12] [INFO] Booting worker with pid: 12
[2023-08-20 07:18:21 +0000] [13] [INFO] Booting worker with pid: 13
[2023-08-20 07:18:21 +0000] [14] [INFO] Booting worker with pid: 14
192.168.64.4 - - [20/Aug/2023:09:18:38 +0200] "GET /space-manage/2 HTTP/1.0" 200 22650 "http://192.168.1.6:8123/list/keyword/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:38 +0200] "GET /api/invite-link/ HTTP/1.0" 200 2 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/pagination.py:200: UnorderedObjectListWarning: Pagination may yield inconsistent results with an unordered object_list: <class 'cookbook.models.UserSpace'> QuerySet.
  paginator = self.django_paginator_class(queryset, page_size)
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api/user-space/?page=1&page_size=25 HTTP/1.0" 200 344 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api-import-open-data/ HTTP/1.0" 200 946 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api/space/2/ HTTP/1.0" 200 309 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api/group/ HTTP/1.0" 200 72 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api/food-inherit-field/ HTTP/1.0" 200 267 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
192.168.64.4 - - [20/Aug/2023:09:18:39 +0200] "GET /api/user-file/ HTTP/1.0" 200 2 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"
{'selected_version': 'fr', 'selected_datatypes': ['food', 'unit', 'category', 'property', 'store', 'conversion'], 'update_existing': True, 'use_metric': True}
192.168.64.4 - - [20/Aug/2023:09:18:50 +0200] "POST /api-import-open-data/ HTTP/1.0" 500 145 "http://192.168.1.6:8123/space-manage/2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"

Thanks in advance for your help

@djstini
Copy link

djstini commented Aug 28, 2023

Same Issue here,

A small Red Error Notice shows in Frontend exclaiming "Failure: There was an error creating a resource!"

Install is also relatively fresh and apart from this import it works flawlessly so far.

@djstini
Copy link

djstini commented Aug 28, 2023

also i think this is the main open issue for the problem described here: TandoorRecipes/recipes#2514

djstini added a commit to djstini/open-tandoor-data that referenced this issue Sep 1, 2023
@djstini
Update de.json to remove duplicates
db5fcc7 
As described in
Issue TandoorRecipes#10 
TandoorRecipes/recipes#2595
TandoorRecipes/recipes#2514
@djstini djstini mentioned this issue Sep 1, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@vabene1111 @JohnMoor @djstini @Petitgnoll6