Support migration of user data

[privatemaker]

Is your feature request related to a problem? Please describe.

As a user migrating from one NC to another, I should be able to bring my data with me.

Describe the solution you'd like

This should be done properly with the user_migration app as outlined in the docs

Describe alternatives you've considered

I figured out a hacky way to migrate data from Instance 1: Deck / Tasks -> Instance 2: Tasks and explained here. I recall about 9 months ago I was able to move data from Tasks: Lists -> Deck: Boards on the same instance within Tasks app. Am I imagining this or did some ACL / perms change somewhere?

Additional context

No response

[raimund-schluessler]

If I understand it correctly, this is about migrating data out of the Deck app. Migrating data from the Tasks / Calendar app should be covered by the user_migration, as these apps are just CalDAV clients and use the backend provided by the server.

Deck, however, brings its own backend and I don't know whether this is covered by user_migration, but I guess its not.

I am transfering this issue to the Deck repository, as exporting data from Deck should be discussed there.

[raimund-schluessler]

Reading what you wrote in the forum, I see that you exported the Deck boards with the Tasks app. This exports from the Deck backend into a ICS file. However, importing this with the Calendar app, imports into the CalDAV backend from server, so it will not show up in the Deck app anymore.

Since this really is a Deck issue, I have to refer you to the Deck devs.

[privatemaker]

This exports from the Deck backend into a ICS file

@raimund-schluessler the ICS file I export is from Tasks app, but can also be from Calendar I suppose. Newer versions of Deck seem to export a CSV file, older version there is no way to export AFAICT.

importing this with the Calendar app, imports into the CalDAV backend from server

I figured something like this is happening. So if I understand CalDAV interop between Calendar and Deck is not bi-directional as Deck Boards are immediately viewable in Calendar, but not the other way around. It just is a bit confusing from a user perspective but makes sense from a engineering perspective.

One cannot import Calendars events into Decks or tasks due to how these API's flow.

so it will not show up in the Deck app anymore.

Right I stated that in my forum post, hence why I opened issue here 😄

[mvdw]

The problem remains: As a user migrating from one NC to another, I should be able to bring my data with me.
I succeeded to export a big JSON with all my data with the "occ deck:export" command, but how do I import this in my new nextcloud server?

[NeroSec]

Its crazy, i am still not able to export/import a Deck... To import it want a json, when export in the app it gives a CSV, export with OCC gives a json, try to import this json fails because:
`It's not a valid config file.
You can get more info on https://deck.readthedocs.io/en/latest/User_documentation_en/#6-import-boards
Please provide a valid config json file: deck.json
Invalid config file
The property owner is required
Valid schema:
{
"type": "object",
"properties": {
"uidRelation": {
"type": "object",
"comment": "Relationship between Trello and Nextcloud usernames",
"example": {
"johndoe": "admin"
}
},
"owner": {
"type": "string",
"required": true,
"comment": "Nextcloud owner username"
}
}
}

Please provide a valid data json file: `
This problem exist since deck is existing. Why is it so difficult to implement this?

[JimMadge]

I think you might be encountering the issue in #5213.

deck/lib/Service/Importer/BoardImportCommandService.php

Lines 85 to 94 in 877c52f

	if (is_string($config)) {
	if (!is_file($config)) {
	throw new NotFoundException('It\'s not a valid config file.');
	}
	$config = json_decode(file_get_contents($config));
	if (!$config instanceof \stdClass) {
	throw new NotFoundException('Failed to parse JSON.');
	}
	$this->setConfigInstance($config);
	}

The default argument for --config seems to be config.json.

deck/lib/Command/BoardImport.php

Lines 58 to 64 in 877c52f

	->addOption(
	'config',
	null,
	InputOption::VALUE_REQUIRED,
	'Configuration json file.',
	'config.json'
	)

So, if that file isn't present you get the "It's not a valid config file." message.

I got around this by passing an empty string to --config. For example, php occ deck:import --config= --data=/path/to/data.json

[mvdw]

The problem remains: As a user migrating from one NC to another, I should be able to bring my data with me. I succeeded to export a big JSON with all my data with the "occ deck:export" command, but how do I import this in my new nextcloud server?

OK, so let me answer my own question: with "occ deck:import".
The change in this issue will solve the confusing error message, so it will probably work.
Too bad the export command is not in the documentation... and it would be good to describe the migration use case at https://deck.readthedocs.io/en/latest/User_documentation_en/#6-import-boards

[epifanio]

Hi, I am on NextcloudAIO and need to migrate deck cards for all the users from the old istance on nextclolud- both running 28.0.5.
I am having the same issue, able to export but not to import.
I have tried the following:

1747d1fc75fc:/var/www/html$ ./occ deck:import --config=data.json 
Starting import...
Failed to parse JSON.
You can get more info on https://deck.readthedocs.io/en/latest/User_documentation_en/#6-import-boards
Please provide a valid config json file: 

Any clue on how to migrate deck data? I can't find any config.json and the data.json was generated by the deck export command using default values for a single user:

./occ deck:export username > data.json

Thanks for your precious help!

[mvdw]

Hi @epifanio , IIUC you have to give an empty config file, and point to the data as follows (I did not test this):

./occ deck:import --config=  --data=data.json

[epifanio]

Hi, thanks for your reply - I tried this way, but the app then complains about my data.json :

1747d1fc75fc:/var/www/html$ ./occ deck:import --config=  --data=/var/www/html/data.json 
Starting import...
Please provide a valid data json file: 

I genmerated the file using occ deck:export username

1747d1fc75fc:/var/www/html$ head data.json 
{
    "version": "1.12.2",
    "boards": {
        "2": {
            "id": 2,
            "title": "Personal",
            "owner": {
                "primaryKey": "username",
                "uid": "username",
                "displayname": "name familyname",
...

it is quite a large json (1.8 mb), could be it is corrupted when I transfered from one server to the other, i will try scp to move it again ..

[epifanio]

i checked with the json file exported from an other user and the import worked (terminated with some error and returned an empy deck, but the import process started) while with the export of my main user (the one that assign cards) it doesn't like the json file generated during the export