Add Uppy Companion to wopi example #6702

dschmidt · 2023-07-03T16:50:49Z

Description

Add Uppy Companion to wopi example, so cloud imports work as soon as the importer app is shipped with oC Web.

Related Issue

Motivation and Context

Cloud imports need the Uppy Companion microservice.

How Has This Been Tested?

Tested locally with my own build of oC Web.

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Technical debt
Tests only (no source changes)

Checklist:

Code changes
Unit tests added
Acceptance tests added
Documentation ticket raised:

update-docs · 2023-07-03T16:50:52Z

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

wkloucek · 2023-07-03T17:05:57Z

deployments/examples/ocis_wopi/.env

+COMPANION_DOMAIN=
+COMPANION_ONEDRIVE_KEY=
+COMPANION_ONEDRIVE_SECRET=


How would this look like for the continuous deployments? How is ist related to WOPI / Office integration in oCIS?

To me it feels like this needs a dedicated example, which probably has no continuous deployment because of the missing secret management for our continuous deployments

I asked @micbar and he wanted me to add it to the wopi example, so it's "full featured".

But will it work when COMPANION_ONEDRIVE_SECRET and COMPANION_ONEDRIVE_KEY is unset?

Well, OneDrive integration obviously won't ... but e.g. https://uppy.io/docs/url/ would

ahh ok, would be nice to add this as comments to the parameters. I didn't know and thought it's needed to make it work

... we most likely will need to add a setting to enable certain integrations in the ui, but the way the import is configured is about to change anyway, so it's probably not worth adding it now.

Hm yeah, well - I thought that was kinda self explanatory, no other variables in .env have that kind of comment although they are empty and everything works

i enabled only "Public Link".

deployments/examples/ocis_wopi/docker-compose.yml

tbsbdr · 2023-07-24T10:01:59Z

@micbar I'd like to reference the wopi deployment example in the releasenotes, but would not drive documentation further for this feature. ok for you?

micbar · 2023-07-24T10:23:35Z

I am still not decided if we should deploy it in the continouus deployments at all.

It was a big effort doing it for the example deployment.

dschmidt · 2023-07-24T18:48:07Z

We could also just deploy the public link provider...

mmattel · 2023-08-07T15:25:24Z

@micbar SonarCloud seems to be "unwilling" to continue, not only here. I tried before with a rebase and it "hung". I think the important two are green...

micbar · 2023-08-07T15:52:22Z

Example is incomplete. Not sufficient. We need to check what we can do with the config file templating.

nicholas-wilson-au · 2023-09-20T11:47:30Z

This is highlighted in our release notes https://doc.owncloud.com/docs/next/ocis_release_notes.html and is still open. We should remove this from the release notes if there is no way for our customers to use this feature.

marciofeld · 2023-10-19T00:33:34Z

I tried to deploy the Upload Companion in my test environment, but nothing changed on the Web Interface. I'm pretty sure I made everything correct.

Is that working with v4.0.0+?

dschmidt · 2023-10-19T09:10:52Z

I tried to deploy the Upload Companion in my test environment, but nothing changed on the Web Interface. I'm pretty sure I made everything correct.

Is that working with v4.0.0+?

I'm sorry, this PR was probably misleading. We changed the way the importer is configured after this PR was sent and then noone made a decision how to progress here, so it got stale and outdated.
In the long run the env var might do the right thing again, until then you need to provide your own config.json - and have a companionUrl setting there.

{
...
  "external_apps": [
    {
      "id": "importer",
      "path": "web-app-importer",
      "config": {
        "companionUrl": "https://your.companion.url/",
        "webdavCloudType": "owncloud"
      }
    }
  ]
...
}

mmattel · 2023-10-19T10:44:14Z

@dschmidt can you fix that PR so it can be merged and the example becomes available?

dschmidt · 2023-10-19T10:58:44Z

I can, but I need a decision we actually want that.

Quoting @micbar :

I am still not decided if we should deploy it in the continouus deployments at all.

mmattel · 2023-10-19T11:12:13Z

This is somehow rediculous.
On the one hand we add it to the changelog
Then we provide a PR making it avaiable, but the PR is somehow not complete
A user asks about a working example
We provide that here
And then we discuss about making the PR complete for deployment
I dont get that ...?

micbar · 2023-10-19T11:33:46Z

@mmattel Like i said there is a technical blocker.

marciofeld · 2023-10-19T13:36:04Z

I tried to deploy the Upload Companion in my test environment, but nothing changed on the Web Interface. I'm pretty sure I made everything correct.
Is that working with v4.0.0+?

I'm sorry, this PR was probably misleading. We changed the way the importer is configured after this PR was sent and then noone made a decision how to progress here, so it got stale and outdated. In the long run the env var might do the right thing again, until then you need to provide your own config.json - and have a companionUrl setting there.
{
...
  "external_apps": [
    {
      "id": "importer",
      "path": "web-app-importer",
      "config": {
        "companionUrl": "https://your.companion.url/",
        "webdavCloudType": "owncloud"
      }
    }
  ]
...
}

Thanks @dschmidt !
Could you let me know the path of this config.json file?

dschmidt · 2023-10-19T13:45:30Z

In the Web dev setup we do it like this:

https://github.com/owncloud/web/blob/b7f1c5dd5ddd7faa62e1fddbb8ae18e67093d011/docker-compose.yml#L16
https://github.com/owncloud/web/blob/b7f1c5dd5ddd7faa62e1fddbb8ae18e67093d011/docker-compose.yml#L69

so basically you need to mount it to your container and use the WEB_UI_CONFIG_FILE env var to point oCIS to it
I would recommend to download your config.json from https://your.ocis.url/config.json, then add the section I pasted above and then mount it to your container...

I'm sorry this is so hard right now...

marciofeld · 2023-10-19T21:42:53Z

Thanks @dschmidt ! I successfully managed to deploy the Cloud Import Feature.

I have an OwnCloud v10 remote instance and I tried to import files using a generated Public URL from it, but it didn't work.
I tried with and without Password, but in both cases it returned a error message Connection with Companion failed, as you can see in the screenshot below:

Please find the latest Companion container log lines below:

::ffff:172.31.11.165 - - [19/Oct/2023:21:33:41 +0000] "OPTIONS /webdavPublicLink/list/?publicLinkURL=<PUBLIC_URL_HIDDEN> HTTP/1.1" 204 0 "<INFINITESCALE_URL_HIDDEN>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
companion: 2023-10-19T21:33:42.110Z [info] companion.client.version uppy client version @uppy/companion-client=3.2.0
companion: 2023-10-19T21:33:42.111Z [warn] 8e508542-6949-4229-bf2d-e291d49d5e16 provider.middleware.invalid invalid provider options detected. Provider will not be loaded
::ffff:172.31.11.165 - - [19/Oct/2023:21:33:42 +0000] "GET /webdavPublicLink/list/?publicLinkURL=<PUBLIC_URL_HIDDEN> HTTP/1.1" 400 11 "<INFINITESCALE_URL_HIDDEN>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"

dschmidt · 2023-10-19T22:55:49Z

Thanks @dschmidt ! I successfully managed to deploy the Cloud Import Feature.

Cool, good to hear!

Please find the latest Companion container log lines below:

::ffff:172.31.11.165 - - [19/Oct/2023:21:33:41 +0000] "OPTIONS /webdavPublicLink/list/?publicLinkURL=<PUBLIC_URL_HIDDEN> HTTP/1.1" 204 0 "<INFINITESCALE_URL_HIDDEN>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
companion: 2023-10-19T21:33:42.110Z [info] companion.client.version uppy client version @uppy/companion-client=3.2.0
companion: 2023-10-19T21:33:42.111Z [warn] 8e508542-6949-4229-bf2d-e291d49d5e16 provider.middleware.invalid invalid provider options detected. Provider will not be loaded
::ffff:172.31.11.165 - - [19/Oct/2023:21:33:42 +0000] "GET /webdavPublicLink/list/?publicLinkURL=<PUBLIC_URL_HIDDEN> HTTP/1.1" 400 11 "<INFINITESCALE_URL_HIDDEN>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"

The error message comes from here: https://github.com/owncloud/uppy/blob/f3ce884e43d6dd8732a0cafb8d32ee8ba9a12d9b/packages/%40uppy/companion/src/server/provider/index.js#L118 -> https://github.com/owncloud/uppy/blob/f3ce884e43d6dd8732a0cafb8d32ee8ba9a12d9b/packages/%40uppy/companion/src/server/provider/index.js#L25

So apparently you don't seem to have server.host and server.protocol configured. See Companion documentation here: https://uppy.io/docs/companion/#server

marciofeld · 2023-10-20T19:44:38Z

Thanks again @dschmidt !
I've been reading all the Uppy Companion docs and also all the OCIS docker-compose examples.

I'm still in doubt about how to solve this last error since I would like to import data from OwnCloud v10 to Infinite Scale v4, where my OwnCloud relies on EBS Volumes to store data while my Infinite Scale relies on S3 Buckets.

On Companion docs, there are environment variables available for these services: BOX, DROPBOX, FACEBOOK, GOOGLE DRIVE, INSTAGRAM, ONEDRIVE and ZOOM. I don't have any option to insert an API Key for External URL, which is the option I might use to import files from the external OwnCloud instance.

How could I proceed? Or in this case, Uppy Companion won't be able to help me importing data from OwnCloud?

micbar · 2023-10-21T21:28:19Z

We added a feature to uppy companion where you can import from an ownCloud 10 public link. It is not yet upstream.

Isn't it @dschmidt ?

dschmidt · 2023-10-21T21:38:07Z

We added a feature to uppy companion where you can import from an ownCloud 10 public link. It is not yet upstream.

Isn't it @dschmidt ?

Correct, upstream is working on it. I think they haven't decided whether they want to integrate it fully into their product, but they are working on providing the necessary apis at least.

transloadit/uppy#4621

micbar · 2023-12-11T09:55:14Z

Note to myself: Maybe change the ocis config to make it configurable via ENV

sonarqubecloud · 2024-02-01T14:46:17Z

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

micbar · 2024-02-01T14:49:00Z

@wkloucek @rhafer @dschmidt The three different web config files are not so nice.

Will be improved by #8339

dragonchaser

lgtm

Add Uppy Companion to wopi example

dschmidt · 2024-02-06T14:52:38Z

🎉

Thanks for taking this over the finish line

Add Uppy Companion to wopi example

dschmidt requested review from micbar and wkloucek July 3, 2023 16:50

wkloucek reviewed Jul 3, 2023

View reviewed changes

deployments/examples/ocis_wopi/docker-compose.yml Outdated Show resolved Hide resolved

mmattel force-pushed the uppy-companion branch from a0e3453 to a1ff566 Compare August 7, 2023 10:46

micbar force-pushed the uppy-companion branch from a1ff566 to 8935dfb Compare August 7, 2023 14:37

micbar self-assigned this Dec 11, 2023

Add uppy companion to wopi example

754f6cc

micbar force-pushed the uppy-companion branch from 8935dfb to e2fb018 Compare February 1, 2024 11:34

micbar force-pushed the uppy-companion branch 2 times, most recently from 6a994fc to f0c7983 Compare February 1, 2024 11:53

Add comments to .env file

3860ed4

micbar force-pushed the uppy-companion branch from f0c7983 to 3860ed4 Compare February 1, 2024 14:43

micbar requested a review from wkloucek February 1, 2024 14:46

dragonchaser approved these changes Feb 6, 2024

View reviewed changes

micbar merged commit d3d4d59 into master Feb 6, 2024
2 checks passed

delete-merged-branch bot deleted the uppy-companion branch February 6, 2024 14:44

ownclouders pushed a commit that referenced this pull request Feb 6, 2024

Merge pull request #6702 from owncloud/uppy-companion

1e2a65f

Add Uppy Companion to wopi example

ownclouders pushed a commit that referenced this pull request Feb 7, 2024

Merge pull request #6702 from owncloud/uppy-companion

11f5291

Add Uppy Companion to wopi example

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Uppy Companion to wopi example #6702

Add Uppy Companion to wopi example #6702

dschmidt commented Jul 3, 2023

update-docs bot commented Jul 3, 2023

wkloucek Jul 3, 2023 •

edited

Loading

dschmidt Jul 3, 2023

wkloucek Jul 3, 2023

dschmidt Jul 3, 2023

wkloucek Jul 3, 2023

dschmidt Jul 3, 2023

dschmidt Jul 3, 2023

micbar Feb 1, 2024

tbsbdr commented Jul 24, 2023

micbar commented Jul 24, 2023

dschmidt commented Jul 24, 2023

mmattel commented Aug 7, 2023 •

edited

Loading

micbar commented Aug 7, 2023

nicholas-wilson-au commented Sep 20, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

mmattel commented Oct 19, 2023

dschmidt commented Oct 19, 2023

mmattel commented Oct 19, 2023

micbar commented Oct 19, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

marciofeld commented Oct 20, 2023

micbar commented Oct 21, 2023

dschmidt commented Oct 21, 2023

micbar commented Dec 11, 2023

sonarqubecloud bot commented Feb 1, 2024

micbar commented Feb 1, 2024

dragonchaser left a comment

dschmidt commented Feb 6, 2024

Add Uppy Companion to wopi example #6702

Add Uppy Companion to wopi example #6702

Conversation

dschmidt commented Jul 3, 2023

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

update-docs bot commented Jul 3, 2023

wkloucek Jul 3, 2023 • edited Loading

Choose a reason for hiding this comment

dschmidt Jul 3, 2023

Choose a reason for hiding this comment

wkloucek Jul 3, 2023

Choose a reason for hiding this comment

dschmidt Jul 3, 2023

Choose a reason for hiding this comment

wkloucek Jul 3, 2023

Choose a reason for hiding this comment

dschmidt Jul 3, 2023

Choose a reason for hiding this comment

dschmidt Jul 3, 2023

Choose a reason for hiding this comment

micbar Feb 1, 2024

Choose a reason for hiding this comment

tbsbdr commented Jul 24, 2023

micbar commented Jul 24, 2023

dschmidt commented Jul 24, 2023

mmattel commented Aug 7, 2023 • edited Loading

micbar commented Aug 7, 2023

nicholas-wilson-au commented Sep 20, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

mmattel commented Oct 19, 2023

dschmidt commented Oct 19, 2023

mmattel commented Oct 19, 2023

micbar commented Oct 19, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

marciofeld commented Oct 19, 2023

dschmidt commented Oct 19, 2023

marciofeld commented Oct 20, 2023

micbar commented Oct 21, 2023

dschmidt commented Oct 21, 2023

micbar commented Dec 11, 2023

sonarqubecloud bot commented Feb 1, 2024

Quality Gate passed

micbar commented Feb 1, 2024

dragonchaser left a comment

Choose a reason for hiding this comment

dschmidt commented Feb 6, 2024

wkloucek Jul 3, 2023 •

edited

Loading

mmattel commented Aug 7, 2023 •

edited

Loading