Add robots.txt file

[jvillafanez]

Description

Add the robots.txt file

Related Issue

owncloud/ocis#1314

Motivation and Context

How Has This Been Tested?

Checked with ocis-web without pulling fresh assets. After generating the embed files and building ocis, the "/robots.txt" is accessible with the contents.
It's expected that a new web release will put everything into the right place.

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Technical debt
• Tests

Checklist:

• Code changes
• Unit tests added
• Acceptance tests added
• Documentation ticket raised:

Open tasks:

• ...

[update-docs]

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

[CLAassistant]

All committers have signed the CLA.

[kulmann]

What about an integration into oc10 with the web app? Does the robots.txt of oc10 itself apply then or do we need to serve this robots.txt there as well? robots.txt would need to be added to the array in

web/packages/web-integration-oc10/lib/Controller/FilesController.php

Line 81 in 05f8735

$permittedPaths = ["css", "img", "js", "themes", "index.html", "manifest.json", "oidc-callback.html", "oidc-silent-redirect.html"];

then.

[ownclouders]

Results for oC10iPhone2 https://drone.owncloud.com/owncloud/web/18756/46/1
The following scenarios passed on retry:

• webUITrashbinRestore/trashbinRestore.feature:14

[ownclouders]

Results for oC10Files1 https://drone.owncloud.com/owncloud/web/18756/12/1
The following scenarios passed on retry:

• webUIFilesActionMenu/versions.feature:36

[fschade]

Thanks, I think It misses a copy step here

web/rollup.config.js

Line 66 in 94ed542

targets: [

[jvillafanez]

I don't think it's needed for the OC10 integration. OC10 already has its own robots.txt file so it's ownCloud the one that should serve the file.
In addition, the robots.txt should be placed in the root of the of the server, so it should be placed in https://my.server/robots.txt, and that URL is controlled by OC10, so the web app shouldn't need to bother

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[ownclouders]

Results for oC10SharingInternalUsersSharingIndicator https://drone.owncloud.com/owncloud/web/18761/28/1
The following scenarios passed on retry:

• webUISharingInternalUsersSharingIndicator/shareWithUsers.feature:154