Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for COS as storage #8

Merged
merged 3 commits into from
Mar 15, 2023

Conversation

shahulsonhal
Copy link
Collaborator

Loki COS Integration: Update Loki documentation

@@ -333,3 +333,54 @@ storage_config:
kms_key_id: 0987dcba-09fe-87dc-65ba-ab0987654321
```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe another empty line for consistency.


When using IBM Cloud Object Storage (COS) as object storage, the following permissions are needed:

- `Content Reader`
Copy link
Collaborator

@tareqmamari tareqmamari Mar 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so far, we requested for Writer and Content Reader, that should be sufficient.
Writer permissions should be sufficient.


storage_config:
cos:
bucket: <bucket>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to fix the bucket field name.bucket vs bucketnames.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed and agreed to be consistent with s3 and keep bucketnames.


storage_config:
cos:
bucket: <bucket>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here as well, we need to fix the bucket field name.bucket vs bucketnames.

@@ -109,6 +110,14 @@ Resources: `*`
Resources: `arn:aws:iam::<aws_account_id>:role/<role_name>`


### IBM Cloud Object Storage

When using IBM Cloud Object Storage (COS) as object storage, the following permissions are needed:
Copy link
Collaborator

@tareqmamari tareqmamari Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since it is one permission, let's rephrase this statement:

When using IBM Cloud Object Storage (COS) as object storage, IAM Writer role is needed.

@@ -42,6 +42,9 @@ S3 is AWS's hosted object store. It is a good candidate for a managed object sto
Blob Storage is Microsoft Azure's hosted object store. It is a good candidate for a managed object store, especially when you're already running on Azure, and is production safe.
You can authenticate Blob Storage access by using a storage account name and key or by using a Service Principal.

### IBM Cloud Object Storage (COS)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a link to IBM Cloud Object Storage landing page?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We felt is wiered to add link in the header, so we added cos landing page link in description.


storage_config:
cos:
bucket: <bucket>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to fix the bucket field name.bucket vs bucketnames.

auth_endpoint: <iam_endpoint_for_authentication>
```

The role should have a policy with the following permissions attached.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, it is sufficient to indicate that the IAM Writer role is needed, let's avoid such details which might confuse people.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they want to have more advanced access, they should refer to IAM.

Copy link
Collaborator Author

@shahulsonhal shahulsonhal Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the policy json

Copy link
Collaborator

@adityacs adityacs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@adityacs adityacs merged commit 99f230c into amankrsingh2000:ibmcloud_cos_loki Mar 15, 2023
@shahulsonhal shahulsonhal deleted the cos-doc branch March 15, 2023 11:56
adityacs pushed a commit that referenced this pull request Mar 17, 2023
* Add documentation for COS as storage

* Address review comments
adityacs added a commit that referenced this pull request Mar 17, 2023
* cos client creation code and unit tests

* Added GetObject,PutObject and IsObjectNotFoundErr code and unit tests

Co-authored-by: Suruthi-G-K <[email protected]>

* add ListObject and DeleteObject methods and unit tests

Rebasing the code

Update config and test files with new changes

Update unit test for List

Arrange methods belong to mockCosClient together

Updated test cases to run independently

Co-authored-by: tareqmamari <[email protected]>

* add support for IAM API Key Authentication

Added code to support IAM API Key Authentication

Update the conditions for api key

use default auth endpoint for cos

Use default auth endpoint instead of forcing to pass auth endpoint always
for COS storage provider.

Replace const to defaultCOSAuthEndpoint

Replace DEFAULT_COS_AUTH_ENDPOINT with defaultCOSAuthEndpoint

Also fixed error fmt.Errorf call needs 6 args but has 7 args

Added new cases in Test_COSConfig

Added unit test for IAM API Key Authentication

iam api key auth test formatting

These are just minor formatting changes for IAM API key authentication
tests.

lowercase clifags cos prefix and rename to hedgedCOS

Co-authored-by: tareqmamari <[email protected]>
Co-authored-by: amankrsingh2000 <[email protected]>

* add documentation for COS storage (#8)

* Add documentation for COS as storage

* Address review comments

---------

Co-authored-by: Aman Kumar Singh <[email protected]>
Co-authored-by: Suruthi-G-K <[email protected]>
Co-authored-by: “athira <[email protected]>
Co-authored-by: tareqmamari <[email protected]>
Co-authored-by: “athira <[email protected]>
Co-authored-by: Shahul <[email protected]>
adityacs pushed a commit that referenced this pull request Mar 17, 2023
* Add documentation for COS as storage

* Address review comments
tareqmamari pushed a commit that referenced this pull request Mar 18, 2023
* Add documentation for COS as storage

* Address review comments
shahulsonhal added a commit that referenced this pull request Mar 28, 2023
* Add documentation for COS as storage

* Address review comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants