Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-5045. Create acceptance test for using rclone with s3 protocol #7571

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

ptlrs
Copy link
Contributor

@ptlrs ptlrs commented Dec 12, 2024

Please describe your PR in detail:
This PR adds a robot test to validate the rclone tool with ozone
This PR depends on apache/ozone-docker-runner#37 to install the rclone tool to the docker runner image

Although the default path to the rclone config works when running the robot test on the local machine, we explicitly use the /tmp dir to save the rclone config file as the default path is not writable when using the Github CI.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5045

How was this patch tested?

Tested locally by running the robot test

CI will fail until apache/ozone-docker-runner#37 is merged
CI:
https://github.com/ptlrs/ozone/actions/runs/12306745127
https://github.com/ptlrs/ozone/actions/runs/12358744782
https://github.com/ptlrs/ozone/actions/runs/12628265823
https://github.com/ptlrs/ozone/actions/runs/12637768640
https://github.com/ptlrs/ozone/actions/runs/12656460230
https://github.com/ptlrs/ozone/actions/runs/12657589951

@adoroszlai
Copy link
Contributor

CI will fail until apache/ozone-docker-runner#37 is merged
CI: https://github.com/ptlrs/ozone/actions/runs/12306745127

You can use the image from your fork of ozone-docker-runner:

ghcr.io/ptlrs/ozone-runner:0698816c8342823939c001b06bb77bf880479367

It is tagged with the hash of of commit. Replace image version with 0698816c8342823939c001b06bb77bf880479367 at:

<docker.ozone-runner.version>20241212-1-jdk21</docker.ozone-runner.version>

and image name at:

OZONE_RUNNER_IMAGE: ghcr.io/apache/ozone-runner

Otherwise, if you would just like review, use [skip ci] in the commit message to avoid unnecessary CI run.

@adoroszlai
Copy link
Contributor

@ptlrs ozone-runner with rclone is published with tag 20241216-1-jdk21

@ptlrs
Copy link
Contributor Author

ptlrs commented Dec 16, 2024

Thank you @adoroszlai

@ptlrs ptlrs force-pushed the HDDS-5045-Create-acceptance-test-for-using-rclone-with-s3-protocol branch from 54a8b62 to 8e9fd7d Compare December 16, 2024 18:15
@jojochuang
Copy link
Contributor

Looks like some tests are failing

@ptlrs ptlrs force-pushed the HDDS-5045-Create-acceptance-test-for-using-rclone-with-s3-protocol branch from 8e9fd7d to e06775d Compare January 6, 2025 07:16
@ptlrs
Copy link
Contributor Author

ptlrs commented Jan 6, 2025

Although the default path to the rclone config works when running the robot test on the local machine, we explicitly use the /tmp dir to save the rclone config file as the default path is not writable when using the Github CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants