Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic cloudapi support #150

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Add basic cloudapi support #150

wants to merge 16 commits into from

Conversation

bcl
Copy link
Collaborator

@bcl bcl commented Nov 13, 2024

This adds support for cloudapi in the existing version of osbuild-composer

  • cloudapi openapi version shown as part of composer-cli status show
  • Start a cloudapi compose using a local blueprint.toml file
  • Start a cloudapi compose with local blueprint and upload.toml
  • Add support for --wait to cloudapi compose start and to composer-cli wait command

Copy link

This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale label Dec 14, 2024
@bcl bcl removed the Stale label Dec 17, 2024
@bcl bcl marked this pull request as draft December 17, 2024 17:04
@bcl
Copy link
Collaborator Author

bcl commented Dec 17, 2024

Needs to copy arch from the blueprint, if set, to the cloudapi request instead of using the host arch.

Copy link

This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jan 17, 2025
@bcl bcl removed the Stale label Jan 22, 2025
@bcl bcl force-pushed the main-cloudapi-part1 branch from 7341469 to 1a4a012 Compare February 3, 2025 23:58
@bcl bcl marked this pull request as ready for review February 6, 2025 01:01
@bcl bcl force-pushed the main-cloudapi-part1 branch from 1a4a012 to f3f3464 Compare February 6, 2025 01:02
@supakeen supakeen self-requested a review February 6, 2025 07:43
@achilleas-k achilleas-k self-requested a review February 6, 2025 09:54
bcl added 10 commits February 6, 2025 09:42
- PostRaw and PostJSON functions
- GetJSON function
- Exists function with test support
- ErrorToString function to parse a cloudapi error json response into a string
- Mock cloud http client for testing
- Tests

Related: RHEL-60148
Related: RHEL-60148
This allows you to build weldr-client with a custom version, eg. for
development so that the NEVRA will always be greater than the last
build.

Use it like this:

BASEVER=$(grep ^VERSION Makefile | awk '{ print $3 }')
VERSION=$BASEVER.$(date +%Y%m%d%H%M) make build-in-podman
This will check to see if the cloudapi socket is available, and if so it
will show the Cloud API version number and name of the service.

Related: RHEL-60148
Retrieve the cloud API's openapi info.

Related: RHEL-60148
This function uses the cloud API to start a compose using a blueprint
that is included in the compose request instead of one that has been
previously uploaded.

Related: RHEL-60124
When a local file is used in place of the blueprint name composer-cli
will use the cloud API to upload the blueprint and start the compose.

Currently this expects the osbuild-composer server to have the
'localsave' option enabled in its osbuild-composer.service file.

Resolves: RHEL-60124
The upload options are passed via the cloud API's image request
ImageOptions:

https://github.com/osbuild/osbuild-composer/blob/main/internal/cloudapi/v2/openapi.v2.yml#L795

the cloud client code does not do any validation of the format or
content, that is all handled by the server which will return an error
describing what went wrong if there is a problem.

Related: RHEL-60124
The upload format for the cloud API server is different from that used
by the weldr API. It is described in the cloud API's openapi schema
document in the UploadOptions section:

https://github.com/osbuild/osbuild-composer/blob/main/internal/cloudapi/v2/openapi.v2.yml#L1015

Related: RHEL-60124
The ExecutionError already prints 'ERROR' so there is no need for it in
the strings passed to it.

Related: RHEL-60148
@bcl bcl force-pushed the main-cloudapi-part1 branch from f3f3464 to a83140d Compare February 6, 2025 17:55
bcl added 5 commits February 7, 2025 11:13
Related: RHEL-60123
Related: RHEL-60148
Related: RHEL-60123
The composes built with the cloud API are not stored in the weldr API
store so the UUID from the user may belong to one or the other. Check
cloud API first, then weldr API and return the weldr API error if it is
not found in either place.

Note that the result printed is different. Cloud API uses 'success' and
'failure' and WELDR API uses 'FINISHED' and 'FAILED'.

Related: RHEL-60124
Related: RHEL-60134
@bcl bcl force-pushed the main-cloudapi-part1 branch from a83140d to 06dc9d6 Compare February 7, 2025 19:14
@bcl
Copy link
Collaborator Author

bcl commented Feb 7, 2025

Added tests for ComposeInfo and ComposeWait

Copy link
Member

@supakeen supakeen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review, will continue later.

// used to query the server.
func NewClient(ctx context.Context, socket HTTPClient, socketPath string) Client {
// TODO
// - check for valid server path
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll assume this is about the socketPath, what kind of validation do you want to do?

// InitClientUnixSocket configures the client to use a unix domain socket
// This configures the cloud.Client with the socket path
// It must be called before using any of the cloud.Client functions.
func InitClientUnixSocket(ctx context.Context, socketPath string) Client {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this returns a new client, I'd call it NewClientFromUnixSocket or such but that's minor.

if route[0] == '/' {
route = route[1:]
}
return fmt.Sprintf("%s://%s/%s", c.protocol, c.host, route)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to use net/url's URL.String() here.


// IsStringInSlice returns true if the string is present, false if not
// slice must be sorted
func IsStringInSlice(slice []string, s string) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be slices.Contains instead or was that introduced later than our minimum version?

}

// GetContentFilename returns the filename from a content disposition header
func GetContentFilename(header string) (string, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be mime.ParseMediaType; you'll get out a map that contains "filename" as the key.

}

// AppendQuery adds the query string to the current url using ? for the first and & for subsequent ones
func AppendQuery(url, query string) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can also be done through net/url.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants