Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DFX Workload Calls #40

Merged
merged 4 commits into from
Mar 31, 2022
Merged

Conversation

Chaffelson
Copy link
Collaborator

Set minimum CDPCLI version to 0.9.48-beta
Set Static Credentials to allow optional access / private key with an access_token
Establish new feature group WORKLOAD_SERVICES with first option of dfworkload
Modify CDPCLI client builder to fetch and use the Bearer token for a workload service transparently from the Environment in passed parameters
Improve CDP CRN validation to be more specific to different CRN types
Update terminology in df.py to reflect separation of DataFlow Deployment from DataFlow Service
Add new functions to df.py to support workload Deployments, this relies on the dfworkload extensions in CDPCLI

Signed-off-by: Daniel Chaffelson [email protected]

@Chaffelson Chaffelson marked this pull request as ready for review January 11, 2022 10:17
@Chaffelson Chaffelson requested a review from wmudge January 11, 2022 10:32
@Chaffelson Chaffelson changed the title Draft: Support DFX Workload Calls Support DFX Workload Calls Jan 21, 2022
src/cdpy/common.py Outdated Show resolved Hide resolved
src/cdpy/common.py Outdated Show resolved Hide resolved
src/cdpy/df.py Outdated Show resolved Hide resolved
src/cdpy/df.py Outdated Show resolved Hide resolved
src/cdpy/df.py Outdated Show resolved Hide resolved
@Chaffelson Chaffelson linked an issue Feb 4, 2022 that may be closed by this pull request
Copy link
Member

@wmudge wmudge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review @asdaraujo comments - they look good to me. Also, is there a reference to the API docs for the df extension used in create_deployment?

Signed-off-by: Daniel Chaffelson <[email protected]>
Signed-off-by: Daniel Chaffelson <[email protected]>
Add support for readyflows
Remove comment support from df_service until string limitations are inline with rest of platform
Add support for terminating deployed flows when disabling the DFX Service

Signed-off-by: Daniel Chaffelson <[email protected]>
Improved application deployment playbook to only attempt Kafka flow deployment when Kafka Datahub is found
Improved cdpy CRN validation by moving substring definitions to a constant and simplifying the validation logic
Improved cdpy.df.describe_service to only run if the dataflow CRN is correctly resolved
Force renamed readyflowCRN for an imported ReadyFlow to be addedReadyflowCrn to improve usability
Renamed some tasks to improve distinction between similar but different activities like enabling the DF service vs deploying a DF flow
Corrected various minor documentation points for DF modules
Normalized responses to readyflow_info, deployment_info, and customflow_info to use listings of the full description of objects to simplify user experience

Signed-off-by: Daniel Chaffelson <[email protected]>
@Chaffelson Chaffelson requested review from wmudge and asdaraujo March 24, 2022 20:03
Copy link
Contributor

@asdaraujo asdaraujo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wmudge wmudge merged commit 6f293c2 into cloudera-labs:devel Mar 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Add 'tags' parameter to DF.enable_service()
3 participants