Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(k8s): introduce Kubernetes lease lock mechanism #707

Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
82dfdc7
added the new roles class that handles registering the cluster role &…
instamenta Sep 26, 2024
ff4a619
add initial method wrappers for the kubernetes client to handle clust…
instamenta Sep 26, 2024
263b6be
added login, register and delete for cluster roles
instamenta Sep 27, 2024
4c12d28
improved implementation, refactored added better error handling and m…
instamenta Sep 30, 2024
8c58428
refactoring, tying-up into over-all implementation and adding tests
instamenta Sep 30, 2024
c0d03f8
Merge remote-tracking branch 'origin/main' into 00043-introduce-clust…
instamenta Sep 30, 2024
743dee3
merge with main
instamenta Oct 18, 2024
5755d0a
added manager, transformed to ts existing code
instamenta Oct 18, 2024
3f8fda1
add lease quire, release and renew logic to all write commands
instamenta Oct 18, 2024
06bae40
add mechanism to enable tests not to crash
instamenta Oct 18, 2024
f489a13
bug fixes
instamenta Oct 18, 2024
223d560
fix integration test
instamenta Oct 18, 2024
3a1dd86
Merge branch 'main' into 00043-introduce-cluster-lock-so-that-solo-do…
instamenta Oct 18, 2024
e494bd3
fix 403 error
instamenta Oct 18, 2024
328ff8d
merge with main
instamenta Oct 18, 2024
17f0b55
merge with main
instamenta Oct 18, 2024
e9bcdfb
Merge remote-tracking branch 'origin/00043-introduce-cluster-lock-so-…
instamenta Oct 18, 2024
2262ff0
add retrying with limit and better encapsulated logic
instamenta Oct 18, 2024
13e7702
fix login and improve handling
instamenta Oct 21, 2024
8ec2b1c
fix
instamenta Oct 21, 2024
ba19592
introduce lease wrapper class to simplify the lease implementation
instamenta Oct 21, 2024
1d71020
fixing cli messages, and failing tests
instamenta Oct 21, 2024
a28245a
remove lease aquire from methods that are read only
instamenta Oct 21, 2024
f8ff765
encapsulate the lease wrapper logic inside the lease manager class
instamenta Oct 21, 2024
926a72d
formatting, fixing import's file extensions and removing obsolete --n…
instamenta Oct 21, 2024
9e6bc0f
merge with main
instamenta Oct 22, 2024
3793c8e
merge with main
instamenta Oct 22, 2024
7b29944
Merge branch 'main' into 00043-introduce-cluster-lock-so-that-solo-do…
instamenta Oct 22, 2024
f958697
merge with main
instamenta Oct 23, 2024
6eb1ac1
Merge remote-tracking branch 'origin/00043-introduce-cluster-lock-so-…
instamenta Oct 23, 2024
2799bbf
Merge branch 'main' into 00043-introduce-cluster-lock-so-that-solo-do…
instamenta Oct 23, 2024
237a4c1
merge with main and fix node commands to support lease
instamenta Oct 24, 2024
e28b116
add tests and support for lease
instamenta Oct 24, 2024
3e6e1b5
fix tests and code
instamenta Oct 24, 2024
de6968c
remove use of process.pwd()
instamenta Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
added the new roles class that handles registering the cluster role &…
… cluster role binding

Signed-off-by: instamenta <[email protected]>
instamenta committed Sep 26, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 82dfdc78c8d6db23bf7cdb49f220cba23082300d
25 changes: 24 additions & 1 deletion src/commands/flags.mjs
Original file line number Diff line number Diff line change
@@ -765,6 +765,27 @@ export const mirrorNodeVersion = {
type: 'string'
}
}

/** @type {CommandFlag} **/
export const clusterRoleUsername = {
constName: 'clusterRoleUsername',
name: 'cluster-role-username',
definition: {
describe: 'The username for the cluster role',
type: 'string'
}
}

/** @type {CommandFlag} **/
export const clusterRolePassword = {
constName: 'clusterRolePassword',
name: 'cluster-role-password',
definition: {
describe: 'The password for the cluster role',
type: 'string'
}
}

/** @type {CommandFlag[]} **/
export const allFlags = [
accountId,
@@ -826,7 +847,9 @@ export const allFlags = [
tlsPublicKey,
updateAccountKeys,
valuesFile,
mirrorNodeVersion
mirrorNodeVersion,
clusterRoleUsername,
clusterRolePassword
]

/**
5 changes: 4 additions & 1 deletion src/commands/index.mjs
Original file line number Diff line number Diff line change
@@ -22,6 +22,7 @@ import { NetworkCommand } from './network.mjs'
import { NodeCommand } from './node.mjs'
import { RelayCommand } from './relay.mjs'
import { AccountCommand } from './account.mjs'
import { RolesCommand } from './roles.mjs'
import * as flags from './flags.mjs'

/**
@@ -37,6 +38,7 @@ function Initialize (opts) {
const relayCmd = new RelayCommand(opts)
const accountCmd = new AccountCommand(opts)
const mirrorNodeCmd = new MirrorNodeCommand(opts)
const rolesCmd = new RolesCommand(opts)

return [
InitCommand.getCommandDefinition(initCmd),
@@ -45,7 +47,8 @@ function Initialize (opts) {
NodeCommand.getCommandDefinition(nodeCmd),
RelayCommand.getCommandDefinition(relayCmd),
AccountCommand.getCommandDefinition(accountCmd),
MirrorNodeCommand.getCommandDefinition(mirrorNodeCmd)
MirrorNodeCommand.getCommandDefinition(mirrorNodeCmd),
RolesCommand.getCommandDefinition(rolesCmd)
]
}

42 changes: 41 additions & 1 deletion src/commands/prompts.mjs
Original file line number Diff line number Diff line change
@@ -20,8 +20,8 @@ import fs from 'fs'
import { FullstackTestingError, IllegalArgumentError } from '../core/errors.mjs'
import { ConfigManager, constants } from '../core/index.mjs'
import * as flags from './flags.mjs'
import * as helpers from '../core/helpers.mjs'
import { resetDisabledPrompts } from './flags.mjs'
import * as helpers from '../core/helpers.mjs'

async function prompt (type, task, input, defaultValue, promptMessage, emptyCheckMessage, flagName) {
try {
@@ -269,6 +269,44 @@ export async function promptTlsClusterIssuerType (task, input) {
}
}

export async function promptClusterRoleUsername (task, input) {
if (!input) {
input = await task.prompt(ListrEnquirerPromptAdapter).run({
type: 'text',
message: 'Enter the cluster role username (minimum 3 characters):'
})
}

if (!input) {
throw new FullstackTestingError('Username cannot be empty.')
}

if (input.length < 3) {
throw new FullstackTestingError('Username must be at least 3 characters long.')
}

return input
}

export async function promptClusterRolePassword (task, input) {
if (!input) {
input = await task.prompt(ListrEnquirerPromptAdapter).run({
type: 'password',
message: 'Enter the cluster role password (more than 6 characters):'
})
}

if (!input) {
throw new FullstackTestingError('Password cannot be empty.')
}

if (input.length < 6) {
throw new FullstackTestingError('Password must be more than 6 characters long.')
}

return input
}

export async function promptEnableHederaExplorerTls (task, input) {
return await promptToggle(task, input,
flags.enableHederaExplorerTls.definition.defaultValue,
@@ -473,6 +511,8 @@ export function getPromptMap () {
.set(flags.grpcEndpoints.name, promptGrpcEndpoints)
.set(flags.endpointType.name, promptEndpointType)
.set(flags.mirrorNodeVersion.name, promptMirrorNodeVersion)
.set(flags.clusterRoleUsername.name, promptClusterRoleUsername)
.set(flags.clusterRolePassword.name, promptClusterRolePassword)
}

// build the prompt registry
224 changes: 224 additions & 0 deletions src/commands/roles.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
/**
* Copyright (C) 2024 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the ""License"");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an ""AS IS"" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
'use strict'
import { BaseCommand } from './base.mjs'
import { FullstackTestingError, IllegalArgumentError } from '../core/errors.mjs'
import { flags } from './index.mjs'
import { Listr } from 'listr2'
import * as prompts from './prompts.mjs'
import { constants } from '../core/index.mjs'
import * as k8s from '@kubernetes/client-node'

export class RolesCommand extends BaseCommand {
/**
* @returns {string}
*/
static get DEPLOY_CONFIGS_NAME () {
return 'deployConfigs'
}

/**
* @returns {CommandFlag[]}
*/
static get DEPLOY_FLAGS_LIST () {
return [
flags.namespace,
flags.clusterRoleUsername,
flags.clusterRolePassword
]
}

/**
* Check if ClusterRole exists, and if not, create it
* @param {string} roleName - The name of the ClusterRole to create
* @returns {Promise<void>}
*/
async ensureClusterRole (roleName) {
try {
const clusterRoleExists = await this.k8.getClusterRole(roleName)

if (clusterRoleExists) {
return this.logger.info(`ClusterRole ${roleName} already exists.`)
}

const clusterRoleBody = new k8s.V1ClusterRole()
clusterRoleBody.apiVersion = 'rbac.authorization.k8s.io/v1'
clusterRoleBody.kind = 'ClusterRole'
clusterRoleBody.metadata = { name: roleName }
clusterRoleBody.rules = [{
apiGroups: [''],
resources: ['pods'],
verbs: ['get', 'list', 'watch', 'create', 'delete']
}]

await this.k8.createClusterRole(clusterRoleBody)

this.logger.info(`ClusterRole ${roleName} created.`)
} catch (e) {
throw new FullstackTestingError(`Error ensuring ClusterRole: ${e.message}`, e)
}
}

/**
* Create a user by adding a secret with username and password
* @param {string} username
* @param {string} password
* @param {string} namespace - The namespace to create the secret in
* @returns {Promise<void>}
*/
async createUserSecret (username, password, namespace) {
const data = {
username: Buffer.from(username).toString('base64'),
password: Buffer.from(password).toString('base64')
}

try {
await this.k8.createSecret(`${username}-credentials`, namespace, 'Opaque', data, {}, true)
this.logger.info(`User ${username} created in namespace ${namespace}`)
} catch (e) {
throw new FullstackTestingError(`Error creating user: ${e.message}`, e)
}
}

/**
* Bind a ClusterRole to the user (using RoleBinding)
* @param {string} roleName - The ClusterRole name
* @param {string} username - The username
* @returns {Promise<void>}
*/
async bindRoleToUser (roleName, username) {
const clusterRoleBinding = new k8s.V1ClusterRoleBinding()
clusterRoleBinding.apiVersion = 'rbac.authorization.k8s.io/v1'
clusterRoleBinding.kind = 'ClusterRoleBinding'
clusterRoleBinding.metadata = new k8s.V1ObjectMeta()
clusterRoleBinding.metadata.name = `${username}-rolebinding`

clusterRoleBinding.subjects = [{
kind: 'User',
name: username,
apiGroup: 'rbac.authorization.k8s.io'
}]

clusterRoleBinding.roleRef = {
kind: 'ClusterRole',
name: roleName,
apiGroup: 'rbac.authorization.k8s.io'
}

try {
await this.k8.createClusterRoleBinding(clusterRoleBinding)
this.logger.info(`Bound ClusterRole ${roleName} to user ${username}`)
} catch (e) {
throw new FullstackTestingError(`Error binding role to user: ${e.message}`, e)
}
}

/**
* @param {Object} argv
* @returns {Promise<boolean>}
*/
async add (argv) {
const tasks = new Listr([
{
title: 'Initialize',
task: async (ctx, task) => {
this.configManager.update(argv)

await prompts.execute(task, this.configManager, [
flags.namespace,
flags.clusterRoleUsername,
flags.clusterRolePassword
])

const config = {
namespace: this.configManager.getFlag(flags.namespace)
}

ctx.config = /** @type {MirrorNodeDeployConfigClass} **/ this.getConfig(
RolesCommand.DEPLOY_CONFIGS_NAME, RolesCommand.DEPLOY_FLAGS_LIST)

if (!(await this.k8.hasNamespace(ctx.config.namespace))) {
throw new FullstackTestingError(`Namespace ${config.namespace} does not exist`)
}

this.logger.debug('Initialized config', { config })
}
},
{
title: 'Ensure ClusterRole',
task: () => this.ensureClusterRole('solo-user-role')
},
{
title: 'Create User',
task: (ctx) => this.createUserSecret('new-user', 'new-password', ctx.config.namespace)
},
{
title: 'Bind Role to User',
task: () => this.bindRoleToUser('solo-user-role', 'new-user')
}
], {
concurrent: false,
rendererOptions: constants.LISTR_DEFAULT_RENDERER_OPTION
})

try {
await tasks.run()
} catch (e) {
throw new FullstackTestingError(`Error in adding role: ${e.message}`, e)
}

return true
}

/**
* @param {RolesCommand} accountCmd
* @returns {{command: string, desc: string, builder: Function}}
*/
static getCommandDefinition (accountCmd) {
if (!accountCmd || !(accountCmd instanceof RolesCommand)) {
throw new IllegalArgumentError('An instance of AccountCommand is required', accountCmd)
}
return {
command: 'role',
desc: 'Manage cluster roles in solo',
builder: yargs => {
return yargs
.command({
command: 'add',
desc: 'Add new user',
builder: y => flags.setCommandFlags(y,
flags.namespace
),
handler: argv => {
accountCmd.logger.debug('==== Running \'role add\' ===')
accountCmd.logger.debug(argv)

accountCmd.add(argv)
.then(r => {
accountCmd.logger.debug('==== Finished running \'role add\' ===')
if (!r) process.exit(1)
})
.catch(err => {
accountCmd.logger.showUserError(err)
process.exit(1)
})
}
})
}
}
}
}
Loading