Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cloud Security][CIS GCP]Migration for new fields #167545

Merged
merged 47 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
e4e173d
added base update (still rough) for GCP Organization option + Updates…
animehart Sep 21, 2023
90af2ed
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Sep 21, 2023
83bf2dc
more update on GCP Organization
animehart Sep 22, 2023
85cc99b
fix conflict
animehart Sep 22, 2023
54a3b20
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 22, 2023
c0ab168
added more unit test
animehart Sep 23, 2023
45ca6d8
Merge branch 'cis-gcp-organization' of github.com:animehart/kibana in…
animehart Sep 23, 2023
3366c97
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Sep 23, 2023
d8ef6b2
a little fix or update for the unit test
animehart Sep 24, 2023
383e3d0
fix conflict
animehart Sep 24, 2023
aacb6e8
version update checks fix
animehart Sep 26, 2023
14853ae
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 26, 2023
a253c63
updates
animehart Sep 27, 2023
38a6cef
Merge branch 'cis-gcp-organization' of github.com:animehart/kibana in…
animehart Sep 27, 2023
f0d6ff5
Fix conflict
animehart Sep 27, 2023
eef51dc
fix
animehart Sep 27, 2023
5c4bde8
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 27, 2023
e1ffc0c
update steps to change based if user provided prj id or not
animehart Sep 28, 2023
ee94563
Merge branch 'cis-gcp-organization' of github.com:animehart/kibana in…
animehart Sep 28, 2023
a555ed5
check fail fix
animehart Sep 28, 2023
e8280b7
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Sep 28, 2023
a6a18af
fix conflict
animehart Sep 28, 2023
19bfd44
Merge branch 'cis-gcp-organization' of github.com:animehart/kibana in…
animehart Sep 28, 2023
2efd21e
Merge branch 'main' into cis-gcp-organization
animehart Sep 28, 2023
5e29f55
migration attempt
animehart Sep 28, 2023
85d8725
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 28, 2023
11b23ea
fixed migrations to fit serverless model
kfirpeled Sep 28, 2023
325f233
Update to_v8_11_0.ts
kfirpeled Sep 28, 2023
9afd4b7
fix conflict
animehart Sep 28, 2023
dd0d759
more conflict fix
animehart Sep 28, 2023
eb365ae
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 28, 2023
8a4137b
added test and reverted a little change i made on previous push
animehart Sep 29, 2023
4cc3556
Merge branch 'cis-gcp-migration' of github.com:animehart/kibana into …
animehart Sep 29, 2023
c03a6af
fix
animehart Sep 29, 2023
b360793
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 29, 2023
8c39269
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Sep 29, 2023
bdc83a3
change test name
animehart Sep 29, 2023
b78cde7
Merge branch 'cis-gcp-migration' of github.com:animehart/kibana into …
animehart Sep 29, 2023
30ffab7
pr comments
animehart Oct 2, 2023
7505f62
Merge branch 'main' into cis-gcp-migration
animehart Oct 2, 2023
006b3e3
Merge branch 'main' into cis-gcp-migration
animehart Oct 2, 2023
c56e2d1
Merge branch 'main' into cis-gcp-migration
animehart Oct 2, 2023
cd31e4e
Merge branch 'main' into cis-gcp-migration
animehart Oct 2, 2023
605d23d
fix
animehart Oct 3, 2023
a60c814
Merge branch 'cis-gcp-migration' of github.com:animehart/kibana into …
animehart Oct 3, 2023
9c75094
test
animehart Oct 3, 2023
2108a0d
update test file
animehart Oct 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions x-pack/plugins/fleet/server/saved_objects/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,14 @@ import {
migratePackagePolicyToV8110,
} from './migrations/security_solution/to_v8_11_0';

import { migrateCspPackagePolicyToV8110 } from './migrations/cloud_security_posture';

import { migrateOutputEvictionsFromV8100, migrateOutputToV8100 } from './migrations/to_v8_10_0';

import { migrateSyntheticsPackagePolicyToV8100 } from './migrations/synthetics/to_v8_10_0';

import { migratePackagePolicyEvictionsFromV8100 } from './migrations/security_solution/to_v8_10_0';

import {
migratePackagePolicyEvictionsFromV81102,
migratePackagePolicyToV81102,
} from './migrations/security_solution/to_v8_11_0_2';

import {
migrateAgentPolicyToV7100,
migratePackagePolicyToV7100,
Expand Down Expand Up @@ -78,6 +75,10 @@ import {
} from './migrations/security_solution';
import { migratePackagePolicyToV880 } from './migrations/to_v8_8_0';
import { migrateAgentPolicyToV890 } from './migrations/to_v8_9_0';
import {
migratePackagePolicyToV81102,
migratePackagePolicyEvictionsFromV81102,
} from './migrations/security_solution/to_v8_11_0_2';

/*
* Saved object types and mappings
Expand Down Expand Up @@ -346,6 +347,10 @@ const getSavedObjectTypes = (): { [key: string]: SavedObjectsType } => ({
type: 'data_backfill',
backfillFn: migratePackagePolicyToV81102,
},
{
kfirpeled marked this conversation as resolved.
Show resolved Hide resolved
type: 'data_backfill',
backfillFn: migrateCspPackagePolicyToV8110,
},
],
schemas: {
forwardCompatibility: migratePackagePolicyEvictionsFromV81102,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

export { migrateCspPackagePolicyToV8110 } from './to_v8_11_0';
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { SavedObjectModelTransformationContext } from '@kbn/core-saved-objects-server';

import { migrateCspPackagePolicyToV8110 as migration } from './to_v8_11_0';

describe('8.11.0 Cloud Security Posture Package Policy migration', () => {
const policyDoc = (
accountType: string,
isAccountTypeCorrect: boolean,
packageName: string
): any => {
return {
id: 'mock-saved-csp-object-id',
attributes: {
name: 'cloud_security_posture_test',
package: {
name: packageName,
title: '',
version: '',
},
id: 'ID_123',
policy_id: '',
enabled: true,
namespace: '',
revision: 0,
updated_at: '',
updated_by: '',
created_at: '',
created_by: '',
inputs: [
{
type: accountType,
enabled: true,
streams: [
{
vars: {
...(isAccountTypeCorrect && {
'gcp.account_type': { value: 'single-account', type: 'text' },
}),
},
},
],
config: {},
},
],
},
type: ' nested',
};
};

it('adds gcp.account_type to policy, set to single', () => {
const initialDoc = policyDoc('cloudbeat/cis_gcp', false, 'cloud_security_posture');
const migratedDoc = policyDoc('cloudbeat/cis_gcp', true, 'cloud_security_posture');
expect(migration(initialDoc, {} as SavedObjectModelTransformationContext)).toEqual({
attributes: migratedDoc.attributes,
});
});

it('if there are no type cloudbeat/cis_gcp, do not add gcp.account_type', () => {
const initialDoc = policyDoc('cloudbeat/cis_aws', false, 'cloud_security_posture');
const migratedDoc = policyDoc('cloudbeat/cis_aws', false, 'cloud_security_posture');
expect(migration(initialDoc, {} as SavedObjectModelTransformationContext)).toEqual({
attributes: migratedDoc.attributes,
});
});

it('if there are no cloud_security_posture package, do not change the doc', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth adding another test case - when the value is already set to something else, we don't set it to single-account

const initialDoc = policyDoc('cloudbeat/cis_gcp', false, 'NOT_cloud_security_posture');
const migratedDoc = policyDoc('cloudbeat/cis_gcp', false, 'NOT_cloud_security_posture');
expect(migration(initialDoc, {} as SavedObjectModelTransformationContext)).toEqual({
attributes: migratedDoc.attributes,
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { SavedObjectModelDataBackfillFn } from '@kbn/core-saved-objects-server';

import type { PackagePolicy } from '../../../../common';

export const migrateCspPackagePolicyToV8110: SavedObjectModelDataBackfillFn<
PackagePolicy,
PackagePolicy
> = (packagePolicyDoc) => {
if (packagePolicyDoc.attributes.package?.name !== 'cloud_security_posture') {
return { attributes: packagePolicyDoc.attributes };
}

const updatedAttributes = packagePolicyDoc.attributes;

const gcpPackage = updatedAttributes.inputs.find((input) => input.type === 'cloudbeat/cis_gcp');

if (gcpPackage) {
const isGcpAccountTypeExists = gcpPackage.streams[0]?.vars?.hasOwnProperty('gcp.account_type');

if (!isGcpAccountTypeExists) {
const migratedPolicy = { 'gcp.account_type': { value: 'single-account', type: 'text' } };
gcpPackage.streams[0].vars = { ...(gcpPackage.streams[0].vars || {}), ...migratedPolicy };
}
}

return {
attributes: updatedAttributes,
};
};