Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EMT-339: policy response schema, views and tests #65962

Closed
wants to merge 6 commits into from
Closed

EMT-339: policy response schema, views and tests #65962

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9de6691
EMT-339: update schema, views and tests
nnamdifrankie May 11, 2020
ddedf86
EMT-339: fix build
nnamdifrankie May 11, 2020
6089a2e
Merge branch 'master' into EMT-339_update_schema_views
elasticmachine May 11, 2020
dd9d00a
EMt-339: clean up and review comments.
nnamdifrankie May 11, 2020
c7e9d41
Merge branch 'EMT-339_update_schema_views' of github.com:nnamdifranki…
nnamdifrankie May 11, 2020
440fd0d
Merge branch 'master' into EMT-339_update_schema_views
elasticmachine May 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 74 additions & 30 deletions x-pack/plugins/endpoint/common/generate_data.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import {
HostMetadata,
HostOS,
HostPolicyResponse,
HostPolicyResponseActions,
HostPolicyResponseActionStatus,
PolicyData,
} from './types';
Expand Down Expand Up @@ -558,88 +557,108 @@ export class EndpointDocGenerator {
endpoint: {
policy: {
applied: {
actions: {
configure_elasticsearch_connection: {
message: 'elasticsearch communications configured successfully',
actions: [
{
name: 'configure_elasticsearch_connection',
message: 'elasticsearch comes configured successfully',
status: HostPolicyResponseActionStatus.success,
},
configure_kernel: {
{
name: 'configure_kernel',
message: 'Failed to configure kernel',
status: HostPolicyResponseActionStatus.failure,
},
configure_logging: {
{
name: 'configure_logging',
message: 'Successfully configured logging',
status: HostPolicyResponseActionStatus.success,
},
configure_malware: {
{
name: 'configure_malware',
message: 'Unexpected error configuring malware',
status: HostPolicyResponseActionStatus.failure,
},
connect_kernel: {
{
name: 'connect_kernel',
message: 'Successfully initialized minifilter',
status: HostPolicyResponseActionStatus.success,
},
detect_file_open_events: {
{
name: 'detect_file_open_events',
message: 'Successfully stopped file open event reporting',
status: HostPolicyResponseActionStatus.success,
},
detect_file_write_events: {
{
name: 'detect_file_write_events',
message: 'Failed to stop file write event reporting',
status: HostPolicyResponseActionStatus.success,
},
detect_image_load_events: {
{
name: 'detect_image_load_events',
message: 'Successfuly started image load event reporting',
status: HostPolicyResponseActionStatus.success,
},
detect_process_events: {
{
name: 'detect_process_events',
message: 'Successfully started process event reporting',
status: HostPolicyResponseActionStatus.success,
},
download_global_artifacts: {
{
name: 'download_global_artifacts',
message: 'Failed to download EXE model',
status: HostPolicyResponseActionStatus.success,
},
load_config: {
{
name: 'load_config',
message: 'successfully parsed configuration',
status: HostPolicyResponseActionStatus.success,
},
load_malware_model: {
{
name: 'load_malware_mode',
message: 'Error deserializing EXE model; no valid malware model installed',
status: HostPolicyResponseActionStatus.success,
},
read_elasticsearch_config: {
{
name: 'read_elasticsearch_config',
message: 'Successfully read Elasticsearch configuration',
status: HostPolicyResponseActionStatus.success,
},
read_events_config: {
{
name: 'read_events_config',
message: 'Successfully read events configuration',
status: HostPolicyResponseActionStatus.success,
},
read_kernel_config: {
{
name: 'read_kernel_config',
message: 'Succesfully read kernel configuration',
status: HostPolicyResponseActionStatus.success,
},
read_logging_config: {
{
name: 'read_logging_config',
message: 'field (logging.debugview) not found in config',
status: HostPolicyResponseActionStatus.success,
},
read_malware_config: {
{
name: 'read_malware_config',
message: 'Successfully read malware detect configuration',
status: HostPolicyResponseActionStatus.success,
},
workflow: {
{
name: 'workflow',
message: 'Failed to apply a portion of the configuration (kernel)',
status: HostPolicyResponseActionStatus.success,
},
download_model: {
{
name: 'download_model',
message: 'Failed to apply a portion of the configuration (kernel)',
status: HostPolicyResponseActionStatus.success,
},
ingest_events_config: {
{
name: 'ingest_events_config',
message: 'Failed to apply a portion of the configuration (kernel)',
status: HostPolicyResponseActionStatus.success,
},
},
],
id: this.commonInfo.endpoint.policy.id,
policy: {
id: this.commonInfo.endpoint.policy.id,
Expand All @@ -648,23 +667,43 @@ export class EndpointDocGenerator {
response: {
configurations: {
events: {
concerned_actions: ['download_model'],
concerned_actions: this.randomHostPolicyResponseActionNames(),
status: this.randomHostPolicyResponseActionStatus(),
},
logging: {
concerned_actions: this.randomHostPolicyResponseActions(),
concerned_actions: this.randomHostPolicyResponseActionNames(),
status: this.randomHostPolicyResponseActionStatus(),
},
malware: {
concerned_actions: this.randomHostPolicyResponseActions(),
concerned_actions: this.randomHostPolicyResponseActionNames(),
status: this.randomHostPolicyResponseActionStatus(),
},
streaming: {
concerned_actions: this.randomHostPolicyResponseActions(),
concerned_actions: this.randomHostPolicyResponseActionNames(),
status: this.randomHostPolicyResponseActionStatus(),
},
},
},
artifacts: {
global: {
version: '1.4.0',
identifiers: [
{
name: 'endpointpe-model',
sha256: 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
},
],
},
user: {
version: '1.4.0',
identifiers: [
{
name: 'user-model',
sha256: 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
},
],
},
},
status: this.randomHostPolicyResponseActionStatus(),
version: policyVersion,
},
Expand All @@ -673,7 +712,12 @@ export class EndpointDocGenerator {
event: {
created: ts,
id: this.seededUUIDv4(),
kind: 'policy_response',
kind: 'state',
category: 'host',
type: 'change',
module: 'endpoint',
action: 'endpoint_policy_response',
dataset: 'endpoint.policy',
},
};
}
Expand Down Expand Up @@ -722,7 +766,7 @@ export class EndpointDocGenerator {
return uuid.v4({ random: [...this.randomNGenerator(255, 16)] });
}

private randomHostPolicyResponseActions(): Array<keyof HostPolicyResponseActions> {
private randomHostPolicyResponseActionNames(): string[] {
return this.randomArray(this.randomN(8), () =>
this.randomChoice([
'load_config',
Expand Down
61 changes: 28 additions & 33 deletions x-pack/plugins/endpoint/common/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -611,47 +611,27 @@ export enum HostPolicyResponseActionStatus {
}

/**
* The details of a given action
* Host Policy Response Applied Action
*/
export interface HostPolicyResponseActionDetails {
export interface HostPolicyResponseAppliedAction {
name: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we could have a permissive list of possible actions that we could use here and everywhere else where we do string[]. Some like this:

type HostPolicyActionName = 
  | 'download_model'
  | 'ingest_events_config'
  | 'workflow'
  | 'configure_elasticsearch_connection'
  | 'configure_kernel'
  | 'configure_logging'
  | 'configure_malware'
  | 'connect_kernel'
  | 'detect_file_open_events'
  | 'detect_file_write_events'
  | 'detect_image_load_events'
  | 'detect_process_events'
  | 'download_global_artifacts'
  | 'load_config'
  | 'load_malware_model'
  | 'read_elasticsearch_config'
  | 'read_events_config'
  | 'read_kernel_config'
  | 'read_logging_config'
  | 'read_malware_config'
  | string;

Then on this line (and other areas below):

Suggested change
name: string;
name: HostPolicyActionName;

Tried it quickly on my IDE and seems to work ok even with custom unknown strings:

image

status: HostPolicyResponseActionStatus;
message: string;
}

/**
* A known list of possible Endpoint actions
*/
export interface HostPolicyResponseActions {
download_model: HostPolicyResponseActionDetails;
ingest_events_config: HostPolicyResponseActionDetails;
workflow: HostPolicyResponseActionDetails;
configure_elasticsearch_connection: HostPolicyResponseActionDetails;
configure_kernel: HostPolicyResponseActionDetails;
configure_logging: HostPolicyResponseActionDetails;
configure_malware: HostPolicyResponseActionDetails;
connect_kernel: HostPolicyResponseActionDetails;
detect_file_open_events: HostPolicyResponseActionDetails;
detect_file_write_events: HostPolicyResponseActionDetails;
detect_image_load_events: HostPolicyResponseActionDetails;
detect_process_events: HostPolicyResponseActionDetails;
download_global_artifacts: HostPolicyResponseActionDetails;
load_config: HostPolicyResponseActionDetails;
load_malware_model: HostPolicyResponseActionDetails;
read_elasticsearch_config: HostPolicyResponseActionDetails;
read_events_config: HostPolicyResponseActionDetails;
read_kernel_config: HostPolicyResponseActionDetails;
read_logging_config: HostPolicyResponseActionDetails;
read_malware_config: HostPolicyResponseActionDetails;
}

/**
* policy configurations returned by the endpoint in response to a user applying a policy
*/
export type HostPolicyResponseConfiguration = HostPolicyResponse['endpoint']['policy']['applied']['response']['configurations'];

interface HostPolicyResponseConfigurationStatus {
status: HostPolicyResponseActionStatus;
concerned_actions: Array<keyof HostPolicyResponseActions>;
concerned_actions: string[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you implement the suggestion above, this would then be:

Suggested change
concerned_actions: string[];
concerned_actions: HostPolicyActionName[]

}

/**
* Host Policy Response Applied Artifact
*/
interface HostPolicyResponseAppliedArtifact {
name: string;
sha256: string;
}

/**
Expand All @@ -674,6 +654,11 @@ export interface HostPolicyResponse {
created: number;
kind: string;
id: string;
category: string;
type: string;
module: string;
action: string;
dataset: string;
};
agent: {
version: string;
Expand All @@ -685,7 +670,7 @@ export interface HostPolicyResponse {
version: string;
id: string;
status: HostPolicyResponseActionStatus;
actions: Partial<HostPolicyResponseActions>;
actions: HostPolicyResponseAppliedAction[];
policy: {
id: string;
version: string;
Expand All @@ -698,6 +683,16 @@ export interface HostPolicyResponse {
streaming: HostPolicyResponseConfigurationStatus;
};
};
artifacts: {
global: {
version: string;
identifiers: HostPolicyResponseAppliedArtifact[];
};
user: {
version: string;
identifiers: HostPolicyResponseAppliedArtifact[];
};
};
};
};
};
Expand Down
10 changes: 6 additions & 4 deletions x-pack/plugins/endpoint/public/applications/endpoint/store/hosts/middleware.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,16 +80,18 @@ export const hostMiddlewareFactory: ImmutableMiddlewareFactory<HostState> = core
version: '1.0.0',
status: HostPolicyResponseActionStatus.success,
id: '17d4b81d-9940-4b64-9de5-3e03ef1fb5cf',
actions: {
download_model: {
actions: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you probably added this only because @parkiino has not yet merged her change that integrates with API, correct (and to suppress ts errors)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes just to get the test to pass.

{
name: 'download_model',
status: 'success',
message: 'Model downloaded',
},
ingest_events_config: {
{
name: 'ingest_events_config',
status: 'failure',
message: 'No action taken',
},
},
],
response: {
configurations: {
malware: {
Expand Down
8 changes: 5 additions & 3 deletions x-pack/plugins/endpoint/public/applications/endpoint/store/hosts/selectors.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ import querystring from 'querystring';
import { createSelector } from 'reselect';
import {
Immutable,
HostPolicyResponseActions,
HostPolicyResponseAppliedAction,
HostPolicyResponseConfiguration,
HostPolicyResponseActionStatus,
ImmutableArray,
} from '../../../../../common/types';
import { HostState, HostIndexUIQueryParams } from '../../types';

Expand Down Expand Up @@ -62,7 +63,8 @@ export const policyResponseFailedOrWarningActionCount: (
Object.entries(applied.response.configurations).map(([key, val]) => {
let count = 0;
for (const action of val.concerned_actions) {
const actionStatus = applied.actions[action]?.status;
const actionStatus = applied.actions.find(policyActions => policyActions.name === action)
?.status;
if (
actionStatus === HostPolicyResponseActionStatus.failure ||
actionStatus === HostPolicyResponseActionStatus.warning
Expand All @@ -81,7 +83,7 @@ export const policyResponseFailedOrWarningActionCount: (
*/
export const policyResponseActions: (
state: Immutable<HostState>
) => undefined | Partial<HostPolicyResponseActions> = createSelector(
) => undefined | ImmutableArray<HostPolicyResponseAppliedAction> = createSelector(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just use Immutable<> instead of ImmutableArray<>. That type will take care of using the appropriate Immutable type in its implementation. I saw Rob provide this feedback recently when I saw him providing the same feedback - the reason being: if Typescript introduces a builtin Immutable Generic, then we will be in a better place to remove our implementation.

detailsPolicyAppliedResponse,
applied => {
return applied?.actions;
Expand Down
6 changes: 3 additions & 3 deletions x-pack/plugins/endpoint/public/applications/endpoint/view/hosts/details/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ const PolicyResponseFlyoutPanel = memo<{
}>(({ hostMeta }) => {
const { show, ...queryParams } = useHostSelector(uiQueryParams);
const responseConfig = useHostSelector(policyResponseConfigurations);
const responseActionStatus = useHostSelector(policyResponseActions);
const responseActions = useHostSelector(policyResponseActions);
const responseAttentionCount = useHostSelector(policyResponseFailedOrWarningActionCount);
const detailsUri = useMemo(
() =>
Expand Down Expand Up @@ -142,10 +142,10 @@ const PolicyResponseFlyoutPanel = memo<{
/>
</h4>
</EuiText>
{responseConfig !== undefined && responseActionStatus !== undefined ? (
{responseConfig !== undefined && responseActions !== undefined ? (
<PolicyResponse
responseConfig={responseConfig}
responseActionStatus={responseActionStatus}
responseActions={responseActions}
responseAttentionCount={responseAttentionCount}
/>
) : (
Expand Down
Loading