Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate test_general_setting documentation to qa-docs #2387

Merged
merged 5 commits into from
Jan 7, 2022

Conversation

fedepacher
Copy link
Contributor

@fedepacher fedepacher commented Dec 22, 2021

Related issue
#2325

Description

As part of epic #1796, this PR adds the missing documentation and migrates the current documentation to the new format used by qa-docs.

The schema used is the one defined in issue #1694

Generated documentation

test_general_settings_enabled.json

{
    "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2",
    "type": "integration",
    "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.",
    "tier": 0,
    "modules": [
        "vulnerability_detector"
    ],
    "components": [
        "manager"
    ],
    "daemons": [
        "wazuh-modulesd",
        "wazuh-db",
        "wazuh-analysisd"
    ],
    "os_platform": [
        "linux"
    ],
    "os_version": [
        "Arch Linux",
        "Amazon Linux 2",
        "Amazon Linux 1",
        "CentOS 8",
        "CentOS 7",
        "CentOS 6",
        "Ubuntu Focal",
        "Ubuntu Bionic",
        "Ubuntu Xenial",
        "Ubuntu Trusty",
        "Debian Buster",
        "Debian Stretch",
        "Debian Jessie",
        "Debian Wheezy",
        "Red Hat 8",
        "Red Hat 7",
        "Red Hat 6"
    ],
    "references": [
        "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html",
        "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#enabled"
    ],
    "tags": [
        "settings",
        "vulnerability",
        "vulnerability_detector"
    ],
    "name": "test_general_settings_enabled.py",
    "id": 1,
    "group_id": 0,
    "path": "tests/integration/test_vulnerability_detector/test_general_settings/test_general_settings_enabled.py",
    "tests": [
        {
            "description": "Check if the `enabled ` option of the vulnerability detector module is working correctly. To do this, it checks the `ossec.log` file for the message indicating that the vulnerability detector is enabled or disabled.",
            "wazuh_min_version": "4.2.0",
            "parameters": [
                {
                    "tags_to_apply": {
                        "type": "string",
                        "brief": "Tags used for use cases."
                    }
                },
                {
                    "custom_callback": {
                        "type": "string",
                        "brief": "Custom callback for the use case."
                    }
                },
                {
                    "custom_error_message": {
                        "type": "string",
                        "brief": "The message shows the vulnerability detector state."
                    }
                },
                {
                    "get_configuration": {
                        "type": "fixture",
                        "brief": "Get configurations from the module."
                    }
                },
                {
                    "configure_environment": {
                        "type": "fixture",
                        "brief": "Configure a custom environment for testing."
                    }
                },
                {
                    "restart_modulesd": {
                        "type": "callable",
                        "brief": "Restart the `wazuh-modulesd` daemon."
                    }
                }
            ],
            "assertions": [
                "Verify that when the `enabled` option is set to `yes`, the vulnerability detector module is running.",
                "Verify that when the `enabled` option is set to `no`, the vulnerability detector module is stopped."
            ],
            "input_description": [
                "Two use cases are found in the test module and include parameters for `enabled` option (`yes` and `no`)."
            ],
            "expected_output": [
                "r'(.*)wazuh-modulesd:vulnerability-detector(.*)'",
                "r'DEBUG: Module disabled. Exiting...'",
                "Vulnerability detector is disabled",
                "Vulnerability detector is enabled"
            ],
            "name": "test_enabled",
            "inputs": [
                "get_configuration0-tags_to_apply0-callback_detect_vulnerability_detector_enabled-Vulnerability detector is disabled",
                "get_configuration0-tags_to_apply1-callback_detect_vulnerability_detector_disabled-Vulnerability detector is enabled",
                "get_configuration1-tags_to_apply0-callback_detect_vulnerability_detector_enabled-Vulnerability detector is disabled",
                "get_configuration1-tags_to_apply1-callback_detect_vulnerability_detector_disabled-Vulnerability detector is enabled"
            ]
        }
    ]
}

test_general_settings_ignore_time.json

{
    "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2",
    "type": "integration",
    "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.",
    "tier": 0,
    "modules": [
        "vulnerability_detector"
    ],
    "components": [
        "manager"
    ],
    "daemons": [
        "wazuh-modulesd",
        "wazuh-db",
        "wazuh-analysisd"
    ],
    "os_platform": [
        "linux"
    ],
    "os_version": [
        "Arch Linux",
        "Amazon Linux 2",
        "Amazon Linux 1",
        "CentOS 8",
        "CentOS 7",
        "CentOS 6",
        "Ubuntu Focal",
        "Ubuntu Bionic",
        "Ubuntu Xenial",
        "Ubuntu Trusty",
        "Debian Buster",
        "Debian Stretch",
        "Debian Jessie",
        "Debian Wheezy",
        "Red Hat 8",
        "Red Hat 7",
        "Red Hat 6"
    ],
    "references": [
        "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html",
        "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#ignore-time"
    ],
    "tags": [
        "settings",
        "vulnerability",
        "vulnerability_detector"
    ],
    "name": "test_general_settings_ignore_time.py",
    "id": 1,
    "group_id": 0,
    "path": "tests/integration/test_vulnerability_detector/test_general_settings/test_general_settings_ignore_time.py",
    "tests": [
        {
            "description": "Check if an alert is not fired during the ignore time interval. To do this, it inserts a custom vulnerability and vulnerable package, it checks the initial vulnerability alert, advances the time clock before the set time, and check that the alert has not been generated. Finally, it advances the time clock just after the set time and checks that the alert has been generated.",
            "wazuh_min_version": "4.2.0",
            "parameters": [
                {
                    "get_configuration": {
                        "type": "fixture",
                        "brief": "Get configurations from the module."
                    }
                },
                {
                    "configure_environment": {
                        "type": "fixture",
                        "brief": "Configure a custom environment for testing."
                    }
                },
                {
                    "restart_modulesd": {
                        "type": "callable",
                        "brief": "Restart the `wazuh-modulesd` daemon."
                    }
                },
                {
                    "prepare_agent": {
                        "type": "fixture",
                        "brief": "Add a mock agent, add a package to it and insert a vulnerability for that package."
                    }
                }
            ],
            "assertions": [
                "Verify that alerts do not appear before ignore time was finished."
            ],
            "input_description": [
                "Three use cases are found in the test module and include ignore time intervals of 3600s, 60m, and 1h. The file real_nvd_feed.json is used to check for vulnerabilities."
            ],
            "expected_output": [
                "Alert did not appear at the start of the test",
                "Alert appeared before ignore_time was finished",
                "Alert did not appear at the end of the test"
            ],
            "name": "test_ignore_time",
            "inputs": [
                "get_configuration0",
                "get_configuration1",
                "get_configuration2"
            ]
        }
    ]
}

test_general_settings_interval.json

{
    "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2",
    "type": "integration",
    "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.",
    "tier": 0,
    "modules": [
        "vulnerability_detector"
    ],
    "components": [
        "manager"
    ],
    "daemons": [
        "wazuh-modulesd",
        "wazuh-db",
        "wazuh-analysisd"
    ],
    "os_platform": [
        "linux"
    ],
    "os_version": [
        "Arch Linux",
        "Amazon Linux 2",
        "Amazon Linux 1",
        "CentOS 8",
        "CentOS 7",
        "CentOS 6",
        "Ubuntu Focal",
        "Ubuntu Bionic",
        "Ubuntu Xenial",
        "Ubuntu Trusty",
        "Debian Buster",
        "Debian Stretch",
        "Debian Jessie",
        "Debian Wheezy",
        "Red Hat 8",
        "Red Hat 7",
        "Red Hat 6"
    ],
    "references": [
        "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html",
        "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#ignore-time"
    ],
    "tags": [
        "settings",
        "vulnerability",
        "vulnerability_detector"
    ],
    "name": "test_general_settings_interval.py",
    "id": 1,
    "group_id": 0,
    "path": "tests/integration/test_vulnerability_detector/test_general_settings/test_general_settings_interval.py",
    "tests": [
        {
            "description": "Check if modulesd waits `interval` between one vulnerability detector scan and another. To do this, it checks in the `ossec.log` file appears the corresponding message.",
            "wazuh_min_version": "4.2.0",
            "parameters": [
                {
                    "get_configuration": {
                        "type": "fixture",
                        "brief": "Get configurations from the module."
                    }
                },
                {
                    "configure_environment": {
                        "type": "fixture",
                        "brief": "Configure a custom environment for testing."
                    }
                },
                {
                    "restart_modulesd": {
                        "type": "callable",
                        "brief": "Restart the `wazuh-modulesd` daemon."
                    }
                }
            ],
            "assertions": [
                "Verify that the Vulnerability Detector process thread sleeps the time set, checking `ossec.log` message."
            ],
            "input_description": [
                "Test cases are defined in the list interval_values and interval_units. This test gets their configuration of the wazuh_interval.yaml file."
            ],
            "expected_output": [
                "Missing sleep between scans"
            ],
            "name": "test_interval",
            "inputs": [
                "1s",
                "1m",
                "1h",
                "1d",
                "2s",
                "2m",
                "2h",
                "2d",
                "5s",
                "5m",
                "5h",
                "5d"
            ]
        }
    ]
}

test_general_settings_run_on_start.json

{
    "copyright": "Copyright (C) 2015-2021, Wazuh Inc.\nCreated by Wazuh, Inc. <[email protected]>.\nThis program is free software; you can redistribute it and/or modify it under the terms of GPLv2",
    "type": "integration",
    "brief": "Wazuh is able to detect vulnerabilities in the applications installed in agents using the Vulnerability Detector module. This software audit is performed through the integration of vulnerability feeds indexed by Redhat, Canonical, Debian, Amazon Linux and NVD Database.",
    "tier": 0,
    "modules": [
        "vulnerability_detector"
    ],
    "components": [
        "manager"
    ],
    "daemons": [
        "wazuh-modulesd",
        "wazuh-db",
        "wazuh-analysisd"
    ],
    "os_platform": [
        "linux"
    ],
    "os_version": [
        "Arch Linux",
        "Amazon Linux 2",
        "Amazon Linux 1",
        "CentOS 8",
        "CentOS 7",
        "CentOS 6",
        "Ubuntu Focal",
        "Ubuntu Bionic",
        "Ubuntu Xenial",
        "Ubuntu Trusty",
        "Debian Buster",
        "Debian Stretch",
        "Debian Jessie",
        "Debian Wheezy",
        "Red Hat 8",
        "Red Hat 7",
        "Red Hat 6"
    ],
    "references": [
        "https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html",
        "https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#run-on-start"
    ],
    "tags": [
        "settings",
        "vulnerability",
        "vulnerability_detector"
    ],
    "name": "test_general_settings_run_on_start.py",
    "id": 1,
    "group_id": 0,
    "path": "tests/integration/test_vulnerability_detector/test_general_settings/test_general_settings_run_on_start.py",
    "tests": [
        {
            "description": "Check if modulesd detects the vulnerability detector scan after starting. To do this, it checks If the parameter run_on_start is set to 'yes'. Modulesd will have to report the vulnerability detector scan. In case of the value 'no', do not report anything.",
            "wazuh_min_version": "4.2.0",
            "parameters": [
                {
                    "get_configuration": {
                        "type": "fixture",
                        "brief": "Get configurations from the module."
                    }
                },
                {
                    "configure_environment": {
                        "type": "fixture",
                        "brief": "Configure a custom environment for testing."
                    }
                },
                {
                    "restart_modulesd": {
                        "type": "callable",
                        "brief": "Restart the `wazuh-modulesd` daemon."
                    }
                }
            ],
            "assertions": [
                "Verify that when the `run_on_start` option is set to `yes`, the vulnerability detector module starts when service starts.",
                "Verify that when the `run_on_start` option is set to `no`, the vulnerability detector module has not started."
            ],
            "input_description": [
                "Two use cases are found in the test module and include parameters for `run_on_start` option (`yes` and `no`). The test case uses the custom_nvd_feed.json file as input file to start scanning for vulnerabilities."
            ],
            "expected_output": [
                "Could not find vulnerability starting scan log",
                "Found starting scan log when run on start is disabled"
            ],
            "name": "test_run_on_start",
            "inputs": [
                "run_on_start_yes",
                "run_on_start_no"
            ]
        }
    ]
}

Tests

  • Python codebase satisfies PEP-8 style style guide. pycodestyle --max-line-length=120 --show-source --show-pep8 file.py.
  • The qa-docs tool does not raise any error.


references:
- https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/index.html
- https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#ignore-time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#ignore-time
- https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/vuln-detector.html#interval

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in this link

the wazuh_interval.yaml file.

expected_output:
- 'Missing sleep between scans'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 'Missing sleep between scans'
- r'.* Sleeping for (.*)...'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in this link

Comment on lines 147 to 149
- 'Alert did not appear at the start of the test'
- 'Alert appeared before ignore_time was finished'
- 'Alert did not appear at the end of the test'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 'Alert did not appear at the start of the test'
- 'Alert appeared before ignore_time was finished'
- 'Alert did not appear at the end of the test'
- r''.* is vulnerable to .*'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in this link

Comment on lines 130 to 131
- 'Could not find vulnerability starting scan log'
- 'Found starting scan log when run on start is disabled'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 'Could not find vulnerability starting scan log'
- 'Found starting scan log when run on start is disabled'
- 'Starting vulnerability scan'
- 'Vulnerability scan finished'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in this link

Copy link

@roronoasins roronoasins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some callbacks were missing, but good job!

real_nvd_feed.json is used to check for vulnerabilities.

expected_output:
- r''.* is vulnerable to .*'
Copy link

@roronoasins roronoasins Dec 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add time travel callback

Suggested change
- r''.* is vulnerable to .*'
- r''.* is vulnerable to .*'
- r'.*Sending FIM event: (.+)$'

Copy link
Contributor Author

@fedepacher fedepacher Dec 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in this link

roronoasins
roronoasins previously approved these changes Dec 24, 2021
Copy link

@roronoasins roronoasins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@snaow snaow merged commit 5a91619 into master Jan 7, 2022
@snaow snaow deleted the 2325-qa-docs-migrate-test-general-settings branch January 7, 2022 08:35
@snaow snaow mentioned this pull request Jan 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants