forked from cisagov/Malcolm
-
Notifications
You must be signed in to change notification settings - Fork 62
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
documentation links work in progress
- Loading branch information
Showing
2 changed files
with
112 additions
and
110 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
|  | ||
|
|
||
| ## <a name="Overview"></a>Overview | ||
|
|
||
|  | ||
|
|
||
| Malcolm processes network traffic data in the form of packet capture (docs/PCAP) files or Zeek logs. A [sensor](live-analysis.md#Hedgehog) (packet capture appliance) monitors network traffic mirrored to it over a SPAN port on a network switch or router, or using a network TAP device. [Zeek](https://www.zeek.org/index.html) logs and [Arkime](https://molo.ch/) sessions are generated containing important session metadata from the traffic observed, which are then securely forwarded to a Malcolm instance. Full PCAP files are optionally stored locally on the sensor device for examination later. | ||
|
|
||
| Malcolm parses the network session data and enriches it with additional lookups and mappings including GeoIP mapping, hardware manufacturer lookups from [organizationally unique identifiers (docs/OUI)](http://standards-oui.ieee.org/oui/oui.txt) in MAC addresses, assigning names to [network segments](host-and-subnet-mapping.md#SegmentNaming) and [hosts](host-and-subnet-mapping.md#HostNaming) based on user-defined IP address and MAC mappings, performing [TLS fingerprinting](#https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967), and many others. | ||
|
|
||
| The enriched data is stored in an [OpenSearch](https://opensearch.org/) document store in a format suitable for analysis through two intuitive interfaces: OpenSearch Dashboards, a flexible data visualization plugin with dozens of prebuilt dashboards providing an at-a-glance overview of network protocols; and Arkime, a powerful tool for finding and identifying the network sessions comprising suspected security incidents. These tools can be accessed through a web browser from analyst workstations or for display in a security operations center (SOC). Logs can also optionally be forwarded on to another instance of Malcolm. | ||
|
|
||
|  | ||
|
|
||
| For smaller networks, use at home by network security enthusiasts, or in the field for incident response engagements, Malcolm can also easily be deployed locally on an ordinary consumer workstation or laptop. Malcolm can process local artifacts such as locally-generated Zeek logs, locally-captured PCAP files, and PCAP files collected offline without the use of a dedicated sensor appliance. | ||
|
|
||
| ## <a name="TableOfContents"></a>Table of Contents | ||
|
|
||
| * [Automated Build Workflows Status](#BuildBadges) | ||
| * [Quick start](quickstart.md#QuickStart) | ||
| * [Getting Malcolm](quickstart.md#GetMalcolm) | ||
| * [User interface](quickstart.md#UserInterfaceURLs) | ||
| * [Overview](#Overview) | ||
| * [Components](components.md#Components) | ||
| * [Supported Protocols](protocols.md#Protocols) | ||
| * [Development](development.md#Development) | ||
| * [Building from source](development.md#Build) | ||
| * [Pre-Packaged installation files](development.md#Packager) | ||
| * [Preparing your system](#Preparing) | ||
| * [Recommended system requirements](#SystemRequirements) | ||
| * [System configuration and tuning](malcolm-config.md#ConfigAndTuning) | ||
| * [`docker-compose.yml` parameters](malcolm-config.md#DockerComposeYml) | ||
| * [Linux host system configuration](#HostSystemConfigLinux) | ||
| * [macOS host system configuration](#HostSystemConfigMac) | ||
| * [Windows host system configuration](#HostSystemConfigWindows) | ||
| * [Running Malcolm](#Running) | ||
| * [OpenSearch instances](opensearch-instances.md#OpenSearchInstance) | ||
| * [Authentication and authorization for remote OpenSearch clusters](#OpenSearchAuth) | ||
| * [Configure authentication](authsetup.md#AuthSetup) | ||
| * [Local account management](authsetup.md#AuthBasicAccountManagement) | ||
| * [Lightweight Directory Access Protocol (LDAP) authentication](authsetup.md#AuthLDAP) | ||
| - [LDAP connection security](#AuthLDAPSecurity) | ||
| * [TLS certificates](authsetup.md#TLSCerts) | ||
| * [Starting Malcolm](running.md#Starting) | ||
| * [Stopping and restarting Malcolm](running.md#StopAndRestart) | ||
| * [Clearing Malcolm's data](running.md#Wipe) | ||
| * [Temporary read-only interface](#ReadOnlyUI) | ||
| * [Capture file and log archive upload](upload.md#Upload) | ||
| - [Tagging](upload.md#Tagging) | ||
| - [Processing uploaded PCAPs with Zeek and Suricata](upload.md#UploadPCAPProcessors) | ||
| * [Live analysis](#LiveAnalysis) | ||
| * [Using a network sensor appliance](live-analysis.md#Hedgehog) | ||
| * [Monitoring local network interfaces](#LocalPCAP) | ||
| * [Manually forwarding logs from an external source](#ExternalForward) | ||
| * [Arkime](#Arkime) | ||
| * [Zeek log integration](arkime.md#ArkimeZeek) | ||
| - [Correlating Zeek logs and Arkime sessions](arkime.md#ZeekArkimeFlowCorrelation) | ||
| * [Help](#ArkimeHelp) | ||
| * [Sessions](arkime.md#ArkimeSessions) | ||
| * [PCAP Export](#ArkimePCAPExport) | ||
| * [SPIView](#ArkimeSPIView) | ||
| * [SPIGraph](arkime.md#ArkimeSPIGraph) | ||
| * [Connections](#ArkimeConnections) | ||
| * [Hunt](arkime.md#ArkimeHunt) | ||
| * [Statistics](#ArkimeStats) | ||
| * [Settings](#ArkimeSettings) | ||
| * [OpenSearch Dashboards](dashboards.md#Dashboards) | ||
| * [Discover](#Discover) | ||
| - [Screenshots](#DiscoverGallery) | ||
| * [Visualizations and dashboards](dashboards.md#DashboardsVisualizations) | ||
| - [Prebuilt visualizations and dashboards](#PrebuiltVisualizations) | ||
| - [Screenshots](#PrebuiltVisualizationsGallery) | ||
| - [Building your own visualizations and dashboards](dashboards.md#BuildDashboard) | ||
| + [Screenshots](#NewVisualizationsGallery) | ||
| * [Search Queries in Arkime and OpenSearch](queries-cheat-sheet.md#SearchCheatSheet) | ||
| * [Other Malcolm features](#MalcolmFeatures) | ||
| - [Automatic file extraction and scanning](file-scanning.md#ZeekFileExtraction) | ||
| - [Automatic host and subnet name assignment](host-and-subnet-mapping.md#HostAndSubnetNaming) | ||
| + [IP/MAC address to hostname mapping via `host-map.txt`](host-and-subnet-mapping.md#HostNaming) | ||
| + [CIDR subnet to network segment name mapping via `cidr-map.txt`](host-and-subnet-mapping.md#SegmentNaming) | ||
| + [Defining hostname and CIDR subnet names interface](host-and-subnet-mapping.md#NameMapUI) | ||
| + [Applying mapping changes](#ApplyMapping) | ||
| - [OpenSearch index management](index-management.md#IndexManagement) | ||
| - [Event severity scoring](severity.md#Severity) | ||
| + [Customizing event severity scoring](#SeverityConfig) | ||
| - [Zeek Intelligence Framework](zeek-intel.md#ZeekIntel) | ||
| + [STIX™ and TAXII™](zeek-intel.md#ZeekIntelSTIX) | ||
| + [MISP](zeek-intel.md#ZeekIntelMISP) | ||
| - [Anomaly Detection](#AnomalyDetection) | ||
| - [Alerting](alerting.md#Alerting) | ||
| + [Email Sender Accounts](#AlertingEmail) | ||
| - ["Best Guess" Fingerprinting for ICS Protocols](ics-best-guess.md#ICSBestGuess) | ||
| - [Asset Management with NetBox](netbox.md#NetBox) | ||
| - [CyberChef](#CyberChef) | ||
| - [API](api.md#API) | ||
| + [Examples](api-examples.md#APIExamples) | ||
| * [Ingesting Third-party Logs](#ThirdPartyLogs) | ||
| * [Malcolm installer ISO](malcolm-iso.md#ISO) | ||
| * [Installation](malcolm-iso.md#ISOInstallation) | ||
| * [Generating the ISO](#ISOBuild) | ||
| * [Setup](#ISOSetup) | ||
| * [Time synchronization](#ConfigTime) | ||
| * [Hardening](hardening.md#Hardening) | ||
| * [Compliance Exceptions](#ComplianceExceptions) | ||
| * [Installation example using Ubuntu 22.04 LTS](ubuntu-install-example.md#InstallationExample) | ||
| * [Upgrading Malcolm](malcolm-upgrade.md#UpgradePlan) | ||
| * [Modifying or Contributing to Malcolm](#Contributing) |