Skip to content

Latest commit

 

History

History
341 lines (226 loc) · 17.5 KB

CHANGELOG.next.asciidoc

File metadata and controls

341 lines (226 loc) · 17.5 KB

Beats version HEAD

Breaking changes

Affecting all Beats - The Elasticsearch output now enables compression by default. This decreases network data usage by an average of 70-80%, in exchange for 20-25% increased CPU use and ~10% increased ingestion time. The previous default can be restored by setting the flag compression_level: 0 under output.elasticsearch. 36681 - Elastic-agent-autodiscover library updated to version 0.6.4, disabling metadata for deployment and cronjob. Pods that will be created from deployments or cronjobs will not have the extra metadata field for kubernetes.deployment or kubernetes.cronjob, respectively. 36877

Auditbeat

Filebeat

  • Switch types of log.file.device, log.file.inode, log.file.idxhi, log.file.idxlo and log.file.vol fields to strings to better align with ECS and integrations. 36697

Heartbeat

Metricbeat

  • System module now collects the number of threads per process. The elastic-agent-system-metrics was updated to v0.7.0 as the number of threads is collected by it.

Osquerybeat

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats - Support for multiline zookeeper logs 2496 - Add checks to ensure reloading of units if the configuration actually changed. 34346 - Fix namespacing on self-monitoring 32336 - Fix namespacing on self-monitoring 32336 - Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964 - Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031 - 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider - 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field - Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640 - Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820 - Do not print context cancelled error message when running under agent 36006 - Fix recovering from invalid output configuration when running under Elastic-Agent 36016 - Improve StreamBuf append to improve performance when reading long lines from files. 35928 - Eliminate cloning of event in deepUpdate 35945 - Fix ndjson parser to store JSON fields correctly under target 29395 - Support build of projects outside of beats directory 36126 - Add default cgroup regex for add_process_metadata processor 36484 32961 - Fix environment capture by add_process_metadata processor. 36469 36471 - syslog processor - Fix the ability to use when conditions on the processor. 36762 - upgrade elastic-agent-libs to v0.6.0, allows beat running as a windows service to receive more than one change request. 36896

Auditbeat

Filebeat

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fix filestream false positive log error "filestream input with ID 'xyz' already exists" 31767

  • Fix error message formatting from filestream input. 35658

  • Fix error when trying to use include_message parser 35440

  • Fix handling of IPv6 unspecified addresses in TCP input. 35064 35637

  • Fixed a minor code error in the GCS input scheduler where a config value was being used directly instead of the source struct. 35729

  • Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. 35772

  • Fix CEL input JSON marshalling of nested objects. 35763 35774

  • Fix metric collection in GCPPubSub input. 35773

  • Fix end point deregistration in http_endpoint input. 35899 35903

  • Fix duplicate ID panic in filestream metrics. 35964 35972

  • Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. 35996

  • Fix handling of NUL-terminated log lines in Fortinet Firewall module. 36026 36027

  • Make redact field configuration recommended in CEL input and log warning if missing. 36008

  • Fix handling of region name configuration in awss3 input 36034

  • Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

  • Fix panic when sqs input metrics getter is invoked 36101 36077

  • Make CEL input’s now global variable static for evaluation lifetime. 36107

  • Update mito CEL extension library to v1.5.0. 36146

  • Filter out duplicate paths resolved from matching globs. 36253 36256

  • Fix handling of TCP/UDP address resolution during metric initialization. 35064 36287

  • Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

  • Remove erroneous error log in GCPPubSub input. 36296

  • Fix Filebeat Cisco module with missing escape character 36325 36326

  • Fix panic when redact option is not provided to CEL input. 36387 36388

  • Remove 'onFilteredOut' and 'onDroppedOnPublish' callback logs 36299 36399

  • Added a fix for Crowdstrike pipeline handling process arrays 36496

  • Ensure winlog input retains metric collection when handling recoverable errors. 36479 36483

  • Revert error introduced in 35734 when symlinks can’t be resolved in filestream. 36557

  • Fix ignoring external input configuration in take_over: true mode 36378 36395

  • Add validation to http_endpoint config for empty URL 36816 36772

Heartbeat

  • Fix panics when parsing dereferencing invalid parsed url. 34702

  • Fix retries to trigger on a down monitor with no previous state. 36842

Metricbeat

  • in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value 32305

  • Fix and improve AWS metric period calculation to avoid zero-length intervals 32724

  • Add missing cluster metadata to k8s module metricsets 32979 33032

  • Add GCP CloudSQL region filter 32943

  • Fix logstash cgroup mappings 33131

  • Remove unused elasticsearch.node_stats.indices.bulk.avg_time.bytes mapping 33263

  • Make generic SQL GA 34637

  • Collect missing remote_cluster in elasticsearch ccr metricset 34957

  • Add context with timeout in AWS API calls 35425

  • Fix EC2 host.cpu.usage 35717

  • Resolve statsd module’s prematurely halting of metrics parsing upon encountering an invalid packet. 35075

  • Fix the gap in fetching forecast API metrics at the end of each month for Azure billing module 36142

  • Add option in SQL module to execute queries for all dbs. 35688

  • Fix Azure Monitor empty metricnamespace. 36295

  • Fix GCP compute metadata. 36338

  • Add support for api_key authentication in elasticsearch module 36274

  • Add remaining dimensions for azure storage account to make them available for tsdb enablement. 36331

  • Add missing 'TransactionType' dimension for Azure Storage Account. 36413

  • Add log error when statsd server fails to start 36477

Osquerybeat

Packetbeat

Winlogbeat

Elastic Logging Plugin

Added

Affecting all Beats

  • Upgrade to Go 1.20.10. 36846

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • When running under Elastic-Agent the status is now reported per Unit instead of the whole Beat 35874 36183

  • Add warning message to SysV init scripts for RPM-based systems that lack /etc/rc.d/init.d/functions. 35708 36188

  • Mark translate_sid processor is GA. 36279 36280

  • dns processor: Add support for forward lookups (A, AAAA, and TXT). 11416 36394

  • Mark syslog processor as GA, improve docs about how processor handles syslog messages. 36416 36417

  • Add support for AWS external IDs. 36321 36322

  • [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor 36506 Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

Auditbeat

Filebeat

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Add oracle authentication messages parsing 35127

  • Add sanitization capabilities to azure-eventhub input 34874

  • Add support for CRC validation in Filebeat’s HTTP endpoint input. 35204

  • Add support for CRC validation in Zoom module. 35604

  • Add execution budget to CEL input. 35409

  • Add XML decoding support to HTTPJSON. 34438 35235

  • Add delegated account support when using Google ADC in httpjson input. 35507

  • Allow specifying since when to read journald entries. 35408

  • Add metrics for filestream input. 35529

  • Add support for collecting httpjson metrics. 35392

  • Add XML decoding support to CEL. 34438 35372

  • Mark CEL input as GA. 35559

  • Add metrics for gcp-pubsub input. 35614

  • [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. 35674

  • Allow non-AWS endpoints for awss3 input. 35496 35520

  • Under elastic-agent the input metrics will now be included in agent diagnostics dumps. 35798

  • Add Okta input package for entity analytics. 35611

  • Expose harvester metrics from filestream input 35835 33771

  • Add device support for Azure AD entity analytics. 35807

  • Improve CEL input performance. 35915

  • Adding filename details from zip to response for httpjson 33952 34044

  • Added support for min/max template functions in httpjson input. 36094 36036

  • Add clean_session configuration setting for MQTT input. 16204

  • Add fingerprint mode for the filestream scanner and new file identity based on it 34419 35734

  • Add file system metadata to events ingested via filestream 35801 36065

  • Add support for localstack based input integration testing 35727

  • Allow parsing bytes in and bytes out as long integer in CEF processor. 36100 36108

  • Add support for registered owners and users to AzureAD entity analytics provider. 36092

  • Add support for endpoint resolver in AWS config 36208

  • Added support for Okta OAuth2 provider in the httpjson input. 36273

  • Add support of the interval parameter in Salesforce setupaudittrail-rest fileset. 35917 35938

  • Add device handling to Okta input package for entity analytics. 36049

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 36286

  • [Azure] Add input metrics to the azure-eventhub input. 35739

  • Reduce HTTPJSON metrics allocations. 36282

  • Add support for a simplified input configuraton when running under Elastic-Agent 36390

  • Make HTTPJSON response body decoding errors more informative. 36481

  • Allow fine-grained control of entity analytics API requests for Okta provider. 36440 36492

  • Add support for expanding journald.process.capabilities into the human-readable effective capabilities in the ECS process.thread.capabilities.effective field. 36454 36470

  • Allow fine-grained control of entity analytics API requests for AzureAD provider. 36440 36441

  • For request tracer logging in CEL and httpjson the request and response body are no longer included in event.original. The body is still present in http.{request,response}.body.content. 36531

  • Added support for Okta OAuth2 provider in the CEL input. 36336 36521

  • Improve error logging in HTTPJSON input. 36529

  • Disable warning message about ingest pipeline loading when running under Elastic Agent. 36659

  • Add input metrics to http_endpoint input. 36402 36427

  • Remove Event Normalization from GCP PubSub Input. 36716

  • Update mito CEL extension library to v1.6.0. 36651

  • Added support for new features & removed partial save mechanism in the Azure Blob Storage input. 35126 36690

  • Improve template evaluation logging for HTTPJSON input. 36668

  • Add CEL partial value debug function. 36652

  • Added support for new features and removed partial save mechanism in the GCS input. 35847 36713

  • Re-use buffers to optimise memory allocation in fingerprint mode of filestream 36736

  • Allow http_endpoint input to receive PUT and PATCH requests. 36734

  • Add cache processor. 36786

  • Avoid unwanted publication of Azure entity records. 36753

  • Avoid unwanted publication of Okta entity records. 36770

Auditbeat

Libbeat

Heartbeat - Added status to monitor run log report. - Capture and log the individual connection metrics for all the lightweight monitors

Metricbeat

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms 36647

Osquerybeat

Packetbeat

  • Improve efficiency of sniffers by deduplicating interface configurations. 36574 36576

  • Bump Windows Npcap version to v1.76. 36539 36549

Packetbeat

Winlogbeat

Functionbeat

Winlogbeat

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues