-
Notifications
You must be signed in to change notification settings - Fork 4.9k
/
Copy pathCHANGELOG.asciidoc
4499 lines (3067 loc) · 187 KB
/
CHANGELOG.asciidoc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
// Use these for links to issue and pulls. Note issues and pulls redirect one to
// each other on Github, so don't worry too much on using the right prefix.
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/
[[release-notes-7.0.0-rc2]]
=== Beats version 7.0.0-rc2
https://github.com/elastic/beats/compare/v7.0.0-rc1...v7.0.0-rc2[Check the HEAD diff]
==== Breaking changes
*Auditbeat*
- Process dataset: Only report processes with executable. {pull}11232[11232]
- Shorten entity IDs. {pull}11405[11405]
*Metricbeat*
- Add connection and request timeouts for HTTP helper. {pull}11032[11032]
==== Bugfixes
*Affecting all Beats*
- Fixed OS family classification in `add_host_metadata` for Amazon Linux, Raspbian, and RedHat Linux. {issue}9134[9134] {pull}11494[11494]
*Filebeat*
- Don't apply multiline rules in Logstash json logs. {pull}11346[11346]
- Fix panic in add_kubernetes_metadata processor when key `log` does not exist. {issue}11543[11543] {pull}11549[11549]
==== Added
*Auditbeat*
- Login dataset: Add event category and type. {pull}11339[11339]
*Filebeat*
- Add support for MySQL 8.0, Percona 8.0 and MariaDB 10.3. {pull}11417[11417]
[[release-notes-7.0.0-rc1]]
=== Beats version 7.0.0-rc1
https://github.com/elastic/beats/compare/v7.0.0-beta1...v7.0.0-rc1[Check the HEAD diff]
==== Breaking changes
*Affecting all Beats*
- On Google Cloud Engine (GCE) the add_cloud_metadata will now trim the project
info from the cloud.machine.type and cloud.availability_zone. {issue}10968[10968]
- Add `cleanup_timeout` option to docker autodiscover, to wait some time before removing configurations after a container is stopped. {issue}10374[10374] {pull}10905[10905]
- Empty `meta.json` file will be treated as a missing meta file. {issue}8558[8558]
- Rename `migration.enabled` config to `migration.6_to_7.enabled`. {pull}11284[11284]
- Initialize the Paths before the keystore and save the keystore into `data/{beatname}.keystore`. {pull}10706[10706]
- Beats Xpack now checks for Basic license on connect. {pull}11296[11296]
*Auditbeat*
- Process dataset: Only report processes with executable. {pull}11232[11232]
*Filebeat*
- Set `ecs: true` in user_agent processors when loading pipelines with Filebeat 7.0.x into Elasticsearch 6.7.x. {issue}10655[10655] {pull}10875[10875]
*Metricbeat*
- Migrate docker module to ECS. {pull}10927[10927]
*Functionbeat*
- Correctly extract Kinesis Data field from the Kinesis Record. {pull}11141[11141]
==== Bugfixes
*Affecting all Beats*
- Reconnections of Kubernetes watchers are now logged at debug level when they are harmless. {pull}10988[10988]
- Add missing host.* fields to fields.yml. {pull}11016[11016]
- Include ip and boolean type when generating index pattern. {pull}10995[10995]
- Using an environment variable for the password when enrolling a beat will now raise an error if the variable doesn't exist. {pull}10936[10936]
- Cancelling enrollment of a beat will not enroll the beat. {issue}10150[10150]
- Allow to configure Kafka fetching strategy for the topic metadata. {pull}10682[10682]
*Auditbeat*
- Package: Disable librpm signal handlers. {pull}10694[10694]
- Login: Handle different bad login UTMP types. {pull}10865[10865]
- System module: Fix and unify bucket closing logic. {pull}10897[10897]
- User dataset: Numerous fixes to error handling. {pull}10942[10942]
*Filebeat*
- Fix errors in filebeat Zeek dashboard and README files. Add notice.log support. {pull}10916[10916]
- Fix a bug when converting NetFlow fields to snake_case. {pull}10950[10950]
- Add on_failure handler for Zeek ingest pipelines. Fix one field name error for notice and add an additional test case. {issue}11004[11004] {pull}11105[11105]
- Fix issue preventing docker container events to be stored if the container has a network interface without ip address. {issue}11225[11225] {pull}11247[11247]
- Change URLPATH grok pattern to support brackets. {issue}11135[11135] {pull}11252[11252]
- Add support for iis log with different address format. {issue}11255[11255] {pull}11256[11256]
*Heartbeat*
- Fix checks for TCP send/receive data {pull}11118[11118]
*Metricbeat*
- Migrate docker autodiscover to ECS. {issue}10757[10757] {pull}10862[10862]
- Fix issue in kubernetes module preventing usage percentages to be properly calculated. {pull}10946[10946]
- Fix for not reusable http client leading to connection leaks in Jolokia module {pull}11014[11014]
- Fix parsing error using GET in Jolokia module. {pull}11075[11075] {issue}11071[11071]
- Collect metrics when EC2 instances are not in running state. {issue}11008[11008] {pull}11023[11023]
- Change ECS field cloud.provider to aws. {pull}11023[11023]
- Add documentation about jolokia autodiscover fields. {issue}10925[10925] {pull}10979[10979]
- Add missing aws.ec2.instance.state.name into fields.yml. {issue}11219[11219] {pull}11221[11221]
- Fix ec2 metricset to collect metrics from Cloudwatch with the same timestamp. {pull}11142[11142]
- Fix potential memory leak in stopped docker metricsets {pull}11294[11294]
*Packetbeat*
- Avoid reporting unknown MongoDB opcodes more than once. {pull}10878[10878]
*Winlogbeat*
- Prevent Winlogbeat from dropping events with invalid XML. {pull}11006[11006]
- Fix Winlogbeat escaping CR, LF and TAB characters. {issue}11328[11328] {pull}11357[11357]
*Functionbeat*
==== Added
*Affecting all Beats*
- Add ip fields to default_field in Elasticsearch template. {pull}11035[11035]
*Auditbeat*
- Move System module to beta. {pull}10800[10800]
*Filebeat*
- Add ISO8601 timestamp support in syslog metricset. {issue}8716[8716] {pull}10736[10736]
- Add support for loading custom NetFlow and IPFIX field definitions to netflow input. {pull}10945[10945] {pull}11223[11223]
- Added categorization fields for SSH login events in the system/auth fileset. {pull}11334[11334]
*Metricbeat*
- Add filters and pie chart for AWS EC2 dashboard. {pull}10596[10596]
==== Known Issue
*Journalbeat*
- Journalbeat requires at least systemd v233 in order to follow entries after journal changes (rotation, vacuum).
[[release-notes-7.0.0-beta1]]
=== Beats version 7.0.0-beta1
https://github.com/elastic/beats/compare/v7.0.0-alpha2...v7.0.0-beta1[Check the HEAD diff]
==== Breaking changes
*Affecting all Beats*
- Embedded html is not escaped anymore by default. {pull}9914[9914]
- Remove port settings from Logstash and Redis output. {pull}9934[9934]
- Rename `process.exe` to `process.executable` in add_process_metadata to align with ECS. {pull}9949[9949]
- Import ECS change https://github.com/elastic/ecs/pull/308[ecs#308]:
leaf field `user.group` is now the `group` field set. {pull}10275[10275]
- Update the code of Central Management to align with the new returned format. {pull}10019[10019]
- Docker and Kubernetes labels/annotations will be "dedoted" by default. {pull}10338[10338]
- Remove --setup command line flag. {pull}10138[10138]
- Remove --version command line flag. {pull}10138[10138]
- Remove --configtest command line flag. {pull}10138[10138]
- Move output.elasticsearch.ilm settings to setup.ilm. {pull}10347[10347]
- ILM will be available by default if Elasticsearch > 7.0 is used. {pull}10347[10347]
*Auditbeat*
- Rename `process.exe` to `process.executable` in auditd module to align with ECS. {pull}9949[9949]
- Rename `process.cwd` to `process.working_directory` in auditd module to align with ECS. {pull}10195[10195]
- Change data type of `process.pid` and `process.ppid` to number in JSON output
of the auditd module. {pull}10195[10195]
- Change data type of `file.uid` and `file.gid` to string in JSON output of the
FIM module. {pull}10195[10195]
- Field `file.origin` changed type from `text` to `keyword`. {pull}10544[10544]
- Rename user fields to ECS in auditd module. {pull}10456[10456]
- Rename `event.type` to `auditd.message_type` in auditd module because event.type is reserved for future use by ECS. {pull}10536[10536]
- Rename `auditd.messages` to `event.original` and `auditd.warnings` to `error.message`. {pull}10577[10577]
*Filebeat*
- Rename many `kibana.log.*` fields to map to ECS. {pull}9301[9301]
- Modify apache/error dataset to follow ECS. {pull}8963[8963]
- Rename many `traefik.access.*` fields to map to ECS. {pull}9005[9005]
- Fix parsing of GC entries in elasticsearch server log. {issue}9513[9513] {pull}9810[9810]
- Rename `read_timestamp` to `event.created` for Redis input. {pull}9924[9924]
- Rename a few `elasticsearch.audit.*` fields to map to ECS. {pull}9293[9293]
- Rename `read_timestamp` to `event.created` for all Filebeat modules using it. {pull}10139[10139]
- Rename many `iis.error.*` fields to map to ECS. {pull}9955[9955]
- Adjust fileset `haproxy.log` to map to ECS. {pull}10143[10143]
- Rename a few `logstash.*` fields to map to ECS, remove logstash.slowlog.message. {pull}9935[9935]
- Rename a few `mongodb.*` fields to map to ECS. {pull}10009[10009]
- Rename a few `mysql.*` fields to map to ECS. {pull}10008[10008]
- Rename a few `nginx.error.*` fields to map to ECS. {pull}10007[10007]
- Rename many `auditd.log.*` fields to map to ECS. {pull}10192[10192]
- Filesets with multiple ingest pipelines added in {pull}8914[8914] only work with Elasticsearch >= 6.5.0 {pull}10001[10001]
- Remove service.name from Elastcsearch module. Replace by service.type. {pull}10042[10042]
- Remove numeric coercions for `user.id` and `group.id`. IDs should be `keyword`. {pull}10233[10233]
- Add grok pattern to support redis 5.0.3 log timestamp. {issue}9819[9819] {pull}10033[10033]
- Now save the 'first seen' timestamp in `event.created` (previously `read_timestamp`),
instead of saving the parsed date. Now aligned with `event.created` semantics elsewhere. {pull}10139[10139]
- Rename `mysql.error.thread_id` and `mysql.slowlog.id` to `mysql.thread_id`. {pull}10161[10161]
- Remove `mysql.error.timestamp` and `mysql.slowlog.timestamp`. {pull}10161[10161]
- Migrate multiple fields to `event.duration`, from modules "apache", "elasticsearch",
"haproxy", "iis", "kibana", "mysql", "nginx", "postgresql" and "traefik",
including `http.response.elapsed_time` (ECS). {pull}10188[10188], {pull}10274[10274]
- Rename multiple fields to `http.response.body.bytes`, from modules "apache", "iis",
"kibana", "nginx" and "traefik", including `http.response.content_length` (ECS). {pull}10188[10188]
- Change type from haproxy.log fileset fields from text to keyword: response.captured_headers, request.captured_headers, `raw_request_line`, `mode`. {pull}10397[10397]
- Change type of field backend_url and frontend_name in traefik.access metricset to type keyword. {pull}10401[10401]
- Ingesting Elasticsearch audit logs is only supported with Elasticsearch 6.5.0 and above {pull}10352[10352]
- Migrate Elasticsearch audit logs fields to ECS {pull}10352[10352]
- Several text fields in the Logstash module are now indexed as `keyword` fields with `text` multi-fields (ECS). {pull}10417[10417]
- Several text fields in the Elasticsearch module are now indexed as `keyword` fields with `text` multi-fields (ECS). {pull}10414[10414]
- Move dissect pattern for traefik.access fileset from Filbeat to Elasticsearch. {pull}10442[10442]
- The `elasticsearch/deprecation` fileset now indexes the `component` field under `elasticsearch` instead of `elasticsearch.server`. {pull}10445[10445]
- Remove field `kafka.log.trace.full` from kafka.log fielset. {pull}10398[10398]
- Change field `kafka.log.class` for kafka.log fileset from text to keyword. {pull}10398[10398]
- Address add_kubernetes_metadata processor issue where old source field is
still used for matcher. {issue}10505[10505] {pull}10506[10506]
- Change type of haproxy.source from text to keyword. {pull}10506[10506]
- Rename `event.type` to `suricata.eve.event_type` in Suricata module because event.type is reserved for future use by ECS. {pull}10575[10575]
- Populate more ECS fields in the Suricata module. {pull}10006[10006]
- Rename setting `filebeat.registry_flush` to `filebeat.registry.flush`. {pull}10504[10504]
- Rename setting `filebeat.registry_file_permission` to `filebeat.registry.file_permission`. {pull}10504[10504]
- Remove setting `filebeat.registry_file` in favor of `filebeat.registry.path`. The registry file will be stored in a sub-directory by now. {pull}10504[10504]
*Heartbeat*
- Remove monitor generator script that was rarely used. {pull}9648[9648]
- monitor IDs are now configurable. Auto generated monitor IDs now use a different formula based on a hash of their config values. If you wish to have continuity with the old format of monitor IDs you'll need to set the `id` property explicitly. {pull}9697[9697]
- A number of fields have been aliased to their relevant counterparts in the `url.*` field. Existing visualizations should mostly work. The fields that have been moved are `monitor.scheme -> url.scheme`, `monitor.host -> url.domain`, `resolve.host -> url.domain`, `http.url -> url.full`, `tcp.port -> url.port`. In addition to these moves the new fields `url.username`, `url.password`, `url.path`, and `url.query` are now present. It should be noted that the `url.password` field does not contain actual password values, but rather the text `<hidden>` {pull}9570[9570].
- The included Kibana HTTP dashboard is now removed in favor of the Uptime app in Kibana. {pull}10294[10294]
*Journalbeat*
- Rename read_timestamp to event.created to align with ECS. {pull}10043[10043], {pull}10139[10139]
- Rename host.name to host.hostname to align with ECS. {pull}10043[10043]
- Fix typo in the field name `container.id_truncated`. {pull}10525[10525]
- Rename `container.image.tag` to `container.log.tag`. {pull}10561[10561]
- Change type of `text` fields to `keyword`. {pull}10542[10542]
*Metricbeat*
- Migrate system process metricset fields to ECS. {pull}10332[10332]
- Refactor Prometheus metric mappings {pull}9948[9948]
- Removed Prometheus stats metricset in favor of just using Prometheus collector {pull}9948[9948]
- Migrate system socket metricset fields to ECS. {pull}10339[10339]
- Renamed direction values in sockets to ECS recommendations, from incoming/outcoming to inbound/outbound. {pull}10339[10339]
- Adjust Redis.info metricset fields to ECS. {pull}10319[10319]
- Change type of field docker.container.ip_addresses to `ip` instead of `keyword`. {pull}10364[10364]
- Rename http.request.body field to http.request.body.content. {pull}10315[10315]
- Adjust php_fpm.process metricset fields to ECS. {pull}10366[10366]
- Adjust mongodb.status metricset to to ECS. {pull}10368[10368]
- Refactor munin module to collect an event per plugin and to have more strict field mappings. `namespace` option has been removed, and will be replaced by `service.name`. {pull}10322[10322]
- Change the following fields from type text to keyword: {pull}10318[10318]
- ceph.osd_df.name
- ceph.osd_tree.name
- ceph.osd_tree.children
- kafka.consumergroup.meta
- kibana.stats.name
- mongodb.metrics.replication.executor.network_interface
- php_fpm.process.request_uri
- php_fpm.process.script
- Add `service.name` option to all modules to explicitly set `service.name` if it is unset. {pull}10427[10427]
- Update a few elasticsearch.* fields to map to ECS. {pull}10350[10350]
- Update a few logstash.* fields to map to ECS. {pull}10350[10350]
- Update a few kibana.* fields to map to ECS. {pull}10350[10350]
- Update rabbitmq.* fields to map to ECS. {pull}10563[10563]
- Update haproxy.* fields to map to ECS. {pull}10558[10558] {pull}10568[10568]
- Collect all EC2 meta data from all instances in all states. {pull}10628[10628]
- Fix MongoDB dashboard that had some incorrect field names from `status` Metricset {pull}9795[9795] {issue}9715[9715]
*Packetbeat*
- Adjust Packetbeat `http` fields to ECS Beta 2 {pull}9645[9645]
- `http.request.body` moves to `http.request.body.content`
- `http.response.body` moves to `http.response.body.content`
- Changed Packetbeat fields to align with ECS. {issue}7968[7968]
- Removed trailing dot from domain names reported by the DNS protocol. {pull}9941[9941]
*Winlogbeat*
- Adjust Winlogbeat fields to map to ECS. {pull}10333[10333]
*Functionbeat*
- Correctly normalize Cloudformation resource name. {issue}10087[10087]
- Functionbeat can now deploy a function for Kinesis. {10116}10116[10116]
- Allow functionbeat to use the keystore. {issue}9009[9009]
==== Bugfixes
*Affecting all Beats*
- Fix config appender registration. {pull}9873[9873]
- Gracefully handle TLS options when enrolling a Beat. {issue}9129[9129]
- The backing off now implements jitter to better distribute the load. {issue}10172[10172]
- Fix TLS certificate DoS vulnerability. {pull}10302[10302]
- Fix panic and file unlock in spool on atomic operation (arm, x86-32). File lock was not released when panic occurs, leading to the beat deadlocking on startup. {pull}10289[10289]
- Fix encoding of timestamps when using disk spool. {issue}10099[10099]
- Fix stopping of modules started by kubernetes autodiscover. {pull}10476[10476]
- Fix a issue when remote and local configuration didn't match when fetching configuration from Central Management. {issue}10587[10587]
- Fix unauthorized error when loading dashboards by adding username and password into kibana config. {issue}10513[10513] {pull}10675[10675]
- Fix exclude_labels when there are dotted keys {pull}10154[10154]
- Fix registry handle leak on Windows (https://github.com/elastic/go-sysinfo/pull/33). {pull}9920[9920]
*Auditbeat*
- Enable System module config on Windows. {pull}10237[10237]
*Filebeat*
- Support IPv6 addresses with zone id in IIS ingest pipeline.
{issue}9836[9836] error log: {pull}9869[9869], access log: {pull}9955[9955].
- Support haproxy log lines without captured headers. {issue}9463[9463] {pull}9958[9958]
- Make elasticsearch/audit fileset be more lenient in parsing node name. {issue}10035[10035] {pull}10135[10135]
- Fix bad bytes count in `docker` input when filtering by stream. {pull}10211[10211]
- Fixed data types for roles and indices fields in `elasticsearch/audit` fileset {pull}10307[10307]
- Ensure `source.address` is always populated by the nginx module (ECS). {pull}10418[10418]
- Support mysql 5.7.22 slowlog starting with time information. {issue}7892[7892] {pull}9647[9647]
*Heartbeat*
- Made monitors.d configuration part of the default config. {pull}9004[9004]
- Fixed rare issue where TLS connections to endpoints with x509 certificates missing either notBefore or notAfter would cause the check to fail with a stacktrace. {pull}9566[9566]
*Journalbeat*
- Do not stop collecting events when journal entries change. {pull}9994[9994]
*Metricbeat*
- Fix panics in vsphere module when certain values where not returned by the API. {pull}9784[9784]
- Fix pod UID metadata enrichment in Kubernetes module. {pull}10081[10081]
- Fix issue that would prevent collection of processes without command line on Windows. {pull}10196[10196]
- Fixed data type for tags field in `docker/container` metricset {pull}10307[10307]
- Fixed data type for tags field in `docker/image` metricset {pull}10307[10307]
- Fixed data type for isr field in `kafka/partition` metricset {pull}10307[10307]
- Fixed data types for various hosts fields in `mongodb/replstatus` metricset {pull}10307[10307]
- Added function to close sql database connection. {pull}10355[10355]
- Fix issue with `elasticsearch/node_stats` metricset (x-pack) not indexing `source_node` field. {pull}10639[10639]
*Packetbeat*
- Fix DHCPv4 dashboard that wouldn't load in Kibana. {issue}9850[9850]
- Fixed a crash when using af_packet capture {pull}10477[10477]
*Winlogbeat*
- Close handle on signalEvent. {pull}9838[9838]
*Functionbeat*
- Ensure that functionbeat is logging at info level not debug. {issue}10262[10262]
- Add the required permissions to the role when deployment SQS functions. {issue}9152[9152]
==== Added
*Affecting all Beats*
- Update field definitions for `http` to ECS Beta 2 {pull}9645[9645]
- Add `agent.id` and `agent.ephemeral_id` fields to all beats. {pull}9404[9404]
- Add `name` config option to `add_host_metadata` processor. {pull}9943[9943]
- Add `add_labels` and `add_tags` processors. {pull}9973[9973]
- Add missing file encoding to readers. {pull}10080[10080]
- Introduce `migration.enabled` configuration. {pull}9805[9805]
- Add alias field support in Kibana index pattern. {pull}10075[10075]
- Add `add_fields` processor. {pull}10119[10119]
- Add Kibana field formatter to bytes fields. {pull}10184[10184]
- Document a few more `auditd.log.*` fields. {pull}10192[10192]
- Support Kafka 2.1.0. {pull}10440[10440]
- Add ILM mode `auto` to setup.ilm.enabled setting. This new default value detects if ILM is available {pull}10347[10347]
- Add support to read ILM policy from external JSON file. {pull}10347[10347]
- Add `overwrite` and `check_exists` settings to ILM support. {pull}10347[10347]
- Generate Kibana index pattern on demand instead of using a local file. {pull}10478[10478]
- Calls to Elasticsearch X-Pack APIs made by Beats won't cause deprecation logs in Elasticsearch logs. {9656}9656[9656]
- Allow to unenroll a Beat from the UI. {issue}9452[9452]
- Release Jolokia autodiscover as GA. {pull}9706[9706]
- Allow Central Management to send events back to kibana. {issue}9382[9382]
*Auditbeat*
- Add system module. {pull}9546[9546]
- Add `user.id` (UID) and `user.name` for ECS. {pull}10195[10195]
- Add `group.id` (GID) and `group.name` for ECS. {pull}10195[10195]
- System module `process` dataset: Add user information to processes. {pull}9963[9963]
- Add system `package` dataset. {pull}10225[10225]
- Add system module `login` dataset. {pull}9327[9327]
- Add `entity_id` fields. {pull}10500[10500]
- Add seven dashboards for the system module. {pull}10511[10511]
*Filebeat*
- Add `convert_timezone` option to Elasticsearch module to convert dates to UTC. {issue}9756[9756] {pull}9761[9761]
- Added module for parsing Google Santa logs. {pull}9540[9540]
- Added netflow input type that supports NetFlow v1, v5, v6, v7, v8, v9 and IPFIX. {issue}9399[9399]
- Add option to modules.yml file to indicate that a module has been moved {pull}9432[9432].
- Add support for ssl_request_log in apache2 module. {issue}8088[8088] {pull}9833[9833]
- Add support for iis 7.5 log format. {issue}9753[9753] {pull}9967[9967]
- Add service.type field to all Modules. By default the field is set with the module name. It can be overwritten with `service.type` config. {pull}10042[10042]
- Add support for MariaDB in the `slowlog` fileset of `mysql` module. {pull}9731[9731]
- Apache module's error fileset now performs GeoIP lookup, like the access fileset. {pull}10273[10273]
- Elasticsearch module's slowlog now populates `event.duration` (ECS). {pull}9293[9293]
- HAProxy module now populates `event.duration` and `http.response.bytes` (ECS). {pull}10143[10143]
- Teach elasticsearch/audit fileset to parse out some more fields. {issue}10134[10134] {pull}10137[10137]
- Add convert_timezone to nginx module. {issue}9839[9839] {pull}10148[10148]
- Add support for Percona in the `slowlog` fileset of `mysql` module. {issue}6665[6665] {pull}10227[10227]
- Added support for ingesting structured Elasticsearch audit logs {pull}10352[10352]
- Added support for ingesting structured Elasticsearch slow logs {pull}10445[10445]
- Added support for ingesting structured Elasticsearch deprecation logs {pull}10445[10445]
- New iptables module that receives iptables/ip6tables logs over syslog or file. Supports Ubiquiti Firewall extensions. {issue}8781[8781] {pull}10176[10176]
- Added support for ingesting structured Elasticsearch server logs {pull}10428[10428]
- Populate more ECS fields in the Suricata module. {pull}10006[10006]
- Add module zeek. {issue}9931[9931] {pull}10034[10034]
*Heartbeat*
- Autodiscover metadata is now included in events by default. So, if you are using the docker provider for instance, you'll see the correct fields under the `docker` key. {pull}10258[10258]
*Journalbeat*
- Migrate registry from previously incorrect path. {pull}10486[10486]
*Metricbeat*
- Add `key` metricset to the Redis module. {issue}9582[9582] {pull}9657[9657] {pull}9746[9746]
- Add `socket_summary` metricset to system defaults, removing experimental tag and supporting Windows {pull}9709[9709]
- Add docker `event` metricset. {pull}9856[9856]
- Add 'performance' metricset to x-pack mssql module {pull}9826[9826]
- Add DeDot for kubernetes labels and annotations. {issue}9860[9860] {pull}9939[9939]
- Add more meaningful metrics to 'performance' Metricset on 'MSSQL' module {pull}10011[10011]
- Rename some fields in `performance` Metricset on MSSQL module to match the updated documentation from Microsoft {pull}10074[10074]
- Add AWS EC2 module. {pull}9257[9257] {issue}9300[9300]
- Release windows Metricbeat module as GA. {pull}10163[10163]
- Release traefik Metricbeat module as GA. {pull}10166[10166]
- Release Elastic stack modules (Elasticsearch, Logstash, and Kibana) as GA. {pull}10094[10094]
- List filesystems on Windows that have an access path but not an assigned letter {issue}8916[8916] {pull}10196[10196]
- Add `nats` module. {issue}10071[10071]
- Release uswgi Metricbeat module GA. {pull}10164[10164]
- Release php_fpm module as GA. {pull}10198[10198]
- Release Memcached module as GA. {pull}10199[10199]
- Release etcd module as GA. {pull}10200[10200]
- Release Ceph module as GA. {pull}10202[10202]
- Release aerospike module as GA. {pull}10203[10203]
- Release kubernetes apiserver and event metricsets as GA {pull}10212[10212]
- Release Couchbase module as GA. {pull}10201[10201]
- Release RabbitMQ module GA. {pull}10165[10165]
- Release envoyproxy module GA. {pull}10223[10223]
- Release mongodb.metrics and mongodb.replstatus as GA. {pull}10242[10242]
- Release mysql.galera_status as GA. {pull}10242[10242]
- Release postgresql.statement as GA. {pull}10242[10242]
- Release RabbitMQ Metricbeat module GA. {pull}10165[10165]
- Release Dropwizard module as GA. {pull}10240[10240]
- Release Graphite module as GA. {pull}10240[10240]
- Release kvm module as beta. {pull}10279[10279]
- Release http.server metricset as GA. {pull}10240[10240]
- Release Nats module as GA. {pull}10281[10281]
- Release munin module as GA. {pull}10311[10311]
- Release Golang module as GA. {pull}10312[10312]
- Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. {pull}10222[10222]
- Add support for MySQL 8.0 and tests also for Percona and MariaDB. {pull}10261[10261]
- Rename 'db' Metricset to 'transaction_log' in MSSQL Metricbeat module {pull}10109[10109]
- Add process arguments and the path to its executable file in the system process metricset {pull}10332[10332]
- Added 'server' Metricset to Zookeeper Metricbeat module {issue}8938[8938] {pull}10341[10341]
- Release AWS module as GA. {pull}10345[10345]
- Add overview dashboard to Zookeeper Metricbeat module {pull}10379[10379]
*Packetbeat*
- Add `network.community_id` to Packetbeat flow events. {pull}10061[10061]
- Add aliases for flow fields that were renamed. {issue}7968[7968] {pull}10063[10063]
- Add support to decode mysql prepare statement command. {pull}8084[8084]
*Functionbeat*
- Mark Functionbeat as GA. {pull}10564[10564]
[[release-notes-7.0.0-alpha2]]
=== Beats version 7.0.0-alpha2
https://github.com/elastic/beats/compare/v7.0.0-alpha1...v7.0.0-alpha2[Check the HEAD diff]
==== Breaking changes
*Affecting all Beats*
- Update add_cloud_metadata fields to adjust to ECS. {pull}9265[9265]
- Automaticall cap signed integers to 63bits. {pull}8991[8991]
- Rename beat.timezone to event.timezone. {pull}9458[9458]
- Use _doc as document type. {pull}9056[9056]
- Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927]
- On systems with systemd, the Beats log is now written to journald by default rather than file. To revert this behaviour override BEAT_LOG_OPTS with an empty value. {pull}8942[8942].
*Auditbeat*
- Remove warning for deprecated option: "filters". {pull}9002[9002]
*Filebeat*
- Allow beats to blacklist certain part of the configuration while using Central Management. {pull}9099[9099]
- Remove warnings for deprecated options: "spool_size", "publish_async", "idle_timeout". {pull}9002[9002]
- Rename many `haproxy.*` fields to map to ECS. {pull}9117[9117]
- Rename many `iis.access.*` fields to map to ECS. {pull}9084[9084]
- IIS module's user agent string is no longer encoded (`+` replaced with spaces). {pull}9084[9084]
- Rename many `system.syslog.*` fields to map to ECS. {pull}9135[9135]
- Rename many `nginx.access.*` fields to map to ECS. {pull}9081[9081]
- Rename many `system.auth.*` fields to map to ECS. {pull}9138[9138]
- Rename many `apache2.access.*` fields to map to ECS. {pull}9245[9245]
- Rename `apache2` module to `apache`. {pull}9402[9402]
*Metricbeat*
- Allow beats to blacklist certain part of the configuration while using Central Management. {pull}9099[9099]
- Remove warning for deprecated option: "filters". {pull}9002[9002]
*Packetbeat*
- Renamed the flow event fields to follow Elastic Common Schema. {pull}9121[9121]
- Renamed several client and server fields. IP, port, and process metadata are
now contained under the client and server namespaces. {issue}9303[9303]
*Functionbeat*
- The CLI will now log CloudFormation Stack events. {issue}8912[8912]
- Function concurrency is now set to 5 instead of unreserved. {pull}8992[8992]
==== Bugfixes
*Affecting all Beats*
- Propagate Sync error when running SafeFileRotate. {pull}9069[9069]
- Fix autodiscover configurations stopping when metadata is missing. {pull}8851[8851]
- Log events at the debug level when dropped by encoding problems. {pull}9251[9251]
- Refresh host metadata in add_host_metadata. {pull}9359[9359]
- When collecting swap metrics for beats telemetry or system metricbeat module handle cases of free swap being bigger than total swap by assuming no swap is being used. {issue}6271[6271] {pull}9383[9383]
- Adding logging traces at debug level when the pipeline client receives the following events: onFilteredOut, onDroppedOnPublish. {pull}9016[9016]
- Ignore non index fields in default_field for Elasticsearch. {pull}9549[9549]
- Update Kibana index pattern attributes for objects that are disabled. {pull}9644[9644]
- Enforce validation for the Central Management access token. {issue}9621[9621]
- Update to Golang 1.11.4. {pull}9627[9627]
*Auditbeat*
*Filebeat*
- Correctly parse `December` or `Dec` in the Syslog input. {pull}9349[9349]
- Fix installation of haproxy dashboard. {issue}9307[9307] {pull}9313[9313]
- Don't generate incomplete configurations when logs collection is disabled by hints. {pull}9305[9305]
- Stop runners disabled by hints after previously being started. {pull}9305[9305]
- Fix saved objects in filebeat haproxy dashboard. {pull}9417[9417]
- Use `log.source.address` instead of `log.source.ip` for network input sources. {pull}9487[9487]
- Rename many `redis.log.*` fields to map to ECS. {pull}9315[9315]
- Rename many `icinga.*` fields to map to ECS. {pull}9294[9294]
- Rename many `postgresql.log.*` fields to map to ECS. {pull}9308[9308]
- Rename many `kafka.log.*` fields to map to ECS. {pull}9297[9297]
- Add `convert_timezone` option to Logstash module to convert dates to UTC. {issue}9756[9756] {pull}9797[9797]
*Metricbeat*
- Fix issue preventing diskio metrics collection for idle disks. {issue}9124[9124] {pull}9125[9125]
- Fix panic on docker healthcheck collection on dockers without healthchecks. {pull}9171[9171]
- Fix issue with not collecting Elasticsearch cross-cluster replication stats correctly. {pull}9179[9179]
- The `node.name` field in the `elasticsearch/node` metricset now correctly reports the Elasticsarch node name. Previously this field was incorrectly reporting the node ID instead. {pull}9209[9209]
*Packetbeat*
- Fix issue with process monitor associating traffic to the wrong process. {issue}9151[9151] {pull}9443[9443]
==== Added
*Affecting all Beats*
- Unify dashboard exporter tools. {pull}9097[9097]
- Add cache.ttl to add_host_metadata. {pull}9359[9359]
- Add support for index lifecycle management (beta). {pull}7963[7963]
- Always include Pod UID as part of Pod metadata. {pull}9517[9517]
- Autodiscovery no longer requires that the `condition` field be set. If left unset all configs will be matched. {pull}9029[9029]
- Add geo fields to `add_host_metadata` processor. {pull}9392[9392]
*Filebeat*
- Added the `redirect_stderr` option that allows panics to be logged to log files. {pull}8430[8430]
- Added `detect_null_bytes` selector to detect null bytes from a io.reader. {pull}9210[9210]
- Added `syslog_host` variable to HAProxy module to allow syslog listener to bind to configured host. {pull}9366[9366]
- Added support on Traefik for Common Log Format and Combined Log Format mixed which is the default Traefik format {issue}8015[8015] {issue}6111[6111] {pull}8768[8768].
- Add support for multi-core thread_id in postgresql module {issue}9156[9156] {pull}9482[9482]
*Heartbeat*
- Add last monitor status to dashboard table. Further break out monitors in dashboard table by monitor.ip. {pull}9022[9022]
- Add central management support. {pull}9254[9254]
*Journalbeat*
- Add cursor_seek_fallback option. {pull}9234[9234]
*Metricbeat*
- Add settings to disable docker and cgroup cpu metrics per core. {issue}9187[9187] {pull}9194[9194] {pull}9589[9589]
- The `elasticsearch/node` metricset now reports the Elasticsearch cluster UUID. {pull}8771[8771]
- Add service.type field to Metricbeat. {pull}8965[8965]
- Support GET requests in Jolokia module. {issue}8566[8566] {pull}9226[9226]
- Add freebsd support for the uptime metricset. {pull}9413[9413]
- Add `host.os.name` field to add_host_metadata processor. {issue}8948[8948] {pull}9405[9405]
- Add more TCP statuses to `socket_summary` metricset. {pull}9430[9430]
- Remove experimental tag from ceph metricsets. {pull}9708[9708]
- Add MS SQL module to X-Pack {pull}9414[9414
==== Deprecated
*Metricbeat*
- event.duration is now in nano and not microseconds anymore. {pull}8941[8941]
[[release-notes-7.0.0-alpha1]]
=== Beats version 7.0.0-alpha1
https://github.com/elastic/beats/compare/v6.5.0...v7.0.0-alpha1[View commits]
==== Breaking changes
*Affecting all Beats*
- Dissect syntax change, use * instead of ? when working with field reference. {issue}8054[8054]
*Auditbeat*
- Use `initial_scan` action for new paths. {pull}7954[7954]
- Rename beat.name to agent.type, beat.hostname to agent.hostname, beat.version to agent.version.
- Rename `source.hostname` to `source.domain` in the auditd module. {pull}9027[9027]
*Filebeat*
- Rename `fileset.name` to `event.name`. {pull}8879[8879]
- Rename `fileset.module` to `event.module`. {pull}8879[8879]
- Rename source to log.file.path and log.source.ip {pull}8902[8902]
- Remove the deprecated `prospector(s)` option in the configuration use `input(s)` instead. {pull}8909[8909]
- Rename `offset` to `log.offset`. {pull}8923[8923]
- Rename `source_ecs` to `source` in the Filebeat Suricata module. {pull}8983[8983]
==== Bugfixes
*Affecting all Beats*
- Fixed `-d` CLI flag by trimming spaces from selectors. {pull}7864[7864]
- Fixed Support `add_docker_metadata` in Windows by identifying systems' path separator. {issue}7797[7797]
- Do not panic when no tokenizer string is configured for a dissect processor. {issue}8895[8895]
- Start autodiscover consumers before producers. {pull}7926[7926]
*Filebeat*
- Fixed a memory leak when harvesters are closed. {pull}7820[7820]
- Fix improperly set config for CRI Flag in Docker Input {pull}8899[8899]
- Just enabling the `elasticsearch` fileset and starting Filebeat no longer causes an error. {pull}8891[8891]
- Fix macOS default log path for elasticsearch module based on homebrew paths. {pul}8939[8939]
*Heartbeat*
- Heartbeat now always downloads the entire body of HTTP endpoints, even if no checks against the body content are declared. This fixes an issue where timing metrics would be incorrect in scenarios where the body wasn't used since the connection would be closed soon after the headers were sent, but before the entire body was. {pull}8894[8894]
- `Host` header can now be overridden for HTTP requests sent by Heartbeat monitors. {pull}9148[9516]
*Metricbeat*
- Fix golang.heap.gc.cpu_fraction type from long to float in Golang module. {pull}7789[7789]
- Add missing namespace field in http server metricset {pull}7890[7890]
- Fix race condition when enriching events with kubernetes metadata. {issue}9055[9055] {issue}9067[9067]
*Packetbeat*
- Fixed the mysql missing transactions if monitoring a connection from the start. {pull}8173[8173]
==== Added
*Affecting all Beats*
- Add field `host.os.kernel` to the add_host_metadata processor and to the
internal monitoring data. {issue}7807[7807]
- Add debug check to logp.Logger {pull}7965[7965]
- Count HTTP 429 responses in the elasticsearch output {pull}8056[8056]
- Allow Bus to buffer events in case listeners are not configured. {pull}8527[8527]
- Dissect will now flag event on parsing error. {pull}8751[8751]
- add_cloud_metadata initialization is performed asynchronously to avoid delays on startup. {pull}8845[8845]
- Add DeDot method in add_docker_metadata processor in libbeat. {issue}9350[9350] {pull}9505[9505]
*Filebeat*
- Make inputsource generic taking bufio.SplitFunc as input {pull}7746[7746]
- Add custom unpack to log hints config to avoid env resolution {pull}7710[7710]
- Make docker input check if container strings are empty {pull}7960[7960]
- Keep unparsed user agent information in user_agent.original. {pull}8537[8537]
- Allow to force CRI format parsing for better performance {pull}8424[8424]
*Heartbeat*
- Add automatic config file reloading. {pull}8023[8023]
*Journalbeat*
- Add the ability to check against JSON HTTP bodies with conditions. {pull}8667[8667]
*Metricbeat*
- Add metrics about cache size to memcached module {pull}7740[7740]
- Add experimental socket summary metricset to system module {pull}6782[6782]
- Collect custom cluster `display_name` in `elasticsearch/cluster_stats` metricset. {pull}8445[8445]
- Test etcd module with etcd 3.3. {pull}9068[9068]
- All `elasticsearch` metricsets now have module-level `cluster.id` and `cluster.name` fields. {pull}8770[8770] {pull}8771[8771] {pull}9164[9164] {pull}9165[9165] {pull}9166[9166] {pull}9168[9168]
- All `elasticsearch` node-level metricsets now have `node.id` and `node.name` fields. {pull}9168[9168] {pull}9209[9209]
*Packetbeat*
- Add support to decode HTTP bodies compressed with `gzip` and `deflate`. {pull}7915[7915]
- Added support to calculate certificates' fingerprints (MD5, SHA-1, SHA-256). {issue}8180[8180]
- Support new TLS version negotiation introduced in TLS 1.3. {issue}8647[8647].
[[release-notes-6.7.1]]
=== Beats version 6.7.1
https://github.com/elastic/beats/compare/v6.7.0...v6.7.1[View commits]
==== Breaking changes
*Affecting all Beats*
- Initialize the Paths before the keystore and save the keystore into `data/{beatname}.keystore`. {pull}10706[10706]
==== Bugfixes
*Affecting all Beats*
- Remove IP fields from default_field in Elasticsearch template. {pull}11399[11399]
[[release-notes-6.7.0]]
=== Beats version 6.7.0
https://github.com/elastic/beats/compare/v6.6.2...v6.7.0[View commits]
==== Breaking changes
*Affecting all Beats*
- Port settings have been deprecated in redis/logstash output and will be removed in 7.0. {pull}9915[9915]
- Update the code of Central Management to align with the new returned format. {pull}10019[10019]
- Allow Central Management to send events back to kibana. {issue}9382[9382]
- Fix panic if fields settting is used to configure `hosts.x` fields. {issue}10824[10824] {pull}10935[10935]
- Introduce query.default_field as part of the template. {pull}11205[11205]
- Beats Xpack now checks for Basic license on connect. {pull}11296[11296]
*Filebeat*
- Filesets with multiple ingest pipelines added in {pull}8914[8914] only work with Elasticsearch >= 6.5.0 {pull}10001[10001]
- Add grok pattern to support redis 5.0.3 log timestamp. {issue}9819[9819] {pull}10033[10033]
- Ingesting Elasticsearch audit logs is only supported with Elasticsearch 6.5.0 and above {pull}8852[8852]
- Remove `ecs` option from user_agent processors when loading pipelines with Filebeat 6.7.x into Elasticsearch < 6.7.0. {issue}10655[10655] {pull}11362[11362]
*Heartbeat*
- Remove monitor generator script that was rarely used. {pull}9648[9648]
==== Bugfixes
*Affecting all Beats*
- Fix TLS certificate DoS vulnerability. {pull}10303[10303]
- Fix panic and file unlock in spool on atomic operation (arm, x86-32). File lock was not released when panic occurs, leading to the beat deadlocking on startup. {pull}10289[10289]
- Adding logging traces at debug level when the pipeline client receives the following events: onFilteredOut, onDroppedOnPublish. {pull}9016[9016]
- Do not panic when no tokenizer string is configured for a dissect processor. {issue}8895[8895]
- Fix a issue when remote and local configuration didn't match when fetching configuration from Central Management. {issue}10587[10587]
- Add ECS-like selectors and dedotting to docker autodiscover. {issue}10757[10757] {pull}10862[10862]
- Fix encoding of timestamps when using disk spool. {issue}10099[10099]
- Include ip and boolean type when generating index pattern. {pull}10995[10995]
- Using an environment variable for the password when enrolling a beat will now raise an error if the variable doesn't exist. {pull}10936[10936]
- Cancelling enrollment of a beat will not enroll the beat. {issue}10150[10150]
- Remove IP fields from default_field in Elasticsearch template. {pull}11399[11399]
*Auditbeat*
- Package: Disable librpm signal handlers. {pull}10694[10694]
- Login: Handle different bad login UTMP types. {pull}10865[10865]
- Fix hostname references in System module dashbords. {pull}11064[11064]
- User dataset: Numerous fixes to error handling. {pull}10942[10942]
*Filebeat*
- Support IPv6 addresses with zone id in IIS ingest pipeline. {issue}9836[9836] error log: {pull}9869[9869] access log: {pull}10029[10029]
- Fix bad bytes count in `docker` input when filtering by stream. {pull}10211[10211]
- Fixed data types for roles and indices fields in `elasticsearch/audit` fileset {pull}10307[10307]
- Cover empty request data, url and version in Apache2 module{pull}10846[10846]
- Fix a bug with the convert_timezone option using the incorrect timezone field. {issue}11055[11055] {pull}11164[11164]
- Change URLPATH grok pattern to support brackets. {issue}11135[11135] {pull}11252[11252]
- Add support for iis log with different address format. {issue}11255[11255] {pull}11256[11256]
- Add fix to parse syslog message with priority value 0. {issue}11010[11010]
*Heartbeat*
- `Host` header can now be overridden for HTTP requests sent by Heartbeat monitors. {pull}9148[9516]
- Fix checks for TCP send/receive data {pull}10777[10777]
*Journalbeat*
- Do not stop collecting events when journal entries change. {pull}9994[9994]
*Metricbeat*
- Fix MongoDB dashboard that had some incorrect field names from `status` Metricset {pull}9795[9795] {issue}9715[9715]
- Fix issue that would prevent collection of processes without command line on Windows. {pull}10196[10196]
- Fixed data type for tags field in `docker/container` metricset {pull}10307[10307]
- Fixed data type for tags field in `docker/image` metricset {pull}10307[10307]
- Fixed data type for isr field in `kafka/partition` metricset {pull}10307[10307]
- Fixed data types for various hosts fields in `mongodb/replstatus` metricset {pull}10307[10307]
- Added function to close sql database connection. {pull}10355[10355]
- Fix parsing error using GET in Jolokia module. {pull}11075[11075] {issue}11071[11071]
*Winlogbeat*
- Fix Winlogbeat escaping CR, LF and TAB characters. {issue}11328[11328] {pull}11357[11357]
*Functionbeat*
- Correctly extract Kinesis Data field from the Kinesis Record. {pull}11141[11141]
- Add the required permissions to the role when deployment SQS functions. {issue}9152[9152]
==== Added
*Affecting all Beats*
- Add ip fields to default_field in Elasticsearch template. {pull}11035[11035]
- Add `cleanup_timeout` option to docker autodiscover, to wait some time before removing configurations after a container is stopped. {issue}10374[10374] {pull}10905[10905]
*Auditbeat*
- System module `process` dataset: Add user information to processes. {pull}9963[9963]
- Add system `package` dataset. {pull}10225[10225]
- Add system module `login` dataset. {pull}9327[9327]
- Add `entity_id` fields. {pull}10500[10500]
- Add seven dashboards for the system module. {pull}10511[10511]
*Filebeat*
- Add field log.source.address and log.file.path to replace source. {pull}9435[9435]
- Support mysql 5.7.22 slowlog starting with time information. {issue}7892[7892] {pull}9647[9647]
- Add support for ssl_request_log in apache2 module. {issue}8088[8088] {pull}9833[9833]
- Add support for iis 7.5 log format. {issue}9753[9753] {pull}9967[9967]
- Add support for MariaDB in the `slowlog` fileset of `mysql` module. {pull}9731[9731]
- Add convert_timezone to nginx module. {issue}9839[9839] {pull}10148[10148]
- Add support for Percona in the `slowlog` fileset of `mysql` module. {issue}6665[6665] {pull}10227[10227]
- Added support for ingesting structured Elasticsearch audit logs {pull}8852[8852]
- New iptables module that receives iptables/ip6tables logs over syslog or file. Supports Ubiquiti Firewall extensions. {issue}8781[8781] {pull}10176[10176]
- Populate more ECS fields in the Suricata module. {pull}10006[10006]
*Heartbeat*
- Made monitors.d configuration part of the default config. {pull}9004[9004]
- Autodiscover metadata is now included in events by default. So, if you are using the docker provider for instance, you'll see the correct fields under the `docker` key. {pull}10258[10258]
*Metricbeat*
- Add field `event.dataset` which is `{module}.{metricset}`.
- Add more TCP statuses to `socket_summary` metricset. {pull}9430[9430]
- Remove experimental tag from ceph metricsets. {pull}9708[9708]
- Add `key` metricset to the Redis module. {issue}9582[9582] {pull}9657[9657]
- Add DeDot for kubernetes labels and annotations. {issue}9860[9860] {pull}9939[9939]
- Add docker `event` metricset. {pull}9856[9856]
- Release Ceph module as GA. {pull}10202[10202]
- Release windows Metricbeat module as GA. {pull}10163[10163]
- Release traefik Metricbeat module as GA. {pull}10166[10166]
- List filesystems on Windows that have an access path but not an assigned letter {issue}8916[8916] {pull}10196[10196]
- Release uswgi Metricbeat module GA. {pull}10164[10164]
- Release php_fpm module as GA. {pull}10198[10198]
- Release Memcached module as GA. {pull}10199[10199]
- Release etcd module as GA. {pull}10200[10200]
- Release kubernetes apiserver and event metricsets as GA {pull}10212[10212]
- Release Couchbase module as GA. {pull}10201[10201]
- Release aerospike module as GA. {pull}10203[10203]
- Release envoyproxy module GA. {pull}10223[10223]
- Release mongodb.metrics and mongodb.replstatus as GA. {pull}10242[10242]
- Release mysql.galera_status as Beta. {pull}10242[10242]
- Release postgresql.statement as GA. {pull}10242[10242]
- Release RabbitMQ Metricbeat module GA. {pull}10165[10165]
- Release Dropwizard module as GA. {pull}10240[10240]
- Release Graphite module as GA. {pull}10240[10240]
- Release http.server metricset as GA. {pull}10240[10240]
- Add support for MySQL 8.0 and tests also for Percona and MariaDB. {pull}10261[10261]
- Release use of xpack.enabled: true flag in Elasticsearch and Kibana modules as GA. {pull}10222[10222]
- Release Elastic stack modules (Elasticsearch, Logstash, and Kibana) as GA. {pull}10094[10094]
- Add remaining memory metrics of pods in Kubernetes metricbeat module {pull}10157[10157]
- Added 'server' Metricset to Zookeeper Metricbeat module {issue}8938[8938] {pull}10341[10341]
- Add overview dashboard to Zookeeper Metricbeat module {pull}10379[10379]
*Functionbeat*
- Mark Functionbeat as GA. {pull}10564[10564]
- Functionbeat can now deploy a function for Kinesis. {pull}10116[10116]
- Allow functionbeat to use the keystore. {issue}9009[9009]
==== Deprecated
*Filebeat*
- Deprecate field source. Will be replaced by log.source.address and log.file.path in 7.0. {pull}9435[9435]
*Metricbeat*
- Deprecate field `metricset.rtt`. Replaced by `event.duration` which is in nano instead of micro seconds.
*Packetbeat*
- Support new TLS version negotiation introduced in TLS 1.3. {issue}8647[8647].
==== Known Issue
*Journalbeat*
- Journalbeat requires at least systemd v233 in order to follow entries after journal changes (rotation, vacuum).
[[release-notes-6.6.2]]
=== Beats version 6.6.2
https://github.com/elastic/beats/compare/v6.6.1...6.6.2[View commits]
==== Bugfixes
*Auditbeat*
- System module: Fix and unify bucket closing logic. {pull}10897[10897]
*Filebeat*
- Fix a bug when converting NetFlow fields to snake_case. {pull}10950[10950]
*Metricbeat*
- Fix issue in kubernetes module preventing usage percentages to be properly calculated. {pull}10946[10946]
*Packetbeat*
- Avoid reporting unknown MongoDB opcodes more than once. {pull}10878[10878]
*Winlogbeat*
- Prevent Winlogbeat from dropping events with invalid XML. {pull}11006[11006]
[[release-notes-6.6.1]]
=== Beats version 6.6.1
https://github.com/elastic/beats/compare/v6.6.0...6.6.1[View commits]
==== Breaking changes
*Affecting all Beats*
- Fix stopping of modules started by kubernetes autodiscover. {pull}10476[10476]
*Auditbeat*
- Enable System module config on Windows. {pull}10237[10237]
*Filebeat*
- Fix bad bytes count in `docker` input when filtering by stream. {pull}10211[10211]
- Add `convert_timezone` option to Logstash module to convert dates to UTC. {issue}9756[9756] {pull}9797[9797]
- Add `convert_timezone` option to Elasticsearch module to convert dates to UTC. {issue}9756[9756] {pull}9761[9761]
- Make elasticsearch/audit fileset be more lenient in parsing node name. {issue}10035[10035] {pull}10135[10135]
*Journalbeat*
- Fix fields.yml indentation of audit group which had the effect of creating an incomplete Elasticsearch index template. {pull}10556[10556]
*Metricbeat*
- Fix issue with `elasticsearch/node_stats` metricset (x-pack) not indexing `source_node` field. {pull}10639[10639]
*Packetbeat*
- Fixed a crash when using af_packet capture {pull}10477[10477]
*Functionbeat*
- Ensure that functionbeat is logging at info level not debug. {issue}10262[10262]
==== Added
*Filebeat*
- Teach elasticsearch/audit fileset to parse out some more fields. {issue}10134[10134] {pull}10137[10137]
*Journalbeat*
- Migrate registry from previously incorrect path. {pull}10486[10486]
[[release-notes-6.6.0]]
=== Beats version 6.6.0
https://github.com/elastic/beats/compare/v6.5.4...6.6[View commits]