OpenSearch 2.7.0 includes a number of new and enhanced features for your search, observability, and analytics workloads. This release delivers production-ready functionality for segment replication, searchable snapshots, and the ability to add multiple data sources to a single dashboard — features that were previously released as experimental. Also included are new flat object fields for more efficient indexing, integrated observability features within OpenSearch Dashboards, new geospatial functionality, and updates designed to simplify administration of indexes and clusters.
- You can now choose segment replication as a data replication strategy for production workloads. Segment replication can yield improved indexing throughput and lower resource utilization at the expense of increased network utilization and refresh times.
- Searchable snapshots also move from experimental to production-ready with this release, allowing users to search indexes that are stored as snapshots within remote repositories in real time.
- The third feature moving out of experimental status is support for multiple data sources in OpenSearch Dashboards. Dynamically manage data sources across multiple OpenSearch clusters, combine visualizations into a single dashboard, and more.
- New flat object fields let you store complex JSON objects in an index without indexing all subfields separately, offering improved resource utilization.
- You can now access observability features from OpenSearch Dashboards, create your own observability dashboards, add event analytics visualizations, and more without leaving the Dashboards environment.
- Geospatial functionality receives upgrades as new shape-based filters let you filter your geospatial data by drawing a rectangle or polygon over a selected area of the map. This release also enables OpenSearch to display maps in local languages.
- OpenSearch makes managing multiple indexes simpler with the addition of component templates. These templates can help you abstract common index settings, mappings, and aliases into reusable building blocks.
- Another time-saving upgrade for OpenSearch administrators comes with the availability of dynamic tenant management in OpenSearch Dashboards. Admins can view, configure, and enable or disable tenancy within Dashboards and effect those changes without a restart.
- You can now use the Performance Analyzer plugin to identify hot shards within an index, so you can take action and mitigate potential impacts on performance and availability.
OpenSearch 2.7.0 includes the following experimental features. Experimental features are disabled by default. For instructions on how to enable them, refer to the documentation for the feature.
- A new correlation engine is added to the Security Analytics toolkit, enabling a knowledge graph that can be used to identify, store, and recall connected events data to help you identify patterns and investigate relationships.
- The experimental ML Framework gains a new automatic reloading mechanism for ML models, allowing you to auto-reload deployed models when a cluster restarts or when a node rejoins the cluster.
- Ml-commons introduced an API breaking change in 2.7 which changes the API response data. FE is updated accordingly. With this change, FE will no longer be compatible with OpenSearch version < 2.7. (#154)
- Version 2.7.0 of OpenSearch Dashboards offers administrators the ability to manually override the version of Node.js used to run the software. If an administrator chooses this option, the change has the potential to affect the launch logic for OpenSearch Dashboards, in which case the administrator may need to modify their infrastructure code or apply a workaround to enable Dashboards to launch properly. The workaround involves making a change to the CLI shell scripts as described in this issue opensearch-project/OpenSearch-Dashboards#3955 and rebuilding the artifacts for Dashboards as described here https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/DEVELOPER_GUIDE.md#building-artifacts.
OpenSearch and OpenSearch Dashboards 2.7.0 includes the following feature, enhancement, bug fixes, infrastructure, documentation, maintenance and refactoring updates.
OpenSearch Release Notes.
OpenSearch Dashboards Release Notes.
Documentations Release Notes
- Implemented support for configuring up to 10 data filters for query and bucket-level monitors using the visual editor. (#504)
- Implemented support for >, >=, <, and <= query operators for the doc-level monitor visual editor. (#508)
- InjectSecurity - inject User object in UserInfo in threadContext. (#396)
- Add localization for opensearch base map (#312)
- Add geo shape query filter (#319)
- Support adding static label to document layer (#322)
- Add shape filter UI button (#329)
- Add tooltip to draw shape filter (#330)
- Support adding field label to document layer (#336)
- Add search query ability on map(#370)
- [Feature] Exposing Metrics for Search Comparison Tool. (#162)
- Error Prevention: Add close action. (#728)
- Error Prevention: Add index priority action. (#729)
- Error Prevention: Add notification, shrink, allocation, and rollup. (#732)
- Error Prevention: Add transition action. (#744)
- Error Prevention: Add snapshot action. (#745)
- Feature: Enhancement of JSON validation on JSON editor of index mapping (#606)
- Feature: Enable component templates management and simulation (#662)
- Adding name and description to org.opensearch.opensearch-job-scheduler pom (#338)
- Adding Decoupling snapshots (#324)
- Adding generatePomFileForPluginZipPublication as dependency of publishNebulaPublicationToSnapshotsRepository (#360)
- Adding groupId to pom section of build.gradle (#363)
- Observability Visualizations enabled to add to OpenSearch Dashboards / Dashboards #353
- Display Observability Dashboards in OpenSearch Dashboards / Dashboards listing #355
- Observability Left-Navigation now grouped at "first level" of OpenSearch Dashboards Left-Navigation fly-out #354
- Dynamic tenancy configurations (#2607)
- Dynamic tenancy configurations (#1394)
- Giving admin priority over backendrole filtering. (#850)
- Run prettier command against all files (#444)
- Enhance layer visibility status display (#299)
- Introduce disable tooltip on hover property (#313)
- Update tooltip behavior change (#317)
- Update max supported layer count (#332)
- BWC for document layer label textType (#340)
- Add mapbox-gl draw mode (#347)
- Add support to draw rectangle shape to filter documents (#348)
- Avoid trigger tooltip from label (#350)
- Remove cancel button on draw shape and use Escape to cancel draw (#359)
- Add support to build GeoShapeFilterMeta and GeoShapeFilter (#360)
- Update listener on KeyUp (#364)
- Add geoshape filter while render data layers (#365)
- Add filter bar to display global geospatial filters (#371)
- Update draw filter shape ui properties (#372)
- Change font opacity along with OpenSearch base map layer (#373)
- Toast Warning Message if OpenSearch base map is used in conflicting regions (#382)
- Add before layer id when adding documents label (#387)
- Support .opensearch-knn-model index as system index with security enabled (#827)
- Add model auto deploy feature (#852)
- Add memory consumption estimation for models in profile API (#853)
- Add text docs ML input (#830)
- Add allow custom deployment plan setting; add deploy to all nodes field in model index (#818)
- Add exclude nodes setting (#813)
- Set model state as partially loaded if unload model from partial nodes (#806)
- Adding CIRCUIT_BREAKER_COLLECTOR_EXECUTION_TIME, CIRCUIT_BREAKER_COLLECTOR_ERROR, CLUSTER_MANAGER_METRICS_ERROR in StatExceptionCode (420)
- Adding Shard HotSpot feature in RCA (295)
- Clock skew tolerance for oidc token validation (#2482)
- Adding index template permissions to kibana_server role (#2503)
- Add a test in order to catch incorrect handling of index parsing during Snapshot Restoration (#2384)
- Expand Dls Tests for easier verification of functionality (#2634)
- New system index[.ql-datasources] for ppl/sql datasource configurations (#2650)
- Allows for configuration of LDAP referral following (#2135)
- Replace legacy template with index template (#1359)
- Add loginEndPointWithPath (#1358)
- Add new actions for ppl and datasource apis (#1395)
- Split up a value into multiple cookie payloads (#1352)
- [main]Changes in DataSourceService and DataSourceMetadataStorage interface (#1414)
- Added SINH function to V2 engine (#1437)
- Added RINT function to V2 engine (#1439)
- Add
sec_to_timeFunction To OpenSearch SQL (#1438) - Add
WEEKDAYFunction to SQL Plugin (#1440) - Add
YEARWEEKFunction To OpenSearch SQL (#1445) - Add
EXTRACTFunction To OpenSearch SQL Plugin (#1443) - Add
STR_TO_DATEFunction To The SQL Plugin (#1444) - Add The
TO_SECONDSFunction To The SQL Plugin (#1447) - Added Arithmetic functions to V2 engine (#1448)
- Added SIGNUM function to V2 engine (#1442)
- Add
TIMESTAMPADDFunction To OpenSearch SQL Plugin (#1453) - Add
TimestampdiffFunction To OpenSearch SQL (#1472) - Add Nested Support in Select Clause (#1490) (#1518)
- Issue with percolate query transforming documents with object type fields. (#844)
- Notification security fix. (#852)
- Fix SNS regex for validation on notification channel to support SNS FIFO topics. (#381)
- Modified autofollow stats to rely on single source for failed indices (#736)
- Update UpdateAutoFollowPatternIT "test auto follow stats" to wait for 60 seconds (#745)
- Update imports from common to core package (#761)
- Adding a proxy mode connection setup for CCR (#795)
- Handled exception under multi-field mapping update (#789)
- Handled batch requests for replication metadata update under cluster state (#778)
- Fix: fixed filters not reset when index pattern changed (#234)
- Fix property value undefined check (#276)
- Show scroll bar when panel height reaches container bottom (#295)
- Add custom layer visibility config to render (#297)
- Fix color picker component issue (#305)
- Fix: layer filter setting been reset unexpectedly (#327)
- Fix data query in dashboard mode when enable around map filter (#339)
- Sync maplibre layer order after layers rendered (#353)
- Fix CI flow. (#33)
- Upgrade json2csv to fix missing value for nested field with false/0 (#78)
- Preserve chart legends in reports (#81)
- Add tesseract and foreign object rendering (#86)
- Update jsdom version to remove request dependency (#89)
- [Bug][Build] Export config file (#182)
- Removing stats.yml until an alternate can be found that can publish P… (#110)
- Shrink action Fix. (#718)
- Fix link destination and make the link open in a new window. (#652)
- Fix the alias deletion issue in security mode.(#704)
- Add downgrade logic when fetching indexes (#684)
- Fix bugs during internal bugathon and fix the alias deletion issue in security mode. (#704)
- Throw errors on model deletion failures (#834)
- Change to old method to fix missing method createParentDirectories (#759)
- Fix delete model API (#861)
- Fix breaking changes of Xcontent namespace change (#838)
- Change the ziputil dependency to fix a potential security concern (#824)
- Fix checkstyle version (#792)
- Typo fix and minor improvement in maven-publish GHA workflow (#757)
- Fixed an issue that model status is displayed as
Not respondingwhile the model status is loading (#146)
- Convert empty httpEntity to {} to avoid DeliveryStatus initialization exception (#648)
- Fix AdmissionControl class loading issue in Netty/PA communication (#414)
- Fix GC metric not collected in RCA (287)
- Fix ShardEvents and ShardBulkDocs null metrics in RCA (283)
- Fix for integ test failures. (#363)
- Support multitenancy for the anonymous user (#2459)
- Fix error message when system index is blocked (#2525)
- Fix of OpenSSLTest is not using the OpenSSL Provider (#2301)
- Add chmod 0600 to install_demo_configuration bash script (#2550)
- Fix SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder" (#2564)
- Fix lost privileges during auto initializing of the index (#2498)
- Fix NPE and add additional graceful error handling (#2687)
- Fix No blank backend role before adding a new one in Create User page (#1384)
- Fix "Get started" image is not adaptive to the browser window size. (#1396)
- Integ Test Refactoring (#1383)
- Exclude OpenSearch system index when IT cleanup (#1381)
- Ensure Nested Function Falls Back to Legacy Engine Where Not Supported (#1549)
- Adding reflections as a dependency (#1559)
- Fix null response from pow/power and added missing integration testing (#1459)
- Created untriaged issue workflow. (#809)
- Update XContent imports. (#854)
- Update NodeInfo constructor. (#869)
- Added support for running Integtest on Remote clusters (#733)
- Add CHANGELOG (#342)
- Add stats API for maps and layers in maps plugin (#362)
- Fix cypress test failed due to OSD dashboard create button updated (#393)
- Fix Node.js and Yarn installation in CI (#64)
- Publish snapshots to maven via GHA (#233)
- Update snapshot version and fix compilation issues (#237)
- Add CHANGELOG (#238)
- Add filter type to filtering release configs (#792)
- Add CHANGELOG (#800)
- Bump byte-buddy version from 1.12.22 to 1.14.2 (#804)
- Add 2.6.0 to BWC Version Matrix ((#810))
- Bump numpy version from 1.22.x to 1.24.2 (#811)
- Update BWC Version with OpenSearch Version Bump ((#813))
- Add GitHub action for secure integ tests (#836)
- Bump byte-buddy version to 1.14.3 (#839)
- Set gradle dependency scope for common-utils to testFixturesImplementation (#844)
- Add client setting to ignore warning exceptions (#850)
- Avoid clearing settings after each test (#159)
- Add GHA to publish to maven repository (#237)
- Add reflection dependency (#136)
- Add CHANGELOG (#135)
- Automate releases based on tagging (#1436)
- Fix maven branch (#1449)
- Rename all SSO references to SS4O (#1470)
- [BACKPORT 2.x] Fix 2.x imports (#1483)
- Getting Jackson,JUnit, Log4j dependency version from core (#417)
- Upgrade checkstyle to 9.3 (#395)
- Publish snapshots to maven via GHA (#385)
- Add publish snapshots to maven via GHA (#651)
- Resolving json5 to latest version. (#478)
- Add auto-release workflow (#1339)
- Added 2.7 release notes. (#864)
- Added 2.7 release notes. (#530)
- Added 2.7 release notes. (#407)
- Drafted 2.7 release notes. (#34)
- Added 2.7 release notes. (#755)
- 2.7 release note (#694)
- Add docker-compose file for starting cluster with dedicated ML node (#799)
- Add 2.7.0 release notes (#652)
- Added 2.7 release notes. (#)
- Added 2.7 release notes. (#523)
- Fix the format of the codeowners file (#2469)
- Documentation and other papercuts for datasource api launch (#1534)
- Increment version to 2.7. (#823)
- Bumped version to 2.7. (#505)
- Increment version to 2.7.0-SNAPSHOT. (#371)
- Bump to 2.7.0 version (#356)
- Bump version to 2.7. (#28)
- Bump version 2.7.0 (#75)
- New Maintainers - Sean Li and Louis Chu (#172)
- Added untriaged issue workflow (#164)
- Bumped version to 2.7. (#695)
- Update to Gradle 8.0.2 (#348)
- [AUTO] Increment version to 2.7.0-SNAPSHOT (#636)
- Bump snakeyaml version (#1465)
- Modify namespace from xcontent common to core (#410)
- Baseline repo groups (#52)
- Bumped version to 2.7. (#387)
- Bumped version to 2.7. (#508)
- Update kafka client to 3.4.0 (#2484)
- Update to gradle 8.0.2 (#2520)
- XContent Refactor (#2598)
- Update json-smart to 2.4.10 and update spring-core to 5.3.26 (#2630)
- Update certs for SecuritySSLReloadCertsActionTests (#2679)
- Refactor AWSSigV4 auth to support different AWSCredentialProviders (#1389)
- [AUTO] Increment version to 2.7.0-SNAPSHOT (#1368)
- Update usage of Strings.toString (#1404)
- Deprecated Spring IoC and using Guice instead (#1177) (#1410)
- Bump backport version. (#1009)
- Resolve table function based on StorageEngine provided function resolver (#1424)
- Rework on
OpenSearchDataType: parse, store and use mapping information (#1455) - Update to account for XContent refactor in 2.x (#1485)
- Replace non-ASCII characters in code and docs. (#1486)
- Add publish snapshots to maven via GHA (#1496)
- Refactoring datasource changes to a new module. (#1511)
- #639: Allow metadata fields and score opensearch function (#228) (#1509)
- Revert enabled field in source_to_query_index_mapping. (#812)
- Fixed xContent dependencies due to OSCore changes. (#839)
- Update config index schema if needed at the start of each monitor execution. (#849)
- Fixed xContent dependencies due to OSCore changes. (#392)
- Move zoom and coordinates as separate component (#309)
- Move coordinates to footer (#315)
- Refactor tooltip setup as component (#320)
- Refactor get field options and add field label option on UI (#328)
- Replace Set in org.opensearch.common.collect with java.util references. (#717)
- Fixed xContent dependencies due to OSCore changes. (#721)
- Refractor e2e test to be the same structure as FTRepo (#663)
- Replace Map, List, and Set in org.opensearch.common.collect with java.util references (#816)
- Index template cleanup. (#317)
- Handle monitor or monitor index not found during detector deletion. (#384)
- Handle index not exists for detector search and delete. (#396)
- Show required field mappings only for enabled rules. (#418)
- Fix reduced margin in Get started popup. (#466)
- Empty states for Overview page. (#467)
- Common data store for the rules. (#474)
- Communicate to users when the detector is initializing. (#487)
- Create global state object for async requests. (#493)
- Provide empty states for Findings and Alerts page. (#494)
- Refactor and move field mapping to first the page of create detector feature. (#501)
- Create detector refactor alert triggers per mocks. (#503)
- Update detector details component. (#504)
- Deleting detectors should delete all related dashboards (including index patterns and visualisations). (#515)