Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using re2 for Timelion regular expressions #55208

Merged
merged 20 commits into from
Mar 12, 2020
Merged

Conversation

kobelb
Copy link
Contributor

@kobelb kobelb commented Jan 17, 2020

No description provided.

@kobelb kobelb requested a review from a team as a code owner January 17, 2020 17:55
@kobelb kobelb requested review from a team and timroes January 17, 2020 17:56
@kobelb kobelb changed the title Using re2 for timelion regular expressions Using re2 for Timelion regular expressions Jan 17, 2020
@kobelb kobelb added v8.0.0 v7.6.0 release_note:skip Skip the PR/issue when compiling release notes labels Jan 17, 2020
@kobelb
Copy link
Contributor Author

kobelb commented Jan 17, 2020

@elasticmachine merge upstream

@kobelb
Copy link
Contributor Author

kobelb commented Jan 21, 2020

@elasticmachine merge upstream

@kobelb
Copy link
Contributor Author

kobelb commented Jan 21, 2020

This might be the reason why using native modules becomes a non-starter... Compiling portable binaries sucks...

 │ proc [kibana]  FATAL  Error: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /dev/shm/workspace/kibana/build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64/node_modules/re2/build/Release/re2.node)

@jbudz
Copy link
Member

jbudz commented Jan 27, 2020

@kobelb 👀 , ready for review?

@kobelb
Copy link
Contributor Author

kobelb commented Jan 27, 2020

@jbudz not yet... working on getting the actually portable binaries ready, and automating their creation using Travis

@kobelb kobelb added v7.6.1 and removed v7.6.0 labels Feb 4, 2020
@kobelb kobelb requested a review from a team February 7, 2020 23:09
@kobelb
Copy link
Contributor Author

kobelb commented Feb 10, 2020

@elasticmachine merge upstream

Copy link
Contributor

@flash1293 flash1293 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides the failing type check, regexes are still applied correctly.

@sulemanof Could you check whether this works on Windows due to the native extension?

This theoretically breaks existing timelion visualizations because backtracking doesn't work anymore, right? I don't think it's a big deal because it's just an edge case in a small feature, but it's worth a release note IMHO

@kobelb kobelb merged commit 728d073 into elastic:master Mar 12, 2020
@kobelb kobelb deleted the node-re2 branch March 12, 2020 18:38
@kobelb kobelb added the v7.7.0 label Mar 12, 2020
kobelb added a commit to kobelb/kibana that referenced this pull request Mar 12, 2020
* Using re2 for Timelion's regexs

* Patching native modules

* Restructuring to be more generic

* Fixing download_node_builds_task tests

* Updating linux sha after properly gzipping the archive

* Using a Centos7 machine with devtoolset-6. That's what node does

* Using new archives which Travis built for us

* Not using a standard import to prevent Kibana from not starting up

If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse

* Isolating the require even further

* Detecting the package version mismatches, thanks Larry!

Co-authored-by: Elastic Machine <[email protected]>
@kobelb kobelb added the v6.8.8 label Mar 12, 2020
kobelb added a commit to kobelb/kibana that referenced this pull request Mar 12, 2020
* Using re2 for Timelion's regexs

* Patching native modules

* Restructuring to be more generic

* Fixing download_node_builds_task tests

* Updating linux sha after properly gzipping the archive

* Using a Centos7 machine with devtoolset-6. That's what node does

* Using new archives which Travis built for us

* Not using a standard import to prevent Kibana from not starting up

If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse

* Isolating the require even further

* Detecting the package version mismatches, thanks Larry!

Co-authored-by: Elastic Machine <[email protected]>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Mar 12, 2020
* master:
  Endpoint: Change the input type for @kbn/config-schema to work with more schemas (elastic#60007)
  Using re2 for Timelion regular expressions (elastic#55208)
  [Monitoring] Re-enable logstash tests (elastic#59815)
  fix karma debug typo (elastic#60029)
  Adds telemetry support to alerting and actions plugins (elastic#58081)
kobelb added a commit that referenced this pull request Mar 12, 2020
* Using re2 for Timelion's regexs

* Patching native modules

* Restructuring to be more generic

* Fixing download_node_builds_task tests

* Updating linux sha after properly gzipping the archive

* Using a Centos7 machine with devtoolset-6. That's what node does

* Using new archives which Travis built for us

* Not using a standard import to prevent Kibana from not starting up

If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse

* Isolating the require even further

* Detecting the package version mismatches, thanks Larry!

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit that referenced this pull request Mar 12, 2020
* Using re2 for Timelion's regexs

* Patching native modules

* Restructuring to be more generic

* Fixing download_node_builds_task tests

* Updating linux sha after properly gzipping the archive

* Using a Centos7 machine with devtoolset-6. That's what node does

* Using new archives which Travis built for us

* Not using a standard import to prevent Kibana from not starting up

If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse

* Isolating the require even further

* Detecting the package version mismatches, thanks Larry!

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit that referenced this pull request Mar 12, 2020
* Using re2 for Timelion's regexs

* Patching native modules

* Restructuring to be more generic

* Fixing download_node_builds_task tests

* Updating linux sha after properly gzipping the archive

* Using a Centos7 machine with devtoolset-6. That's what node does

* Using new archives which Travis built for us

* Not using a standard import to prevent Kibana from not starting up

If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse

* Isolating the require even further

* Detecting the package version mismatches, thanks Larry!

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
spalger added a commit that referenced this pull request Mar 13, 2020
spalger added a commit that referenced this pull request Mar 13, 2020
spalger added a commit that referenced this pull request Mar 13, 2020
spalger added a commit that referenced this pull request Mar 13, 2020
@spalger
Copy link
Contributor

spalger commented Mar 13, 2020

Reverted as this causes compilation errors locally in certain python environments, which aren't documented or validated and don't have any suggested way methods for being fixed. It seems like we should be using the prebuilt versions of re2 unless otherwise instructed from an environment variable or something.

Macos Version: 10.15.3
which python: /usr/bin/python
python --version: Python 2.7.16

The errors I'm getting locally when I try to bootstrap:

Installing dependencies in [kibana]:

$ node ./preinstall_check
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
warning Resolution field "@types/[email protected]" is incompatible with requested version "@types/[email protected]"
warning Resolution field "@types/[email protected]" is incompatible with requested version "@types/node@^12.0.2"
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
warning "@kbn/analytics > @kbn/babel-preset > [email protected]" has unmet peer dependency "styled-components@>= 2".
warning "workspace-aggregator-11a6ccfb-7322-4002-ba55-0490022a825a > x-pack > @elastic/[email protected]" has incorrect peer dependency "react-router-dom@^4.0.0".
warning "workspace-aggregator-11a6ccfb-7322-4002-ba55-0490022a825a > x-pack > @mapbox/[email protected]" has incorrect peer dependency "mapbox-gl@>=0.27.0 <=0.51.0".
warning "workspace-aggregator-11a6ccfb-7322-4002-ba55-0490022a825a > x-pack > [email protected]" has incorrect peer dependency "react@^0.14.8 || ^15".
warning "workspace-aggregator-11a6ccfb-7322-4002-ba55-0490022a825a > x-pack > [email protected]" has incorrect peer dependency "react-dom@^0.14.8 || ^15".
warning "workspace-aggregator-11a6ccfb-7322-4002-ba55-0490022a825a > x-pack > [email protected]" has unmet peer dependency "react-is@>= 16.8.0".
[##-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 56/789[5/5] 🔨  Building fresh packages...
[8/61] ⠐ re2
[-/61] ⠐ waiting...
[-/61] ⠐ waiting...
[-/61] ⠐ waiting...
error /Users/spalger/kbn-dev/master/kibana/node_modules/re2: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /Users/spalger/kbn-dev/master/kibana/node_modules/re2
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | x64
gyp http GET https://nodejs.org/download/release/v10.19.0/node-v10.19.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v10.19.0/node-v10.19.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v10.19.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v10.19.0/SHASUMS256.txt
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/Users/spalger/kbn-dev/master/kibana/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/spalger/kbn-dev/master/kibana/node_modules/re2/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/spalger/kbn-dev/master/kibana/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/spalger/.node-gyp/10.19.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/spalger/.node-gyp/10.19.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/spalger/kbn-dev/master/kibana/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/spalger/.node-gyp/10.19.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/spalger/kbn-dev/master/kibana/node_modules/re2',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

gyp: No Xcode or CLT version detected!
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/spalger/kbn-dev/master/kibana/node_modules/node-gyp/lib/configure.js:345:16)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Darwin 19.3.0
gyp ERR! command "/Users/spalger/.nvm/versions/node/v10.19.0/bin/node" "/Users/spalger/kbn-dev/master/kibana/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /Users/spalger/kbn-dev/master/kibana/node_modules/re2





[bootstrap] failed:

Error: Command failed with exit code 1: yarn install --non-interactive
    at makeError (/Users/spalger/kbn-dev/master/kibana/packages/kbn-pm/dist/index.js:33439:11)
    at handlePromise (/Users/spalger/kbn-dev/master/kibana/packages/kbn-pm/dist/index.js:32377:26)
    at process._tickCallback (internal/process/next_tick.js:68:7)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

kobelb added a commit to kobelb/kibana that referenced this pull request May 8, 2020
kobelb added a commit to kobelb/kibana that referenced this pull request May 26, 2020
kobelb added a commit that referenced this pull request Jun 10, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit to kobelb/kibana that referenced this pull request Jun 10, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (elastic#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit to kobelb/kibana that referenced this pull request Jun 11, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (elastic#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit that referenced this pull request Jun 11, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit that referenced this pull request Jun 18, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
kobelb added a commit that referenced this pull request Jun 18, 2020
* Revert "Revert "Using re2 for Timelion regular expressions (#55208)""

This reverts commit c90293d.

* Updating re2 to 1.14.0. Still need to update build patching

* Extract the gzip to the destination, supporting multiple extract methods

* Adding 'node' to jest's moduleFileExtensions

'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring

Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes reverted v7.7.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants