Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding in natural language queries extension #468

Merged
merged 59 commits into from
Jun 19, 2023

Conversation

BennuFire
Copy link
Contributor

No description provided.

@BennuFire BennuFire changed the base branch from master to develop June 5, 2023 08:25
@nielsdejong nielsdejong added the 2.3 label Jun 5, 2023
@nielsdejong nielsdejong changed the title Feature/query translator dev Adding in natural language queries extension Jun 8, 2023
@nielsdejong
Copy link
Collaborator

Add filter for OpenAPI key storage in state

src/modal/SaveModal.tsx Outdated Show resolved Hide resolved
src/dashboard/DashboardThunks.ts Show resolved Hide resolved
src/extensions/query-translator/README.md Outdated Show resolved Hide resolved
src/extensions/query-translator/clients/OpenAiClient.ts Outdated Show resolved Hide resolved
src/extensions/query-translator/clients/OpenAiClient.ts Outdated Show resolved Hide resolved
src/extensions/query-translator/clients/const.ts Outdated Show resolved Hide resolved
src/extensions/query-translator/component/LoadingIcon.tsx Outdated Show resolved Hide resolved
…l inside customPrepopulation function to activate after calling the function itsef, to prevent race conditions
@nielsdejong nielsdejong merged commit 37d16e2 into develop Jun 19, 2023
nielsdejong added a commit that referenced this pull request Jul 11, 2023
* Picking up redesign

* Fix some modals

* Some more modals

* Update all modals

* Migrate card footer and graph actions

* Migrate NeoField select to Dropdown

* wip: needle integration

* feat: integrate tailwind with needle preset

* chore: tidy hmr work

* Update classes ; fix Dropdowns

* Improve modals ; misc improvements

* Fix title bar gutter width

* Add tooltips to sidebar ; fix tabs bar position

* Fix color picker

* Bump Needle to fix icon misplacement

* Fix node color scheme dropdown

* Fix Cypress tests

* Migration from mui4 to 5.

* Aria label warnings and other errors

* Bump yaml from 2.2.1 to 2.2.2

Bumps [yaml](https://github.com/eemeli/yaml) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/eemeli/yaml/releases)
- [Commits](eemeli/yaml@v2.2.1...v2.2.2)

---
updated-dependencies:
- dependency-name: yaml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fix selectors not working

* Override themes default breakpoints

* Remove extra divider ; fix typos

* Migrate NeoField text input to Needle

* Fix node property graph dropdown ; Fix text input label placement

* Fix wronge merge parts

* Bumping versions

* Removing all Mui4 references
Bugs and console errors

* Cleaning key Warnings

* Extra key Warnings

* Console fixes

* Add Theme and Color picker migration out of Mui4

* Fix dark bg headers

* Footer margin

* Removing margin on view footer rule

* Last bump of dependencies (up for rollback)

* Console errors on Example modal

* fix test init

* merge lost on 2.2.1

* fix tests

* Param bug

* Fix rule-based styling modal

* Selector warning

* Fixed color picker in Advanced settings

* Fix CardActions position

* Fix drag handles position

* Fix render tooltip dev (#448)

Setting minimum cluster

* Changed inline source maps to source maps

* Architectural review (#450)

* First draft refactor

* revert yarn

* Fix on selectors, reducers and thunks

* Cleaning the house 1

* ReProcessing TreeMap

* ReProcessing Sunburst

* ReProcessing Circle

* No re-execution without query change.
TODO Settings that require re-execute it

* I see dead Flaky tests

* Test 2

* Trying cypress wait after this

* Test should

* Test should 2

* Test should 3

* Test should 4

* Test should 5

* Test should final

* Test should final 2

* Index to Id
Fix page change

* Test password revert

* Console error on i

* Index input 99999

* div on p warning

* Bar chart re render clean

* Line chart re render clean

* TEST

* Pie chart re render clean
Re execute query on chart type change

* Sankey chart re render clean
Test again

* test re

* test query time

* test last

* test legacy

* Here we go again

* Here we go again vendetta

* new action

* sup file

* sup pass

* no im

* Cypress config

* tok

* Cypress dashboard ci

* Flaky solution

* remove test recordings

* Renamed reportId --> id

* Removed TETST comment

* small fixes

* Changed getReportIndex to getReportKey

* Fix comments

---------

Co-authored-by: Niels de Jong <[email protected]>

* Bring back new extensions (Workflows & Sidebar) from fork (#452)

* fix wrong import

* fix missing functio in drilldownMap

* Update package.json

* Removed outdated deploy script

* added actions into new graph

* Reworking map to use WHO geoJson, to move those in the neodash CDN

* removing old geojson

* Moved geojson files to CDN

* Work in progress on node editing

* second draft for polygon map, todo: remove choropleth when polygonmap is ready

* Updated graph

* Second draft of the map, to fix data bindign process and legend styòing

* Changed binding logic to actually remove unbinded data from legend calculation

* Removed Choropleth, updated object/file naming conventions

* Updated names

* Updated names

* fixed clusterMarkers and removed older lines

* Adding TODO

* WIP on graph interactivity

* Better adding of nodes

* Added expansion

* Relationship and node editing interface (not complete)

* bootstrapping alert logic, now we have a button that can trigger the rendering and defined a new field in the state called extensionsConfig to manage the current and future extensions configurations

* Expanding nodes (v0.1)

* working on alert interface

* Even cleaner graph visualization code

* Added logic for query inside drawer, added generalized selectors/reducer/actions for extensions and built skeleton for alerting drilldown flow

* Editing and exploration is done

* changed some file names and added title for extensions (binded within the state)

* Final version of exploration/editing in the graph

* Fixed locking for new code structure

* Usuability fix for label entry component

* Final fixes to new graph interactivity

* embedded graph report inside dialog

* fixed startup db selection for alert settings

* Fixed curvature errors

* Fixed example modal for the graph

* Changed Report in alert inspector to ReportWrapper

* added settings for extensions adnd binded some of them to the UI

* Style fixes for sidebar extension

* Added maxRecords option

* added validation on date picker to fix wrong casting to neo4j date

* fixed strange colouring behaviour in the date picker text field by settingmaxDate to 9999-12-31

* still fixing parameter selection, to continue

* Small fixes for exploration in graphs

* abstracted entity inspection modal from graphChart and reused inside drawer, added some styling for inspection modal

* Added comment

* removed useless import and disabled editing on graphChart by default

* Final style, interface fixes for alerts

* Halfway through review/refactor alerts extension

* Finished review/refactor of alerts extension

* Changed component to enable mapping of regional data and commented code, refactored some code to improve readability

* Added skeleton

* Added workflow editor screen

* wip

* Workflow editor modal

* bootstrapping logic for pluggable extensions and defined workflow logic

* fix wrong state access

* removed useless log

* defined reducer logic for workflows

* defined function to abbreviate numbers in MapUtils.ts, enabled re-render on AreaMap resize and to improve styling of the component

* Added 'Add step' screen

* Added workflow listing UI

* wip

* Added more skeleton components

* WIP

* connecting state to workflow list and modal

* added button to open the workflow list modal to the drawer

* defined action to change name of a workflow by providing current name and new name

* refactored workflow state from map to list and tested on UI with hardcoded indexes

* potpurri

* when disabling the node sidebar, the drawer will be closed. Added logic to plugnew buttons without touching the dashboard.

* changed getExtensionDrawerButtons logic

* Changed workflow icon

* fix on table view for objects

* Renaming some of the interface variables

* Renamings + better error message

* just comments and some easy refactors

* Added correct editing (renaming, step change, delete, add) to UI

* Workflow integration with runner modal

* Added placeholder for workflow running logic

* Fix for determining node click position in firefox

* connected reportQueryRunner to workflows, not suitable, to create a sequential runner

* fixed report query runner behaviour to be really asynchronous (now we can await it's end)

* Started testing SSO in editor mode

* Fixed race-condition that made exploration statistics glitchy in graphs

* Removed old log statements

* now the drawerButton has the responsibility to trigger the workflow

* Added comment

* added stop logic inside workflow runner and added some comments, to continue commenting the component

* Removed alerts extension

* Restore original GH actions

* Restore original GH actions

* Restore original files

* now a failed uery will stop the workflow execution

* Removed branding

* Revert branding

* Remove alerts

* More cleanup

* Cleanup

* Cleanup

* chenged stupid names in workflows and added some comments

* Added SSO login for editor mode

* Cleanup

* fixing strange behavior on field selection in areaMap

* restored choropleth map

* adding missing parts to areaMap

* added example, to write correct query

* Added option in AreaMap Config to hide legend and the reset view button won't be available if the drillDown isn't enabled

* Added docstrings

* moved areamap to extension/advancedcharts

* added example for areaMAp and restored choropleth in docs

* bootstrapping areaMap docs and added config settings to enable three letter country cod es in areaMap

* Revert sample dashboard

* Fix incorrect merge

* Added docs on graph exploration/editing

* added documentation for areaMap, to add image. Changed name fo setting in areaMap to make it more understandable

* added steps to test GDS and changed error definition logic in workflowRunner

* added images for docs and changed example for areaMap

* Fix dead links in docs

* Final touched to area map docs

* from now, when a workflow is running, you can't start the others workflow or modify the current one

* Fixed drilldown styling for area map

* Removed old console statements

* bootstrapping color logic for drawer button

* added status management for the total workflow inside the workflowRunner to add more colors to the UI, binded stop Button to UI

* added db selector for workflows

* fixed strange behaviour at workflow startup and added logic to manage correctly the state

* Added release notes template

* Added example link

* Changes bennu

* adding link to pr

* Final changes

* Finalized release notes

* Added missing docs

* Added images

* Fixed graph exploration in fullscreen view

* Finalized release notes

* added icons for workflowListModal and changed workflow trigger logic

* now the play button won't open the runner modal

* installed show-more-text library to manage text too long

* Fixed version number

* Results in workflow runner view

* bootstrapping drill down button

* added drill down for node sidebar

* added comments and removed unused file in node-sidebar

* added scatteplot first draft and some small refactorings for code quality in line chart

* working on scatterploy

* Added styling for results

* added state Management to scatterplot, keeping two versions to enable discussion

* bump version

* Merge

* Update changelog.md

* to fix legend and add compatibility with datetime for intensity

* working on scatterplot

* Almost done here, to read the todo on top of the component

* Refactoring

* Refactor

* Final refactoring

* Cleanup code

* Refactoring and style fixed for scatter plot

* Added tSNE workflows

* Added fixes for demo dashboard

* fixing canvas tooltip bug and moved data logic for barChart inside useEffect and useState to prevent calculation every re-render

* Removed old log statements

* Removed log statement

* Fixed style on workflow selection screen

* Refactor bar-chart coed

* Specific t-SNE period & persitence to AnalysisTSNE nodes

* removed step library and added query editor for workflows

* Style fixes

* added comments

* reviewing code

* refactoring components for review

* readded index parameter wrongly removed during review

* removed useless todo

* added comments

* fixed casting error that was crashing the whole application in the nodesidecard

* Removed old TODO

* Fixed typo

* Cleaned up TODOs

* Some style fixes and version bumps

* more mini style fixes

* starting refactoring for mergr

* Fixed master deployment script

* Bumped version numbers

* Consolidated extension format for 2.3

* Fixed dynamic extension filter

* Refactoring actions model to use Needle

* Finalized extensions

* Fixed action on map tooltip clikc

* Commented out scatter plot

* Updating SSO settings

* Removed duplicate code

* Fixed deploy script, revert default SSO setting

* Bumped all versions to 2.3

* Bumped all versions to 2.3

* NeoDash 2.3 migrations and version bumps

---------

Co-authored-by: Alfred Rubin <[email protected]>
Co-authored-by: BennuFire <[email protected]>
Co-authored-by: Stuart Green <[email protected]>
Co-authored-by: alfredorubin96 <[email protected]>

* Fix for cypress tests

* Add option to change table column width specification from relative to fixed (#451)

* Relative Vs Fixed Column width enhancement for Table chart

* Fix whitespaces, setting labels

* Cleanup

---------

Co-authored-by: Harold Agudelo <[email protected]>
Co-authored-by: Niels de Jong <[email protected]>

* Added config-based custom styling override (#454)

* Dynamic style

* Style 1

* Empty style

* no logo

* Load of style needle

* Header configs

* Revert dev

* Extra configs

* Clean up Github jobs

* Add URL option to override confirmation screen for shared dashboards (#476)

* Merge

* Merge

* Revert unused changes to code

* Removed unused console log statement

* No Race

---------

Co-authored-by: Brahm Prakash Mishra <[email protected]>
Co-authored-by: Bennu <[email protected]>

* Adding in natural language queries extension (#468)

* starting working on queryTranslator

* added comments and starting work on openAiClient definiton

* defined thunk to connect each report with the model

* working on abstract class

* fixing state inconsistency when reloading and commenting

* more comments and added card query thunk inside translation

* Updating interface

* Styling of extensions

* Style tweak

* Added dual-language support for card settings content

* refactoring some variable names, added reducer for lastMessage

* refactoring some variable names, added reducer for lastMessage

* plugged sessionStorage, it will be flushed after a new connection

* refactoring folder name, added comments

* refactoring folder name, added comments

* moved docs inside the developer guide

* added reportexamples in the user message

* Break out query component

* added migration for new extension architecture from 2.2 to 2.3 and added automatic fetch of cardIds to test openAi features in  queryTranslator

* Revert

* Generalizing

* Props drilling

* Plugged translation on switch

* UX improvements

* Better UX

* added correct error throwing in modelClient

* UX fix

* Minor UX improvement for settings of extension

* Added docs template

* added function in queryTranslation to get back the current validationStep

* Added skeleton for injecting a pre-report-populating translation function

* Add skeleton for pre-report population logic

* Bumped OpenAI lib version

* Fixes

* Fixed glitchy caching in english editor

* Fix bugs

* moved model client inside SessionStorage, fixed error in query-translator utils and fixed error while updating client settings from querytranslator.tsx

* added doc about natural language queries

* Added custom running icon

* Pushing new logic for session storage

* Pushing non-broken test code

* updated the doc

* added logic to report to work with prepopulation functions

* plugged everything

* Minor style tweaks

* Added pics and also how it works section

* Status handling for GPT config screen

* removed useless readme file in query-translator. Changed function call inside customPrepopulation function to activate after calling the function itsef, to prevent race conditions

* changing some bits of code based on github conversations and removed apiKey from dashboard state when saving

---------

Co-authored-by: Alfred Rubin <[email protected]>
Co-authored-by: Niels de Jong <[email protected]>
Co-authored-by: aleksandarneo4j <[email protected]>

* Final style tweaks for Neo4j Design Language / Align docs for 2.3 (#456)

* Added multiselect clearable tweak

* Fix on/off boolean setting

* Fix GraphChart display

* Bump Needle ; various style improvements

* Reset z-index scale using Needle Design Tokens

* Migrate anchor links to TextLink

* Make Tooltip not hoverable

* Pixel-perfect style tweaks + align border softness with Needle

* Fix drag handle area

* Replaced some inline style with tailwind classes

* Update doc screenshots

* Fix CardAdd button background

* Make multiline Field use Textarea

* Draft release notes

* Final tweaks

* Updated release notse

* Fix release notes

---------

Co-authored-by: Harold Agudelo <[email protected]>
Co-authored-by: Marius Conjeaud <[email protected]>

* Update

* Downsize images

* Update documentation and info messages

* Final changes after the 2.3.0 Pre-release (#482)

* added file for workflows adoc, waiting for component refactoring to support Needle

* Style docs

* Added warning icon to sidebar when no provider is selected

* UX improvements for settings and error message produced by GPT extension

* Trying to make some visual difference between the cypher/english editor....

* Style tweak

* Tweaking style

* fixed error that wasn't creating correctly the userMessage in OpenAiClient and removed some useless comments and added APOC as requirement

* fixed wrong error throw in NeoGraphChartFitViewButton

* Added comments and added TODO in NeoOverrideCardQueryEditor

* loose equality for parameter loading from saved dashboard

* when on error - reopen the settings to avoid an infinitely looping loading screen

* added todo

* Fixed bug where driver is recreated too many times

* removed a console log

---------

Co-authored-by: Alfred Rubin <[email protected]>
Co-authored-by: Bennu <[email protected]>
Co-authored-by: Niels de Jong <[email protected]>
Co-authored-by: aleksandarneo4j <[email protected]>

* Add sentry.io

* Final fixes for 2.3.0 (#499)

* added file for workflows adoc, waiting for component refactoring to support Needle

* Style docs

* Added warning icon to sidebar when no provider is selected

* UX improvements for settings and error message produced by GPT extension

* Trying to make some visual difference between the cypher/english editor....

* Style tweak

* Tweaking style

* fixed error that wasn't creating correctly the userMessage in OpenAiClient and removed some useless comments and added APOC as requirement

* fixed wrong error throw in NeoGraphChartFitViewButton

* Added comments and added TODO in NeoOverrideCardQueryEditor

* loose equality for parameter loading from saved dashboard

* when on error - reopen the settings to avoid an infinitely looping loading screen

* added todo

* Fixed bug where driver is recreated too many times

* removed a console log

* added function to create from apoc metadata relationship pattterns, parked in side the const file for now

---------

Co-authored-by: Alfred Rubin <[email protected]>
Co-authored-by: Bennu <[email protected]>
Co-authored-by: aleksandarneo4j <[email protected]>

* Bump tough-cookie from 4.1.2 to 4.1.3 in /gallery (#503)

Bumps [tough-cookie](https://github.com/salesforce/tough-cookie) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/salesforce/tough-cookie/releases)
- [Changelog](https://github.com/salesforce/tough-cookie/blob/master/CHANGELOG.md)
- [Commits](salesforce/tough-cookie@v4.1.2...v4.1.3)

---
updated-dependencies:
- dependency-name: tough-cookie
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Feature/add codecov (#502)

* Try code coverage report

* Fix develop test pipeline

* New try code coverage

* Exclude cypress folder from test coverage

* Test adding click on Advanced settings to see coverage

* Click on Advanced settings for all report types

---------

Co-authored-by: Marius Conjeaud <[email protected]>
Co-authored-by: Niels de Jong <[email protected]>

* Fix minor issues before 2.3.0 release (#504)

* Added top margin for new param select

* Final fixes for 2.3.0 release

* Fix for luma threshold handling of non-hex colors

* Added fix for when seperating overlapping markers is not required

* Fixed bug where page names with dots in them were not rendered as suggestions in the action modal

* Finalized release notes

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Marius Conjeaud <[email protected]>
Co-authored-by: konsalex <[email protected]>
Co-authored-by: Bennu <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: MariusC <[email protected]>
Co-authored-by: Alfred Rubin <[email protected]>
Co-authored-by: Stuart Green <[email protected]>
Co-authored-by: alfredorubin96 <[email protected]>
Co-authored-by: BlackRaven <[email protected]>
Co-authored-by: Brahm Prakash Mishra <[email protected]>
Co-authored-by: aleksandarneo4j <[email protected]>
Co-authored-by: KiZach <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants