Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete rewite of the browser part (js, css) of the UI #165

Merged
merged 32 commits into from
Nov 10, 2023

Conversation

pgiraud
Copy link
Member

@pgiraud pgiraud commented Nov 30, 2022

With this PR the previous frameworks:

  • backbone,
  • foundation,
  • rickshaw,
  • bower.

Are replaced by:

  • VueJS,
  • Vuetify,
  • d3,
  • vitejs.

To test this, you have two options (yet to be documented):

  • In production mode : execute npm run build. Built assets will be generated and put in powa/static/dist [1]. Then run powa as you would normally do.
  • In debug mode : execute npm run dev in a terminal. This should launch a VITE service on http://localhost:5173/. Then run powa in dev mode in an other terminal.

I'm looking forward for your feedback.

[1] the built assets are meant to be committed along with releases.

@pgiraud
Copy link
Member Author

pgiraud commented Dec 1, 2022

I forgot to mention that we tried to keep it iso-functional. This is the first step for possible future changes to the design and layout a bit to improve the usability.

For testing purpose, in order to make sure that the information displayed in the interface is correct, it's possible to run 2 versions of powa-web at the same time. Running with the same repository data.

In https://github.com/powa-team/powa-podman/blob/master/dev/powa-dev-demo.yml one can add the following section:

powa-web-dev-master: 
  image: powateam/powa-web-dev
  container_name: powa-dev-web-master
  command: "python3 powa-web-dev/run_powa.py"
  ports:
    - 8889:8888
  volumes: 
    - '${POWA_WEB_MASTER_GIT}:/usr/local/src/powa-web-dev'
  links:
    - powa-repository
    - remote-primary
    - remote-standby

Clone powa-web project in a new folder (ie. powa-web-master). The idea is to have one powa-web folder positioned on the current PR branch and one powa-web-master positioned on upstream master branch.

Then run containers with the following command:

POWA_WEB_MASTER_GIT=../../powa-web-master POWA_WEB_GIT=../../powa-web POWA_COLLECTOR_GIT=../../powa-collector podman-compose -f powa-dev-demo.yml up

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch 2 times, most recently from 6965a73 to d6f479b Compare December 2, 2022 19:58
@rjuju
Copy link
Member

rjuju commented Dec 7, 2022

First, thanks again for working on that subjet.

I don't really understand the PR though, I waited a bit to make sure that no more commits would be added and it looks like way more than simply the rewrite to get rid of some js stack and use a new one instead. I think that keeping the scope of this first change as small as possible would be easier to review.

About the change itself, I tried to look at the new components but honetly I don't even understand what those are supposed to be used for based on their home pages.

Naively, I see that you get rid of foundation (which I think is a good thing), but I was thinking that relying on bootstrap instead would be the logical replacement. As far as I can see it's not the case, and I'm a bit worried about that. Or is bootstrap part / dependency of some of the new components?

In debug mode : execute npm run dev in a terminal. This should launch a VITE service on http://localhost:5173/. Then run powa in dev mode in an other terminal.

this is quite scary. There's no way to have the python server simply pass the js files like it was the case before?

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch from 7563a2d to ab982eb Compare December 8, 2022 08:17
@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch 2 times, most recently from cf67f20 to 624c48c Compare December 20, 2022 08:54
@pgiraud
Copy link
Member Author

pgiraud commented Dec 20, 2022

First, thanks again for working on that subjet.

I don't really understand the PR though, I waited a bit to make sure that no more commits would be added and it looks like way more than simply the rewrite to get rid of some js stack and use a new one instead. I think that keeping the scope of this first change as small as possible would be easier to review.

I wanted to propose this PR even though that there were some additional small fixes to be made. I could have waited a bit more but I wanted the team to have a look at the new look and feel.

To be honest, I don't think it would have been possible to do get rid of only some js stack, ie. one at a time.
The initial goal was to get rid of backbone and foundation.

The choice made :

  • VueJS : for the core web user interface framework,
  • Vuetify : for the layout and components (Grids, Cards, etc.).

Then as a consequence, the choice of ViteJS was straightforward. It replaces bower and is reponsible for the building and dependency management.

Rickshaw which was used to display the graphs could have been kept at first even though compatibility hacks would have probably been required. I decided to get rid of it as well and replaced it with d3.

In summary, it didn't seem possible to split this huge change into smaller steps. If this is a blocker I can of course consider splitting into smaller commits.

About the change itself, I tried to look at the new components but honetly I don't even understand what those are supposed to be used for based on their home pages.

The only thing to know is that with VueJS it's now possible to have the component code all in one place (aka. Single File Components). In my opinion, it makes things a lot easier to understand.

Naively, I see that you get rid of foundation (which I think is a good thing), but I was thinking that relying on bootstrap instead would be the logical replacement. As far as I can see it's not the case, and I'm a bit worried about that. Or is bootstrap part / dependency of some of the new components?

In this rewrite, we don't use bootstrap at all. Instead we're using vuetify because it is more popular and has a more complete and up to date set of components available.

In debug mode : execute npm run dev in a terminal. This should launch a VITE service on http://localhost:5173/. Then run powa in dev mode in an other terminal.

this is quite scary. There's no way to have the python server simply pass the js files like it was the case before?

No, I don't think it's possible.
Please note that in the previous version, the libraries' code had to be included in the application code. Now they are installed in node modules via npm. It makes future upgrade easier.

@rjuju
Copy link
Member

rjuju commented Apr 18, 2023

Thanks for the commits that show how upgrading components is done! Could you also add some documentation (maybe a README.dev or similar) that points out the basics, like

  • how to setup a dev environment from scratch for the powa-web specific part (ie. ignoring the postgres / extension part)
  • how to check for lib updates and how to update them
  • how to run powa-web locally for dev purpose, how to teardown anything that was spawned and how to check that everything that was spawned is stopped (I'm mostly thinking about the extra vite service thing, maybe others)

And also document what needs to be done to actually host powa-web in a production style way (this one should eventually be documented in https://github.com/powa-team/powa/blob/master/docs/components/powa-web/deployment.rst but having it locally would still help).

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch 3 times, most recently from 417d593 to 22878cb Compare April 20, 2023 08:05
@pgiraud
Copy link
Member Author

pgiraud commented Apr 20, 2023

Thanks for the commits that show how upgrading components is done! Could you also add some documentation (maybe a README.dev or similar) that points out the basics, like

  • how to setup a dev environment from scratch for the powa-web specific part (ie. ignoring the postgres / extension part)
  • how to check for lib updates and how to update them
  • how to run powa-web locally for dev purpose, how to teardown anything that was spawned and how to check that everything that was spawned is stopped (I'm mostly thinking about the extra vite service thing, maybe others)

And also document what needs to be done to actually host powa-web in a production style way (this one should eventually be documented in https://github.com/powa-team/powa/blob/master/docs/components/powa-web/deployment.rst but having it locally would still help).

I added a CONTRIBUTING.md file. Please let me know if it is clear enough and works for you.

@rjuju
Copy link
Member

rjuju commented Apr 21, 2023

I added a CONTRIBUTING.md file. Please let me know if it is clear enough and works for you.

Thanks a lot, it looks great!

After a quick look at it, I'm quite happy with the requirements and the general workflow, as nodes 16 doesn't look like a crazy recent version, and the various scenario needed for the common usages are straightforward (and way better than what they currently are). I will try to have a more thorough look at the PR shortly but it overall looks great! Again, I want to emphasize how grateful I'm for this work, congrats to all people involved.

About merging that PR, and then all the other improvements you already made on the UI (which are also really welcome), I was thinking integrating that with the upcoming v5 version of powa that I'm already currently working on, which should hopefully be released somewhere around this summer. Would that work for you? As far as I can see there are only a few minor conflicts, which I can take care of if you prefer. If that's ok for you I can build and maintain specific "v5" versions of various podman images so you can easily work on that branch. The powa-archivist part is probably going to change quite often, but trashing it and recreating it regularly shouldn't be too much of a problem, especially since most of the changes that would impact the UI are already done.

The reason for that is that this rewrite is a huge improvement. It will probably change quite a lot the general UI/UX so it would be weird to drop that in a minor version release. Also, I'm imagining that you may need a bit of time to adjust some details and/or add other changes, and since we only maintain a single major version it will give additional time for that while still having the possibility to release fixes for the current v4.

@pgiraud
Copy link
Member Author

pgiraud commented Apr 21, 2023

I added a CONTRIBUTING.md file. Please let me know if it is clear enough and works for you.

Thanks a lot, it looks great!

After a quick look at it, I'm quite happy with the requirements and the general workflow, as nodes 16 doesn't look like a crazy recent version, and the various scenario needed for the common usages are straightforward (and way better than what they currently are). I will try to have a more thorough look at the PR shortly but it overall looks great! Again, I want to emphasize how grateful I'm for this work, congrats to all people involved.

Thanks.

About merging that PR, and then all the other improvements you already made on the UI (which are also really welcome), I was thinking integrating that with the upcoming v5 version of powa that I'm already currently working on, which should hopefully be released somewhere around this summer. Would that work for you? As far as I can see there are only a few minor conflicts, which I can take care of if you prefer. If that's ok for you I can build and maintain specific "v5" versions of various podman images so you can easily work on that branch. The powa-archivist part is probably going to change quite often, but trashing it and recreating it regularly shouldn't be too much of a problem, especially since most of the changes that would impact the UI are already done.

Should I try rebasing the current branch onto your v5 branch in anticipation?
Having v5 versions of podman images would make things a lot easier. Thanks.

The reason for that is that this rewrite is a huge improvement. It will probably change quite a lot the general UI/UX so it would be weird to drop that in a minor version release. Also, I'm imagining that you may need a bit of time to adjust some details and/or add other changes, and since we only maintain a single major version it will give additional time for that while still having the possibility to release fixes for the current v4.

The earlier we can work on the UI rewrite on top of v5, the better.

@rjuju
Copy link
Member

rjuju commented Apr 21, 2023

Should I try rebasing the current branch onto your v5 branch in anticipation?

Assuming you're ok with the plan I mentioned sure!

Having v5 versions of podman images would make things a lot easier. Thanks.

I just pushed a new powateam/powa-archivist-git:v5 image. I think that's the only specific image you need to work on the v5, as you need a local copy of powa-web (on this branch rebased over v5) and powa-collector (on v5 branch). Let me know if it's working as intended.

The earlier we can work on the UI rewrite on top of v5, the better.

Agreed.

@pgiraud
Copy link
Member Author

pgiraud commented Apr 24, 2023

I tried running powa in dev mode with podman, but I got the following errors (found in container logs) in the 3 container running "powa-archivist:v5" image:

exec container process `/usr/local/bin/docker-entrypoint.sh`: Exec format error

FYI, I'm running the following command (from powa-podman/dev directory):

POWA_WEB_GIT=../../powa-web POWA_COLLECTOR_GIT=../../powa-collector podman-compose -f powa-dev-demo.yml up

@rjuju
Copy link
Member

rjuju commented Apr 24, 2023

Ah, sorry I built the image on an arm machine and that's incompatible with an amd64 machine. I just pushed a new version, can you check if that solves the problem?

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch from 22878cb to 4d1a423 Compare April 24, 2023 09:19
@pgiraud
Copy link
Member Author

pgiraud commented Apr 24, 2023

Thanks.

The current branch is now rebase on v5. It seems to work well.

@rjuju
Copy link
Member

rjuju commented Apr 24, 2023

Great news! I will try to review the PR and test it thoroughly. We apparently hit different code path, and my local test setup heavily relies on the newest stuff changed in v5 so hopefully we should get a somewhat full manual testing of the UI between you and me.

@rjuju
Copy link
Member

rjuju commented Apr 25, 2023

I think you used a wrong branch or something during the rebase. The CONTRIBUTING.md isn't there anymore, and the extra commits to bump the dependencies are also missing.

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch from 4d1a423 to c588ca0 Compare April 25, 2023 07:10
@pgiraud
Copy link
Member Author

pgiraud commented Apr 25, 2023

Indeed. Fixed.

@rjuju
Copy link
Member

rjuju commented Apr 25, 2023

Thanks. First initial feedback:

  • first of all, it seems to mostly work quite great :) It's also globally way better looking (the old interface is honestly quite ugly), although this new versions feels a bit like austerity. Maybe that's because of the graphs (see below), or me just being too used to the old version. Not a blocker for now, especially since it's quite better already.

  • the interface if the vite service isn't running, or not reachable yet (or I'm assuming if js is deactivated in the browser or the httpd configuration is broken or...) is quite bad. There's absolutely no hint of what's going on and/or clue of what you should be doing. In my humble opinion a web page shouldn't be totally broken and unreadable if js is deactivated, even if js is necessary to properly view this page. There should be some better looking default page, with a clear message explaining the problem, that is removed when the javascript is loaded. Note also that each page change shows that default page and then switch to a different layout a few hundres of ms later. It makes the browsing experience quite unpleasant.

  • the .husky/pre-commit seems broken. I wanted to fix something unrelated and got:


> [email protected] lint
> eslint --ext .js,.vue --fix powa/static/


Oops! Something went wrong! :(

ESLint: 8.25.0

ESLint couldn't find the config "../.eslintrc.json" to extend from. Please check that the name of the config is correct.

The config "../.eslintrc.json" was referenced from the config file in "/home/rjuju/git/powa/powa-web/powa/static/bower_components/bootstrap/build/.eslintrc.json".

If you still have problems, please stop by https://eslint.org/chat/help to chat with the team.

husky - pre-commit hook exited with code 2 (error)

Maybe you forgot to git-add some file in the tree?

I locally removed the pre-commit to be able to commit some fix, but I'm a bit worried about an undocumented pre-commit hook, especially since it can fail on something different that what it seems to be checking.

  • the graphs used to have some background grid to help reading them. Could they be added back? A totally blank graph looks weird to me, and I find it harder to follow the time without it

  • the config changes red triangles looks a bit scary. Maybe use a different color at least? Note also that they stick to the lower axs which makes the graphs harder to read if you have a lot of them. I kept restarting my db server to fix some unrelated bug, and I have a bunch of those, which can also hide the very small time ticks on the axis

  • the new time selector is nice, but I feel like there are too many configurations. The first one(s) seems a bit unnecessary as the default frequency is 5 min (and it's somewhat recommended to not go have too frequent snapshots). Maybe removing the 5 and 15 min choices would be better. Also, AFAICS the date picker only allows you to pick a date. Isn't there a component available where you could select the time too?

  • the links in the grids are not "real link" anymore but only a js action. I rely a lot on ctrl-click / open in a new tab when using powa to compare stuff easily, enqueue pages to look at or simply share a link. Could it be made a normal link again? As mentioned above, in general I think that normal html should be used when possible instead of relying on js

  • the "Most executed values" graphs (in /server/X/database/bench/query/Y/qual/Z) seems to be broken

  • there are a lot of error in the js console. I get that the 501 errors are for data sources that are not available, I'm assuming that the "Uncaught (in promise) Error: 501 Not Implemented" are a side effect of that, but I see other errors. For instance on the query page:

VM11629:1 Uncaught (in promise) SyntaxError: Unexpected token '<', "<div>

<!-"... is not valid JSON
    at JSON.parse (<anonymous>)
    at store.js:34:27
(anonymous) @ store.js:34
Promise.then (async)
(anonymous) @ store.js:33
(anonymous) @ lodash.js:4967
baseForOwn @ lodash.js:3032
(anonymous) @ lodash.js:4936
forEach @ lodash.js:9410
loadData @ store.js:31
(anonymous) @ main.js:107
VM11630:1 Uncaught (in promise) SyntaxError: Unexpected token '<', "<div>



<"... is not valid JSON
    at JSON.parse (<anonymous>)
    at store.js:34:27
(anonymous) @ store.js:34
Promise.then (async)
(anonymous) @ store.js:33
(anonymous) @ lodash.js:4967
baseForOwn @ lodash.js:3032
(anonymous) @ lodash.js:4936
forEach @ lodash.js:9410
loadData @ store.js:31
(anonymous) @ main.js:107
text.js:7          GET http://127.0.0.1:8888/server/0/metrics/database/powa/query/5057148677207632030/wait_events/?from=2023-04-25%2016%3A11%3A45%2B0800&to=2023-04-25%2017%3A11%3A45%2B0800 501 (Not Implemented)
text_default3 @ text.js:7
(anonymous) @ store.js:32
(anonymous) @ lodash.js:4967
baseForOwn @ lodash.js:3032
(anonymous) @ lodash.js:4936
forEach @ lodash.js:9410
loadData @ store.js:31
(anonymous) @ main.js:107
text.js:2 Uncaught (in promise) Error: 501 Not Implemented
    at responseText (text.js:2:27)
responseText @ text.js:2
Promise.then (async)
(anonymous) @ store.js:33
(anonymous) @ lodash.js:4967
baseForOwn @ lodash.js:3032
(anonymous) @ lodash.js:4936
forEach @ lodash.js:9410
loadData @ store.js:31
(anonymous) @ main.js:107
3content.js:7 Uncaught TypeError: Cannot read properties of undefined (reading '0')
    at Array.<anonymous> (content.js:7:88534)
    at c.trigger (content.js:7:81786)
    at HTMLDocument.<anonymous> (content.js:7:81496)
(anonymous) @ content.js:7
c.trigger @ content.js:7
(anonymous) @ content.js:7

if (row.url) {
window.location.href = [row.url, serialize(store.from, store.to)].join("?");
const destination = [row.url, serialize(store.from, store.to)].join("?");
if (event.ctrlKey) {
Copy link
Member

@rjuju rjuju Apr 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it possible to just put an <a> there? A link is a basic html feature and I don't think there should be multiple lines of code to try to simulate something as basic as that. What about people who want to use alt click to open in a new window or use mouse middle click, or use something like vimperator and need a real <a> to do a batch action on those and so on?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, doesn't this prevent users from only selecting some text to copy?

@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch 2 times, most recently from 89d3ef3 to 691eba7 Compare May 2, 2023 14:01
@rjuju
Copy link
Member

rjuju commented May 7, 2023

Thanks for the work on the changes marker. FTR one feature missing in the rewrite is when you click on the marker. In the older version, it makes the hover tooltip stay active and add a vertical line above (now should be below) the marker so you can easily eyeball the graphs before/after the change. Is that something that can be done with the rewrite?

Note that one annoying thing in the current version is that the same marker for different graphs are independent. Ideally, we'd want to have to hover sticky thing and the vertical line be synced across all graphs.

pgiraud and others added 22 commits November 10, 2023 09:47
Everything now goes in .vue files.

We don't need to manually declare the Vuetify components anymore.

We now expose handler properties as JSON in template.

We can't use VuetifyResolver for treeshaking because of the
external component V-Snackbars. But we actually have our own resolver
which does the same except that it excludes VSnackbars.
Display an error message only when main.js script fails loading.
The xhrm.html and messages.html templates and corresponding ui_method
are not required anymore.
We always use <code> blocks to show code (SQL or explain plans). We
don't use .hljs class since it adds extra styling we don't really want.

Also we reset <code> font size (set by vuetify) since we don't want it
to be smaller.

In grid, we also reset some styling to make sure ellipsis work and
background is not grey.
Otherwise it's loaded for every single component. We actually only want
to override variables using sass preprocessor additionalData option.
We get rid to v-snackbars.

Size is only handled with some flex positioning.
Transition are preserved by making sure shown is set after creating the snackbar element.
dark_theme is stored in the localStorage to keep styling during
the navigation.
We have removed v-main classes in layout.html to manage the dark
and light background on the v-application element.
We copy the stackoverflow highlight dark and light css into a new
highlight.scss file to dynamically manage highlight.css.
We will be able to dynamically import the source file with
SASS 1.64, but this will require Vuetify 3.
Not by using script tags in the HTML body. This will help loading new
data without loading a new page.
By using vue-router (in HTML5 mode) we can reload the page content
without reloading the page itself.

The urls are still compatible with the old mechanism, which means that
an url copied from the url toolbar will still work and display the same
content.

Disabled attribute for breadcrumb items is not required anymore since
vuetify in combination with vue-router disables the item for the current
route.
Update of vuetify help setting the text-transform with a scss variable.

With this change only graphs legend and SQL <code> get a bit smaller.
We made it look more like the old one.
This is required since we don't reload the page but use a single page.
For example, the currently selected database may have changed and the
action buttons may have changed too.
@pgiraud pgiraud force-pushed the switch_backbone_vue_vite branch from ed21a29 to 2349ebe Compare November 10, 2023 08:48
@pgiraud
Copy link
Member Author

pgiraud commented Nov 10, 2023

I just force pushed a fix for the popover.

@rjuju
Copy link
Member

rjuju commented Nov 10, 2023

Great!

I'm pretty happy with the state of the PR, I think we're ready to merge it when you want!

@rjuju rjuju merged commit 15f8d6f into powa-team:v5 Nov 10, 2023
@pgiraud pgiraud deleted the switch_backbone_vue_vite branch November 10, 2023 08:56
@rjuju
Copy link
Member

rjuju commented Nov 10, 2023

And it's now merged! Thanks a lot for this awesome contribution @pgiraud, @MarionGiusti and @pirlgon (and my apologies if I missed anyone).

@rjuju
Copy link
Member

rjuju commented Nov 16, 2023

@pgiraud some minor complain, I noticed that if multiple configuration changes are detected in the same snapshot, the timeline will show 2 events with the same timestamp (making it hard to spot and see) rather than merging them in the same hover tooltip. Is that something you can fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants