Replace react-beautiful-dnd

[mgmeyers]

As nice as it is, it seems development has stalled. There are a few key things that it doesn't support that Kanban needs:

Nested drag containers
This would allow capping the lane heights to the view height and reduce the amount of scrolling required overall

Adjusting autoscroll speed
Dragging cards and lanes between boards causes each board to scroll when near the board edges. It would be nice to adjust this behavior and fine tune it to our use case.

[mgmeyers]

https://github.com/react-dnd/react-dnd
https://github.com/SortableJS/Sortable

[pjeby]

Don't forget https://github.com/azuqua/react-dnd-scrollzone, which adds scrolling to react-dnd, and https://github.com/SortableJS/react-sortablejs, which adds Sortable to React. :) (Slight downside to Sortable, is that its default method of swapping items rather than making room, and its lack of transitions, make it rather less pleasant as far as UI goesI.)

I rather like the fact that both the options you're considering are HTML5-based, or can be, as that means it will be ultimately possible to drag cards or lanes between vaults(!) as well as dropping them into non-Obsidian apps. (Including dropping to Speare and Trello!) Proper scrolling would also mean Speare-style UI would be possible, and dragging between different-length lists would be a lot easier.

[mgmeyers]

Ah, good point about sortable. I hadn't noticed that nuance in how it swaps everything. I wonder if it can be tweaked to work more like beautiful-dnd.

My plan is to do a simple PoC with each of these to see how close we can get them to the current look and feel of the kanban plugin.

[pjeby]

Yeah, on the other hand, a limitation of react-dnd-scrollzone is that it will only be applicable within a Kanban view, since you have to explicitly define your scroll regions, whereas Sortable's autoscroll plugin automatically finds parent scroll regions.

You can get Sortable to hide that target and animate (see e.g. SortableJS/Sortable#663 (comment)), and with revertOnSpill you can get the item to go back to its original position if dropped somewhere other than a drop target. That gets it to looking mostly like react-beautiful-dnd. I haven't figured out how to get the dropzone (the "ghost" as they call it) to shrink or disappear when you're not over a drop target; it looks like some JS is needed to add some additional classes to the ghost element. There's a library that actually works for that (drip-drop) that would need to be used on the list and board to change styles in such a way as to make the ghost disappear when you're not in a valid drop target,

You can see my progress here:

https://replit.com/@pjeby/clean-sortable

The main issues it has are the lack of drop/spill animation (i.e. when you let go the item just instantly appears), and some weird flutteriness when you drag quickly (where all the items you pass over can be in transitional movement at the same time).

(This repl also doesn't test scrolling; it's just a proof of concept that sortable's less-than-stellar UX defaults can be improved upon, even without creating a plugin. (Which is something I didn't look into.))

[mgmeyers]

So dnd-kit seems really promising, though I worry about it still being in beta. Don't want to swap out the existing DnD lib only to run into other roadblocks. That being said, the demos contain all the functionality that we really need: https://5fc05e08a4a65d0021ae0bf2-htnzcdxubh.chromatic.com/?path=/story/presets-sortable-multiple-containers--many-items

[pjeby]

One downside to dnd-kit versus the others is the HTML5 API, which would allow you to drag cards out of the window and drop them into other apps -- including Kanban views in other Obsidian vaults, as well as Trello and various other dnd-accepting apps. Accepting drag and drop of other apps' data in any position in a lane (instead of just the bottom) would basically come for free as well. (You could even include the contents of a linked note as a file attachment!)

Dunno how important you consider those features, but they'd be nice to have, and both Sortable and react-dnd would support them, while rb-dnd and dnd-kit do not and cannot. (Well, technically, you could probably make dropping in from other apps possible, but dragging between Obsidian vaults would not be possible except by selecting text, and the attachment thing would be right out.)

[pjeby]

Also, react-dnd + react-virtualized can do stuff like this -- which isn't quite the tree view you imagined, but it certainly looks like react-dnd is a stable base for building fancy stuff on.

Given the existence of react-dnd-scrolling and the comment in react-dnd's docs that "Luckily, React DnD is designed to work great with any virtual React data list components because it doesn't keep any state in the DOM," I would count all these factors as a strong recommendation for react-dnd as the first choice to try as a replacement. (The fact that react-dnd-scrolling has explicit examples for combining it with both react-dnd and react-virtualized should be a big help with doing virtualization of Trello-style full-height columns, which in turn should be a big help to inital loading time of large boards.)

[mgmeyers]

Ah, very good points. I'll test out react-dnd then.

[pjeby]

Here's a really good example, though it only demos cards moving, not lanes: https://codesandbox.io/s/github/erikthedeveloper/react-example-kanban-board?file=/src/Card.js

The code is extremely simple, and I'm in the middle of experimenting with it in the Kanban plugin. So far I have a board that works perfectly except for the lack of any drag and drop whatsoever. 😁 But it is way more performant and the code is way simpler, which makes it look to me like the source of a lot of the problems was rbdnd and/or the lack of using props.children as a rendering approach. My draft renders all the lanes and items as vdom directly from the Kanban component, without needing any renderLane/renderItem jank, and the result works out with really high performance and doesn't do much GC. I'll have to see how well that continues once I put drag and drop back in, of course. 😉

[mgmeyers]

Ah, nice! It seems you're a little ahead of me on this then, so I'll let you continue. One thing I was curious about: because this uses the HTML5 backend, could we get rid of using portals to handle dragging between boards?

[pjeby]

I imagine we could? Dunno why we'd want to, though. Cross-context DnD means we would copy by default rather than moving. And while it might be handy to support cross-context move, that needs more work at the HTML5 level.

And I just found out that while react-dnd is based on the HTML5 level, it severely limits what you can do with the dataTransfer object by default, and I'm not sure it supports drop effects (e.g. move vs. copy). I was just looking into that, and haven't seen if anybody has a good workaround. It's likely they can be accomplished by subclassing the HTML5 backend, though.

In the meantime, I made cards draggable, but they can't be dropped anywhere. 😁 I had to turn items back into a functional component so I could useDrag(), as the wrapper method was too complex. But I managed to preserve single-item refreshing, so that's good. I need to add some more params to useDrag() and then add some useDrop() stuff and see how it works, maybe throw up a PR so you can join the fun? 😉

[pjeby]

So... after playing around with react-dnd a bit, here's my assessment:

The good news is, overall integration is pretty straightforward, and gets rid of a lot of the cruft we had with rbdnd, and is a cleaner overall structure.

The bad news is, to get the same nice animations and UX, we have to add them ourselves.

For the last couple hours I've been messing with trying to get items to slide over to leave a space for the drop, without actually moving the item and making it look like it's not there (which is what most of the react-dnd samples do, probably because it's the easiest way to do it -- but in our situation it would be constantly saving the file with a new position while you dragged it, and I don't think that's a good idea).

In order to get similar animation, it looks like we'll need to inject a placeholder during dragover, and shift items down (or lanes right) with transform: translate. I was trying to do it in a simpler way by animating the dragged-over item's margins, but I couldn't get it to work properly. (It kept restarting the animation, I think, as well as there being some issues with calculating the position at which to decide if the space should be above or below the item.)

In principle, this shouldn't be that difficult. In practice, I'm having trouble separating what's me not knowing React and or CSS animation and position-calculating vs. me not knowing react-dnd. The react-dnd docs are difficult to find things in, and its type declarations leave some things to be desired.

So...? Dunno where to go from here, really. Do you want me to throw what I've got in a PR so you can have a look and take your own shot at it, or do you want to try a different library? My overall feeling is that this is quite doable, it's just that the animation stuff is just barely out of reach of my current skill level. I'm sure I'll get it worked out eventually, but you might be able to do it a heck of a lot more quickly.

Honestly, all I've done is:

• Strip out all references to rbdnd, except for MetadataSettings
• Change Kanban to render DragableLane items directly with Draggableitem children nested in them (i.e. top-level component does everything except the actual rendering) and have DraggableLane simply render its props.children inside the lane.
• Change DraggableLane and DraggableItem back to function components and add a few useDrag/useDrop calls
• Make DragDropApp use an rdnd wrapper instead of a rbdnd one
• Spend an inordinate amount of time trying to figure out how to animate on hover in a way that looks like rbdnd 😉

I haven't actually implemented dropping, because that part actually looks ridiculously simple. Well, not simple, but straightforward, since I've previously implemented HTML5 drop for the add item area. But I'm assuming that we need to 1) get reasonably similar animation, and 2) can't use the "actually move it while dragging" approach used in the typical examples for react-dnd. So trying to meet those requirements first seems like a good idea from a risk-reduction POV.

[mgmeyers]

So, two thoughts on this:

1. Do you think the dataTransfer limitations warrant checking out a different library (eg, react-sortable)?
2. I wonder if we could wire up framer-motion to work with react-dnd: https://www.framer.com/api/motion/

• Framer could give us some really slick animations without a whole lot of work, though it would require that react-dnd allow us to use framer components for all of the interactive components.

If the answer to these is no, or it seems framer would be overkill, I'd be happy to take a crack at the animations using CSS.

[pjeby]

Sortable has a different set of challenges; if you look at my example I ended up using another library (drip-drop) to detect when the drag left the drop area. But it does have closer to the right sort of animation.

The actual animation needed for basic dragover behavior isn't that complex, I'm just completely unused to doing CSS animation, even less so in the context of react components. And there is a bit of weirdness around the events that rdnd exposes for dragover: you have a "collect" event that lets you get state of the drag, and there's a separate "hover", and it sort of seems like you need to use collect to find out when you're no longer being dragged over, but hover to find out when it starts, and it's kind of funky.

The thing that rbdnd does that I was trying to emulate, was done by just adding a transform: translate to every item past the hover point, and add an invisible placeholder at the end for those shifted items to move into. That requires changing props on every item in the column, though, and that was where things got tricky, due to my lack of React-HOC-Fu, as it were.

In truth, I don't have time to work on this right now; I was just curious and didn't think my experimentation was going to cause you to wait. I didn't want to hold you up, especially since I don't actually have time to seriously work on this right now.

Based on my experience with the different DND libraries I've played with to date (rbdnd, rdnd, sortable, and drip-drop), I'm actually thinking that if I had to choose, I would strongly consider just using drip-drop, if I could figure out how to integrate it with React. Or perhaps just not directly integrate it with React and set up event listeners on the view's content element instead. The benefit being that I wouldn't have to work around the limited event data and constraints of the various other libraries. Given that Sortable and rdnd basically both require you to do a good bit of "roll your own" animation, it would actually be easier to roll my own from drip-drop or even the unadorned HTML5 drag-drop API, as I would be able to look directly at event targets.

That is, I could just have a drag-drop databridge sending state into the Kanban from Obsidian event capturing handlers, and then the Kanban would just render the placeholders and animation. The hard parts I'm running into with rdnd is that in theory, it gives you everything you need to do this, but in practice the events are kind of funky and you have to combine info from several places to do it -- you can't just have the lane ID where the drop is going because the API doesn't tell you (even though it knows internally) what child item is being hovered over. The items know if they're being hovered over, but it isn't easy to tell the parent... especially since you know when you're hovering, but not necessarily when you stop. And so on.

Sortable has much the same problems, just different.

In general it seems the choice is between:

• Deal with a high-level wrapper over HTML5 (Sortable, rdnd) and have to work around its events to be able to add animation
• Use a non-HTML5 library (like dnd-kit or rbdnd or react-dragula) and get full service (but no cross-vault or cross-app dragging)
• Use a low-level wrapper like drip-drop
• Use something else entirely (like this)

When I do have some time to play with this again, I think I will make a new branch using the same basis as for my rdnd experiments: i.e. strip out rbdnd and make a pure functional top-level render, then work on making the animations work without actually using any kind of DND library, but instead have a custom state specifically for the Kanban plugin, and just manually send events into it to get animation working. Then, see if I can figure out how to get rdnd to give the events needed, so that issue is isolated, and can also be compared to setting up drip-drop or plain HTML5 events to get the same state info. That should give a good indication of how good any given HTML5 drag-drop library's API is.

Second, I think I would also experiment with virtualization and Trello-style columns, perhaps before adding the drag-drop back in, again to make sure that the mechanics of positioning and animation can work without the actual drag-drop library getting in the way.

But, I do not think that any of these thoughts of mine should block you from doing whatever you want to do with the project. I'm actually pretty happy with the state of things right now with rbdnd, and TBH virtualizing the lists is a bigger priority for me featurewise than anything else. (Of course that feature depends on the drag-drop library, so it's all pretty circular.)

I'm too tired to think about this any more, and have way too many things I should be working on instead of this, so I'm going to take a break from coding on it for a few days. I definitely did not want to pre-empt you from working on any of this, so please don't let me stop you.

[mgmeyers]

Hey @pjeby totally understand. Thanks for all the work you put into this!

[pjeby]

Since my last comment, I did take a few minutes to look more at dnd-kit and browse its internal architecture a bit. It does look like the best programming interface and best internal architecture of all the libraries. The autoscrolling in its demos seems erratic, however, and I don't know how fixable that is.

It also looks like its architecture might be able to support writing a custom sensor for the HTML5 drag-drop API, as the internals are far more decoupled than the other toolkits, and it looks like it should be possible to reuse its hover and drop animations in such a case. The way it works is that sensors add event listeners and properties to the targeted drag or drop components, so a sensor could in principle add e.g. draggable and dragstart as part of that, and it's something that I could maybe do a PR later for.

It'll probably be necessary to reverse some of my function->class refactors, no matter what library ends up being used, since most of them integrate via hooks rather than HOCs or components. At the time I was doing those changes, I wasn't familiar enough with functional components and was trying to rule out causes of re-rendering, but the experiments I did do with r-dnd showed that the true source of the re-renders (well, the source of the mysterious re-renders) was the creation of new component types on each render (to integrate with rb-dnd).

Me switching to static component classes fixed that, but only because it exposed the component type issue. I've also got my eye on an experiment to see if I can push markdown rendering into a useEffect in the MarkdownRenderer component to make initial loads faster, and if making the Mark.mark() stuff an effect will make searches more responsive. If that turns out well (whenever I get around to doing it), I'll send a PR.

[mgmeyers]

So I played around with dnd-kit quite a bit. It's definitely not as straightforward as I'd hoped. It's hard to tell yet if it'd actually work, but I'm getting close: https://github.com/mgmeyers/dnd-playground/blob/master/src/App.tsx

This definitely makes me appreciate the developer experience of react-beautiful-dnd.

[mgmeyers]

So, I was able to get a workable prototype of almost everything we need using dnd-kit. It's definitely not as smooth and stable as beautiful dnd, but there may be room for improvement. The one thing I've yet to figure out is virtualization. It seems to be a tricky thing to do with dynamic height list items.

https://github.com/mgmeyers/dnd-playground/

Screen.Recording.2021-06-19.at.4.49.58.PM.mov

[pjeby]

Might need to rip off some animation logic from rbdnd, I suppose. 😉 Seriously, though, it's looking good.

As far as virtual lists go, have you seen this one: https://github.com/dwqs/react-virtual-list ? It does automatic measuring, but it's not clear when/if it will remeasure when items change. This one -- https://github.com/miketalbot/react-virtual-dynamic-list -- says it supports item sizes changing. And so does this one: https://virtuoso.dev/auto-resizing/

I did also look at the two big names in virtualization -- react-virtualized and react-window -- but neither of those seem to do much more than lip service to dynamic sizing that isn't calculated by the app. Very much a "roll your own" kind of thing. RVDL and Virtuoso seem more tailored to this type of scenario.

[mgmeyers]

So, I'd done a ton of experimentation over the last few days and tested: dnd-kit, react-dnd, and react-smooth-dnd.

It's been a frustrating experience, to say the least.

dnd-kit almost got us where we needed to go, but there's a fundamental flaw in its design: each draggable item uses context and will update every time dragging start or ends. With a board of 160 cards, this results in a 150-200ms delay when picking up or dropping a card. We could scope each lane two its own nested context, but you can't drag/drop between contexts out of the box (not sure if it could be done at all). https://github.com/mgmeyers/dnd-playground/blob/alt-approach/src/App.tsx

react-dnd has some weird quirks, and doesn't allow for drop animations when using a custom drag layer. More importantly, its mobile isn't great.

react-smooth-dnd is pretty decent, but doesn't support virtualization. Also, drag scrolling when there are nested containers is pretty frustrating.

I'm going to table this investigation for now so I can focus on adding more features and fixing bugs. One avenue I have yet to explore: building a custom solution from the ground up using something like framer-motion: https://codesandbox.io/s/framer-motion-drag-to-reorder-pkm1k?file=/src/Example.tsx This would obviously be a lot more work, but would have fewer limitations.

[pjeby]

Hm. I thought you were doing wrapper components such that though the outermost div of each item would change, the children wouldn't? i.e. the expensive parts would stay unchanged? Also, wouldn't virtualization keep all but the visible cards from updating, anyway? (IOW, there'd be an upper limit on overhead.) Or is the problem that the delay is coming from DOM relayout?

[mgmeyers]

Hm. I thought you were doing wrapper components such that though the outermost div of each item would change, the children wouldn't? i.e. the expensive parts would stay unchanged? Also, wouldn't virtualization keep all but the visible cards from updating, anyway? (IOW, there'd be an upper limit on overhead.) Or is the problem that the delay is coming from DOM relayout?

So the issue with dnd-kit is that all draggable / droppable components are tied to context that updates on drag start and end. In my tests, this resulted in the entire react tree rerendering. This doesn't cause the dom to rerender, but from what I can tell dnd-kit measures the dimensions/position of every drag/droppable element, which seems to be a bottleneck.

I couldn't leave this alone, though, and ended up writing my own DnD engine from scratch:

Screen.Recording.2021-06-28.at.5.02.06.PM.mov

The code is a complete mess at the moment, but so far it seems a lot more performant than dnd-kit: https://github.com/mgmeyers/dnd-playground/tree/from-scratch