feature request: uniform DSL for state/data management

[Korede-TA]

i'm not sure if this is unrealistic or out-of-scope for this project, but please hear me out

context:
umbrella supports a number of different rendering targets including browser DOM, SVG and even the Canvas API. the hiccup s-expression DSL abstracts away how these rendering targets work and allow for a more uniform programming model that's easier to reason about. it also allows for these different targets to be composed nicely.

also, umbrella provides a number of different state and dataflow mechanisms including atom with it's nice undo/redo helpers and rstream for reactive state management.

my question/request:
could there be a uniform data management DSL that wraps atom, rstream and the likes which makes it easier to reason about and compose the different state management mechanisms available in umbrella?

(this question if sort of a follow-up to an earlier one i asked here: #238)

[postspectacular]

Hey @Korede-TA - firstly welcome! 👋

This is a well-known feature request of sorts, and even though I've got some very loose ideas for a layer of such syntax sugar glazing, it's been on my own back burner for a long time, and will realistically be approached/addressed earlier by others than by myself. This is not to say that there wouldn't be any interest in such an addition, but I don't think I have the bandwidth to create & maintain this right now. Also, personally, I see rstream as the unifying and most flexible solution for state management and there already is full integration with the atom package in the form of fromAtom() and fromView() stream creation adapters. In terms of updating atoms, several rstream constructs can also help to make these updates more coordinated (e.g. queue update requests, sync with requestAnimationFrame(), use pubsub for unified event handling, or thi.ng/transducers-patch for declarative state updates). So far I've mainly shied away from trying to define some additional unifying layer/DSL for all these options, since a) I'm still often forming (and changing!) my own opinions for how & when to use certain constructs and b) even though these parts all technically compose easily, some approaches aren't semantically compatible (or sometimes even just superfluous).

Finally, I always saw hiccup more as a general purpose data format rather than specific DSL. Think about it more as XML being a superset of HTML. Hiccup, as format/convention, is ideal for defining UI components, but could also be used for any other tree-like structure. Also, there're some minor differences in how certain details are interpreted between the existing hiccup-consuming packages. E.g. hiccup-canvas vs. hiccup-svg or hdom vs. rdom. Some of these differences are the result of time/age and learning, but in other situations also exist by purpose due to their different use cases (e.g. again canvas vs. SVG).

In general, all the packages offered here are provided to help others (like myself) to explore the opportunity space rather than offering turn-key solutions. Of course most of them are (readily usable & useful!), but none of us really have definitive answers to big questions like state management or component definitions, no matter how large & popular the other currently existing solutions are. So the search is still on. I'm also very much of the view (learned the hard way) that each application I've ever built has some specifics which do not easily fit off-the-shelf solutions. Therefore, my (so far unfulfilled) hope has always been that others would more pro-actively join in that search space exploration rather than just being consumers/users of these packages. This is NOT a criticism of the current community/users but it just has uncovered less findings and new ideas than I've originally have hoped for...

So with all that said, let's please NOT close this issue and use it to at least start figuring out what such a unifying solution would need to address in your and everyone else's view?! You mentioned to make it easier to reason about the different solutions, or abstract them away...

For me the two biggest conceptual gaps & issues w/ atom & rstream are:

• centralized vs. decentralized state
• atom-based undo/redo in dynamic dataflow topology (which is only imperatively defined/editable)

The first is partially addressed in the thi.ng/rstream-graph package (as an example), where a single atom is used to store each node's state. For the second aspect, so far I've always ended up employing some form of the Command pattern combined with a watch attached to the atom, which then performs the necessary re-wiring of associated rstreams...

[Korede-TA]

thanks for the super thorough answer @postspectacular!

i totally agree with the sentiment about exploring this search space of functionality. what you've noted about rstream being the closest to a generic state management solution makes a lot of sense as well, and i'm eager to see what other kinds of patterns could be teased out from different combinations of these tools.

i've really enjoyed the tiny bit of work i've done with umbrella so far so i'm definitely going to be using it a lot more and will try my best to be pro-active about surfacing any findings i make!

[allforabit]

Here's an interesting looking project that I think moves towards something quite unified: https://effector.now.sh/ and may be of interest. I've only really read up about it however and haven't actually used it but there's some good ideas in there that could provide some inspiration. I think umbrella would have all the building blocks to do something similar. Also of note is that one of it's inspirations is re-frame (https://effector.now.sh/docs/introduction/prior-art), a clojurescript framework that I know inspired some of the libs here. Here's a fairly detailed write up on how it's made: https://dev.to/yumauri/e-wee-ctor-writing-tiny-effector-from-scratch-1-1kap