• Recent Articles
• By Topic
• By Year
• About me

Applying Clojure core.async to JavaFX

01.05.2014 Permalink

If you ever created a non-trivial enterprise rich client application with many UI forms you will have noticed that it is quite a challenge. Given the conditions that apply this is no surprise:

• UI toolkits like Swing, JavaFX or others are packed with mutable state.
• The layout and configuration of widgets is usually very verbose or requires the time-consuming use of point-and-click designer tools.
• Every piece of presentation logic is triggered on the basis of callbacks.
• Actions that change UI must do so by causing side-effects.
• To make a UI responsive we have to deal with multiple threads.
• Testing presentation logic is often very hard if it's mixed up with code that accesses UI. This forces us to employ UI robots and to maintain brittle test scripts.
• A usable UI is often the result of numerous iterations until user needs are sufficiently satisfied. Thus changes in UI code happen often.

As a result rich client code often exhibits overwhelming complexity and quickly becomes unmaintainable. Considering the importance a user interface has for product acceptance and the amount of code necessary to produce a sufficient user experience, this is an unsatisfying situation.

In the past, I worked on several rich clients in OO languages like C++ and Java. The best ideas I was able to find in OO land beyond the very basic - but still somehow applicable - separation of responsibilities into Model, View and Controller (MVC) were Model-View-Presenter and Presentation Model. Both patterns essentially address the problem of how to make presentation logic unit-testable without using GUI robots and brittle test scripts.

An improvement, but still not satisfying

In a customer project two years ago, I used separate classes for data - the Model - and actions - the Controller. I built a textual UI form DSL and a corresponding code generator that produced View classes, and added a full-blown data binding between Model and View. In addition I invented a UI interface to give the Controller a mockable API for UI operations, e.g. creating a new UI form or showing a message box.

Here's an overview of this design:

This already allowed the team to produce unit-testable client code without having to deal with many of the tricky details of GUI development, but the client still heavily used callbacks in disguise of "action methods", and these methods did their work through in-place mutation in Model and Controller instances. Even worse, in order to execute long-running actions, developers still had to create two additional levels of callbacks, one for the long-running operation and the other to merge the results back into the view. This was the forecourt to callback hell, and it is still the default way in JavaFX and Swing to keep the application responsive to user input. So the code expressing the presentation logic was still messy, nonetheless it was the best I was able to come up with by that time.

And now for something completely different

Through learning Clojure I came across ideas like Functional Reactive Programming (FRP) and Communicating Sequential Processes (CSP). These approaches allow us to treat some of these problems in a different way.

In the last few months, I tried to apply FRP in my experimental library visuals, which is based on a home-grown implementation of Signals (a.k.a Behaviours), Eventsources (a.k.a Eventstreams) and corresponding operations, which I re-implemented in fall 2014. You can find some links to papers about FRP on the GitHub page of reactnet. Coincidentally, I started my work on reactor around the same time that James Reeves started to work on reagi, and shortly after that Rich Hickey published core.async.

I applied FRP to enterprise-style UI forms in a very direct manner: Each visual component of a UI toolkit (e.g. a window, textfield, button, table, ...) provides mutable properties that allow to register listeners. This concept is strikingly similar to what FRP "behaviours" do in practice. So I adapted these properties of visual components to conform to reactor's Signal protocol (which represents a "behaviour"). Button presses or input-focus changes can likewise be seen as Eventsources, so consequently I registered numerous listeners that feed into individual Eventsource instances. This finally enabled me to represent all mutable data and events in the unified terms of FRP.

Although I experienced some relief from the resulting uniformity, I eventually got stuck, because my FRP implementation didn't seem to offer a natural way to express the interaction between UI forms. Similarly, I didn't see a way to handle long-running actions without resorting to some kind of callback mechanism. Perhaps my FRP implementation was simply missing the important Inversion of Control feature core.async provides. To summarize: I consider this experiment as a learning exercise, but not as a result to base future work on.

Update 31.10.2014: As it turned out, reactor was missing the flatmap combinator. I discovered this very recently while approaching GUI form interaction with my new reactor re-implementation. So, it is possible to solve this problem by using an FRP library that offers to dynamically create and connect new eventsources, which is essentially what a flatmap does with a passed-in function.

A Process-and-Channel oriented approach

Inspired by David Nolens article about CSP in ClojureScript and my recent learnings about how UI is managed in 55 years old IBM RPG I switched with my current async-ui prototype over to CSP and core.async. Here's a picture giving a rough overview of the design:

The rich client consists of processes and the state of the toolkit. The term "toolkit" refers for example to JavaFX or Swing. A process is a core.async go block. There are two predominant types of processes:

• One is the single toolkit oriented process, started by invoking the run-tk function. Its purpose is to keep the toolkit state in sync with the data that represents views. It reads view data from a central channel and causes effects to the UI toolkit, e.g. creating a visual component tree or updating data within such a tree.
• The other is a view oriented process, started by invoking the run-view function. Its purpose is to update the view data according to events it reads from the events channel each view has.

A view is a map that contains a specification of the visual component tree, the domain data, a mapping between both, validation rules and validation results. This data represents the state of a UI form. In addition each view has its own events channel and references the root of a toolkits visual component tree (a JavaFX Stage or a Swing JFrame instance). For each view, one process is started, which processes events for that view. "Event processing" means that the data contained in the event is merged into the view state, validation is applied, an individual handler is invoked and the resulting view is passed via the central channel to the toolkit oriented process.

An event is a map created by toolkit specific event listeners that write them into the events channel of the corresponding view.

The concrete UI toolkit like JavaFX or Swing is kept behind the toolkit protocol. It is mainly implemented by a Builder and a Binding. The Builder takes the specification of a form and translates it to JavaFX or Swing API calls in order to create an actual visual component tree. The Binding registers listeners that put an event onto the views own events channel and creates setters that update the visual components properties with the data contained in the view.

An example

The example I have prepared so far shows a little master-detail scenario. Here's how the master form looks like when run with JavaFX:

To start the view process the following expression suffices: You can see that two function-vars are referenced, one points to the factory function that creates the initial data that represents the view. The other one points to the event handler function.

The visual component tree and the mapping is expressed by the factory function:

The event handler is another Clojure function, and by means of the go block an asynchronous process: The go block is needed here because the master may start another asynchronous view process for displaying and handling the detail view. Please note that the communication between master and detail is almost like an ordinary function invocation. The master view process is paused until the detail process finishes.

Benefits

As it turns out, the channels establish a strict separation between the toolkit oriented process, which handles synchronization of view state with the visual component tree, and the view oriented process, which applies events to the view state and pushes updates into the central toolkit channel. While the toolkit oriented process must deal with all the dirty details of the JavaFX or Swing API, and has to keep mutable state in sync with the view state, the view oriented process solely deals with data transformation based on events, which themselves are simple maps. This is an important property, because it allows the frontend developer to stick to Clojure core functions to get his work done.

Unit testing presentation logic becomes a no-brainer, since we can feed events into the views channel and must only assert that the view state changes as expected.

The interaction between different views uses ad-hoc channels as returned by a go expression. By immediately starting a blocking read, view processes can wait for each other without blocking the UI event thread.

The channel infrastructure can also be used for dealing with long-running actions like calling remote services, which can easily go either into their own future, or use a non-blocking callback based API. Upon availability of the results the process feeds these into the events channel of a view, which handles them like any other event.

Conclusion

While I still think that FRP allows for very elegant solutions in the area of graphics animation or gaming, its core concepts "behaviour" and "eventsource" seem not crucial for solving the wide-spread problems in enterprise GUIs. But I'd like to be proven wrong on this point.

Update 31.10.2014: After I found out how form interaction can be expressed by means of FRP combinators, I consider it as one viable option to create typical enterprise "forms-over-data" GUIs. However, it took quite some time to grok this. For me, a helpful key insight was that eventsources are the way how asynchronous processes convey their results.

The idea of asynchronous processes is better visible in CSP because it materializes as go blocks in the code, thus after I dissected the whole problem into separated processes which are only coupled loosely through channels that carry dumb data, things became much easier.

My doubt about whether it makes sense to create a full-blown Clojure library for JavaFX programming has grown. While I'm convinced that the ideas and the design I used (like separated processes connected via channels, view representation by pure data, an explicit builder and a binding) are a good foundation, I'm afraid that project specific requirements between different applications vary in numerous details. A library that provides the level of comfort I consider as necessary inevitably becomes a batteries-included framework with lots of assumptions and implicit behaviour. My experience with those frameworks in Java-land tells me that such a "claim of omnipotence" almost always leads to pain and horrendous work-arounds. I still have to make up my mind if there is any piece in this picture that can be extracted to form a useful library.

Meanwhile, please feel free to exploit these ideas and the code I pushed to my GitHub repo.