Six years of professional Clojure development

10.05.2021 Permalink

Over the last couple of years me and my colleagues here at doctronic have been busy creating and maintaining more than a dozen of individual software systems for our clients.

We have the privilege to almost exclusively use Clojure and ClojureScript, as this is still the valid strategic technology decision made by doctronic shortly before I joined them in 2015. In this post I'd like to list some of the strengths and weaknesses in everyday real-life use of the language and its ecosystem.

Stability of the language and libraries: Anyone who ever created software of practical value for paying customers knows that reliability of core language constructs and libraries is important. It supports predictability, and eventually saves you from insanity whenever you want to benefit from performance improvements or bug fixes. Upgrading to a new version of a library or even Clojure itself is risk-free. The authors and maintainers really do care about stability. If you listened to talks of Rich Hickey or Stuart Halloway you know that this is no coincidence. "Don't break things!" is part of the culture.

Code size: If you compare some of the idiomatic code snippets in Clojure with the equivalents in more popular languages you might conject that whole Clojure code bases might in general be smaller compared to what you can achieve in other languages. Now, after having worked more than 20 years with imperative OO languages and more than 5 years with Clojure I can firmly state that you'll achieve the same functionality with a fraction of the amount of code. This impacts costs: My feeling is that we now build systems with only half of the staff that I used to have on projects with a comparable scope.

Finding staff: Clojure's popularity is still far, far behind imperative general purpose languages like Java or Python, so fewer people feel encouraged to invest the time to learn the concepts and gain practice. It seems that this makes hiring harder. doctronic addressed this challenge by positioning itself as a supporter of the language in Germany through organizing and sponsoring the annual :clojureD conference. This helps greatly with staffing. But even if a company has a lower visibility, the fact that it uses Clojure will attract aficionados of advanced programming languages. If these developers do enjoy their workplace they will usually stay around longer with their employer. So I'd argue that you gain quality in skills and a lower turnover rate, but it might take more patience to build up a team of Clojure developers.

Teaching the language to apprentices: doctronic constantly employs apprentices so I was able to accompany some of them and help them learn Clojure. Young minds with very few programming experience are remarkably fast in picking up the language and become productive, it seems. I assume there are two main reasons for this: There's actually a whole lot of things they don't need to master compared to OO land. And they don't need to unlearn anything, in other words, they don't need to overcome old habits of doing things.

Some discipline required: Clojure is a language that gives us a wide range of options to implement solutions. And it imposes very little ceremony. For example, there is no static type system that checks your code upon compilation, but you can get some of the benefits for your data at runtime by using Clojure spec. This also serves as documentation and possibly a generator for test data. It is up to the team to decide when and where more checks, restrictions and tests have to be added. If the team misses out on this the code base might be harder to understand and maintain than necessary. In essence: with great freedom comes great responsibility.

Navigating the code: When maintaining or extending an existing code base you often need to find specific places where a function or a piece of data is referenced. To support this kind of search for specific identifiers it is important that naming is self-explaining and consistent, even more so because there are no links established via a type system. To mitigate this weakness in Clojure, you should use qualified keywords whereever possible.

Documentation becomes more important: When you're trying to understand a function that invokes other, non-core functions you'll need a clear idea what kind of data these functions expect and return. Without any data type declarations the code itself often doesn't reveal how the data that flows through you functions looks like. You'll need either Clojure spec or more thorough documentation to mitigate this problem. It is not so rare that you need to test functions of interest isolated in the REPL to get a clear idea how the data looks like.

The promise of purity: Any practical software application must cause some side-effects. So even if Clojure strongly encourages developers to create pure functions your systems will contain substantial parts that either depend on some environment or cause side-effects. The trick is to separate these pieces sharply and keep the unpure parts small. Without any discipline you could easily spread side-effects everywhere and end up with the typical imperative mess. Clojure makes it easy to create a mostly pure implementation but it does not enforce this. To ensure this and other qualities the teams at doctronic conduct code reviews for almost every commit before it is promoted into the master branch.

Reusability: Here's where Clojure really shines. Because most functions are pure and operate on few common data structures (mostly maps, vectors and sets) it is very easy to write reusable code. We created many internal libraries just by separating the candidate functions into their own namespaces within the regular project source tree. To finally establish the library we move the code from the project repo into a new Git repo and include the resulting library Jar in the project.clj as dependency. Thus, we have a very lean process that results in production-quality resusable assets.

Startup time: Starting a fresh JVM with a Clojure Uber-Jar to bring up an application takes noticable time. I assume that JVM class loading causes this delay. So if you consider to create a command line tool that has a short net execution time you wouldn't want this runtime overhead.

Runtime stability: Clojure applications are in general very stable. I remember that in bigger Java projects from my past we always had to do some load testing in order to detect programming flaws like memory leaks or race conditions. With Clojure, we do load testing only to find real performance issues.

Ok, time to come to an end before this post becomes too long. The list is perhaps not complete, but I guess I included the most important aspects that recently popped into my mind. When I decided to fully jump on this train in 2015 I expected more unpleasant surprises. Now, six years later I have proven to myself that Clojure and ClojureScript are a practical and sound choice for real world software development, and I still enjoy using them.


QMK and touch typing

20.05.2020 Permalink

Back in 2012 I bought my first keyboard with mechanical Cherry MX Brown switches, the yellow edition of a Ducky Shine 3 TKL. By that time I had virtually no clue about the world of keyboards, but the product felt nice to me and it looked extraordinary:

In my opinion a good mechanical keyboard makes a difference so last year I tried to find a nice one for my daughter, as she was beginning to spend more time on her PC. I was lucky enough to buy a Ducky x Varmilo Miya Pro which has a very fine build quality and typing sound.

During the search I was a bit surprised that it is actually a matter of luck and/or patience to acquire a high quality mechanical keyboard. There are so many nice keyboards out there, as r/MechanicalKeyboards demonstrates on a daily basis. The nice pictures and stories got me hooked, so I started to explore the world of mechanical keyboards more in depth. Eventually I decided that I wanted to tackle one of my long lasting weaknesses: the lack of consistent typing speed. Isn't this the perfect excuse to spend some money on a new keyboard?

I was never a good typist, I hardly used touch typing, although I tried more than once to properly learn it. I suspect my failures of using touch typing in everyday work were caused by the position of the modifiers (Ctrl, Shift, Alt, Super) and especially the nav cluster (arrow keys, page up, page down), constantly drawing my right hand away from the home row (the row with the keys for A, S, D, F and so on).

My exploration led me to the idea that a programmable keyboard with a 60% form factor (a keyboard without numpad, nav cluster or function key row) might bring me closer to my goal. After some more research I bought a KB Paradise V60 Type R because it featured an ATmega32U4 micro processor which is known to support the open source firmware QMK. This firmware allows you to freely program your keyboard with up to 32 layers using the programming language C. In essence, this allows you to make it truly your very own keyboard.

The 60% form factor eliminated the nav cluster, so I assigned the arrow keys to I, J, K and L, but the modifier position was still disturbing.

QMK provides a feature called Mod-Tap which allows you to assign a modifier to any key. If you just tap the key, you'll get the character, but if you hold it and press it in conjunction with another key you'll get the modified behaviour. I assigned the modifiers to my home row like this:

ASDFGHJKL;
CtrlSuperAltFnFnAltSuperCtrl

It took a bit of tweaking of QMK parameters and a few hours to really get used to it. But after all, my fingers stayed on the home row and my latest effort to properly learn touch typing succeeded!

But with this change the modifier keys on the bottom row of a classical keyboard became useless to me, so this leads me to think about keyboard layouts in a more general way. I am quite aware of Atreus and other split keyboards as a potentially more ergonomic arrangement of keys.

So this story might continue in the very near future...


DIY desktop with the i3 window manager

01.07.2018 Permalink

Linux offers several desktop environments and window managers, so if you don't want to stick to the default of your distro you're free to explore alternatives. I started using Ubuntu Linux in 2010 on my business notebook. Back then Ubuntu came with its Unity desktop environment, but this was eventually dropped in favor of Gnome 3 in 2017, featuring a virtual desktop feature called workspaces.

I usually have to handle at least a dozen of running applications, among them several terminals, Thunderbird, Slack, Emacs, a twin-panel file manager named Krusader, at least four different browser based apps and so on. Often I need to switch quickly between applications. To manage the corresponding windows I made heavy use of Gnome workspaces, giving me a multiple of desktop real estate.

Over time I observed that:

So there was room for improvement.

Looking for a solution, I came across a specific class of windows managers. A tiling window manager promises to automate arrangement of application windows to some degree. The main idea is that the window manager avoids any overlap and automatically splits the available screen space horizontally or vertically when a new window opens. Among the popular implementations i3 seemed to be the best choice for me, so I gave it a try. The following screenshot shows how i3 arranges multiple terminal windows.

i3 has some characteristics that I appreciate:

Please note, that the last item points to a substantial implication: i3 is not a desktop environment, and I had to add numerous features until the system was convenient to use. In particular the status bar required some research and time until it looked like this:

The following table shows which Linux tools I used together with a bit of Bash scripting. This illustrates that learning i3 is only the tip of the iceberg. I had to pick up with many more tools that make up a usable desktop experience on an X window system.

Feature Tool
Notebook display brightness control xbacklight or xrandr
Notebook display night light redshift or xrandr
Turn external monitors on/off xrandr
Screen lock i3lock, xdpyinfo, compton
Screen saver xset (dpms feature)
Wallpaper feh
Audio and music control pavucontrol, pactl, clementine
Bluetooth control blueman-applet
Networking control nm-applet
Switch keyboard layout, adjust autorepeat setxkbmap, xmodmap, inputplug, xset (r feature)
Application launcher dmenu
USB storage automount udiskie
Screenshot tool flameshot
System information in status bar i3blocks
Provide keyring keys gnome-keyring-daemon

After all, it was a fun project, and it eventually improved my productivity on Linux. The whole configuration including all scripts is hosted on Github.


ClojureBridge Germany 2015

05.03.2015 Permalink

Programming languages breed cultures. They do it by means of their advocates who talk about them and differentiate them from other languages. They do it by how members of their communities interact with each other and with the people from the "outside". But I should be more precise: A programming language itself can't do anything. People create cultures, and using a programming language or not is just one type of boundary to make a distinction between "us" and "them".

Learning a programming language and its ecosystem sometimes requires quite a bit of frustration tolerance. The way members of a community interact with each other, the way how welcoming it is to people looking for guidance or people with different ideas makes a great deal of how attractive it is in the long term. But if you add fear of rejection or hostility to the hard work of learning (and sometimes failing), the community scares away even those who could potentially bring substantial contributions.

When I started to learn Clojure I recognized the openess within the community and a strong will to nurture a healthy mode of interaction among people. The other thing I immediatley enjoyed about the community is that its members have -- in summary -- the best ratio of good-vs-crap ideas in software engineering that I experienced in programming so far. In fact, those ideas changed fundamentally how I think about system design.

And then I came across ClojureBridge, a type of event that was new to me. Odds were high that I was missing an important idea, simply because its adoption in the Clojure community usually is a strong sign that it's valuable.

So I tried to understand the deeper purpose of ClojureBridge, I digged into diversity, understood how the lack of it was connected to how racism, sexism, ableism and all these other -isms work, and that such an event is one type of action to keep this insanity from growing further.

By the end of 2014, I played around with the idea of organizing the first ClojureBridge event in Germany, as I could rely on getting full support from my employer, itemis. I was very pleased to find out, that Silvia (innoQ) and Gerrit (codecentric) already had the same idea in their minds, so we formed the organizing team to distribute the work. Both got immediately full support from their companies, too.

However, one important question remained: will there be enough Clojure trainers available that help the attendees succeed in learning the language? And again, a very pleasing surprise: in only a few days we received more commitments than we have hoped for, especially from regional companies engaged in professional Clojure development like bevuta and doctronic, which -- in addition -- will help as sponsors.

So, we now have a date (June 12th to 13th), a building (thanks to codecentric), strong financial backup and the support of the local Clojure community. For all of this, I am very grateful, and I'm looking forward to the workshop in June!


Ideas for Clojure GUI libraries

15.12.2014 Permalink

Earlier this year I published a prototype to combine JavaFX with core.async to overcome some problems that I never solved in plain Java. Very recently, I sensed some interest in that prototype so I write a public post about my thoughts.

At the moment I have no plans to evolve async-ui as-is into a library, because it deals with numerous problems at once, i.e. it is not simple, and creating a GUI framework typically involves a huge amount of work. As it is today it just demonstrates that the idea of splitting UI code up into the following two conceptual functions

  1. build and update GUI from view state
  2. purely transform view state event by event
is an interesting approach for functional GUI programming, and that it can be combined with CSP style processes connected via channels to avoid nested callbacks, which - by the way - results in a crystal clear separation between GUI API access code and business related presentation logic.

It is the basis of a talk I give in January at ClojureD conf and it will remain a prototype. If, however, it turns out that many people actually want to have a more functional-style GUI programming experience in Clojure I would be willing to team up with others and put some effort in.

My ideas for such an endeavour are as follows:

1) I would rather focus on JavaFX than Swing, because JavaFX has a cleaner API, and Swing support has ended.

2) I would not like to create a "batteries-included" framework, because this would include numerous assumptions and trade-offs that might not fit actual needs and it's usually no fun to bend a framework into directions that its authors didn't anticipate in the first place.

3) One interesting aspect IMO is the transformation of Clojure data into the JavaFX scene graph. This is kind of the JavaFX version of React in browser DOM world. In async-ui this is currently done by the build multimethod that creates and configures controls, and by calling the setter functions to update data in controls. This transformation should be extensible in the sense that developers must be able to add transformation code for any individual control without touching existing namespaces (open-closed principle).

At the moment, with respect to this extensible transformation async-ui has two shortcomings: 1) the specification language in forml namespace is not extensible, 2) interactive development works, but rebuilds a Frame/Stage from scratch, so I would like to see a smoother way of merging changes into the scene graph.

It would take some exploration to identify the piece of non-trivial, reusable transformation related code that can be put into a library. At least such a library might provide some infrastructure and guidance. On top of this infrastructure, a benefit for real-world projects would be default transformations for widely used FX controls.

4) Application state management is a different aspect. Async-ui currently uses local state per Frame/Stage, not global state as in Om. I would like to keep the decision of how state is managed independent from the transformation problem. Cursors as in Om are IMO crucial for managing global state. A general cursor library would definitely be a useful asset. Sean Corfield started a ClojureScript cursor library for reagent which might act as a starting point.

5) Communication between GUI views, presentation logic and long-running code is another distinct aspect. My main requirements would be support for callback free code and a clear separation between GUI access code and presentation logic. I would like to keep the decision for the mechanism out of 3 and 4, but I'm not sure if this is possible.

In a concrete application I currently favor core.async over Rx-style libraries. I experimented with GUI and an Rx-like flatmap here. As you can see, Rx-style combinators help to generalize over different event sources and even treat the result of an asynchronously executed modal dialog just as another event source. But what I'm eventually after is a programming model that allows the programmer to stick to pure functions as much as possible. For this, I would try to get Rx or core.async to work more "in the background". With respect to GUI and asynchronity you get basically the same benefits from any of them, so I'd pick whatever is the more "mainstream" way in Clojure, which is core.async.

To sum it up, I think there a some useful pieces that one could build around Java GUI APIs to make them lend more towards functional programming. I hesitate to invest time here because I'm afraid that hardly anyone does Java GUI programming these days. If you think I'm wrong, drop me a mail or send me a tweet.