The tools of an Agentic Engineer

A lot of great things have origins from the 1970s: Hip Hop redefining music and street culture, Bruce Lee was taking Martial Arts to the next level and the initial development of something called editor macros (also known as Emacs) was happening. I was born in that decade, but that's purely coincidence.

My choice of primary development tool since a couple of years back is that editor from the seventies. It is my choice of development for Python, JavaScript, TypeScript and Lisp dialects such as Clojure and elisp. And today, as an agentic engineer, it turned out to be a great choice for this kind of software development too. With the rise of various CLI, TUI & Desktop based tools for AI development, it would be reasonable to think that this ancient code editor would become obsolete - right?

Not if you knew about the innovative Emacs community. It is driven by passion, support from the community itself and Open Source. These components are usually more resilient and reliable long term than the VC driven startup culture. Emacs is part of the greater Lisp community, where a lot of innovations in general take place. The Clojure community is cutting edge in many aspects of software development including AI.

More Agents

One thing that I have noticed lately is that the more I get into Agentic Engineering, the more I use Emacs. When the focus has shifted from typing code to instruct and review, I have found use of Emacs powers I haven't really needed until now. Tools like Magit (git) and I'm also learning more about the powerful Org Mode. I didn't care that much about Markdown before, but now it is an important part of the development itself. So I just configured my Emacs to have a nice-looking, simple and readable markdown experience.

"More Agentic Engineering, More Emacs"

With Emacs, I use a great AI-tool called Eca and with it I am not limited to any specific vendor for agentic development. Vendor lock-in is something I really want to avoid. The combination of Eca and the power tools mentioned before, makes a very nice Agentic Engineering toolset. Eca is actively developed and has a lot of useful features and a very nice developer experience. It supports standards like AGENTS.md, commands, skills, hooks, sub-agents and use a client-server setup in the same way as the language server protocol. It is Open Source and not only for Emacs. Have a look at the website for support of your favorite editor or IDE. By the way, Eca is developed in Lisp (Clojure).

I have my Eca-setup shared at GitHub, and have also some contributions to the Eca plugins repository.

Human Driven Development

With this setup, the human reviewing can happen in real time, and doesn't have to wait until the end where the amount of code too often is quite overwhelming. The human developer (that's me) can quickly act when noticing that things takes a different route than expected, in a similar way as the stop-the-line principle from the Toyota Way. This is a lean way to reach the end goal quickly: deploying code that is good enough for production and adds value.

I have found that many Agile practices in combination with developer friendly tools fits well with the ideas of Agentic Engineering. Even though I've seen worrying signs of a return of the Waterfall movement.

To summarize: the result of my new Agentic Engineering development-style is that I haven't put my IDE to the side - it's at the very Center of the agentic workflow.

^{Top Photo by me, taken at Åreskutan, Jämtland, Sweden.}

Are we there yet?

Continuing with the work on tooling support for interactive & fun development with Python.

A while ago, I wrote an article about my current attempts to make development in Python more interactive, more "test" driven and more fun.

My North Star is the developer experience in Clojure, where you have everything at your fingertips. Evaluating expressions (i.e. code) is a common thing for Lisp languages in general. I've found that it is sometimes difficult to explain this thing to fellow developers with no experience from Clojure development. The REPL is in most languages an external thing you do in a terminal window, detached from the work you usually do in your IDE. But that's not the case with REPL Driven Development (RDD).

Along the way, I have learned how to write and evaluate Python code within the code editor by using already existing tools and how to configure them. Here's my first post and second post (with setup guides) about it. You'll find information about the basic idea with RDD and guides on how to setup your IDE or code editor for Python development.

Can it be improved?

This has been the way I have done Python development most of the time, described in the posts above. But I have always wanted to find ways to improve this workflow, such as actually see the evaluated result in an overlay right next to the code. Not too long ago, I developed support for it in Emacs and it has worked really well! You can read about it and see examples of it here.

What about AI?

While writing Python code, and doing the RDD workflow, there's often a manual step to add some test or example data to variables and function input parameters. Recently, I got the idea to automate it using an LLM. I wrote a simple code editor command that prompts an AI to generate random (but relevant) values and then populate the variables with those. Nowadays, when I want to test Python code, I can prepare it with example data in a simple way by using a key combination. And then evaluate the code as before.

Are we there yet?

One important thing with RDD, that I haven't been able to figure out until now, is how to modify and evaluate code from an actual running program. This is how things are done in Clojure, you write code and have the running app, service, software constantly being changed while you develop it. Without any restarts. It is modified while it is running. Tools like NRepl does this in the background, with a client-server kind of architecture. I haven't dig that deep into how NRepl works, but believe it is similar to LSP (Language Server Protocol).

The workflow of changing a running program is really cool and something I've only seen before as a Clojure developer. So far I have used IPython as the underlying tool for REPL Driven Development (as described in the posts above).

A solution: the Kernel

In Python, we have something similar to NRepl: Jupyter. Many developers use Notebooks for interactive programming, and that is closely related to what I am trying to achieve. With a standard REPL session, you can add, remove and modify the Python code that lives in the session. That's great. But a sesssion is not the same as an actual running program.

A cool thing with Jupyter is that you can start a Python Kernel that clients can connect to. A client can be a shell, a Notebook - or a Code Editor.

jupyter kernel

By running the jupyter kernel command, you have a running Python program and can interactively add things to it, such as initiating and starting up a Python backend REST service. Being able to connect to the Jupyter kernel is very useful. While connected to the kernel, you can add, remove, modify the Python code - and the kernel will keep on running. This means that you can modify and test your REST service, without any restarts. With this, we are doing truly interactive Python programming. You will get instant feedback on the code you write, by evaluating the code in your editor or when testing the endpoint from a browser or shell.

"Dude, ever heard about debugging?"

Yes, of course. Debugging is closly related to this workflow, but it is not the same thing. Debugging is usually a one way flow. A timeline: you run code, pause the execution with breakpoints where you can inspect things, and then continue until the request is finalized. The RDD workflow, with modifying and evaluating a running program, doesn't have a one-way timeline. It's timeless. And you don't add breakpoints.

REPL Driven tooling support

I have developed tooling support for this new thing with connecting to a Jupyter Kernel, and so far it works well! Python is different from languages like Clojure: namespaces are relative to where the actual module is placed in the folder structure of a repo. This means that those connected to a Kernel need to have the full namespace (i.e. the full python path) for the items to inspect. This is what I have added in the tooling, so I can keep the RDD flow with the nice overlays and all.

I am an Emacs user, and the tooling I've written is for Emacs. But it shouldn't be too difficult to add it to your favorite code editor. Have a look at the code. You might even learn some Lisp along the way.

UPDATE: I have recorded a very much improvised video (with sound) explaining what is happening and how you start things up.

Resources

• The Emacs configuration, with RDD tooling support.

^{Top Photo by David Vujic}

Introducing python-hiccup

"All you need is list, set and dict"

Write HTML with Python

Python Hiccup is a library for representing HTML using plain Python data structures. It's a Python implementation of the Hiccup syntax.

You create HTML with Python, using list or tuple to represent HTML elements, and dict to represent the element attributes. The work on this library started out as a fun coding challenge, and now evolving into something useful for Python Dev teams.

Basic syntax

The first item in the list is the element. The rest is attributes, inner text or children. You can define nested structures or siblings by adding lists (or tuples if you prefer).

["div", "Hello world!"]

Using the html.render function of the library, the output will be HTML as a string: <div>Hello world!</div>

Adding id and classes:["div#foo.bar", "Hello world!"]
The HTML equivalent is: <div id="foo" class="bar">Hello world!</div>

If you prefer, you can define the attributes using a Python dict as an alternative to the compact syntax above: {"id": "foo", "class": "bar"}

Writing a nested HTML structure, using Python Hiccup:
["div", ["span", ["strong", "Hello world!"]]]
The HTML equivalent is:
<div><span><strong>Hello world!</strong></span></div>

Example usage

Server side rendering with FastAPI

Using the example from the FastAPI docs , but without the inline HTML. Instead, using the more compact and programmatic approach with the Python-friendly hiccup syntax.

from python_hiccup.html import render

from fastapi import FastAPI
from fastapi.responses import HTMLResponse

app = FastAPI()


def generate_html_response():
    data = ["html",
           ["head", ["title", "Some HTML in here"]],
           ["body", ["h1", "Look ma! HTML!"]]]

    return HTMLResponse(content=render(data), status_code=200)


@app.get("/items/", response_class=HTMLResponse)
async def read_items():
    return generate_html_response()

PyScript

Add python-hiccup as any other third-party dependency in the package.toml file: packages = ["python-hiccup"]

Write the HTML rendering in your PyScript files:

from pyweb import pydom
from python_hiccup.html import render

pydom["div#main"].html = render(["h1", "Look ma! HTML!"])

That's it!

python-hiccup aims to make HTML rendering programmatic, simple and readable. I hope you will find it useful. The HTML in this blog post was written using python-hiccup.

Resources

• python-hiccup at GitHub
• PyScript and python-hiccup example

^{Top photo by Susan Holt Simpson on Unsplash}

The last Python Architecture you will ever need?

Should features live in several independent Microservices, or is a Monolith the way to go? What kind of setup is the best one for the code in a repo? Organizing is difficult.

"... I'll just put it in a utils folder for now ..."

There's a thing called Polylith. I've written about different aspects of it before. Polylith is an architecture (and a tool) focusing on the Developer and Deployment experience. It is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk.

The Polylith Architecture is a fresh take on how to share code, and offers a nice and simple solution to the Monolith vs Microservice tradeoffs. In addition to that, it is a good fit for functional programming. Some time ago, I got an idea: how about bringing a couple of the good things from there to here? So I developed something that I believe could be useful for many Python teams out there.

I already released a preview in early 2022, it was missing some essential features and the great developer experience wasn't really there yet. Also, I had little knowledge about how to package Python apps & libraries, but have learned a lot since then.

Today, I believe the Python tools for the Polylith Architecture is ready to use in the Real World. You will find version 1 on PyPI.

A couple of Poetry plugins

I have developed it as two different Poetry plugins. One of them - the Multiproject plugin - adds support for workspaces to the popular Poetry tool, by adding a new command called build-project.

The second one - Polylith plugin - adds useful tooling support for the Polylith Architecture itself. With the tooling, you can add components & projects in a simple way and also keep track of what's happening in the workspace.

Have a look at the repo and the docs.

An Architecture well suited for Monorepos

Polylith is using a components-first architecture. The components are building blocks, very much like LEGO. The code is separated from the infrastructure and the building of artifacts. This may sound complicated, but it isn't.

In a way, it is about:

1. thinking about code as LEGO bricks, that can be combined into features.
2. making it easy to reuse code across apps, tools, libraries, serverless functions and services.
3. Keeping it simple.

Have a look at these introductory videos, describing Polylith in Python and the tooling support:

Python with the Polylith Architecture

The Poetry Polylith Plugin

Give it a try! I would love to hear you feedback.

^{Top photo by frank mckenna on Unsplash}

A fresh take on Monorepos in Python

"... What if we had Polylith in Python ..."

For software development in general, it seems to be an ongoing trend towards using Monorepos. You know, organizing code in one big single repo. On the other hand, it seems to be a general trend in going the opposite way too. Developing features into many small repos.

I think that the JavaScript and Python communities in particular, are in favor of the latter approach. Developing features in isolation - in separate repos - and publish versioned libraries to package repositories (such as npm and pypi). That has potential to introduce some headache: versioning, backwards compatibility and keeping dependencies up to date. Probably also duplication of common source code and repeating the deployment infrastructure. You can solve those kind of issues, by using a Monorepo.

🔥 Monorepos

There are already solutions out there on how to develop software in Monorepos. It is my impression that they are mainly about solving the deployment experience. Which is good, because it is a huge problem in a lot of code bases out there. What about the Developer experience?

In the Clojure community, we have a thing called Polylith. It is an architecture, (including a tool) that is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk. The Polylith architecture is focusing on making both the Developer & Deployment experience great. There is also a very nice tool to help you get started with Polylith. Helping you create & keeping track on components, projects and much more.

🤔 So, what is Polylith?

From the official docs:

"... Polylith is a software architecture that applies functional thinking at the system scale. It helps us build simple, maintainable, testable, and scalable backend systems. ..."

A Polylith code-base is structured in a components-first architecture. Similar to LEGO, components are building blocks. A component can be shared across apps, tools, libraries, serverless functions and services. The components live in the same repository; a Polylith monorepo. The Polylith architecture is becoming popular in the Clojure community.

🐍 What about Python?

Again, I find myself copying some existing Python code - a small module creating a "logger" - into a new repo. I think it’s the third time I do this. The logger code is too tiny to even bother packaging as a library, but still very useful in many code projects.

My mind is wandering. I'm daydreaming:

"What if we had Polylith in Python. I would just simply reuse an existing component."

✨We have Polylith in Python now✨

Porting Polylith to Python

A couple of weeks ago, I decided to give porting Polylith to Python a try. I quickly realized that there are some fundamental differences between Python and Clojure to be aware of when implementing an architecture (and developing a tool) like this.

Especially when it comes to namespacing and packaging. Python code is based on the concept of modules. Also, Python code is not compiled into a binary.

Short note on modules, packages, namespaces & libraries

In Python, a file is a module. One or more modules in a folder is a package. If you put a package in a folder, it is now namespaced. One or more packages can also be combined and built into a library to be used as a third party dependency. From my perspective, a Polylith component in Python should be a namespaced package, and not a library.

Almost like Poetry

When trying to figure out how to solve modules, paths & packaging problems I found Poetry. It is a tool that makes Python packaging and dependency management really easy. Out of the box, Poetry supports project based dependencies and references only. But the latest version of Poetry - currently in preview - has support for third party plugins. So I developed a plugin that enables the concept of workspaces, making it possible to reference dependencies outside of a project boundary.

This was the first step. A very important step. Now it became possible to build tooling support for a Python implementation of the Polylith architecture. Earlier this week I released a first version of such a tool. It is a new Poetry plugin, containing the very basic tooling support for the Polylith architecture.

poetry-polylith-plugin

This brand new Poetry plugin will help you create a Polylith workspace and add components, bases & projects to it. Poetry itself will handle the building & packaging. It is still in early development, but you should be able to develop apps, serverless functions and services with it.

As of today, I wouldn’t (at least not yet) recommend building libraries with it, mostly because of how the packaging is done in Poetry & the nature of Python modules. I’ll try to figure out a way to solve this in the future.

There’s also a lot missing & left to do compared to the original poly tool. But hey, you gotta start somewhere 😀

Check out the 10 minute video for a quick overview.

Learning about Polylith

I recommend to read the official docs to learn more about what Polylith is (and what it isn’t) - even though the code examples are about Clojure. You can easily bring the ideas into a Python context, now that we have tooling support for it.

My intentions with this is to introduce the Polylith architecture to the Python community, and I very much hope that it will be useful in your daily work as a developer.

Direct link to the Porting Polylith to Python intro video.

^{Top photo by Katarzyna Kos on Unsplash}

Simple Component Driven ClojureScript

That Component Driven thing

Component Driven is about developing user interfaces and apps in a certain way. In short, it is about grouping relevant code into components, combining compontents into features, combining features into views (or pages) - and combining the views into an app. I don't think it should be that much about the tools.

Just like Test Driven Development, Component Driven Development encourages us to start small. In a way, it also reminds of LEGO when combining building blocks into something useful.

As your app grows and the coding is done deeper into the app structure, developing user interfaces might be more of a challenge. Finding the spot where a new feature is to be added, might require some clicks and navigation. You know, stuff like logging in, a specific app state - or even waiting for data fetched from a third party service.

This is when the idea of Component Driven Development is really useful. To simplify and speed up the development, we need something simple to host our components. A UI that is isolated from any app specific state or dependencies. A place where a component can be rendered without the need to set things up. Having that, it is possible to develop a component in isolation and test it with different input.

Do I need a tool?

Storybook is a tool that will solve all of this, with a bunch of other really nice things like docs, previews, example usage, variants, code snippets and viewports. But it is possible to try out the Component Driven style of creating user interfaces, without going all-in with a new tool.

Start small

I’m suggesting a very simplistic approach, that will enable rendering components in a short-lived isolated view. You throw it away when done.

Here, I’m using shadow-cljs and have defined a new alias that is targeting a browser. I have also set entry points. A setup almost identical to the web app alias. This one will run in a separate port and the output is compiled to a separate public folder. In that one, I have copied the HTML template from the public folder used by the web app.

Click to enlarge the image, or browse an example gist here.

Keep it Simple

The entry point here is a simplistic and simplified version of the one used in the actual web app. But this entry point does only care about the component you are developing right now. Here is where you render and try out a component during development. This is a temporary place to work Component Driven. It’s almost like when evaluating code in a REPL, but with HTML and compiled JavaScript in a browser.

Click to enlarge the image, or browse an example gist here.

Run both aliases. The App in one browser folder, the component in a different one.

The browser is a powerful tool, with nice developer features.

Take it to the next level

At some point, you will probably want something more long-lived than this very simplistic scratch-style approach. That’s where Storybook fits in!

Component Driven ClojureScript with Storybook

With Storybook, you can focus on styling, different viewports and UI events when developing a single component. You write "stories" (i.e. running components in isolation), including documentation and lot of in-browser interactivity. The stories & docs can also be shared, by deploying it as a separate web app.

Recently, I talked about getting started with Storybook at the re:Clojure 2021 conference.

In this less than 25 minutes talk, I demo how you can use Storybook in your ClojureScript app and work Component Driven.

I hope you’ll like it!

Direct link to the Component Driven ClojureScript with Storybook video.

^{Top photo by Brett Jordan on Unsplash}

Can we have that in Python too?

(REPL Driven Development)

REPL Driven what? 😐

REPL Driven Development is about fast feedback loops during development. It is not about typing code into a terminal window. Don’t do that. A Read Eval Print Loop (REPL) is often described as a shell tool that you use to try out a programming language. That’s not what I mean with REPL Driven Development.

What is it then?

With this workflow, the REPL is acting behind the scenes. You don’t have to type code in a shell. Yes, a Read Eval Print Loop process is running, but in an interactive mode. You do all the work in your code editor. That’s where you have autocomplete, syntax highlighting, your favorite color theme and everything.

⚡ The Workflow ⚡

The feedback loop is fast in REPL Driven Development. You evaluate code blocks and functions as soon as you have typed them. The result of an evaluation is printed out on your screen. A common thing with REPL Driven Development is to write some short lived code snippets next to the actual code that you are developing. With this, you are able to quickly run a function with some test data as input and verify the output.

This workflow is similar to Test Driven Development (TDD), but the feedback loop is much faster. After a while, you might discover that those code snippets should actually be a proper unit test. Well, wrap the code in a test function, move it to a separate file and your’e all good.

So, REPL Driven Development is lazy TDD?

Yeah, that’s probably right. Another way of looking at it might be that it is Test Driven Development Deluxe.

The Interactive REPL is an essential tool in the Clojure & ClojureScript ecosystems. It is a superpower I’ve not seen before anywhere else. Once learned, it is something you’ll want in other languages too. But is that even possible, outside of a Lisp environment?

Evaluating vars, state and functions in real time with ClojureScript

Can we have that in Python too?

I think I have found something similar to the Clojure way of writing software, that works really well. You can pass an entire buffer, a selected region or a function from the code editor into a running IPython process, that quickly evaluates the code and prints the result. I have replaced the standard Python shell with IPython. I’ll explain why.

When evaluating code, IPython will output the result while the cursor remains in the code editor. That is what I mean with fast feedback loops. Evaluated functions, variables and imports can also be redefined - without having to restart anything. I’ve configured IPython to auto-reload sub modules when changed, to avoid having to restart the process. That feature is essential for this kind of workflow and the main reason why I’ve switched out the standard Python shell.

REPL Driven development in Python

It would be really cool to have the output pop up in the editor, right next to the actual code (as when evaluating Clojure or ClojureScript). I think that’s mainly a tooling thing, currently not available.

Also, I haven’t yet figured out how to connect to an actual running Python program and redefine functions or variables while the program is running (as you do in Clojure).

However, IPython is closely related to Jupyter, often referred to as an interactive notebook. There’s a thing called the kernel, handling code evaluation and returning result to connected clients. Sounds a bit like NRepl & Cider! I should probably dig deeper into and learn more about the possibilities with Jupyter.

Joyful Python

To summarize, even though there are some limitations, I think the REPL Driven Development workflow makes writing Python code joyful. The fast feedback loop is a great thing to have at your fingertips. Just like TDD, REPL Driven Development can be very helpful when writing testable, simplistic and functional code.

And it is fun too. 😁

Update: have a look at this post of mine from 2022, for more info about how to setup your code editor. You'll also find a 5-minute video in there about this style of development.

^{Top photo by Hitesh Choudhary on Unsplash}

ClojureScript. Amplified.

"… we're going to build a web app with storage, authentication and everything …"

But how?

My favorite building blocks 🤩

I have tried out some tools that I find very useful. As you may have guessed, the code is developed with ClojureScript. A language I appreciate more each day. Developing an app, without the Interactive REPL? That would be taking a huge step backwards.

I have found that Material-UI works really well in the development of the app I'm working on. The components library cover many usage scenarios, it is well documented and there's a lot of code examples to learn from.

Recently, I added Storybook and really like the idea of developing user interfaces according to Component Driven Design. It reminds me of how I experiment with code using Test Driven Development. Just like TDD, I think tools like Storybook and Devcards changes the way we develop, to the better. Start small & build quality in.

Let's begin with a building block from my favorites list: AWS Amplify.

A basic setup for AWS Amplify & ClojureScript

The official AWS Amplify documentation is all about JavaScript. In this post, and in this example code repo, you will find a ClojureScript setup that I think will get you up & running quickly. First, set up AWS Amplify by following the first parts of the Official getting started guide. You will be creating an AWS account and installing a CLI to initialize your setup.

If want more details, have a look at Hey Webpack, Hey ClojureScript. It's a post describing a ClojureScript & Amplify specific setup that is used in the example repo.

Start coding, but where? 🤔

A pitfall might be where to place the auto generated client code from Amplify. I usually put files like the aws-exports.js at the same level as the root namespace.

In the example code repo, the ClojureScript source code lives in the src/main folder. That's also where I've added the aws-exports.js file.

Configure Amplify in the init function of your app.

Login & Logout with ClojureScript & the Amplify CLI

Adding authentication can easily be done by using the Amplify CLI.

From the docs:

"... The Amplify CLI supports configuring many different Authentication and Authorization workflows, including simple and advanced configurations of the login options, triggering Lambda functions during different lifecycle events, and administrative actions which you can optionally expose to your applications ..."

Use the Auth features in Amplify UI

There's a very useful Higher-Order React Component (aka HOC) in the AWS Amplify UI Library, that will render login and signup views.

I've chosen this less-lines-of-code approach by using the withAuthenticator HOC. To customize things like signup fields or styling, there is also a fully customizable Authorization component available in the Amplify UI library. Or you can use your own components.

The views entry point.

The passed in component is wrapped in the withAuthenticator HOC.

If you already have browsed the code in my example repo, you may have noticed the Reagent reactify-component and as-element functions. These functions are needed when passing components between ClojureScript and JavaScript.

Read more about React Components, elements and instances and how the interop is done with Reagent.

Create, read, update and delete

So far, we have used built in features that will render UI views and handle all auth communication with a backend service. To store, read and update data, we can use the Amplify API feature. Use the CLI to create a backend:

I have chosen GraphQL in my example code, but there's also a REST API option. When using GraphQL, all you need to do is to define data models. Amplify will create the backend infrastructure according to your models. In a matter of minutes, there are AWS AppSync endpoints & DynamoDB tables created for your app.

GraphQL

Here's a GraphQL model. An authenticated user should be able to add data to the profile, like a summary or an "About me" text. The user will be able to create, delete and update the data in the profile. Other authenticated users can read the data when they visit the user's profile.

The schema files belongs to the backend, and lives in the amplify folder of your repo.

async ClojureScript

The Amplify Client Library handles the communication with the backend services. All calls are asynchronous. In ClojureScript, you can use core.async. I haven't yet dig into the core.async stuff, so I'm using Promises here. I think it's a quite straight forward approach.

The data going through the wire is JSON. To make the developer experience better, I think it might be a good idea to add conversion functions to-and-from Clojure data structures. Like this one, converting user settings JSON to Clojure data:

Read more about the ^js type hints in ClojureScript.

Finally: Ship it! 🚢

Here's an excerpt from an amplify.yml that defines the build steps. If you add a definition file in your repo, AWS Amplify will use it in the CI/CD process.

This is the amplify.yml that I use in my example code repo.

There's more details at the official AWS docs about build settings and about amazon-linux-install.

Add UI building blocks to the amplified app

Now we have our amplified ClojureScript app. Let's add UI building blocks from Material-UI, a library that is very popular in the JavaScript & React ecosystem. Here, I'm using it via a ClojureScript library called reagent-material-ui. The library makes it possible to write components using Clojure data structures. You can read more about Material-UI and ClojureScript in my previous post Material Design in a Functional World.

Totally unnecessary css styling used here, but why not?

Testing the UI building blocks with ✨ Storybook ✨

I find Storybook very useful when experimenting with components and user interfaces in isolation. Being able to focus on one specific thing. There's many different aspects of developing an app, and it is sometimes difficult to handle all of them at the same time.

Storybook is a place for trying out things like css, styling, different viewports and UI events. You won't try out the AWS Amplify code here. The Interactive REPL is a better tool for that.

You'll find more about Storybook in Component Driven ClojureScript with Storybook.

Finally, check out my GitHub repo for ideas on how to create apps with the building blocks AWS Amplify, Material-UI, Storybook and ClojureScript.

That's a combination I'd like to call ✨ClojureScript. Amplified. ✨

^{Top photo by Drew Patrick Miller on Unsplash}

Material Design in a Functional World

"... Material is a design system created by Google to help teams build high-quality digital experiences for Android, iOS, Flutter, and the web ..."
^{from material.io}

ClojureScript & React is A 💕 Story

With Clojure, you can write functional & minimalistic code. With ClojureScript and Reagent, you can write functional & minimalistic React components. By adding re-frame to it, we also have a powerful way of handling state changes and triggering events.

The code you write in ClojureScript is instantly compiled to JavaScript React components that runs in your browser. The entire JavaScript ecosystem is available with this kind of setup.

"... React components for faster and easier web development. ..."
^{from material-ui.com}

Material-UI

Material-UI is a very popular React framework. And we can use it with ClojureScript. But the JavaScript interop can sometimes be bit of a hassle. Fortunately, there is a great ClojureScript library called reagent-material-ui that simplifies this a lot.

One thing to notice is that the Material-UI components use React Hooks heavily to inject styling and theming into components. This is something to think about when using Reagent. Basically, just use the functional component syntax [:f> my-component] and you’re all good.

Have a look at the reagent-material-ui docs about common pitfalls in the React/Reagent interop and a fully working example in this GitHub repo.

Storybook: a playground for components

Here’s me playing around with the Card component, using ClojureScript and Storybook. I describe the Storybook integration in detail in this post.

The source code is available at GitHub.

No copy-paste?

I guess it can be a bit overwhelming for a Clojure developer to look at example code from the Material-UI site, with JavaScript/JSX syntax that is quite verbose (you’ll be even more overwhelmed by the TypeScript examples).

A JavaScript developer would probably copy-paste the example code and make some adjustments. We can’t do that. But it is worth the effort writing the code from scratch when discovering that the result is way more simplistic than the original. The amount of code written is usually half the size of the original JavaScript code. Yeah, that’s Clojure!

ClojureScript to the left, JavaScript to the right. Less is more.

Check out my GitHub repo with examples on how you can implement Material-UI components in ClojureScript, with hooks and styles - while having fun writing functional code.

^{Top photo by Thierry Lemaitre on Unsplash}

Component Driven ClojureScript with Storybook

Component Driven?

Start small. Develop components in isolation. Don’t forget to pause & reflect on the code you’ve written. Can it be split into several small components?

✨ When you’re ready, combine your components to create features.

✨ Add the features to a view.

✨ Add the view to your app.

You can read more about Component Driven User Interfaces here.

Storybook

I’ve just begun to use Storybook and currently learning about it. What I especially like so far, is that the tool encourages you to write UI components that are independent … and functional?

This makes it possible to develop user interfaces in isolation. Just pass data and actions as parameters into your components. Keep it simple. This way of working reminds me a lot of Polylith, an architecture I have used in backend focused Clojure projects and that I enjoy a lot.

ClojureScript & Storybook?

You will find a fully working example in this repo. I have added Storybook to a ClojureScript web app, using shadow-cljs, with additional npm scripts located in package.json.

Update: a new version of shadow-cljs supports ns-regexp for the npm-module target. No need to manually add new stories!

The code in my GitHub repo is a continuation of the work I’ve described in an earlier post, about combining shadow-cljs with Webpack and AWS Amplify.

The Clojure Interactive REPL and Storybook working well together.

No JavaScript quirks?

Well, yeah some. The JavaScript ecosystem is evolving fast, tools are updated with breaking changes and sometimes that causes compatibility issues. In this case it is Storybook and Webpack 5. But I have solved it with a few extra rows of configuration. Don’t worry.

An example of a Storybook story written in ClojureScript

Have a look at the fully working example at GitHub.

Many thanks to Shaolang Ai, that wrote this blog post about shadow-cljs and Storybook that I’ve learned a lot from.

^{Top photo by Etienne Girardet on Unsplash}

Hey Webpack, Hey ClojureScript

✅ ClojureScript is great.

✅ AWS Amplify is great.

🤔 Why are there errors in my browser console?

ClojureScript. Amplified.

Some time ago, I decided to give AWS Amplify a try for a new project. With Amplify, you can quickly create and deploy web apps and AWS backend services by using your command line.

Even though the Amplify code examples and docs are all about JavaScript, I wasn't too keen on abandoning my favorite REPL powered language: Clojure. 🤩

So, I decided to combine two great things: AWS Amplify & ClojureScript!

Amplifed, yes. But with 🔔.

I have learned a lot from this blog post by Robert J Berger about how to setup AWS Amplify with ClojureScript. New versions of the AWS Amplify libraries are released regurarly. The current Major version include breaking changes that probably will cause ClojureScript web apps to raise errors and stop working.

Until recently, I have had to rollback updates and use earlier versions of the Amplify libraries. Nothing wrong with that. But my inner Alarm Bells have been ringing constantly since noticing the word legacy in the official Amplify docs for those library versions (the aws-amplify-react library and the Amplify Auth feature in particular).

The newer Amplify libraries rely on Webpack specific features under the hood. The example setup described in the official Amplify docs include using react-scripts, that (if I'm not mistaking) will package code with Webpack. That kind of setup is not supported by shadow-cljs, a tool that is very common in ClojureScript projects.

Breaking things

In addition to being stuck using legacy code, the app will break each time I add a third party library from npm. I have learned that it is caused by not having a fixed version for amplify libraries defined in my package.json file. A quick fix is to set all AWS Amplify libraries to a fixed version and use that one forever. A version that is becoming more legacy for each day. 🔔🔔🔔

Fix the broken things

The main issue with AWS Amplify libraries, ClojureScript and shadow-cljs is described in this GitHub thread. There is also a suggested solution hinted by Thomas Heller, the creator of shadow-cljs.

A solution: shadow-cljs & Webpack working together

I have created an example app using AWS Amplify and ClojureScript in this repo. Here, shadow-cljs will only create a target JavaScript file containing requires to all of the third party JavaScript dependencies.

Webpack will use that target to create a JavaScript bundle. In a continuous deploy environment, this can easily be automated by executing two scripts instead of one (the shadow-cljs build and webpack).

This solution makes it possible to use the very latest versions of the AWS Amplify libraries, without any css quirks or being forced to use legacy code. Hey Webpack, Hey ClojureScript!

Nowadays, my inner alarm bells are nice & quiet.

^{Photo by Kamil Pietrzak on Unsplash}

Almost like Clojure

Practicing different programming languages has given me new insights in programming concepts, code styles and tools. My current crush is Clojure. I like the functional and data oriented style of it, the simplicity and minimalism. I think my favorite thing with Clojure, though, is how the REPL is used.

“What’s so cool about that repple thing?” 🧐

Testing at your fingertips 🚦

When using a code editor powered by a REPL, you can instantly evaluate variables, code blocks, functions or the entire file, just by a hitting a key combination. The result pops up right next to the cursor. This means that testing the code you currently write is at your fingertips. You are practicing a Deluxe version of Test Driven Development.

No copy-paste here 👮

It is not necessary to copy-paste code snippets into the actual REPL window or in a shell. In fact, you shouldn’t touch the REPL window at all. Because you don’t want to miss out on all the good stuff like autocomplete, hints and smart navigation that you have in your favorite editor.

For Clojurians only? 💭

I really would like to have a similar workflow in other languages too. It turns out there are some pretty good tools out there, the ones I’ve tried out so far are for JavaScript and ✨Emacs ✨.

Interactive JavaScript development 😍

Here’s me playing around with the node-zookeeper code. The cursor never leaves the editor window, the evaluation result is printed out in the Node.js REPL window to the right. More verbose than I’m used too in Clojure, but I think it works well.

Trying out code and ideas like this is a very nice workflow. I think you will find less need for debugging or console.log statements.

Keep on learning 📚

Learning a new programming language or tool can be really difficult, but also fun when the pieces finally start coming together. The knowledge gathered is something you can bring with you to other areas.

<sup>Have a look at my post about Emacs and my current configuration. The Node.js REPL feature is made possible using the js-comint package.

Photo by Marc-Olivier Jodoin on Unsplash</sup>

Software as building blocks

There seems to be an ongoing trend in software development towards using monorepos. This trend is something I have seen especially in the Clojure community.

Polylith - a monorepo architecture

I like the way Polylith solves how to work with code using a components-first architecture. Similar to LEGO, components are building blocks. A component can be shared across apps, tools, libraries, serverless functions and services.

Read more here: Polylith gitbook

The last architecture you will ever need *

From the Polylith docs:

"... Polylith is a software architecture that applies functional thinking at the system scale. It helps us build simple, maintainable, testable, and scalable backend systems. ..."

Okay, backend systems. What about frontend systems? 🤔

I want to Polylith all the things

Is it possible to use the Polylith architecture for a code base that includes web apps? This is something that I have wanted to find out.

Here's my example repo.

In this repo, I’ve added backend Clojure code, frontend ClojureScript and also some glue in between in the form of cljc files. Cljc is Clojure code that can be consumed by both frontend and backend code. This makes it possible to share code across Clojure and ClojureScript, building things just like with LEGO bricks and baseplates.

All the things?

I'll leave the question if ClojureScript and Clojure really should live in the same ecosystem unanswered and hope to get feedback from you. In my example repo, I have put all components in the same place. Should the building blocks be separated somehow, or is it good enough to have both LEGO and DUPLO in the same box? What are your thoughts about it?

Do we still have the Polylith one-REPL experience?

Well, we can have a two-REPLs experience. One for Clojure, that run on top of the JVM, and one for ClojureScript on top of JavaScript. Running both will make REPL driven backend development and Interactive Web Development possible.

You can add and use new ClojureScript components while the REPL is running. Create the namespace and evaluate the function.

There is one thing that I have no solution for (yet). When creating a new ClojureScript component and evaluating the entire namespace at once, I get a compilation error in the ClojureScript REPL: file not on classpath. The ClojureScript REPL have to be restarted to reload new source paths.

But don't worry, you can still evaluate the individual functions in the namespace and they will be loaded as expected in the ClojureScript REPL.

Tooling support

Polylith has a very nice and useful tool to support creating building blocks and to verify the setup. You can create components, bases and projects - as long as it is Clojure. For ClojureScript, you will have to create components manually.

If you are lazy, like me, just create a component with the poly tool as you would for Clojure, and simply rename the file extension to cljs or cljc afterwards.

Editor support?

Your editor most likely has support for running both Clojure and ClojureScript simultaneously. Emacs is my favourite editor. Start the REPLs with the cider-jack-in-clj&cljs command and you're ready to go!

<sup>Two REPLS running (even though the Cider splash message is confusing)

* The quote is from Joakim Tengstrands and Furkan Bayraktars talk about Polylith at the FuncProg Sweden 2020 meetup.

Photo by Amélie Mourichon on Unsplash</sup>

Wake up, sleepy lambda

AWS Lambda functions with painfully slow start up times is a problem. But there's hope.

Clojure anywhere

After going through some of the 12 Stages of learning Clojure, I have found this Lisp style language to be very nice and it has become my favorite programming language.

Some of the nice things with Clojure are that data is immutable, the REPL is like magic and the code you write looks minimalistic. Also, you can run Clojure code almost anywhere: as backend services, web frontends and even in shell scripts.

Even in Lambda functions?

I found out that it is not that difficult to make Clojure code to also run in AWS Lambda. The code can live in a Java runtime. Lambda events will be routed to a handler function written in Clojure, when the namespace implements a Java Request handler class (here's an example). Yes, there is some interop needed at the entry point of the Lambda code to make it work. But don't worry.

In addition to the Java interop, the source code should also be ahead-of-time compiled and packaged. I've used the uberdeps library that will make the process smooth when using tools.deps.

If you are not familiar with the Java lingo (like me, I have a background in Node.js, frontend and .NET), words like jar, AOT and even Java can be intimidating. I guess it is possible to sidestep all of this by writing the Lambda function in ClojureScript and run it in a Node.js runtime. But I don't want to opt out of the rich ecosystem of Clojure libraries built for the server side.

Sleepy lambda, slow cold starts

It seems that a Lambda running in a Java or .NET runtime often has painfully slow start up times. Setting a timeout of 3 seconds is probably not even enough. A simplistic solution to this problem is to use the AWS provisioned concurrency with a Lambda alias. No code changes required, only configuration and money. I wanted to find out if there are other ways to solve this problem.

What about GraalVM?

I found this great post about Clojure and Lambda written by Esko Luntola. Code that is compiled with GraalVM and running in Docker, solving the issues with slow cold starts and even makes requests in general super fast. Wow!

However, I haven't succeeded in going through the steps described in the post and am stuck in build failures that I don't know how to solve - yet. But I will try this approach some more. Even though it requires some initial setup with configs and Docker containers, this seems to be the way to go.

What about GraalVM in a custom runtime?

When digging deeper in how to run code in AWS Lambda, I found out that you can create your own runtime. To me, this approach looks simplistic and straight forward.

In this repo, I have written a Hello World example with:

• a custom runtime (the file called bootstrap) written in bash (grabbed from the AWS Official docs with some additional error handling).
• Clojure code, with a main function as the entry point. It is not neccessary to implement the Java Request handler class, and the main function returns data via standard output.

^{Input args as a JSON string}

The Clojure code is compiled, and built with GraalVM by using the Native Image feature. Have a look at the Makefile for details.

^{compile and build with GraalVM Native Image}

I've tried to keep things simple and followed the guides at clj-graal-docs and watched Michiel Borkents excellent beginners guide to GraalVM on YouTube.

The custom runtime and the function can be deployed separately. You can reuse the same runtime for several Lambda functions, by creating a Lambda Layer in AWS. The function code can be deployed directly, just upload the zipped file to AWS Lambda.

I like this approach, it works fine in my simplistic hello world example. But I haven't tried it in a real-world scenario. When going beyond an experiment, I think there might be additional resource configuration flags to Native Image required. Probably the setting ReflectionConfigurationFiles.

The tradeoffs?

From what I can see in my totally non-scientific weekend experimental testing is that the code running in my Custom Runtime has a cold start of somewhere between 100 and 300 milliseconds. Good enough. When warmed up, requests are processed between 15-30 milliseconds. Not bad.

When comparing with my other example lambda that is running in a Java runtime (with cold starts usually taking over 2000 milliseconds), the Custom Runtime with GraalVM is way faster.

But once warmed up, the Java runtime is actually super fast, with duration times between 1 - 30 milliseconds. Also, I haven't yet solved how to build all of this in a CI/CD setup. The code was built with GraalVM locally on my machine.

I would very much appreciate your input on the experiments shared in this blog post.

^{Photo by Abdelrahman Hassanein on Unsplash}