The tools of an Agentic Engineer

A lot of great things have origins from the 1970s: Hip Hop redefining music and street culture, Bruce Lee was taking Martial Arts to the next level and the initial development of something called editor macros (also known as Emacs) was happening. I was born in that decade, but that's purely coincidence.

My choice of primary development tool since a couple of years back is that editor from the seventies. It is my choice of development for Python, JavaScript, TypeScript and Lisp dialects such as Clojure and elisp. And today, as an agentic engineer, it turned out to be a great choice for this kind of software development too. With the rise of various CLI, TUI & Desktop based tools for AI development, it would be reasonable to think that this ancient code editor would become obsolete - right?

Not if you knew about the innovative Emacs community. It is driven by passion, support from the community itself and Open Source. These components are usually more resilient and reliable long term than the VC driven startup culture. Emacs is part of the greater Lisp community, where a lot of innovations in general take place. The Clojure community is cutting edge in many aspects of software development including AI.

More Agents

One thing that I have noticed lately is that the more I get into Agentic Engineering, the more I use Emacs. When the focus has shifted from typing code to instruct and review, I have found use of Emacs powers I haven't really needed until now. Tools like Magit (git) and I'm also learning more about the powerful Org Mode. I didn't care that much about Markdown before, but now it is an important part of the development itself. So I just configured my Emacs to have a nice-looking, simple and readable markdown experience.

"More Agentic Engineering, More Emacs"

With Emacs, I use a great AI-tool called Eca and with it I am not limited to any specific vendor for agentic development. Vendor lock-in is something I really want to avoid. The combination of Eca and the power tools mentioned before, makes a very nice Agentic Engineering toolset. Eca is actively developed and has a lot of useful features and a very nice developer experience. It supports standards like AGENTS.md, commands, skills, hooks, sub-agents and use a client-server setup in the same way as the language server protocol. It is Open Source and not only for Emacs. Have a look at the website for support of your favorite editor or IDE. By the way, Eca is developed in Lisp (Clojure).

I have my Eca-setup shared at GitHub, and have also some contributions to the Eca plugins repository.

Human Driven Development

With this setup, the human reviewing can happen in real time, and doesn't have to wait until the end where the amount of code too often is quite overwhelming. The human developer (that's me) can quickly act when noticing that things takes a different route than expected, in a similar way as the stop-the-line principle from the Toyota Way. This is a lean way to reach the end goal quickly: deploying code that is good enough for production and adds value.

I have found that many Agile practices in combination with developer friendly tools fits well with the ideas of Agentic Engineering. Even though I've seen worrying signs of a return of the Waterfall movement.

To summarize: the result of my new Agentic Engineering development-style is that I haven't put my IDE to the side - it's at the very Center of the agentic workflow.

^{Top Photo by me, taken at Åreskutan, Jämtland, Sweden.}

Are we there yet?

Continuing with the work on tooling support for interactive & fun development with Python.

A while ago, I wrote an article about my current attempts to make development in Python more interactive, more "test" driven and more fun.

My North Star is the developer experience in Clojure, where you have everything at your fingertips. Evaluating expressions (i.e. code) is a common thing for Lisp languages in general. I've found that it is sometimes difficult to explain this thing to fellow developers with no experience from Clojure development. The REPL is in most languages an external thing you do in a terminal window, detached from the work you usually do in your IDE. But that's not the case with REPL Driven Development (RDD).

Along the way, I have learned how to write and evaluate Python code within the code editor by using already existing tools and how to configure them. Here's my first post and second post (with setup guides) about it. You'll find information about the basic idea with RDD and guides on how to setup your IDE or code editor for Python development.

Can it be improved?

This has been the way I have done Python development most of the time, described in the posts above. But I have always wanted to find ways to improve this workflow, such as actually see the evaluated result in an overlay right next to the code. Not too long ago, I developed support for it in Emacs and it has worked really well! You can read about it and see examples of it here.

What about AI?

While writing Python code, and doing the RDD workflow, there's often a manual step to add some test or example data to variables and function input parameters. Recently, I got the idea to automate it using an LLM. I wrote a simple code editor command that prompts an AI to generate random (but relevant) values and then populate the variables with those. Nowadays, when I want to test Python code, I can prepare it with example data in a simple way by using a key combination. And then evaluate the code as before.

Are we there yet?

One important thing with RDD, that I haven't been able to figure out until now, is how to modify and evaluate code from an actual running program. This is how things are done in Clojure, you write code and have the running app, service, software constantly being changed while you develop it. Without any restarts. It is modified while it is running. Tools like NRepl does this in the background, with a client-server kind of architecture. I haven't dig that deep into how NRepl works, but believe it is similar to LSP (Language Server Protocol).

The workflow of changing a running program is really cool and something I've only seen before as a Clojure developer. So far I have used IPython as the underlying tool for REPL Driven Development (as described in the posts above).

A solution: the Kernel

In Python, we have something similar to NRepl: Jupyter. Many developers use Notebooks for interactive programming, and that is closely related to what I am trying to achieve. With a standard REPL session, you can add, remove and modify the Python code that lives in the session. That's great. But a sesssion is not the same as an actual running program.

A cool thing with Jupyter is that you can start a Python Kernel that clients can connect to. A client can be a shell, a Notebook - or a Code Editor.

jupyter kernel

By running the jupyter kernel command, you have a running Python program and can interactively add things to it, such as initiating and starting up a Python backend REST service. Being able to connect to the Jupyter kernel is very useful. While connected to the kernel, you can add, remove, modify the Python code - and the kernel will keep on running. This means that you can modify and test your REST service, without any restarts. With this, we are doing truly interactive Python programming. You will get instant feedback on the code you write, by evaluating the code in your editor or when testing the endpoint from a browser or shell.

"Dude, ever heard about debugging?"

Yes, of course. Debugging is closly related to this workflow, but it is not the same thing. Debugging is usually a one way flow. A timeline: you run code, pause the execution with breakpoints where you can inspect things, and then continue until the request is finalized. The RDD workflow, with modifying and evaluating a running program, doesn't have a one-way timeline. It's timeless. And you don't add breakpoints.

REPL Driven tooling support

I have developed tooling support for this new thing with connecting to a Jupyter Kernel, and so far it works well! Python is different from languages like Clojure: namespaces are relative to where the actual module is placed in the folder structure of a repo. This means that those connected to a Kernel need to have the full namespace (i.e. the full python path) for the items to inspect. This is what I have added in the tooling, so I can keep the RDD flow with the nice overlays and all.

I am an Emacs user, and the tooling I've written is for Emacs. But it shouldn't be too difficult to add it to your favorite code editor. Have a look at the code. You might even learn some Lisp along the way.

UPDATE: I have recorded a very much improvised video (with sound) explaining what is happening and how you start things up.

Resources

• The Emacs configuration, with RDD tooling support.

^{Top Photo by David Vujic}

Simple (within parentheses)

• - Hey, what's ClojureScript?
• - It's a thing that compiles Clojure to JavaScript.
• - I get it. What's Clojure?
• - It's a Lisp.
• - ???
• - You know, a functional programming language that uses a lot of parentheses.
• - I … get it? 🤯

Don't worry. In this post, I will share some of the basics that I have learned about Clojure and ClojureScript.

Clojure?

With Clojure, you can write simple, functional and minimalistic code. Some of the cool things with Clojure are:

• data is immutable
• the REPL is a powerful tool
• few keystrokes to make things happen

Clojure runs on top of Java, suitable for all kinds of "backend" things. It also runs on top of JavaScript. Yes, JavaScript - making it possible to develop all kinds of apps, tools, products and websites. There's even an option to run in a .NET runtime, but I haven’t tried out that one yet.

What does it look like?

Even though the syntax may look unfamiliar, you can already see in this silly example the less amount of keystrokes needed using Clojure, compared to JavaScript.

Parenthesis?

Everything you write in Clojure is a list, wrapped in parentheses. That may sound weird. I agree, it is weird. But your eyes will get used to it. In short, the format of writing something in Clojure is:

(operator operands) or even (function arguments)

Here is an example:

... and how it would be written in JavaScript:

If all this is new to you, you might have this expression in your face right now: 😬

That is perfectly normal. I have looked like that a lot during 2020. When digging deeper, you may need to go through The 12 Stages of Learning Clojure.

ClojureScript?

With ClojureScript, you develop web apps and write simple React components. You write code in Clojure, compile it to native JavaScript and run it in your favorite browser. The compilation happens all the time and automagically as you type.

Just like with Java or .NET in the backend, ClojureScript has native access to JavaScript. You can use and interact with your favorite npm packages and React components just as you would with JavaScript.

Reagent and React

I'm going to fast-forward a bit by adding a ClojureScript library called Reagent. With Reagent, you write React components but with Hiccup instead of JSX. Wait, what?

View components are built up using Clojure data structures. The example above creates html elements with Clojure vectors and keywords. This style is called Hiccup. It will save you a lot of keystrokes and you'll have a lot less of scrolling in your code editor.

What about state?

I'll fast-forward again by adding a very nice library called re-frame.

From the re-frame docs:
“... It has reactivity, unidirectional data flow, pristinely pure functions, interceptors, coeffects, conveyor belts, algebraic effects, statechart-friendliness and claims an immaculate hammock conception. All while being both simple and easy. ...” 💥🤯💥

With re-frame, data is stored in one place called the "app db". Your components will fire events and subscribe to changes in the app db. The cool thing is that you write code in a sequential way, making it very readable and easy to understand. However, the syntax of the actual events and subscriptions might take a while to understand, at least if you are new to Clojure in general.

Here, we have a Reagent component that adds a button component, only when the app is disconnected. It is subscribing to changes of disconnected in the app db. React will re-render the components when the app state has changed.

Cool, but can I use my favorite editor?

Yes, most likely. I know there are Clojure plugins for VS Code, Atom, Sublime Text and intelliJ. I use Emacs, because ... I just like it.

There is something important and special going on with developers and their text editors. I think you will be able to find your favorite dev setup. I also think that you will learn a lot of new programming concepts and styles when reading more about this cool language and its libraries.

• Clojure
• ClojureScript
• Reagent
• re-frame

^{Photo by Marcos Paulo Prado on Unsplash}