Showing posts with label Open Source. Show all posts
Showing posts with label Open Source. Show all posts

Saturday, July 4, 2026

Practice what you Preach

I'm the maintainer of the Python tools for the Polylith Architecture, that adds useful tooling support for teams having their code organized according to the Polylith Architecture. So far in 2026, I have made seven Feature releases (i.e. minor versions) and a bunch of smaller ones (i.e. patches).

One of the features that was added this year is the possibility to inspect the coupling between individual parts of the code. In Polylith, code is logically grouped into something called bricks. For Python, these are what we know as namespace packages within the same repo.

Each package has an __init__.py, and you can define the interface of the package there. This is optional, and it is perfectly valid Python to access the modules within a package directly. In Polylith, a stricter approach is recommended: explicitly define the things that should be exposed outside of the package (aka brick).

      from my_package import my_function

      __all__ = ["my_function"]
    

The new feature is exactly about this: visualize and validate the usage of brick interfaces across the repository. With a simple command, you will find out if code is bypassing the explicit interface of a brick. This is still optional, and as long as it is valid Python you can safely ignore this. There's benefits of having code that interacts via explicit interfaces, such as refactoring - it's very easy to move code - and low-risk of breaking things when changing the code within the brick.

      # the existing command visualizing coupling between bricks
      poly deps

      # the addition, that will inspect the brick interfaces
      poly deps --interface

      # or inspect a single brick
      poly deps --interface --brick the_brick
    
Vizualize coupling, bypassed interfaces or circular dependencies.

What about circular dependencies between bricks?

Remember, a brick is a namespace package and that means that it is on a level above the individual Python modules. It can be valid Python to have two packages depend on each other, if the imports are for different modules in the packages. But this is an indication that something might be wrong with the Design. I would even go so far calling it a Bug. Two packages that import code from each other could be refactored into three, where the shared things are in a separate package. This is how the Polylith tool identifies circular dependencies between the bricks. The same command as for validating interfaces is used for this feature (poly deps) and it will inspect the code in the entire repo, to look for any circular brick dependencies. The interface lookups and the circular dependency checks are both internally using the builtin AST parsing.

Control your Agents

Many teams use agents today during development and knowing that agents can produce a lot of code in a fast pace, these two features are very useful if you want to avoid any unexpected side-effects of importing Python code without clear boundaries. Agents can use the poly tool too, just like when doing linting and testing. Static analysis is a great thing to have in the Agentic Engineering toolbox. The Polylith tool includes agent skills, so your agent will understand how to use it.

Practice what I Preach

I knew from before that the source code of Polylith tool bypassed interfaces in a couple of places, but I thought this is fine since it is valid Python. But I realized that my future self will thank me if these risky parts are fixed now and not later. Also, it is good if I actually practice what I preach being the maintainer of the Polylith tool for Python. So I fixed that. In addition, I also learned that two bricks in the repo had circular dependencies caused by a recently added feature. In this case, it was a circle of several bricks that initially made me worried. But the solution was simple: move the parts that caused the circularity into a different brick. I already had an existing brick that made sense to host the couple of Python functions that caused the circular dependency.

After refactoring: no interface violations, no circular dependencies.

Related posts and resources



Top Image generated by Mistral AI with modifications by me.

Sunday, June 21, 2026

An Agent- and Human-Friendly Architecture

"Software Architecture in the agentic era?"

What's needed for an architecture to fit well in the agentic era? Probably many things, but I would say at least simplicity and available context as two very important things to consider. Even if it for sure was not the purpose from the beginning (it was born many years ago), Polylith is about that and is a good choice for Agentic Engineering. That's a nice side-effect of keeping things simple and having all necessary context at your fingertips, which is the guiding star for this software architecture.

A year ago, agent-friendly architectures was not something I had thought about at all. Today, a lot of things has changed. But many ideas about good software remains valid.

If you haven't heard about Polylith, here's an elevator pitch: the main use case is having Microservices in a Monorepo, and share code between the services.

It's an architecture with useful tooling support and great development experience for both humans and agents. All of it Open Source. I'm the maintainer of the tooling support for Python, and teams can use it with their favorite existing tools (such as uv, poetry, pdm, pixi, maturin, hatch). The Polylith tool also includes a set of agent skills, that will add knowledge about Polylith in general and the available commands to your agent. Recently, I also added skills that are specific for migrating a single-repo Python project into a monorepo. I hope that this will help teams to more easily start using this way of structuring code.

The code in a Polylith monorepo is basically building blocks, just like LEGO bricks. Some bricks are small, some are bigger and some are a combination of other bricks. All code lives in a well-structured Monorepo (without symlinks or complex custom setups). The code is organized into smaller reusable parts with clear boundaries between them. The tooling will notify if these boundaries are bypassed or if references are circular.

LEGO bricks and code, what's the connection? In Python, a file is a module. One or more modules in a folder is a package. One or more packages can be combined into a feature. In Polylith, this is called bricks. There are two types of them: components and bases. Components are the main building blocks in Polylith. This is where the business logic lives, the actual features and functionality. Bases are the entry points to your apps or services. A base should ideally be thin, and delegate the business logic to components.

Microservices are great, but the standard setup can introduce new problems: code in many places, duplications, shared code as libraries (that means even more repositories) and different versions of everything. That can be a lot of things to maintain, for agents too.

With a monorepo structured as a Polylith, the agents will have all the necessary context in one single place. It's right there! Agents perform better when things are simple, very much like us humans do. Besides having all needed context, an agent can also use the Polylith tool just like a human would. This will save tokens, and likely speed up the development process even more. The agent skills of the tool will tell the agent how and when to use it. Having all code there at your fingertips is also great for Test & REPL Driven Development, making coding both joyful and interactive.

Link to the Python tools for the Polylith Architecture repo.

Top Photo by Mitchell Hartley on Unsplash

Sunday, June 7, 2026

Refactoring with AI?

With Agentic Engineering there's an opportunity to refactor legacy code, but this aspect of AI isn't that that much talked about and seems to be overlooked by teams out there. So far, the main thing with AI and agents has been to build new things in a fast pace.

After several months of practicing Agentic Engineering, I've realized that code quality - or code health - is more important than ever. Agents get confused by complexity, just like humans. You've probably seen it (I have too) and now there is also research backing it up. Keeping code simple is a good guideline also in the agentic era.

This is where I believe the Polylith Architecture is a good fit, for both humans and agents. The original idea of Polylith was to keep things simple and great Developer Experience for Humans. As a nice side effect, Agents will probably like this setup too. An agent will have the context it needs to build new features from the existing well-organized code in the Monorepo.

Since before, the Polylith tool for Python includes a set of agent skills about the commands and about Polylith in general. But what about migrating an entire existing single-repo project into a Polylith Monorepo? With AI, many steps could be automated here.

I recently added additional skills to the Polylith tooling that are focused on migrating and refactoring an existing Python project into an Agent friendly Monorepo. The migration skills handles the extracting of application code, restructuring and also refactoring of large components into smaller ones.

The agentic flow for the migration of a project is:

  1. discover
  2. analyze-imports
  3. extract-to-base
  4. update-imports
  5. prepare-project
  6. verify-stability
  7. isolate-base-and-big-component
  8. split-big-component
  9. extract-standalone-modules
  10. isolate-shared-and-project-logic
  11. distribute-wiring
  12. split-component-internals
  13. refactor-tests
  14. definition-of-done

Also, the project-specific tooling (such as Project & Dependency management, linting and type checking) is migrated and converted according to what’s already in the monorepo.

Example: you use uv and ruff in the Polylith Monorepo, and migrate a project that uses Poetry and flake8. The migration will convert those to uv and ruff.

I have tested the skills with Claude (Opus 4.7 and 4.8) and Mistral (mistral-large and devstral). One of the tests I ran was converting the FastAPI repo (just for fun and to test the skills out) into Polylith.

my migration prompt

In this repo, there is @./.agents instructions about the Polylith Architecture and the "poly" tool. There is also migration instructions, and those are important here. I want to migrate the project #./project/fastapi according to the migration instructions.

Before running the prompt, I copied the contents of the FastAPI repo into the Polylith projects folder. I also removed the existing .git folder in there, to avoid any confusion during the agentic work.

Hint: the more unit tests the project to migrate has, the better!

The skills are part of the polylith-cli, and you can copy the contents of the .agents file or use a tool like library-skills to add them to your monorepo. Agents are unpredictable and different models behave differently. If you decide to try this thing out, please let me know how the skills works for you.

You'll find info about Polylith for Python here.

Top Photo by Blend Archive on Unsplash

Sunday, March 22, 2026

The tools of an Agentic Engineer

A lot of great things have origins from the 1970s: Hip Hop redefining music and street culture, Bruce Lee was taking Martial Arts to the next level and the initial development of something called editor macros (also known as Emacs) was happening. I was born in that decade, but that's purely coincidence.

My choice of primary development tool since a couple of years back is that editor from the seventies. It is my choice of development for Python, JavaScript, TypeScript and Lisp dialects such as Clojure and elisp. And today, as an agentic engineer, it turned out to be a great choice for this kind of software development too. With the rise of various CLI, TUI & Desktop based tools for AI development, it would be reasonable to think that this ancient code editor would become obsolete - right?

Not if you knew about the innovative Emacs community. It is driven by passion, support from the community itself and Open Source. These components are usually more resilient and reliable long term than the VC driven startup culture. Emacs is part of the greater Lisp community, where a lot of innovations in general take place. The Clojure community is cutting edge in many aspects of software development including AI.

More Agents

One thing that I have noticed lately is that the more I get into Agentic Engineering, the more I use Emacs. When the focus has shifted from typing code to instruct and review, I have found use of Emacs powers I haven't really needed until now. Tools like Magit (git) and I'm also learning more about the powerful Org Mode. I didn't care that much about Markdown before, but now it is an important part of the development itself. So I just configured my Emacs to have a nice-looking, simple and readable markdown experience.

"More Agentic Engineering, More Emacs"

With Emacs, I use a great AI-tool called Eca and with it I am not limited to any specific vendor for agentic development. Vendor lock-in is something I really want to avoid. The combination of Eca and the power tools mentioned before, makes a very nice Agentic Engineering toolset. Eca is actively developed and has a lot of useful features and a very nice developer experience. It supports standards like AGENTS.md, commands, skills, hooks, sub-agents and use a client-server setup in the same way as the language server protocol. It is Open Source and not only for Emacs. Have a look at the website for support of your favorite editor or IDE. By the way, Eca is developed in Lisp (Clojure).

I have my Eca-setup shared at GitHub, and have also some contributions to the Eca plugins repository.

Human Driven Development

With this setup, the human reviewing can happen in real time, and doesn't have to wait until the end where the amount of code too often is quite overwhelming. The human developer (that's me) can quickly act when noticing that things takes a different route than expected, in a similar way as the stop-the-line principle from the Toyota Way. This is a lean way to reach the end goal quickly: deploying code that is good enough for production and adds value.

I have found that many Agile practices in combination with developer friendly tools fits well with the ideas of Agentic Engineering. Even though I've seen worrying signs of a return of the Waterfall movement.

To summarize: the result of my new Agentic Engineering development-style is that I haven't put my IDE to the side - it's at the very Center of the agentic workflow.



Top Photo by me, taken at Åreskutan, Jämtland, Sweden.

Friday, February 7, 2025

FOSDEM 25

FOSDEM, a Conference different from any other I've been to before. I'm very happy for the opportunity to talk, share knowledge and chat with the fellow devs there. I also met old and new friends in Brussels, and learned a little bit more about the nice Belgian beer culture. 😀

The FOSDEM event is free and you don't even register to attend (only speakers do that in a call for speakers process). Just come by the University Campus, located not far from the center of the Inner City.

I travelled towards Belgium with the Night Train from Stockholm, Sweden, and arrived the next morning in Hamburg, Germany. From there, I took the DB ICE Train to Köln/Cologne. At some point, the top speed was about 250 km/h. That's fast! From there, another fast train to Brussels.

The variety of topics and the amount of tracks at FOSDEM was mind blowing, with thousands of participants. The overall vibe was very friendly & laid back. I joined Rust focused talks, JavaScript and UX/Design talks. And, of course, the Python room talks.

My talk was in the Python room and was about Python Monorepos and the Polylith Developer Experience. Everything Python related was on FOSDEM Day 2 and I think my presentation went really well! There was a lot of questions afterwards and I got great feedback from the people attending.

The next day I went back the same route as before. I got a couple extra of hours to spend in Hamburg before onboarding the Night Train back home to Stockholm. A great Weekend Trip!

Here's the recording of my talk from FOSDEM 25:

The video was downloaded from fosdem.org.
Licensed under the Creative Commons Attribution 2.0 Belgium Licence.


Resources

Thursday, January 25, 2024

Simple & Developer-friendly Python Monorepos

🎉 Announcing new features 🎉

Polylith is about keeping Monorepos Simple & Developer-friendly. Today, the Python tools for the Polylith Architecture has support for Poetry, Hatch and PDM - three popular Packaging & Dependency Management tools in the Python community.

In addition to the already existing Poetry plugin that adds tooling support for Polylith, there is also a brand new command line tool available. The CLI has support for both Hatch and PDM. You can also use it for Poetry setups (but the recommended way is to use the dedicated Poetry plugin as before).

To summarize: it is now possible to use the Simple & Developer-friendly Monorepo Architecture of Polylith with many different kinds of Python projects out there.

"Hatch is a modern, extensible Python project manager."
From the Hatch website

🐍 Hatch

To make the tooling fully support Hatch, there is a Hatch build hook plugin to use - hatch-polylith-bricks - that will make Hatch aware of a Polylith Workspace. Hatch has a nice and well thought-through Plugin system. Just add the hook to the build configuration. Nice and simple! The Polylith tool will add the config for you when creating new projects:

[build-system]
requires = ["hatchling", "hatch-polylith-bricks"]
build-backend = "hatchling.build"
"PDM, as described, is a modern Python package and dependency manager supporting the latest PEP standards. But it is more than a package manager. It boosts your development workflow in various aspects."
From the PDM website

🐍 PDM

Just as with Hatch, there are build hooks available to make PDM aware of the Polylith Workspace. Writing hooks for PDM was really simple and I really like the way it is developed. Great job, PDM developers! There is a workspace build hook - pdm-polylith-workspace - and a projects build hook - pdm-polylith-bricks - to make PDM and the Polylith tooling work well together.

This is added to the workspace build-system section pyproject.toml:

[build-system]
requires = ["pdm-backend", "pdm-polylith-workspace"]
build-backend = "pdm.backend"

And the plugin for projects.
This will be added by the poly create project command for you.

[build-system]
requires = ["pdm-backend", "pdm-polylith-bricks”]
build-backend = "pdm.backend"
"Python packaging and dependency management made easy."
From the Poetry website

🐍 Poetry

For Poetry, just as before, add or update these two plugins and you're ready to go!

poetry self add poetry-multiproject-plugin
poetry self add poetry-polylith-plugin

📚 Docs, examples and videos

Have a look at the the Polylith documentation for more information about getting started. You will also find examples, articles and videos there for a quick start. I'm really excited about the new capabilities of the tooling and hope it will be useful for Teams in the Python Community!



Top photo made with AI (DALL-E) and manual additions by a Human (me)

Wednesday, November 15, 2023

A Coding Copilot

When developing software, I spend most of the time thinking & talking about the problems to solve: the what, the why and the how. The thinking-part works best for me when I'm actually away from the desktop, such as walking outdoors or even when grabbing a cup of coffee & a banana by the office kitchen. 🍌

Talking is a really nice thing, we should try that more often in combination with listening to what others have say. Even when not having any human around to chat with, there is always the rubber duck. Speak to it!

Problem Solving

Mob programming takes the Thinking & Talking to the next level. I am, strangely enough, surprised almost every time how great mob and pair programming is for software development. I tend to forget that. It's probably easy to fall back into old familiar behavior.

When figuring out what to try out, the actual typing is done pretty fast for me. A big part of the productivity boost is inspiration and the joy of solving problems with programming. I think this is valid for most creative and problem-solving work.

Everyday practicing

I try hard every day to write clear & concise code. Practicing, learning ... and walking. I really like the Functional approach to programming: separating data from behavior, actions from calculations. I should probably use GitHub Copilot or ChatGPT more, but haven't yet found the need to go all-in (aren't the AI suggestions a bit verbose, or am I doing things wrong?). Those types of in-editor copilots are cool, definitely, and are for sure here to stay. Currently, I find more value in a different kind of automated guidance.

CodeScene

When working on my Open Source Project, the Python tools for the Polylith Architecture, I am guided by tools like SonarCloud and CodeScene. I guess there are some AI involved quite heavily in there too. I especially appreciate the code reviews & even more the CodeScene long-term analysis about the effects of Tech Debt. It's like having a co-pilot, or even a Teacher, guiding you during the development to make you a better software developer than before.

This is different from the kind of review a team of humans usually do when sharing feedback on Pull Requests or even during mob programming sessions.

Even if I sometimes disagree with things the tools report on being a problem, I go ahead and refactor the code anyway. Then, when doing the refactoring, I quickly realize that the code is actually transforming into something simpler and clearer than before. SonarCloud is more into the low level details, and lets me know about code duplications or code smells. We don't want that, no. Better fix that too.

Coding Style

The feedback from these kind of tools is helping me to get closer to writing the type of code I want to write: minimalistic, readable and functional. A coding style that also is a very good fit for the Polylith Architecture, even if that thing isn't about how the code is written. You are encouraged to write in a readable & functional style by the structuring of code and by having all code at your fingertips, ready to experiment with in the REPL or in a Notebook.

CodeScene, SonarCloud and Polylith: all of them helping me in my journey to be a better Developer. Maybe I'll throw in Copilot in there too eventually.

Top Photo by Jamie Templeton on Unsplash

Sunday, April 18, 2021

Confessions of an Open Source Contributor

I am one of the contributors to an Open Source project called zookeeper. It's an Apache ZooKeeper client made for Node.js. You can find the package on npm and the source code at GitHub.

What’s ZooKeeper?

"Apache ZooKeeper ... providing configuration information, naming, synchronization and group services over large clusters in distributed systems ..."

In short, Apache ZooKeeper is made of two parts: a service, and clients that communicate with it. Included in the packaging from Apache, there is a client made for the Java platform and also a C based client. You will most likely find one for your favorite language out there as a third party Open Source project.

Why joining Open Source?

A couple of years ago, I was part of a team in an organisation that use Apache ZooKeeper heavily. The team maintains a Node.js based app that communicates with a ZooKeeper service.

One day, the app began to raise strange warnings and unexpected errors. We looked into it and found out that the reason was probably caused by the client no longer being compatible with the API of the service.

There wasn't much happening in the client's repository:
unanswered questions from users, issues not taken care of and pull requests not being reviewed.

The project was abandoned.

So, we used a different client that seemed fairly updated and stable. The strange behavior in our app was now gone. Great! It worked on our machines and in production. All good!


🙋"Hey, team. I can't install your app on my Windows machine."

Sometimes other teams need to run our app locally for testing and such things. It turned out that our new client didn't support Windows at all.

Wait a minute, isn't Node.js supported in all major platforms? Well, yes. But this one is building a Native Node.js AddOn using bash scripts. Windows don't have that.

🤦‍♂️

Even though it wasn't my fault only, all I could think of is that it was my fault. I have ruined it for the teams at this workplace.

What else to do than try fix it on my spare time? So, I grabbed the source code to try to understand what's going on behind the scenes.

😭



And that's how my Open Source journey began: triggered by guilt and feeling ashamed.

Coding in the Dark

Late nights and weekends were spent trying to figure out how to solve this huge problem, including C/C++ code, CMake, bash and node-gyp.

One day, I figured it out. I had succeeded in making it work on Mac OS X, Linux and on Windows! The next day - a sunny morning in beautiful Stockholm - I went to the office smiling. Exhausted by the lack of sleep, but happy.

😎


Do I have to fix everything? 😟

While struggling with the source code, I realized there were more things that should be fixed. By then, I also realized that this Open Source project suffered from similar issues as the one we had replaced earlier: there wasn't really that much activity in the repo.

This is probably a common pattern: you work with something on a daily basis and contribute to Open Source projects. The years go by, you drift way from the project, simply because you now work with other things unrelated to the project.

How to maintain an Open Source project

Instead of grabbing the next task to spend late nights and weekends with, I decided to learn more about how to maintain Open Source projects.

I found a very useful guideline at GitHub, and I began to add notes about all kinds of things I thought should be done. Other Open Source projects were using labels in a way that I liked, so I added labels like help-wanted and good-first-issue to most of the tasks.

I think labels can help people getting an overview of the tasks and to understand the degree of difficulty.

Every time a question comes up, or someone is filing a bug report, I have replied to most of them within a day or two. I think this is important. The person reporting will know there is someone that has actually received and read the message.

As a result, really nice things have happened. The project has received Pull Requests from people located all over the world. Solving a broad variation of things from typos in the docs, to rewrites of the source code to modern JavaScript.

What's happening today?

Today, I think the project is stable and working well. I don't work nights or weekends. Even though I have moved on and no longer work with ZooKeeper as before, the feedback from users is my main inspiration to keep on working. Also, the amount of users of the client looks pretty good.

It inspires me to do the best I can with the project.



Photo by Justin Lim on Unsplash

Sunday, March 10, 2019

Updates to the Apache ZooKeeper Node.js client

At last, Node.js is platform independent again!

Eh, what?

Apache Zookeeper?
From the docs: "... ZooKeeper is a distributed, open-source coordination service for distributed applications. It exposes a simple set of primitives that distributed applications can build upon to implement higher level services for synchronization, configuration maintenance, and groups and naming. It is designed to be easy to program to, and uses a data model styled after the familiar directory tree structure of file systems. It runs in Java and has bindings for both Java and C. ..."
(the docs)

A Node.js Zookeeper Client
You will find several Node.js implementations when browsing on npmjs.org, and one of them is simply called: zookeeper. Recently, I was added to the list of maintainers of the project, hosted on GitHub.

Background
The project got my attention when we at work were looking for a new ZooKeeper client, because the API of the current one we were using had gone out of date. We were all developing on Mac OS X, and pushed changes to a CI using a Linux docker container. Implementing the new client seemed like a breeze.

Until a developer in a peer team came by our spot at the office:
- Hey, I cloned the app repo and followed your setup guide. But I get errors when installing packages. I think it's the "zookeeper" dependency that fails. Do you know what's wrong?

We quickly learned it was because Windows. We realized that the entire build process - compiling C code to a native Node.js add-on on the fly - used non-Windows compliant features like make, bash and gcc.

Oh sh*t. How can this be solved?
What about bypassing it with Docker? I've learned that Docker on Windows can be painful, because of access right issues between a Windows host and Linux in a container. Windows Subsystem for Linux (WSL) would probably work after some tweaking, but adding WSL as a requirement to run (or develop) an app is ... well ... a bit depressing.

How about fixing the root cause of the problem, the source code? I did choose that path and have spent a lot of time learning about Node.js add-ons, (a little bit of) C and C++, Windows equivalents to bash commands, node-gyp, and how to compile the Apache Zookeeper C Client from source.

Months later, I hit the "Create Pull Request" button. Shortly after, the changes were merged and an updated npm package was published.

Besides supporting Windows, the zookeeper npm package also builds on Node.js 10. Previous versions failed, because the use of deprecated V8 features.

Details of recent changes can be found in the Changelog.

Contribute to the Open Source project
If you are interested in contributing, check out the issues tab at the GitHub repo. Currently there are some issues labeled with "good first issue", go ahead and grab one!