Hi podman, Bye docker

A 🌧️ Saturday.

This feels like a great day to uninstall Docker Desktop on my Mac, and try to go all-in Podman.

What is Podman?

" ... a daemonless container engine for developing, managing, and running OCI Containers ... "

Podman is an open source project and you can run containers as root, or as a non-privileged user. It supports the Docker API and can be used with tools like docker-compose.

What's wrong with docker?

Nothing, docker is a great tool! Free to use for Open Source. The Desktop app for Mac and Windows will continue to be free for small businesses, but require a license for bigger companies. For some time now, I have wanted to find out what the alternatives are to using docker.

Uninstalling docker

I ran docker system prune to delete all existing containers and images.

Then I uninstalled the desktop client. By selecting the "Troubleshoot" menu option I found an uninstall button and pressed it. It required some research (aka googling) to figure out how and where to find an uninstall option.

As you can see in the video, the docker app is still there even if the Desktop client was uninstalled. I just deleted that one, and then deleted the docker-related files and folders that were still in my file system:

Installing podman

I followed the official install guideline to install podman. In addition to that, also installed the Mac OS X specific helper tool to be able to run docker-compose. I learned that from the print-out displayed when the brew install command was completed.

I can now run containers with podman! Here I'm starting a ZooKeeper server running in a container:

podman run --rm -p 2181:2181 zookeeper

Where's my commands?

But wait, my 🧠 is already wired to use the docker command. Do I have to unlearn things? Podman is compatible with the docker commands already, but you can also create an alias:

alias docker=podman

It is now possible to use the docker command as before!

Installing docker-compose

By installing the podman-mac-helper tool as described before in this post, it is possible to use podman with docker-compose.

You might wonder: didn't I uninstall all docker related things? 🤔
Yes, but the docker-compose tool is available to install separately & you'll find it on Homebrew. 🍻

brew install docker-compose

Quirks & workarounds

When trying all of this out, the podman command worked well for building and running containers. However, I got an error when running the docker-compose up command. That's kind of a dealbreaker, isn't it?

The error: listing workers for Build: failed to list workers: Unavailable: connection error: desc = "transport: Error while dialing unable to upgrade to h2c, received 404"

As I understand it (from this conversation on Github), there is something not working as expected when using the latest version(s) of docker-compose and podman.

Luckily, there is a simple workaround to use until the bug is fixed. Prefixing the docker-compose command to disable the buildkit API that is causing the bug. Doing this will make things work very well.

That's it! 👋

^{Top photo by frank mckenna on Unsplash}

What would slackbot do?

I recently found out about Should I deploy today? and laughed. A humorous way of guidning us developers when we hesitate about deploying code to production. This one is extra funny on a Friday or Saturday. Every now and then, memes about (not) releasing code to production on Fridays appear on my Twitter timeline. There's even T-shirts for sale, with No Deploy Fridays printed on them.

Funny because it's true?

I totally get the idea of minimizing the chance of ruining your or someone else's weekend. There's also a potential risk with postponing & stacking up features. The things we develop are there to add value, why make users wait for it?

Ship it!

Fridays shouldn't be different than any of the other working days, at least when it comes to deploying code to production. Just like on a Tuesday, you should probably think twice about deploying stuff just before you'll call it a day. When a fresh release has started causing errors for users, you might have already checked out from the office with loud music in the headphones, the mind focused on something else and unaware of the mess you've left to friends on call.

Deploy with some slack

It's a good idea to monitor the live environment for a while, making sure the deploy has gone well. Keep an eye on it, while working on the next thing. Prepare to act on alerts, errors appearing in your logs or messages in the company Slack.

Have a browser tab with graphs monitoring memory consumption, CPU usage and response times. For how long and what metrics to monitor is depending on the product. From my experience, being attentive to things like this is mostly good enough. When I call it a day a couple of hours later, I can go ahead and play that loud music in my headphones and relax.

When things go 💥

I haven't fact checked this - but unless a deploy is about database schema changes or to an offline machine, it should be relatively easy to roll back. Most deploys are probably easy to undo. If not, find out why and try making it easier.

I would suggest teams to practice: commit, push, merge, release to production and rollback. You'll probably find infrastructure related things to fix or tweak, maybe there's some automation missing? The team will know what to do when things go boom for real.

I've been fortunate working in environments with the basic automation already in place. We, the autonomous dev team, are the ones responsible for pushing features to production. We do that on a daily basis. Before, we hesitated about the deploy-on-a-Friday thing. Now we have better trust in our ways of working, and this Friday we deployed new features, dev experience improvements and bug fixes. Several small deployments went live, the last one about 3-4 hours before we called it a weekend 😀

Automate it

A good deployment process will stop and let the team know about when things break. The obvious code related issues will be caught this way. So, automation will get you far, but not all the way. That's why we'll need the monitoring. More importantly, releasing small portions of code changes is much safer than the ones with lot of code changes in it. Release small and release often.

What would slackbot do?

These days, when I hesitate if a new feature should be deployed right now or not, I'll just ask slackbot what to do.

^{Top photo by Ochir-Erdene Oyunmedeg on Unsplash}

A fresh take on Monorepos in Python

"... What if we had Polylith in Python ..."

For software development in general, it seems to be an ongoing trend towards using Monorepos. You know, organizing code in one big single repo. On the other hand, it seems to be a general trend in going the opposite way too. Developing features into many small repos.

I think that the JavaScript and Python communities in particular, are in favor of the latter approach. Developing features in isolation - in separate repos - and publish versioned libraries to package repositories (such as npm and pypi). That has potential to introduce some headache: versioning, backwards compatibility and keeping dependencies up to date. Probably also duplication of common source code and repeating the deployment infrastructure. You can solve those kind of issues, by using a Monorepo.

🔥 Monorepos

There are already solutions out there on how to develop software in Monorepos. It is my impression that they are mainly about solving the deployment experience. Which is good, because it is a huge problem in a lot of code bases out there. What about the Developer experience?

In the Clojure community, we have a thing called Polylith. It is an architecture, (including a tool) that is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk. The Polylith architecture is focusing on making both the Developer & Deployment experience great. There is also a very nice tool to help you get started with Polylith. Helping you create & keeping track on components, projects and much more.

🤔 So, what is Polylith?

From the official docs:

"... Polylith is a software architecture that applies functional thinking at the system scale. It helps us build simple, maintainable, testable, and scalable backend systems. ..."

A Polylith code-base is structured in a components-first architecture. Similar to LEGO, components are building blocks. A component can be shared across apps, tools, libraries, serverless functions and services. The components live in the same repository; a Polylith monorepo. The Polylith architecture is becoming popular in the Clojure community.

🐍 What about Python?

Again, I find myself copying some existing Python code - a small module creating a "logger" - into a new repo. I think it’s the third time I do this. The logger code is too tiny to even bother packaging as a library, but still very useful in many code projects.

My mind is wandering. I'm daydreaming:

"What if we had Polylith in Python. I would just simply reuse an existing component."

✨We have Polylith in Python now✨

Porting Polylith to Python

A couple of weeks ago, I decided to give porting Polylith to Python a try. I quickly realized that there are some fundamental differences between Python and Clojure to be aware of when implementing an architecture (and developing a tool) like this.

Especially when it comes to namespacing and packaging. Python code is based on the concept of modules. Also, Python code is not compiled into a binary.

Short note on modules, packages, namespaces & libraries

In Python, a file is a module. One or more modules in a folder is a package. If you put a package in a folder, it is now namespaced. One or more packages can also be combined and built into a library to be used as a third party dependency. From my perspective, a Polylith component in Python should be a namespaced package, and not a library.

Almost like Poetry

When trying to figure out how to solve modules, paths & packaging problems I found Poetry. It is a tool that makes Python packaging and dependency management really easy. Out of the box, Poetry supports project based dependencies and references only. But the latest version of Poetry - currently in preview - has support for third party plugins. So I developed a plugin that enables the concept of workspaces, making it possible to reference dependencies outside of a project boundary.

This was the first step. A very important step. Now it became possible to build tooling support for a Python implementation of the Polylith architecture. Earlier this week I released a first version of such a tool. It is a new Poetry plugin, containing the very basic tooling support for the Polylith architecture.

poetry-polylith-plugin

This brand new Poetry plugin will help you create a Polylith workspace and add components, bases & projects to it. Poetry itself will handle the building & packaging. It is still in early development, but you should be able to develop apps, serverless functions and services with it.

As of today, I wouldn’t (at least not yet) recommend building libraries with it, mostly because of how the packaging is done in Poetry & the nature of Python modules. I’ll try to figure out a way to solve this in the future.

There’s also a lot missing & left to do compared to the original poly tool. But hey, you gotta start somewhere 😀

Check out the 10 minute video for a quick overview.

Learning about Polylith

I recommend to read the official docs to learn more about what Polylith is (and what it isn’t) - even though the code examples are about Clojure. You can easily bring the ideas into a Python context, now that we have tooling support for it.

My intentions with this is to introduce the Polylith architecture to the Python community, and I very much hope that it will be useful in your daily work as a developer.

Direct link to the Porting Polylith to Python intro video.

^{Top photo by Katarzyna Kos on Unsplash}

Simple Component Driven ClojureScript

That Component Driven thing

Component Driven is about developing user interfaces and apps in a certain way. In short, it is about grouping relevant code into components, combining compontents into features, combining features into views (or pages) - and combining the views into an app. I don't think it should be that much about the tools.

Just like Test Driven Development, Component Driven Development encourages us to start small. In a way, it also reminds of LEGO when combining building blocks into something useful.

As your app grows and the coding is done deeper into the app structure, developing user interfaces might be more of a challenge. Finding the spot where a new feature is to be added, might require some clicks and navigation. You know, stuff like logging in, a specific app state - or even waiting for data fetched from a third party service.

This is when the idea of Component Driven Development is really useful. To simplify and speed up the development, we need something simple to host our components. A UI that is isolated from any app specific state or dependencies. A place where a component can be rendered without the need to set things up. Having that, it is possible to develop a component in isolation and test it with different input.

Do I need a tool?

Storybook is a tool that will solve all of this, with a bunch of other really nice things like docs, previews, example usage, variants, code snippets and viewports. But it is possible to try out the Component Driven style of creating user interfaces, without going all-in with a new tool.

Start small

I’m suggesting a very simplistic approach, that will enable rendering components in a short-lived isolated view. You throw it away when done.

Here, I’m using shadow-cljs and have defined a new alias that is targeting a browser. I have also set entry points. A setup almost identical to the web app alias. This one will run in a separate port and the output is compiled to a separate public folder. In that one, I have copied the HTML template from the public folder used by the web app.

Click to enlarge the image, or browse an example gist here.

Keep it Simple

The entry point here is a simplistic and simplified version of the one used in the actual web app. But this entry point does only care about the component you are developing right now. Here is where you render and try out a component during development. This is a temporary place to work Component Driven. It’s almost like when evaluating code in a REPL, but with HTML and compiled JavaScript in a browser.

Click to enlarge the image, or browse an example gist here.

Run both aliases. The App in one browser folder, the component in a different one.

The browser is a powerful tool, with nice developer features.

Take it to the next level

At some point, you will probably want something more long-lived than this very simplistic scratch-style approach. That’s where Storybook fits in!

Component Driven ClojureScript with Storybook

With Storybook, you can focus on styling, different viewports and UI events when developing a single component. You write "stories" (i.e. running components in isolation), including documentation and lot of in-browser interactivity. The stories & docs can also be shared, by deploying it as a separate web app.

Recently, I talked about getting started with Storybook at the re:Clojure 2021 conference.

In this less than 25 minutes talk, I demo how you can use Storybook in your ClojureScript app and work Component Driven.

I hope you’ll like it!

Direct link to the Component Driven ClojureScript with Storybook video.

^{Top photo by Brett Jordan on Unsplash}

Functional Python

All in Functional

In a previous post, I had some concerns about doing too much functional programming in Python and that it might not be Pythonic. But I'm not going worry about that here. I'll be writing Functional Python, with some help from a very nice library called toolz.

I have added example code in this GitHub repo and have also recorded a short video (about 8 minutes long). You'll find it embedded in this post, just scroll down a bit.

Functional in a nutshell

The video is very much inspired by one of my favorite recordings about functional programming in general, and about Clojure in specific: Clojure in a nutshell (Func Prog Sweden), by James Trunk. I have learned a lot from that talk and figured it would be fun to copy the workflow and examples that James uses in his excellent talk.

So, I've been translating Clojure code to Python. You'll also notice in the short video that I'm practicing a thing called REPL Driven Development.

Pipe functions with the toolz.thread_last function.

Composing, piping, chaining, threading

What I've learned so far is that it's a smooth ride to apply functional concepts into a language like Python. After all, functions are first class citizens here. Composing functions in a minimalistic - and readable - way is joyful. Yeah, these things also encourage to aim for simplicity in functions. Keeping functions stateless, single purposed and testable. All of that is really good stuff.

Intuitive, Pythonic & Effective?

Are those one-liners, like that thread_last thing that I use heavily in the video, Pythonic? Well, I guess it depends. As with most things, the features of cool toolz should be used wisely.

Yes, Python should be used wisely.

Remember to run import this in a Python shell now and then to remind yourself about and reflect on the Zen of Python 🧘. Does it say anything at all about OOP, or functional?

Here's an improvised 8 minute recording with live coding functional Python, using toolz:

Direct link to the Functional Python video.

^{Top photo by Charlein Gracia on Unsplash}