söndag 20 februari 2022

What would slackbot do?

I recently found out about Should I deploy today? and laughed. A humorous way of guidning us developers when we hesitate about deploying code to production. This one is extra funny on a Friday or Saturday. Every now and then, memes about (not) releasing code to production on Fridays appear on my Twitter timeline. There's even T-shirts for sale, with No Deploy Fridays printed on them.

Funny because it's true?

I totally get the idea of minimizing the chance of ruining your or someone else's weekend. There's also a potential risk with postponing & stacking up features. The things we develop are there to add value, why make users wait for it?

Ship it!

Fridays shouldn't be different than any of the other working days, at least when it comes to deploying code to production. Just like on a Tuesday, you should probably think twice about deploying stuff just before you'll call it a day. When a fresh release has started causing errors for users, you might have already checked out from the office with loud music in the headphones, the mind focused on something else and unaware of the mess you've left to friends on call.

Deploy with some slack

It's a good idea to monitor the live environment for a while, making sure the deploy has gone well. Keep an eye on it, while working on the next thing. Prepare to act on alerts, errors appearing in your logs or messages in the company Slack.

Have a browser tab with graphs monitoring memory consumption, CPU usage and response times. For how long and what metrics to monitor is depending on the product. From my experience, being attentive to things like this is mostly good enough. When I call it a day a couple of hours later, I can go ahead and play that loud music in my headphones and relax.

When things go 💥

I haven't fact checked this - but unless a deploy is about database schema changes or to an offline machine, it should be relatively easy to roll back. Most deploys are probably easy to undo. If not, find out why and try making it easier.

I would suggest teams to practice: commit, push, merge, release to production and rollback. You'll probably find infrastructure related things to fix or tweak, maybe there's some automation missing? The team will know what to do when things go boom for real.

I've been fortunate working in environments with the basic automation already in place. We, the autonomous dev team, are the ones responsible for pushing features to production. We do that on a daily basis. Before, we hesitated about the deploy-on-a-Friday thing. Now we have better trust in our ways of working, and this Friday we deployed new features, dev experience improvements and bug fixes. Several small deployments went live, the last one about 3-4 hours before we called it a weekend 😀

Automate it

A good deployment process will stop and let the team know about when things break. The obvious code related issues will be caught this way. So, automation will get you far, but not all the way. That's why we'll need the monitoring. More importantly, releasing small portions of code changes is much safer than the ones with lot of code changes in it. Release small and release often.

What would slackbot do?

These days, when I hesitate if a new feature should be deployed right now or not, I'll just ask slackbot what to do.



Top photo by Ochir-Erdene Oyunmedeg on Unsplash

onsdag 9 februari 2022

A fresh take on Monorepos in Python

"... What if we had Polylith in Python ..."

For software development in general, it seems to be an ongoing trend towards using Monorepos. You know, organizing code in one big single repo. On the other hand, it seems to be a general trend in going the opposite way too. Developing features into many small repos.

I think that the JavaScript and Python communities in particular, are in favor of the latter approach. Developing features in isolation - in separate repos - and publish versioned libraries to package repositories (such as npm and pypi). That has potential to introduce some headache: versioning, backwards compatibility and keeping dependencies up to date. Probably also duplication of common source code and repeating the deployment infrastructure. You can solve those kind of issues, by using a Monorepo.

🔥 Monorepos

There are already solutions out there on how to develop software in Monorepos. It is my impression that they are mainly about solving the deployment experience. Which is good, because it is a huge problem in a lot of code bases out there. What about the Developer experience?

In the Clojure community, we have a thing called Polylith. It is an architecture, (including a tool) that is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk. The Polylith architecture is focusing on making both the Developer & Deployment experience great. There is also a very nice tool to help you get started with Polylith. Helping you create & keeping track on components, projects and much more.

🤔 So, what is Polylith?

From the official docs:

"... Polylith is a software architecture that applies functional thinking at the system scale. It helps us build simple, maintainable, testable, and scalable backend systems. ..."

A Polylith code-base is structured in a components-first architecture. Similar to LEGO, components are building blocks. A component can be shared across apps, tools, libraries, serverless functions and services. The components live in the same repository; a Polylith monorepo. The Polylith architecture is becoming popular in the Clojure community.

🐍 What about Python?

Again, I find myself copying some existing Python code - a small module creating a "logger" - into a new repo. I think it’s the third time I do this. The logger code is too tiny to even bother packaging as a library, but still very useful in many code projects.

My mind is wandering. I'm daydreaming:

"What if we had Polylith in Python. I would just simply reuse an existing component."

We have Polylith in Python now

Porting Polylith to Python

A couple of weeks ago, I decided to give porting Polylith to Python a try. I quickly realized that there are some fundamental differences between Python and Clojure to be aware of when implementing an architecture (and developing a tool) like this.

Especially when it comes to namespacing and packaging. Python code is based on the concept of modules. Also, Python code is not compiled into a binary.

Short note on modules, packages, namespaces & libraries

In Python, a file is a module. One or more modules in a folder is a package. If you put a package in a folder, it is now namespaced. One or more packages can also be combined and built into a library to be used as a third party dependency. From my perspective, a Polylith component in Python should be a namespaced package, and not a library.

Almost like Poetry

When trying to figure out how to solve modules, paths & packaging problems I found Poetry. It is a tool that makes Python packaging and dependency management really easy. Out of the box, Poetry supports project based dependencies and references only. But the latest version of Poetry - currently in preview - has support for third party plugins. So I developed a plugin that enables the concept of workspaces, making it possible to reference dependencies outside of a project boundary.

This was the first step. A very important step. Now it became possible to build tooling support for a Python implementation of the Polylith architecture. Earlier this week I released a first version of such a tool. It is a new Poetry plugin, containing the very basic tooling support for the Polylith architecture.

poetry-polylith-plugin

This brand new Poetry plugin will help you create a Polylith workspace and add components, bases & projects to it. Poetry itself will handle the building & packaging. It is still in early development, but you should be able to develop apps, serverless functions and services with it.

As of today, I wouldn’t (at least not yet) recommend building libraries with it, mostly because of how the packaging is done in Poetry & the nature of Python modules. I’ll try to figure out a way to solve this in the future.

There’s also a lot missing & left to do compared to the original poly tool. But hey, you gotta start somewhere 😀

Check out the 10 minute video for a quick overview.

Learning about Polylith

I recommend to read the official docs to learn more about what Polylith is (and what it isn’t) - even though the code examples are about Clojure. You can easily bring the ideas into a Python context, now that we have tooling support for it.

My intentions with this is to introduce the Polylith architecture to the Python community, and I very much hope that it will be useful in your daily work as a developer.

Direct link to the Porting Polylith to Python intro video.



Top photo by Katarzyna Kos on Unsplash

onsdag 22 december 2021

Simple Component Driven ClojureScript

That Component Driven thing

Component Driven is about developing user interfaces and apps in a certain way. In short, it is about grouping relevant code into components, combining compontents into features, combining features into views (or pages) - and combining the views into an app. I don't think it should be that much about the tools.

Just like Test Driven Development, Component Driven Development encourages us to start small. In a way, it also reminds of LEGO when combining building blocks into something useful.

As your app grows and the coding is done deeper into the app structure, developing user interfaces might be more of a challenge. Finding the spot where a new feature is to be added, might require some clicks and navigation. You know, stuff like logging in, a specific app state - or even waiting for data fetched from a third party service.

This is when the idea of Component Driven Development is really useful. To simplify and speed up the development, we need something simple to host our components. A UI that is isolated from any app specific state or dependencies. A place where a component can be rendered without the need to set things up. Having that, it is possible to develop a component in isolation and test it with different input.

Do I need a tool?

Storybook is a tool that will solve all of this, with a bunch of other really nice things like docs, previews, example usage, variants, code snippets and viewports. But it is possible to try out the Component Driven style of creating user interfaces, without going all-in with a new tool.

Start small

I’m suggesting a very simplistic approach, that will enable rendering components in a short-lived isolated view. You throw it away when done.

Here, I’m using shadow-cljs and have defined a new alias that is targeting a browser. I have also set entry points. A setup almost identical to the web app alias. This one will run in a separate port and the output is compiled to a separate public folder. In that one, I have copied the HTML template from the public folder used by the web app.

Click to enlarge the image, or browse an example gist here.

Keep it Simple

The entry point here is a simplistic and simplified version of the one used in the actual web app. But this entry point does only care about the component you are developing right now. Here is where you render and try out a component during development. This is a temporary place to work Component Driven. It’s almost like when evaluating code in a REPL, but with HTML and compiled JavaScript in a browser.

Click to enlarge the image, or browse an example gist here.

Run both aliases. The App in one browser folder, the component in a different one.

The browser is a powerful tool, with nice developer features.

Take it to the next level

At some point, you will probably want something more long-lived than this very simplistic scratch-style approach. That’s where Storybook fits in!

Component Driven ClojureScript with Storybook

With Storybook, you can focus on styling, different viewports and UI events when developing a single component. You write "stories" (i.e. running components in isolation), including documentation and lot of in-browser interactivity. The stories & docs can also be shared, by deploying it as a separate web app.

Recently, I talked about getting started with Storybook at the re:Clojure 2021 conference.

In this less than 25 minutes talk, I demo how you can use Storybook in your ClojureScript app and work Component Driven.

I hope you’ll like it!

Direct link to the Component Driven ClojureScript with Storybook video.



Top photo by Brett Jordan on Unsplash

söndag 10 oktober 2021

Functional Python

All in Functional

In a previous post, I had some concerns about doing too much functional programming in Python and that it might not be Pythonic. But I'm not going worry about that here. I'll be writing Functional Python, with some help from a very nice library called toolz.

I have added example code in this GitHub repo and have also recorded a short video (about 8 minutes long). You'll find it embedded in this post, just scroll down a bit.

Functional in a nutshell

The video is very much inspired by one of my favorite recordings about functional programming in general, and about Clojure in specific: Clojure in a nutshell (Func Prog Sweden), by James Trunk. I have learned a lot from that talk and figured it would be fun to copy the workflow and examples that James uses in his excellent talk.

So, I've been translating Clojure code to Python. You'll also notice in the short video that I'm practicing a thing called REPL Driven Development.

Pipe functions with the toolz.thread_last function.

Composing, piping, chaining, threading

What I've learned so far is that it's a smooth ride to apply functional concepts into a language like Python. After all, functions are first class citizens here. Composing functions in a minimalistic - and readable - way is joyful. Yeah, these things also encourage to aim for simplicity in functions. Keeping functions stateless, single purposed and testable. All of that is really good stuff.

Intuitive, Pythonic & Effective?

Are those one-liners, like that thread_last thing that I use heavily in the video, Pythonic? Well, I guess it depends. As with most things, the features of cool toolz should be used wisely.

Yes, Python should be used wisely.

Remember to run import this in a Python shell now and then to remind yourself about and reflect on the Zen of Python 🧘. Does it say anything at all about OOP, or functional?

Here's an improvised 8 minute recording with live coding functional Python, using toolz:

Direct link to the Functional Python video.



Top photo by Charlein Gracia on Unsplash

fredag 8 oktober 2021

A Pythonic Railway?

What is Railway Oriented Programming, and how it can be implemented in Python? Is it even a good fit for Python?

🐍 Python on Rails

I’ve experimented with this Railway thing to find out if it is a good fit. If you haven't already, make sure to watch this excellent talk about Railway Oriented Programming from NDC London by Scott Wlaschin.

As I understand Railway Oriented Programming, it is about adding error handling to code without cluttering the program flow.

The source code in my GitHub repo is an attempt to implement a lightweight, and probably a sloppy, variant of Railway Oriented Programming in Python.

What I'm trying to achieve is to add the functional concepts of Railway Oriented Programming, and try keeping a Pythonic mindset. I believe that means not going all-in functional, but I could be wrong here. 🤔

🚂 Why Railways?

As I understand it, this way of writing code is about adding error handling and still keeping a happy path style in the code. This is done by wrapping - or as I’ve been doing, decorating - functions.

The functions are wrapped (or decorated, as in the examples in my GitHub repo) to catch failures. The output of a failed function call will be the input to the next one. The next function will choose a track based on the input: taking the success track, or the fail track. In general, taking the fail track means bypassing the function and skip to the next one.

By using a two-tracked approach in functions, the error handling will be separated from the program. Functions will be less cluttered with try except error handling and probably also with less of if else flow control clauses. As result, In many cases, this will mean less code in functions.

Less code in a function will probably be easier to understand and reason about. Less branches in a program flow will keep the functions simple. I think less is nice. Yeah, less is more. 😎

Here’s an improvised 5 minute video I’ve made to demo the concepts Railway Oriented Development in Python:

Direct link to the A Pythonic Railway? video.

I would love to hear your feedback about this!



Top photo by Alex Paganelli on Unsplash