Should features live in several independent Microservices, or is a Monolith the way to go? What kind of setup is the best one for the code in a repo? Organizing is difficult.
"... I'll just put it in a utils folder for now ..."
There's a thing called Polylith. I've written about different aspects of it before. Polylith is an architecture (and a tool) focusing on the Developer and Deployment experience. It is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk.
The Polylith Architecture is a fresh take on how to share code, and offers a nice and simple solution to the Monolith vs Microservice tradeoffs. In addition to that, it is a good fit for functional programming. Some time ago, I got an idea: how about bringing a couple of the good things from there to here? So I developed something that I believe could be useful for many Python teams out there.
I already released a preview in early 2022, it was missing some essential features and the great developer experience wasn't really there yet. Also, I had little knowledge about how to package Python apps & libraries, but have learned a lot since then.
Today, I believe the Python tools for the Polylith Architecture is ready to use in the Real World. You will find version 1 on PyPI.
A couple of Poetry plugins
I have developed it as two different Poetry plugins. One of them - the Multiproject plugin - adds support for workspaces to the popular Poetry tool, by adding a new command called build-project.
The second one - Polylith plugin - adds useful tooling support for the Polylith Architecture itself. With the tooling, you can add components & projects in a simple way and also keep track of what's happening in the workspace.
Polylith is using a components-first architecture. The components are building blocks, very much like LEGO. The code is separated from the infrastructure and the building of artifacts. This may sound complicated, but it isn't.
In a way, it is about:
thinking about code as LEGO bricks, that can be combined into features.
making it easy to reuse code across apps, tools, libraries, serverless functions and services.
Keeping it simple.
Have a look at these introductory videos, describing Polylith in Python and the tooling support:
For software development in general, it seems to be an ongoing trend towards using Monorepos. You know, organizing code in one big single repo. On the other hand, it seems to be a general trend in going the opposite way too. Developing features into many small repos.
I think that the JavaScript and Python communities in particular, are in favor of the latter approach. Developing features in isolation - in separate repos - and publish versioned libraries to package repositories (such as npm and pypi). That has potential to introduce some headache: versioning, backwards compatibility and keeping dependencies up to date. Probably also duplication of common source code and repeating the deployment infrastructure. You can solve those kind of issues, by using a Monorepo.
🔥 Monorepos
There are already solutions out there on how to develop software in Monorepos. It is my impression that they are mainly about solving the deployment experience. Which is good, because it is a huge problem in a lot of code bases out there. What about the Developer experience?
In the Clojure community, we have a thing called Polylith. It is an architecture, (including a tool) that is developed by Joakim Tengstrand, Furkan Bayraktar and James Trunk. The Polylith architecture is focusing on making both the Developer & Deployment experience great. There is also a very nice tool to help you get started with Polylith. Helping you create & keeping track on components, projects and much more.
"... Polylith is a software architecture that applies functional thinking at the system scale. It helps us build simple, maintainable, testable, and scalable backend systems. ..."
A Polylith code-base is structured in a components-first architecture. Similar to LEGO, components are building blocks. A component can be shared across apps, tools, libraries, serverless functions and services. The components live in the same repository; a Polylith monorepo. The Polylith architecture is becoming popular in the Clojure community.
🐍 What about Python?
Again, I find myself copying some existing Python code - a small module creating a "logger" - into a new repo. I think it’s the third time I do this. The logger code is too tiny to even bother packaging as a library, but still very useful in many code projects.
My mind is wandering. I'm daydreaming:
"What if we had Polylith in Python. I would just simply reuse an existing component."
✨We have Polylith in Python now✨
Porting Polylith to Python
A couple of weeks ago, I decided to give porting Polylith to Python a try. I quickly realized that there are some fundamental differences between Python and Clojure to be aware of when implementing an architecture (and developing a tool) like this.
Especially when it comes to namespacing and packaging. Python code is based on the concept of modules. Also, Python code is not compiled into a binary.
Short note on modules, packages, namespaces & libraries
In Python, a file is a module. One or more modules in a folder is a package. If you put a package in a folder, it is now namespaced. One or more packages can also be combined and built into a library to be used as a third party dependency. From my perspective, a Polylith component in Python should be a namespaced package, and not a library.
Almost like Poetry
When trying to figure out how to solve modules, paths & packaging problems I found Poetry. It is a tool that makes Python packaging and dependency management really easy. Out of the box, Poetry supports project based dependencies and references only. But the latest version of Poetry - currently in preview - has support for third party plugins. So I developed a plugin that enables the concept of workspaces, making it possible to reference dependencies outside of a project boundary.
This was the first step. A very important step. Now it became possible to build tooling support for a Python implementation of the Polylith architecture. Earlier this week I released a first version of such a tool. It is a new Poetry plugin, containing the very basic tooling support for the Polylith architecture.
poetry-polylith-plugin
This brand new Poetry plugin will help you create a Polylith workspace and add components, bases & projects to it. Poetry itself will handle the building & packaging. It is still in early development, but you should be able to develop apps, serverless functions and services with it.
As of today, I wouldn’t (at least not yet) recommend building libraries with it, mostly because of how the packaging is done in Poetry & the nature of Python modules. I’ll try to figure out a way to solve this in the future.
There’s also a lot missing & left to do compared to the original poly tool. But hey, you gotta start somewhere 😀
Check out the 10 minute video for a quick overview.
Learning about Polylith
I recommend to read the official docs to learn more about what Polylith is (and what it isn’t) - even though the code examples are about Clojure. You can easily bring the ideas into a Python context, now that we have tooling support for it.
My intentions with this is to introduce the Polylith architecture to the Python community, and I very much hope that it will be useful in your daily work as a developer.
In a previous post, I had some concerns about doing too much functional programming in Python and that it might not be Pythonic. But I'm not going worry about that here. I'll be writing Functional Python, with some help from a very nice library called toolz.
I have added example code in this GitHub repo and have also recorded a short video (about 8 minutes long). You'll find it embedded in this post, just scroll down a bit.
Functional in a nutshell
The video is very much inspired by one of my favorite recordings about functional programming in general, and about Clojure in specific: Clojure in a nutshell (Func Prog Sweden), by James Trunk. I have learned a lot from that talk and figured it would be fun to copy the workflow and examples that James uses in his excellent talk.
So, I've been translating Clojure code to Python. You'll also notice in the short video that I'm practicing a thing called REPL Driven Development.
Pipe functions with the toolz.thread_last function.
Composing, piping, chaining, threading
What I've learned so far is that it's a smooth ride to apply functional concepts into a language like Python. After all, functions are first class citizens here. Composing functions in a minimalistic - and readable - way is joyful. Yeah, these things also encourage to aim for simplicity in functions. Keeping functions stateless, single purposed and testable. All of that is really good stuff.
Intuitive, Pythonic & Effective?
Are those one-liners, like that thread_last thing that I use heavily in the video, Pythonic? Well, I guess it depends. As with most things, the features of cool toolz should be used wisely.
Yes, Python should be used wisely.
Remember to run import this in a Python shell now and then to remind yourself about and reflect on the Zen of Python 🧘. Does it say anything at all about OOP, or functional?
Here's an improvised 8 minute recording with live coding functional Python, using toolz:
Yesterday I talked about and used some Open Source tools with funny names. They are not only funny, but also useful in the everyday life of a software developer. I talked about how tools like Chocolatey and Vagrant can make virtualization (a whole lot) easier, and automated.
This video is a minified version of the talk (about 20 minutes) and it is in Swedish. You will find the configuration files and scripts used in the demo at my GitHub page right here.
Let me know if you have any questions or feedback about the talk!
What is EPiServer and what does EPiServer 7.5 do? I (think) that I have aswered those questions during lunch hour today here at Knowit in Stockholm. This video - in swedish - is a minified version of my talk & live coding about the CMS, using asp.net mvc to demo how page types, properties, editor features and pages are wired together (without complicating things too much). Do you have feedback about this presentation? Please contact me by writing a comment at this blog post or at twitter.
"... Let the unit testing drive your design. Write a unit test that specifies what should be done. Do a couple of variations if necessary, and then write the code that does the job. Switch back and forth between the unit tests and the code, refactor and clarify. If you don't trust your code, write another unit test that provokes side effects. Do that until you feel confident. Then go home and have a good night sleep. ..." Unknown (Okay, it's me. I said that. I just thought it would be cool having a quote as an intro to this post)
Test driven JavaScript?
Write a simple test, write a simple function. Done. That was the twitter version. What about the blog version? I think test driven development (TDD) is a good tool for helping you write code with high readability and a design with ease of use in mind (sometimes). As a tool for testing (no matter what programming language you choose) TDD is overrated and quite often misunderstood. Probably because of the word Test in it, don't you think? Before we start, I want you to don't care about stuff like code coverage, what frameworks to use and every line of code should have a corresponding unit test. That last thing is just nonsense. Think about the result, the code that actually will run in the production environment. The result is important. No matter how many nice unit tests there are, the result should be as simple as possible. Simple, as in easy to understand, maintain and refactor. Simple is really difficult! I have failed with that many, many times (and still do, but never give up trying). What I have learned is that having a bunch of unit tests doesn't compensate for unnecessary complex production code. With or without unit tests, complicated code still sucks. Also, don’t start the TDD journey by evaluating all those tools and worrying about continuous integration scenarios. Leave that for later. Do it the Agile way: experiment and learn. "- Cool, I know unit testing already. I use that when writing C#. But TDD with JavaScript, how?" If you come from a C# or Java background and already write unit tests (but have little experience of JavaScript) I think test driven development will help you understand and explore the language. TDD has helped me to break free from the mindset chains of C# and Java style programming. JavaScript is not object oriented like in those languages. There are no classes, it is a language of functions and objects (and functions are objects). Objects can inherit from other objects and every object has a force field surrounding it (aka Closures) that is really powerful. Maybe we could call JavaScript the class less society? ...Because everyone has a function there. Enough. (Want the basics? Check out my talk "Rediscover JavaScript" from dotNetConf.) I have made a 15 minute video on how test driven development can be done with JavaScript and tried to write the code with simplicity in mind. I have avoided the common JavaScript language gotchas, like prototype inheritance and pseudo classes with the new or this keywords. In this video I use only one tool for unit testing (QUnit). Take a look at the video and please share your thoughts about it!
It's about two weeks since aspConf, the online .NET developer conference, took place. This year had thousands of registered participants and about sixty booked speakers. I feel honoured being one of the speakers of aspConf 2012.
Check out the great stuff published atChannel 9,
lots of interesting talks with a wide range of subjects within the .NET developer
world. Is there an easier way to learn new things than this?
My talkQuick Start: Test Driven Development was just published. Lots of thanks to all of you that joined the session! Have a look, please give your feedback and rate the video. Don't hesitate to contact me if you have any questions about Test DrivenDevelopment!
För
snart två veckor sedan kunde man följa aspConf - en utvecklarkonferens
på nätet. Årets konferens hade sextio talare på schemat och flera tusen
anmälda deltagare. I år var jag en av talarna och det känns stort att ha
varit med på detta. På Channel 9 finns nu massvis med bra videor att
kolla in, ett enkelt sätt att lära sig nya grejer!
Nu finns även inspelningen av min presentation Quick Start: Test Driven Development publicerad. Stort tack till er som följde den direktsända presentationen!
Titta, ge din feedback och betygsätt gärna videon (den är strax under en timme lång). Hör av dig till mig om du har frågor om testdriven utveckling!
