Joyful Python with the REPL

REPL Driven Development is a workflow that makes coding both joyful and interactive. The feedback loop from the REPL is a great thing to have at your fingertips.

"If you can improve just one thing in your software development, make it getting faster feedback."
^{Dave Farley}

Just like Test Driven Development (TDD), it will help you write testable code. I have also noticed a nice side effect from this workflow: REPL Driven Development encourages a functional programming style.

REPL Driven Development is an everyday thing among Clojure developers and doable in Python, but far less known here. I'm working on making it an everyday thing in Python development too.

But what is REPL Driven Development?

What is it?

You evaluate variables, code blocks, functions - or an entire module - and get instant feedback, just by a hitting a key combination in your favorite code editor. There's no reason to leave the IDE for a less featured shell to accomplish all of that. You already have autocomplete, syntax highlighting and the color theme set up in your editor. Why not use that, instead of a shell?

Evaluate code and get feedback, without leaving the code editor.

Ideally, the result of an evaluation pops up right next to the cursor, so you don't have to do any context switches or lose focus. It can also be printed out in a separate frame right next to the code. This means that testing the code you currently write is at your fingertips.

Easy setup

With some help from IPython, it is possible to write, modify & evaluate Python code in a REPL Driven way. I would recommend to install IPython globally, to make it accessible from anywhere on your machine.

pip install ipython

Configure IPython to make it ready for REPL Driven Development:

c.InteractiveShellApp.exec_lines = ["%autoreload 2"] c.InteractiveShellApp.extensions = ["autoreload"] c.TerminalInteractiveShell.confirm_exit = False

You will probably find the configuration file here: ~/.ipython/profile_default/ipython_config.py

You are almost all set.

Emacs setup

Emacs is my favorite editor. I'm using a couple of Python specific packages to make life as a Python developer in general better, such as elpy. The auto-virtualenv package will also help out making REPL Driven Developer easier. It will find local virtual environments automatically and you can start coding without any python-path quirks.

Most importantly, set IPython as the default shell in Emacs. Have a look at my Emacs setup for the details.

VS Code setup

I am not a VS Code user. But I wanted to learn how well supported REPL Driven Development is in VS Code, so I added these extensions:

You would probably want to add keyboard shortcuts to get the true interactive feel of it. Here, I'm just trying things out by selecting code, right clicking and running it in an interactive window. It seems to work pretty well! I haven't figured out if the interactive window is picking up the global IPython config yet, or if it already refreshes a submodule when updated.

Evaluating code in the editor with fast feedback loops.
It would be great to have keyboard commands here, though.

Current limitations

In Clojure, you connect to & modify an actually running program by re-evaluating the source code. That is a wonderful thing for the developer experience in general. I haven't been able to do that with Python, and believe Python would need something equivalent to NRepl to get that kind of magic powers.

Better than TDD

I practice REPL Driven Development in my daily Python work. For me, it has become a way to quickly verify if the code I currently write is working as expected. I usually think of this REPL driven thing as Test Driven Development Deluxe. Besides just evaluating the code, I often write short-lived code snippets to test out some functionality. By doing that, I can write code and test it interactively. Sometimes, these code snippets are converted to proper unit tests.

For a live demo, have a look at my five minute lightning talk from PyCon Sweden about REPL Driven Development in Python.

Never too late to learn

I remember it took me almost a year learning & developing Clojure before I actually "got it". Before that, I sometimes copied some code and pasted it into a REPL and then ran it. But that didn't give me a nice developer experience at all. Copy-pasting code is cumbersome and will often fail because of missing variables, functions or imports. Don't do that.

I remember the feeling when figuring out the REPL Driven Development workflow, I finally had understood how software development should be done. It took me about 20 years to get there. It is never too late to learn new things. 😁

^{Top photo by ckturistando on Unsplash}

Can we have that in Python too?

(REPL Driven Development)

REPL Driven what? 😐

REPL Driven Development is about fast feedback loops during development. It is not about typing code into a terminal window. Don’t do that. A Read Eval Print Loop (REPL) is often described as a shell tool that you use to try out a programming language. That’s not what I mean with REPL Driven Development.

What is it then?

With this workflow, the REPL is acting behind the scenes. You don’t have to type code in a shell. Yes, a Read Eval Print Loop process is running, but in an interactive mode. You do all the work in your code editor. That’s where you have autocomplete, syntax highlighting, your favorite color theme and everything.

⚡ The Workflow ⚡

The feedback loop is fast in REPL Driven Development. You evaluate code blocks and functions as soon as you have typed them. The result of an evaluation is printed out on your screen. A common thing with REPL Driven Development is to write some short lived code snippets next to the actual code that you are developing. With this, you are able to quickly run a function with some test data as input and verify the output.

This workflow is similar to Test Driven Development (TDD), but the feedback loop is much faster. After a while, you might discover that those code snippets should actually be a proper unit test. Well, wrap the code in a test function, move it to a separate file and your’e all good.

So, REPL Driven Development is lazy TDD?

Yeah, that’s probably right. Another way of looking at it might be that it is Test Driven Development Deluxe.

The Interactive REPL is an essential tool in the Clojure & ClojureScript ecosystems. It is a superpower I’ve not seen before anywhere else. Once learned, it is something you’ll want in other languages too. But is that even possible, outside of a Lisp environment?

Evaluating vars, state and functions in real time with ClojureScript

Can we have that in Python too?

I think I have found something similar to the Clojure way of writing software, that works really well. You can pass an entire buffer, a selected region or a function from the code editor into a running IPython process, that quickly evaluates the code and prints the result. I have replaced the standard Python shell with IPython. I’ll explain why.

When evaluating code, IPython will output the result while the cursor remains in the code editor. That is what I mean with fast feedback loops. Evaluated functions, variables and imports can also be redefined - without having to restart anything. I’ve configured IPython to auto-reload sub modules when changed, to avoid having to restart the process. That feature is essential for this kind of workflow and the main reason why I’ve switched out the standard Python shell.

REPL Driven development in Python

It would be really cool to have the output pop up in the editor, right next to the actual code (as when evaluating Clojure or ClojureScript). I think that’s mainly a tooling thing, currently not available.

Also, I haven’t yet figured out how to connect to an actual running Python program and redefine functions or variables while the program is running (as you do in Clojure).

However, IPython is closely related to Jupyter, often referred to as an interactive notebook. There’s a thing called the kernel, handling code evaluation and returning result to connected clients. Sounds a bit like NRepl & Cider! I should probably dig deeper into and learn more about the possibilities with Jupyter.

Joyful Python

To summarize, even though there are some limitations, I think the REPL Driven Development workflow makes writing Python code joyful. The fast feedback loop is a great thing to have at your fingertips. Just like TDD, REPL Driven Development can be very helpful when writing testable, simplistic and functional code.

And it is fun too. 😁

Update: have a look at this post of mine from 2022, for more info about how to setup your code editor. You'll also find a 5-minute video in there about this style of development.

^{Top photo by Hitesh Choudhary on Unsplash}

Test Driven Development Deluxe

You have probably already heard about the red-green-refactor workflow when practicing Test Driven Development: start with a failing test, make the test pass, refactor and repeat.

I don’t think I have ever been very strict about the TDD workflow. I usually jump back and forth between the code and the unit test, often begin with trying out naming of functions and less-important things like that. I also find refactoring difficult at that stage, probably because the goal is to write code that does something and that does it well enough. I find it much easier to reflect and come up with refactoring ideas once I see a Pull Request diff on GitHub.

Too much context switch

Until recently, I have failed to find a workflow that is seamless when writing, running and testing code. There is often context switching involved: executing commands that run tests in a terminal or navigating the test runner of an IDE. Sometimes that means a pause with a delay of seconds, or even minutes. The eyes and the brain have lost attention to the code. The focus and flow is gone. Now where was I?

What would Clojure do?

These days, the programming language that makes me 🤩 is Clojure. It has been quite a journey learning how to use the language and how to use the code editor properly. I guess most of us don’t pick up all the good parts of a language or a tool immediately. It takes a while to get there.

An example: my brain hasn’t had the bandwidth enough to get how to use the interactive REPL until recently. There has been a lot of other things to unpack: language syntax, functional programming concepts and keyboard shortcuts. On a positive note: there are opportunities for daily learning and that’s where I am today.

- Wow, I can evaluate an expression in the code editor and the result pops up right next to it!

Star struck. 🤩

The other day I was frustrated with the slow feedback from a CLI test command running in a terminal window. I guess it has something to do with the Java Virtual Machine starting up every time a command is executed. I’ve tried out tools with file watchers, reacting to changes, that speed things up - but there’s still context switching involved, moving your eyes off the editor & the code.

What would a Clojurist do? I think the answer would be:
- Use the REPL.

The REPL, you must use

With Clojure, using the REPL means writing code in the editor, with autocomplete and everything. Also, evaluating the code while in the editor. There is no need to write a statement in a separate terminal window. Your editor has a connection to the interactive REPL (a running server), that will evaluate your code and return the result back to you. With the interactive REPL, you can quickly try things out, without switching to a different tool or view.

This is what some people call REPL Driven Development. Before I learned about it, I have always thought of a REPL as something that is used in a terminal window, away from the editor and away from the code.

Development Deluxe

By combining REPL Driven & Test Driven Development we have a setup that solves the issues with context switching and slow test runs. From what I’ve learned and tried out so far, I think it is a very nice workflow. 🤩

A very nice workflow

With REPL Driven Development, you write code, evaluate it and get instant feedback while keeping your focus on the code. Maybe the quick visual feedback in the editor is good enough to continue working? Great. Or, you write one or two commands wrapped in a runnable Clojure (comment) block just below the functions, to simplify the evaluation.

Everyone needs a coffee break. Commit the code (yes, the commands too) and continue with it later. ☕

Take a break

Your brain has been charged and you feel refreshed again. When returning to the code, you realize that the commands are probably evolving into something similar to a unit test. Should it be a proper unit test? If so, wrap it in a (deftest) function with some assertions and put it in a separate file.

Here’s a tip: give the unit test the same namespace as the actual code, but add a -test suffix to it. When doing so, your code editor (if it is smart) can run the corresponding tests when in a namespace. If you are an Emacs user, simply press the keys C-c C-t n to run the tests.

Even though you have put the unit tests in a separate file, in a separate folder, there is no need to switch context. Your tests run from where you are and you get instant feedback on the result. Very cool.

I think this is Test Driven Development Deluxe. 🤩

☕

<sup>Photo by Felipe Giacometti on Unsplash.

The screenshots are from email-attachments - A Clojure library that makes extracting attachments from email simple.</sup>