fredag 15 mars 2019

Pythons & Windows

For a couple of years now, I have been developing software on machines with Mac OS X or Linux. Installing dev environments and tools for coding in languages like Node.js or Python is usually without any issues. Or is it just luck? At work, our team maintain packages, apps, tools and frameworks written in different languages (JavaScript, C#, Python, Ruby). Everything we develop should be platform independent.

Recently, we built a tool written in Python 3 and the tool is intended to be used by many teams within the organization. So, I asked a fellow developer with a Windows machine to try it out. As it turned out, installing Python on Windows was ... a bit different. 

Why use more than one version of Python?
It's important for us to be able to run both Python 2.7 and 3 on the same machine, because of peer projects using build tools like node-gyp (that still has dependencies to Python 2).

This is what we learned about setting up and run both the Pythons on a Windows machine.

First step: Download and install Python
Download the latest version of Python from https://www.python.org/downloads/
Install Python 3.7.2 or later using the downloaded installer.

What about Python 2.7?
Ok, Python 2.7 was already installed on the Windows machines we used, so we skipped that step. But you can download and install it from the same python.org site, or install the windows-build-tools.

Use PowerShell
PowerShell is cool. To make PowerShell as smooth as Bash, I think you should create aliases for the different Python versions. An alias can be added to the PowerShell profile. Check out the guide from Microsoft about verifying existing profiles and creating new ones.

Adding Python aliases to your PowerShell profile will make it possible to run both Python 2.7 and Python 3 in the same way as in other systems.

But where did the Windows installers put the Pythons? To find out, you can run this command in PowerShell:
where.exe python

Now you're ready to edit the PowerShell profile. Open it by typing notepad $profile, and add these rows:
New-Alias python C:\<my path to Python 2.7>\python.exe
New-Alias python3 C:\<my path to Python 3.7.2 or later>\python.exe 

Note: you may need an additional step, enabling profile scripts to run when a PowerShell window is opened. To do that, open a new PowerShell window in Admin mode (important) and write:
Set-ExecutionPolicy RemoteSigned

When finished, close the admin window, and open a new regular one. 

Almost done 
Verify the aliases, by typing:

python --version
(should output something like 2.7.*)

python3 --version 
(should output something like 3.7.*)



The Python virtual environment on Windows
Creating a virtual environment is the same as in Mac OS X or Linux: Install it with pip install virtualenv (only once), and create a virtual environment in your Python project folder with virtualenv venv.

To activate in bash, I'm used to write source venv/bin/activate

But that doesn't work on Windows. Instead, VirtualEnv for Windows have custom PowerShell scripts prepared for activation. You'll find them in the venv\Scripts folder that virtualenv has created with the virtualenv venv command.

.\venv\Scripts\activate.ps1 will activate the virtual environment in your Python folder.


With this setup, running Python apps will be close to identical to the setups in Mac OS X and Linux.



söndag 10 mars 2019

Updates to the Apache ZooKeeper Node.js client

At last, Node.js is platform independent again!

Eh, what?

Apache Zookeeper?
From the docs: "... ZooKeeper is a distributed, open-source coordination service for distributed applications. It exposes a simple set of primitives that distributed applications can build upon to implement higher level services for synchronization, configuration maintenance, and groups and naming. It is designed to be easy to program to, and uses a data model styled after the familiar directory tree structure of file systems. It runs in Java and has bindings for both Java and C. ..."
(the docs)

A Node.js Zookeeper Client
You will find several Node.js implementations when browsing on npmjs.org, and one of them is simply called: zookeeper. Recently, I was added to the list of maintainers of the project, hosted on GitHub.

Background
The project got my attention when we at work were looking for a new ZooKeeper client, because the API of the current one we were using had gone out of date. We were all developing on Mac OS X, and pushed changes to a CI using a Linux docker container. Implementing the new client seemed like a breeze.

Until a developer in a peer team came by our spot at the office:
- Hey, I cloned the app repo and followed your setup guide. But I get errors when installing packages. I think it's the "zookeeper" dependency that fails. Do you know what's wrong?

We quickly learned it was because Windows. We realized that the entire build process - compiling C code to a native Node.js add-on on the fly - used non-Windows compliant features like make, bash and gcc.

Oh sh*t. How can this be solved?
What about bypassing it with Docker? I've learned that Docker on Windows can be painful, because of access right issues between a Windows host and Linux in a container. Windows Subsystem for Linux (WSL) would probably work after some tweaking, but adding WSL as a requirement to run (or develop) an app is ... well ... a bit depressing.

How about fixing the root cause of the problem, the source code? I did choose that path and have spent a lot of time learning about Node.js add-ons, (a little bit of) C and C++, Windows equivalents to bash commands, node-gyp, and how to compile the Apache Zookeeper C Client from source.

Months later, I hit the "Create Pull Request" button. Shortly after, the changes were merged and an updated npm package was published.

Besides supporting Windows, the zookeeper npm package also builds on Node.js 10. Previous versions failed, because the use of deprecated V8 features.

Details of recent changes can be found in the Changelog.

Contribute to the Open Source project
If you are interested in contributing, check out the issues tab at the GitHub repo. Currently there are some issues labeled with "good first issue", go ahead and grab one!

tisdag 1 augusti 2017

NDC Oslo Video: JavaScript in 2017 - You might (not) need a framework

The sessions from NDC Oslo 2017 are online!


I had a good time in Oslo, got new insights and left the Norwegian capital with inspiration. There were many interesting talks and lots of fellow geeks to chat with. I am also glad that so many people joined my talk. Thank you! I hope you have learned something useful.


Here's the recording of my NDC Oslo 2017 talk:





(and here's the code used in the talk)

Click the Youtube "Like" button if you like it! Also, I would very much appreciate your feedback. Please write your feedback at the comments section of the Youtube page (or this blog post) if you come up with ideas, questions, suggestions or things that should be improved.



Thank you NDC Oslo!

onsdag 4 januari 2017

You might (not) need a JavaScript library

JavaScript today is like:
"Yeah great, and now you need Webpack, Babel, npm, yarn, React ...".

Redux? Relax.

Developing a front end web project of today can be a bit intimidating, especially when the home base is back end development. What about all those frameworks and tools, why use them?

I am currently learning about React, and I like it. React have JSX, ES2017 and a nice logo. That is cool, but my favorite thing with React is how code is organized. A user interface is built with components: small and isolated "packages" of html & JavaScript. That is a pattern I like.

Components with vanilla JavaScript?
When learning about React patterns, I started thinking about how this could be done without using React, ES2017, Webpack or any of the other libraries and frameworks out there. Is that even possible?

Okay, but why?
I want to learn and understand the problems that are solved using a library. Also, I want to find out what problems can be solved without an npm install. One way of doing that is to write all code with plain old vanilla JavaScript, html, css and find out what pain is removed by which library. Also, I think it would be a fun challenge!

Example code
You will find all code referenced in this blog post at my GitHub page

No build step required
So, I spent some late nights coding and learning. The code in the main branch of this repo does not require any build steps or npm package downloads. The "listItem component" is made of two parts: JavaScript in a code file and an html template in a separate file. The render function will create a DOM object containing html from the template, with data that is passed in to it, and return it in a callback.

code from listItem.js:
      function render(props, done) {

        // load the template using a helper
        templates.load('/src/listItem/listItem.html', function (el) {

          // 'el' is the html template element
          el.textContent = props.data;
          el.addEventListener('click', props.onClick);

          // pass the element to be added to the DOM
          if (done) {
            done(el);
          }
        });
      }

      return {
        render: render
      };
    
The listItem.html template:
      <li class="listItem" title="the listItem component"></li>
    
This is of course a very simplistic example with a single tag html template, but I think it already highlights issues: where's the data added? To understand where data is added, we have to read & understand the contents of the render function. I think it would be nice if the data to be rendered is visible in the actual template.

Time to grow a Mustache?
A template render engine can solve that. Here's the same component, using a template engine called Mustache.js. You will find the code in a separate branch of the GitHub repo.
      function render(props, done) {
        // pass data to a modified template loader
        templates.load('/src/listItem/listItem.html', props, function (el) {
          el.addEventListener('click', props.onClick);

          if (done) {
            done(el);
          }
        });
      }
    
compare with the vanilla code

The templates helper now use Mustache to render the html from the template and the data. It is now possible to write html templates with placeholders for data like this:
      <li class="listItem" title="the listItem component">{{data}}</li>
    
code from the templates.js file:
      container.innerHTML = Mustache.render(template, data);
    
More issues?
If you look at the source code in the main branch you'll notice the JavaScript files is written with a coding style called IIFE (immediately invoked function expression). It is used to isolate code and makes it possible to write modules without using any framework. Also, every single file is added with a script tag in the html body of the main page (index.html). Some modules depend on others and have to be added in the correct order. That's not great.
      <script src="src/templates.js"></script>

      <script src="src/listItem/listItem.js"></script>
      <script src="src/list/list.js"></script>
      <script src="src/terminal/terminal.js"></script>
      <script src="src/nav/nav.js"></script>
      <script src="src/logView/logView.js"></script>

      <script src="src/app.js"></script>
    
Solution: JavaScript AMD modules
In a separate branch, I have converted all of the immediately invoked function expressions (IIFE) to AMD modules. I use Require.js that takes care of module loading and dependencies. Instead of a very long list of html script tags, there is only an entry point defined.

from the index.html file
      <script data-main="src/app" src="lib/vendor/requirejs.js"></script>
    
Here is the listItem component as an AMD module:
      define(['templates'], function (templates) {
        function render(props, done) {
          templates.load('/src/listItem/listItem.html', props, function (el) {
            el.addEventListener('click', props.onClick);

            if (done) {
              done(el);
            }
          });
        }
        return {
          render: render
        };
      });
    
compare it with the previous branch

But wait. Don't we have native modules in Javascript now?
Oh, I forgot. It is 2017 and ECMAScript 2015 was released almost two years ago. A nice module system was included in it. Finally there is a common standard in the language! I have rewritten the code to ES2017 style - with arrow functions, the const keyword and most importantly, the ES import/export feature. Now, the listItem component looks like this:
      import load from 'templates';

      export function render(props, done) {
        load('/src/listItem/listItem.html', props, (el) => {
          el.addEventListener('click', props.onClick);

          if (done) {
            done(el);
          }
        });
      }
    
compare the ES2017 code with old school JavaScript

I think the code has improved a bit. ES2017 is great, but there are trade offs to be aware of. Many browsers don't have enough support for this version of JavaScript yet. To make it work in all kinds of browsers and devices we need to introduce a build step: the code need to be compiled from ES2017 to vanilla JavaScript with Babel.

The package.json file in the project has quite a few scripts compared to the original framework-and-build-step-free version. In addition to dependencies like Mustache.js and Require.js, there is a compile step and a Babel polyfill dependency added:
      "scripts": {
        "deps:lib": "mkdir -p -v lib/vendor",
        "deps:requirejs": "cp node_modules/requirejs/require.js ...
        "deps:mustache": "cp node_modules/mustache/mustache.min.js ...

        "deps:polyfill": "cp node_modules/babel-polyfill/dist/polyfill...
        "deps": "npm run deps:lib && npm run deps:requirejs && npm run ...
        "transpile": "babel src --out-dir lib --source-maps",

        "lint": "eslint src",
        "build": "npm run lint && npm run transpile && npm run deps",
        "start": "npm run build && live-server"
      }
    
More frameworks, more problems?
When browsing the page there is now a couple of third party libraries loaded to the client, besides our own modules. This might cause a not so great experience for users with a slow connection.

Bundling & minification
While we're at it, why not add another build step that will bundle all JavaScript files to one single file? This will reduce the number of requests from the browser. With minification we also will get rid of a couple of Kilobytes. The entry point is now one bundled and minified JavaScript file.
      <script data-main="lib/bundle/main"
              src="lib/vendor/requirejs.js"></script>
    
The source code in this branch is compiled from ES2017 to browser friendly AMD modules. With Require.js, there is a tool for bundling & minification included (called R.js) and used in this branch. compare the branches

Heard about Webpack?
The scripts section of the package.json file is quite massive now and probably difficult to understand. By using Webpack, most of those build steps are no longer necessary. Webpack does a lot of things, it's like a swiss army knife (that's both good and bad, I guess).

package.json with Webpack:
      "scripts": {
        "lint": "eslint src",
        "build": "npm run lint && webpack",
        "start": "webpack-dev-server"
      }
    
compare the two branches, with bundling vs with Webpack

Where did it all go, how is that even possible? Okay, I forgot to mention Webpack.config. Sorry. Some of the build magic live in that file now.

So, did Webpack make any difference?
One nice thing with Webpack is that there is no longer any need for Require.js. Webpack will resolve ES2017 modules and convert them to plain vanilla JavaScript before the bundling & minification. Also, Webpack has a local dev server feature (with auto reloading on file change) that I like.

Add React to the mix
This is how the listItem component looks like when converted to React. The template files are gone, everything is written in the JavaScript modules using the JSX syntax. There is no longer need for a custom template loader or mustaches. Compared to the source code in the previous branch, this one has less code. I like less code.
      import React from 'react';

      function ListItem(props) {
        return <li className='listItem' title='the listItem component'
        onClick={props.onClick}>{props.data}</li>;
      }

      export default ListItem;

    
React: Before vs After

Conclusion
By experimenting with one library at a time, I have learned about the value added and also some of the trade offs that comes with using a tool or a framework. Sometimes, plain vanilla JavaScript is just enough, and sometimes a framework or library will make life easier. You might (not) need a JavaScript library.

tisdag 10 maj 2016

Unit test Commerce projects with FakeMaker

FakeMaker is updated with support for unit testing Episerver Commerce code. I am very excited about this and hope it will be useful in your projects. The new version is made with contributions and feedback by users of FakeMaker. Thank you! 

Install the new plugin to FakeMaker - called FakeMaker.Commerce - from NuGet, or check out the source code on GitHub. The package currently suppport basic unit testing scenarios. What kind of features are missing? Contact me by creating a GitHub issue, sending a pull request or just write a comment on this blog.

What is FakeMaker?
FakeMaker takes care of mocking and a simplifies creating fake content, that you can use when testing your code.

Unit testing the EPiServer CMS got a whole lot easier alredy with version 7. However, creating fake content to be used when setting up mocked repositories isn't always that smooth. When mocking of code is all you see on your screen, this little library may help. A bigger screen probably also would, but is probably more expensive. FakeMaker make it easier to write unit tests for mvc controllers and helpers that expect the episerver repositories to return content. And now also for creating fake Commerce content.

How does the Commerce support work?
Just like when creating fake pages in FakeMaker

var page = FakePage.Create("MyPageName");

you can create fake products with FakeMaker.Commerce in the same way:

var product = FakeProduct.Create<ProductContent>("My Fake Product");

Add pages and products to the repository and you are ready to go.

fake.AddToRepository(product);

Get FakeMaker
Read more, have a look at the source code and contribute to the project: https://github.com/DavidVujic/EPiServer-FakeMaker

FakeMaker NuGet package: https://www.nuget.org/packages/FakeMaker/
FakeMaker.Commerce NuGet package: https://www.nuget.org/packages/FakeMaker.Commerce/