Monday, July 17, 2023

GCP Cloud Functions with Python and Polylith

Running a full-blown 24/7 running service is sometimes not a good fit for a feature. Serverless functions (also known as Lambdas or Cloud functions) is an alternative for those single-purpose kind of features. The major cloud providers have their own flavors of Serverless, and Cloud Functions is the one on Google Cloud Platform (GCP).

GCP Cloud Functions & Polylith

Polylith is good choice for this. The main use case for Polylith is to support having one or more Microservices, Serverless functions or apps, and being able to share code & deploy in a simple and developer friendly way.

One thing to notice is that GCP Cloud Functions for Python expects a certain file structure when packaging the thing to deploy. There should be a in the top directory and it should contain the entry point for the function. Here's an example, using the functions-framework from GCP to define the entry point.

  import functions_framework

  def handler(request):
      return "hello world"

You'll find more about Python Cloud Functions on the official GCP docs page.

In addition to the, there should also be a requirements.txt in the top folder, defining the dependencies of the Cloud function. Other than that, it's up to you how to structure the code.


How does this fit in a Polylith workspace?

I would recommend to create a base, containing the entry point.

    poetry poly create base --name my_gcp_function
Never heard about Polylith? Have a look a the documentation.

Polylith will create a namespace package in the bases directory. Put the handler code in the already created file.

Make sure to export the handler function in the of the base. We will use this one later, when packaging the cloud function.

    from my_namespace.my_gcp_function.core import handler

    __all__ = ["handler"]

If you haven’t already, create a project:

    poetry poly create project --name my_gcp_project

Just as with any other Polylith project, add the needed bricks to the packages section.

 packages = [{include=”my_namespace/my_gcp_function”,from="../../bases"}]

 # this is specific for GCP Cloud Functions:
 include = ["", "requirements.txt"]

Packaging a Cloud Function

To package the project as a GCP Cloud Function, we will use the include property to make sure the needed and requirements.txt files are included. These ones will be generated by a deploy script (see further down in this article).

Just as we normally would do to package a Polylith project, we will use the poetry build-project command. Before running that command, we need to make some additional tasks.

Polylith lives in a Poetry context, using a pyproject.toml to define dependencies, but we can export it to the requirements.txt format. In addition to that, we are going to copy the "interface" for the base ( into a file. This file will be put in the root of the built package. GCP will then find the entry point handler function.

To summarize:

  • Export the dependencies into a requirements.txt format
  • Copy the interface into a
  • run build-project
Depending on how you actually deploy the function to GCP, it is probably a good idea to also rename the wheel that has been built to "". A wheel is already a zipped folder, but with a specific file extension.

 # export the dependencies
 poetry export -f requirements.txt --output requirements.txt
 # copy the interface into a new file
 cp ../../bases//messages_gcp_function/ ./

 # build the project
 poetry build-project
 # rename the built wheel
 mv dist/my_gcp_function_project-0.1.0-py3-none-any.whl dist/

That's all you need. You are now ready to write & deploy GCP Cloud Functions from your Polylith workspace. 😄

Additional info

Full example at: python-polylith-example
Docs about Polylith: Python tools for the Polylith Architecture

Top photo by Rodion Kutsaiev on Unsplash

No comments: