Title: Deploying Streamlit App as a Containerized App

  • Understanding Container
  • Deploy App on Container Stack (CStack)




Congratulations on successfully building and deploying Streamlit apps on the Streamlit Community Cloud! As you continue to develop more sophisticated applications, you might encounter challenges related to environment setup, dependency management, scalability, and reproducibility. This is where Docker comes into play. In this article, we'll delve into why Docker is a game-changer for developers, especially those working with Streamlit applications.




1 Understanding the Problem: Why Do We Need Docker?

Before diving into Docker's benefits, it's essential to understand the pain points developers often face without it:

1.1 Environment Inconsistencies

When developing applications, especially in teams, ensuring that every team member's development environment replicates the production environment can be challenging. Differences in operating systems, installed libraries, or versions can lead to bugs that are hard to trace.

1.2 Dependency Hell

Managing dependencies can become cumbersome. Conflicting library versions required by different projects can lead to complex and error-prone setups. Without a containerization tool, isolating these dependencies for each project is tricky.

1.3 Deployment Challenges

Deploying applications across different environments (development, staging, production) often requires manual configuration, increasing the risk of human error. Ensuring consistency across these environments can be time-consuming.

1.4 Scalability Issues

As your Streamlit app grows or needs d traffic, scaling it across multiple servers or services can be daunting without a standardized deployment method.

1.5 Reproducibility

Reproducing bugs or issues in a different environment can be difficult if the setup isn't consistent. This hampers debugging and prolongs development cycles.

  • No worries…. Docker🐳 to the rescue!



2 What is Docker🐳 ?

  • ✦ Now is a good time to ask, what is docker?

    • It will help you solve the problem of varying development environments for app shipping and deployment.
    • Docker made simple… Allowing you to run apps anywhere.
    • Docker🐳 is a technology to create and manage containers.
      • Wait… what?
      • A container is nothing but a software unit, i.e. your node app's code, and its dependencies.
      • Containers allow execution of your app in isolation, anywhere in the world, by anyone, without you worrying about the environment set-up….. Ain’t that just awesome!
      • It is a tool used to create, deploy and run applications by using containers.
      • Containers allows developers to package up an application with all the parts it needs.
      • Containers are isolated from one another and bundle their own software, libraries and configuration file.
  • ✦ Docker is a platform designed to make it easier to create, deploy, and run applications by using containers. Containers allow developers to package an application with all of its dependencies into a standardized unit for software development. This ensures that the application runs seamlessly in any environment, whether it's on your local machine, a colleague's laptop, or a cloud server.

  • At its core, Docker addresses the classic "it works on my machine" problem by providing a consistent environment across various stages of development and deployment.

Difference between Docker Containers and Virtual Machines
  • Docker Containers
    • Docker Containers contain binaries, libraries and configuration files along with the application itself.
    • They don’t contain a guest OS for each container and rely on the underlying OS kernel, which makes the containers lightweight.
    • Containers share resources with other containers in the same host OS and provide OS level process isolation.
  • Virtual Machines
    • Virtual Machines (VMs) run on Hypervisors, which allow multiple Virtual Machines to run on a single Machine along with its own operating system.
    • Each VM has its own copy of an operating system along with the application and necessary binaries, which makes it significantly larger and it requires more resources.
    • They provide Hardware-level process isolation and are slow to boot.



3 How does Docker🐳 do the magic?

  • ✦ Docker🐳 is a container management framework, it allows you to create, share and manage images and containers.

  • ✦ But what are containers and images in the first place and how do they allow us to solve the problem of varying development environments?


3.1 Images

  • ✦ Of course, images are not framed pictures 😁… Just remember three things:
    • Images are blueprints/templates for a container.
    • It contains our code and its dependencies.
    • We can run a container on top of an image, to run your application.

3.2 Containers 📦

  • ✦ You see Docker’s logo in the image up there.

    • It has a whale carrying “Containers” across the sea from one country to another.
    • That is exactly what a container is….
  • ✦ It creates an isolated environment for running your application at the host machine using information (code + dependencies) from the image.

    • It’s a running unit of our application.
    • In more technical term, It is a runtime instance of an image.
  • ✦ With containers and images, we won’t have to worry about installing and uninstalling dependencies on a host / server, before running our application there.

    • We simply create an image (snapshot of your running app i.e. code+depedencies like Python package or even other backend processes like OCR engined) and share it with our colleague. 
    • Our colleague can now simply spin up a container on top of that image and can run your app, without having to worry about anything.

3.3 Docker file

  • ✦ It is a text document that contains necessary commands which on execution helps assemble a Docker Image.
  • Docker image is created using a Docker file.
  • ✦ Imagine this as the recipe to make the Docker Image.

3.4 Docker Engine

  • ✦ The software that hosts the containers is named Docker Engine.

3.5 Docker Hub

  • ✦ DockerHub is a cloud-based registry service provided by Docker, Inc. It allows developers to store, share, and manage Docker images.
    • It makes it easy to find, manage and share container images with others.
    • Think of it as a vast library where you can find pre-built images for a wide range of applications, from simple web servers to complex multi-service applications
  • Key Features of DockerHub
    • Image Repository:
      • DockerHub hosts millions of Docker images, both official and community-contributed.
      • Official images are maintained by Docker and verified for quality and security, making them a reliable choice for production environments.
    • Private Repositories:
      • While DockerHub offers public repositories for free, it also provides private repositories for users who need to keep their images confidential.
      • This is particularly useful for proprietary software or internal tools.
    • Teams and Organizations:
      • DockerHub supports collaborative workflows by allowing you to create teams and organizations.
      • This feature helps manage access control and permissions, ensuring that only authorized users can push or pull images.
    • Search and Discovery:
      • The search functionality in DockerHub makes it easy to find the images you need.
      • Whether you’re looking for a specific version of a database or a particular application framework, DockerHub’s extensive catalog has you covered.



4 Benefits of Using Docker

Docker addresses the aforementioned challenges by providing a robust framework for containerization. Let's explore the key benefits it offers:


4.1 Streamlined Deployment of Streamlit App

Using Docker, deploying a Streamlit app becomes as simple as building and running a container. Whether deploying to cloud providers like AWS, GCP, or Azure, Docker ensures that your app runs consistently across platforms.


4.2 Consistency Across Environments

Docker containers encapsulate an application and its environment. This means the same container can run on a developer's laptop, a testing server, or a production environment without any modifications. This uniformity eliminates discrepancies that arise from varying system configurations.

Example: Suppose you develop a Streamlit app on a Windows machine, but it needs to run on a Linux-based cloud server. Docker ensures that the app runs identically in both environments, sidestepping issues like file path differences or system-specific dependencies.


4.3 Simplified Dependency Management

With Docker, all dependencies are defined within a Dockerfile. This file specifies the base image, necessary libraries, environment variables, and other configurations required to run the application. The Dockerfile ensures that anyone building the container gets the exact dependencies needed, avoiding the dreaded "dependency hell."


4.4 Scalability and Portability

Docker containers are lightweight and can be easily replicated or scaled across multiple servers or services. This makes it easier to handle increased traffic or deploy microservices architectures.

Example: If your Streamlit app gains popularity and needs to handle more users, Docker allows you to spin up multiple container instances effortlessly, balancing the load with orchestration tools like Kubernetes.


4.5 Isolation and Security

Containers run in isolated environments, ensuring that applications and their dependencies do not interfere with each other. This isolation enhances security by limiting the application's access to system resources and preventing malicious code from affecting other containers or the host system.

Example: Running multiple Streamlit apps on the same server becomes safer with Docker, as each app operates within its container, reducing the risk of cross-contamination or unauthorized access.


4.6 Efficient Collaboration

Docker facilitates collaboration by allowing team members to share containers. This ensures that everyone works with the same environment, minimizing setup times and configuration issues.

Example: A data scientist can share a Docker image of a Streamlit app with all its dependencies, ensuring that collaborators can run and modify the app without wrestling with environment setup




5 🚀 ️Let’s Dockerize!

To dockerize an app, you need to follow the following simple steps.

  • Prepare a Docker File (the recipe to create the Docker Image)
    • Create a file with filename Dockerfile (note: without any extension) at the root of our project folder
    • Copy the content below to the newly created Dockerfile
# Use the official lightweight Python image.

# https://hub.docker.com/_/python
# This lien pecifies the base image for the Docker container
# It uses the official Python image from Docker Hub, 
# specifically the version 3.12 with the "slim" variant
# which is a smaller, more lightweight version of the full Python image.
FROM python:3.12-slim

# This ensures that Python output is not buffered, 
# which is useful for real-time logging and debugging.
ENV PYTHONUNBUFFERED True
# This line is needed for the app to work in CStack 
RUN groupadd --gid 1001 app && useradd --uid 1001 --gid 1001 -ms /bin/bash app

#  Sets the working directory for the container to `/home/app`. All subsequent commands will be run from this directory.
WORKDIR /home/app
# Copies the `requirements.txt` file from the local machine to the current working directory in the container (`/home/app`).
COPY requirements.txt ./

RUN pip install -r requirements.txt
# This line is needed for the app to work in CStack 
USER 1001

# Copy local code to the container image.
#  Copies all files from the local directory to the current working directory in the container (`/home/app`), and changes the ownership of the copied files
COPY --chown=app:app . ./

# (Optional) Add any additional commands here

# Run the web service on container startup.
# Informs Docker that the container will listen on port 8501 at runtime. 
# This is used for documentation purposes and does not actually publish the port.
EXPOSE 8501
# Specifies the command to run when the container starts. In this case, it runs a Streamlit application using the `main.py` script.
CMD streamlit run main.py
You don't have to worry about fully understand or writing the Dockerfile from scratch.
  • ✦ Firstly, we can reuse this Dockerfile to deploy our Streamlit app in majority of container hosting services (available from major Cloud Service Providers like AWS, Azure, and Google Cloud) with no to minimal changes.
  • ✦ Secondly, if we need to deploy the container image to agency's infra, there are usually some assistance from the engineer or infra person who we will work with to deploy the built image.
  • Export the versions of the installed packages from the virtual environment into a requirements.txt

    • Navigate to the project folder
    • Activate the virtual environment
    • Run the command pip freeze > requirements
    • This is to allow the same packages to be used to build the image
  • Create a .dockerignore file at the root of our project folder

    • This file is to tell Docker which are the files that it should ignore and not to include into the image during the build process
    • We should include our .env filename in this file that contains our credentials like API Keys
    • We should also include the folder venv that contains all the files and Python interpreter that allow us to test the app locally.
    • Here is how the file would looks:
.env
venv
  • Build the image
    • Use default docker build for local development
    • Below is the syntax for building an image
      • where <image> is the name of the image that we are going to build
      • We recommend to give the image a meaningful and representative name, so we can quickly tell what the image is for.
docker build -t <image> .
  • Run container based on build image locally

    • See the video below on how to run the container from Docker Desktop
    • Remember to set up the environment variables from our .env file using the the pop-up dialog on Docker Desktop
  • Upload the image to DockerHub

    • This is done when we are sure the containerized app works without issues.
    • <username> is your unique DockerHub username. We can find it under the pop up after you clicking on the avatar after logging in to DockerHub
    • Below is the command for building and uploading (more commonly we say "push") the image to DockerHub.
docker buildx build --push --platform linux/amd64 -t <username>/<image> .
Note that --platform linux/amd64 is flag (like a parameter) to allows us to define the target platform for the Docker image.
  • ✦ This should be determined by the hosting server where the Streamlit app will be deployed.
  • linux/amd64 specifies two things:
    • the target platform is Linux operating system running on an amd64 (x86_64) architecture.
    • the amd64 processor architecture is the most common architecture for desktop and server processors, including those from Intel and AMD.



6 Video Walkthrough

6.1 Video 1

Major Timeline

00:00 - Overview
01:17 - Prepare a Docker File (the recipe to create the Docker Image)
03:03 - Export the versions of the installed packages from the virtual environment into a requirements.txt
07:12 - Create a .dockerignore file at the root of our project folder
09:42 - Build the image


6.2 Video 2 (Continue.)

Major Timeline

00:00 - Running Container from the Image locally
03:50 - Upload the Image to Docker Hub