Live Algo Trading on the Cloud – Microsoft Azure

What does live algorithmic trading on the Cloud mean?

Live algorithmic trading on the Cloud means that your trading bots can use the cloud provider’s resources to run 24/7 while being easily maintainable.

This is article, we will demonstrate how to deploy a live algorithmic trading strategy to the cloud server using Microsoft Azure and Docker.

Prerequisite – Basic Guide

What are the pros of deploying your trading strategies to the Cloud?

The pros of deploying trading strategies to the Cloud are:

• Cloud services are easily accessible – they allow the user to access them at any time, from anywhere, and from almost any device.
• Can lower costs – cloud computing can lower the costs and provide you with a good connection and hardware that saves you money in the long run.
• Easy to maintain – you can upload your trading strategies to the cloud and get them running 24/7 with minimal effort.
• Is scalable – you can store almost limitless amounts of data in the cloud.
• Easy to set up – setting up your cloud server is quite easy.

What are the cons of deploying your trading strategies to the Cloud?

The cons of deploying trading strategies to the Cloud are:

• Security risks – your data is being stored by a third-party provider and if a security breach happens your data is at high risk.
• Depends on the internet connection – even though this is the same case when running trading strategies by yourself, the cloud service provider can also suffer from downtime and other natural risks.
• Data confidentiality risk – your privacy is of utmost importance and when using a third-party cloud provider you can’t be too sure that you’re the only one accessing your data or being able to see it.

What is the Cloud Service?

A Cloud Service delivers cloud computing as a service with an intent to deliver affordable, easy and efficient access to diverse resources without the need to have your own hardware or infrastructure.

What is the Cloud used for? 

Cloud services have many purposes for which they could be used for and here are some of the common ones:

• Algorithmic trading – cloud services allow algorithmic traders to run multiple bots that trade the market.
• Big data storage – clouds offer an almost limitless storage capacity which is great when dealing with big data.
• Backup and recovery – clouds can be frequently backed up and you can easily recover data from them. They also offer good disaster recovery.
• Test and Development – cloud services are great when testing and developing your applications and trading strategies.
• Model training – when doing machine learning work, model training can take up to several months, and running it on a cloud can save you computing power and in some cases even money.

What cloud providers are good?

There are many cloud services and picking a quality one is important. In order to make your decisions process easier I will mention a few that are on top of their cloud game:

What is Microsoft Azure?

Microsoft Azure is a cloud provider service that offers various cloud solutions for things such as algorithmic trading, data storage, development, AI and machine learning, app scaling, system monitoring, and more.

Why should I use Microsoft Azure?

• Microsoft Azure is cost-effective (pay as you go)
• Microsoft Azure is scalable
• Offers many cloud solutions
• Has SOTA security practices
• Has a good uptime track record
• Has good documentation with examples
• Is one of the largest and most known cloud providers

Why shouldn’t I use Microsoft Azure?

• Microsoft Azure can be confusing for beginners
• Microsoft Azure support can take a while to respond
• It can be hard to migrate off of Azure
• The interface might be complicated for beginners
• Might have a steep learning curve
• Can get pricey depending on the user’s needs

How to get started with Microsoft Azure?

What is the Azure CLI?

The Azure CLI (Command-Line Interface) is a way for us to interact with Microsoft Azure via code. This way, we don’t need to go over to their website to manually create, start and stop our cloud services.

How to get started with the Azure CLI?

What is Docker?

Docker is an open source containerization platform. It allows us to package our code into containers that can be easily deployed anywhere while being sure that the code in them will work and function in a standardized way.

Kraken exchange trading bot setup

In this article, we will explore a simple trading strategy that will feature the Kraken exchange, Azure CLI, and Docker. Before we set up anything, we need to explain how this strategy works and where you can learn how to code a similar one.

The main idea behind the strategy will be to buy ETH when BTC moves 5% in the last 5 minutes. For this to work, we will pull BTC prices in 5 minute intervals and calculate the percentage difference.

If this difference is more or equal to 5% we will buy ETH, if the requirement isn’t reached we will continue to loop around. The strategy also features error management, logging, and environmental variables.

Error management is used to catch errors that might occur, logging is used so we can obtain data about our bot and what it’s been up to while running on the server and environmental variables allow us to interact with the bot.

How to package a trading algorithm with Docker?

To package a trading algorithm with Docker, we will need to have a file named dockerfile that has all the instructions on how to make our algorithm ready by copying code files we need, installing libraries, passing the environment variables, and more.

Because Docker needs to know what dependencies we want to install for our algorithm, it is good practice to have a requirements.txt file that features all the libraries we need and their versions:

python-dotenv==0.21.0
pandas==1.4.4
krakenex==2.1.0
pykrakenapi==0.3.1

To get your package version you can write pip show <package>.

Now that the requirements are ready, we can move on to specifying our installation and other setup commands inside of the dockerfile. We first create the dockerfile by writing touch dockerfile.

The inside of our dockerfile looks the following way:

FROM python:3.8
RUN mkdir -p /code

COPY ./main.py /code/
COPY ./requirements.txt /code/

WORKDIR /code

# set environment variables
ARG API_KEY
ARG API_SECRET
ARG TRACK
ARG TRADE

ENV KRAKEN_API_KEY $API_KEY
ENV KRAKEN_API_SECRET $API_SECRET
ENV TRACK_TICKER $TRACK
ENV TRADE_TICKER $TRADE

RUN pip install -r requirements.txt
CMD ["python", "/code/main.py"]

In the code above we specify what version of python we’re using, then we create a folder /code to hold our script. Then, we copy over the two files we need to the newly created folder and set it as our working directory. We then provide environment variables that will be set when we build the docker image.

The ARG is your placeholder that is only accessible during the build time and then it vanishes, while the ENV will be accessible through your whole image process. We use the ARG parameter to dynamically pass our environment variables on docker image builds.

What is Azure Container Registry?

Azure Container Registry is a service that allows us to build, store, secure, scan, replicate, and manage container images and artifacts. You can think of it as GitHub but private and for containers.

How to create Azure Container Registry with Azure CLI?

To create an Azure Container Registry with the Azure CLI, we will need to have Azure CLI installed and authenticate our account through it. Then we will create our registry and push the container image to it.

To make sure that your Azure CLI is ready to go, write the following command in your terminal emulator:

az --version

azure-cli                         2.40.0

core                              2.40.0
telemetry                          1.0.8

Dependencies:
msal                            1.18.0b1
azure-mgmt-resource             21.1.0b1

Now, let us login into our account by writing az login. This will open up your browser and ask you to log in. After you do that, you can close the browser.

Before creating the Azure Container Registry we will need a resource group. To create a resource group we write the following :

az group create --name algotrading101Group --location eastus

{
  "id": "/subscriptions/...",
  "location": "eastus",
  "managedBy": null,
  "name": "algotrading101Group",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

How to push a Docker image to Azure Container Registry?

To push a Docker image to the Azure Container Registry, we must first tag the image with the full name of the registry’s login server. Let’s find that full name by writing the following command:

az acr show --name krakenexample101 --query loginServer --output table

Result
---------------------------
krakenexample101.azurecr.io

Now we tag the image, I’ll add :awesome as its tag. To recall the image name, run docker images.

docker tag azure-algotrading101 krakenexample101.azurecr.io/azure-algotrading101:awesome

To make sure the tag was set, run docker images again.

REPOSITORY                                         TAG      
krakenexample101.azurecr.io/azure-algotrading101   awesome

Finally, we can push our Docker image to the Registry by writing the following:

docker push krakenexample101.azurecr.io/azure-algotrading101:awesome

The speed of the push will depend on your internet connection and the size of your image. As this is a small image it shouldn’t take long in most cases. To make sure that the image has been properly pushed, we can inspect the registry for its images:

az acr repository list --name krakenexample101 --output table

Result
--------------------
azure-algotrading101

How to protect your Azure Container Registry?

To protect your Azure Container Registry from cybersecurity attacks, it is best practice to set up a feature named Azure Active Directory. The Active Directory is a service that provides single sign-on, multifactor authentication, and conditional access to guard against cybersecurity attacks.

In the case of our Container Registry, the best practice for many scenarios is to create and configure an Azure Active Directory service principal with pull permissions to the registry. Azure has us covered with an example of how to do this.

You can find the bash script for it in the GitHub repo linked at the bottom of the article. Be sure to edit the script to your liking by changing the first two parameters indicated in the script.

To run the script go to the directory and do:

.\active_directory.sh

To get your registry credentials, run:

az acr show --name krakenexample101 --query loginServer

What are Azure Container Instances?

Azure Container Instances is a service that enables users to deploy containers on the Microsoft Azure public cloud without having to provision or manage any underlying infrastructure.

How to deploy a docker image to Azure Container Instance?

To deploy a docker image to Azure Container Instance, use the az container create command to deploy the container.

Replace <acrLoginServer> with the value, you obtained previously. Replace <service-principal-ID> and <service-principal-password> with the service principal ID and password that you created to access the registry. Replace <aciDnsLabel> with the desired DNS name.

az container create --resource-group algotrading101Group \
	--name azure-algotrading101 --image <acrLoginServer>/azure-algotrading101:awesome \
	--cpu 1 --memory 1 --registry-login-server <acrLoginServer> \
	--registry-username <service-principal-ID> \
	--registry-password <service-principal-password> \
	--ip-address Public --dns-name-label <aciDnsLabel> --ports 80

To verify our deployment we run the following command:

az container show --resource-group algotrading101Group \
   --name krakenexample101 --query instanceView.state

Until you get “Running” as a response, don’t proceed to the next step.

How to view Docker logs with Azure CLI?

To view the output logs of your Docker container with Azure CLI, run the following command:

az container logs --resource-group algotrading101Group --name azure-algotrading101

2022-09-13 21:13:32,340: INFO: --------- Start of current 5 minute period ---------
2022-09-13 21:13:32,343: INFO: 2022-09-13 21:13:32.342519

How to remove a Docker container from Azure with the CLI?

To remove a Docker container from Azure and all other things we created with the Azure CLI, we only need to delete the resource group which will take care of everything else. To delete the group we do the following:

az group delete --name algotrading101Group

Where can I learn more?

Full code