During the series of posts on implementing Continuous Deployment of Multi-Container App in Azure using VSTS and Docker, I had demonstrated how to use Docker compose file to specify the services I we want to run in target deployment environment. In this post we will see how to utilize the features provided by Docker Swarm and Docker compose to manage the Desired State Configuration (DSC) for various services running inside Docker Swarm in Azure provisioned using Azure Container Service (ACS).
Desired State Configuration
Desired State Configuration is a term used to describe how a target environment should look like. When the system is up and running it has a current state. Assume that there is a failure like hardware failure or loss of a node in a cluster. We had some workload running on the failed node. We will have to ensure that the task which was running on the failed node is resumed on some other available node. Usually Orchestrators are capable of handling such things. In our examples so far we have been using using Docker Swarm as an Orchestrator. In this case Swarm would be able to identify the failure of node and reschedule the tasks on other node. The final result is that the state of the system is kept always in sync with what was specified as the desired state. Lets look at he concrete example how we can implement Desired State Configuration with Docker containers.
Docker Compose without Desired State Configuration
I am using version 3 of Docker compose specification. This has support for specifying the constrains as part of the deploy section for each service. The deploy parameters are only considered by Swarm cluster when we use the docker stack deploy command. Below is the current state of docker-compose-deploy.yml file.
I have provisioned a Docker Swarm cluster with 1 master & 3 agent nodes using the approach which I had described during provisioning Docker Swarm with Azure Container Service using ARM template. If I deploy the application and have a look at the state of the cluster using Portainer, I get the output as
Each container which I described in my services section is scheduled on one node by the Swam master.
Docker Compose with Desired State Configuration
Lets make use of some of the features provided by Docker compose particularly with regards to deployment. We start with SQL Server 2017 container. Based on the pre-requisites we know that this image requires at least 2 GB of memory. We will add some buffer and assume that 3 GB memory is good enough for time being. Also assume that the website is quite popular and we would like to start with two replicas of the coremvc image.
As you can see in the above image, we have added the resource limit of 3GB for SQL Server 2017 image. When Swarm master decides to schedule this image as a container on a node it will check that there is at least 3 GB available on the target node.
In the case of coremvc service image, we specify that 2 replicas need to be there. We also specify the update configuration. The parallelism setting is used to control how many parallel instances can be upgraded together. We set this to 1, which means that only 1 running container will be replaced with the newer version of the image at any point of time. We also specify a delay of 5 seconds which will enable us to have the side by side versions of the application running for few seconds before the rolling update is applied to the next container image.
Finally we specify the restart policy for the corewebapi image. We use a delay of 10 seconds to retry between attempts to restart the service. The number of attempts are set to 3 with max_attempts parameter. And we do this within a window of 120 seconds.
Lets use this version of the docker compose file and deploy the services to swarm cluster. I have the continuous deployment setup for my repository. So when I check in the changes to docker compose file, the build and release pipeline is triggered automatically.
We can see the older version of the images with tag 114 are replaced with the new one 115. We have 2 replicas of coremvc image running on nodes ending with 02 and 03. SQL 2017 with memory limit of 3GB is running on node ending with 00.
I am sure you would have faced issues in the past when someone from the release management team made some changes to the environment configuration for a system running in Production. These changes were not documented properly. When the next deployment happened, you might have lost those changes with some later deployment. Other possibility is that the person who made the change may no longer be working with the organization and you don’t really know why the change was made.
Following DevOps practices like Infrastructure as Code and Desired State Configuration can help you and your team stay away from lot of headaches like this. By ensuring that these aspects are part of source control add the traceability into the complete build and release lifecycle. You can repeatedly test the release and deployment cycle right from development environment all the way to Production knowing that the same process has been followed exactly the same way. The process becomes repeatable and can be a huge confidence booster to the team in ensuring deliveries with greater quality.
Conclusion
It is very easy to specify constraints related to target environment using docker compose with the deploy features. In DevOps terminology this is called as Desired State Configuration (DSC). We are describing the target state of the application. It becomes quite easy to manage change using this approach. The code used to describe the desired state is versioned inside the source control which can help us to track the evolution of system over a period of time. Orchestrators like Docker Swarm make it very easy to implement desired state configuration.
There are many types of constraints which can be specified in the compose file. Refer to the documentation for more details on other options which can be specified as part of compose file. Until next time code with passion and strive for excellence.
