Create Release Pipeline using VSTS for multi-container app

Background

This is the continuation of Continuous Deployment of Multi-Container app in Azure using VSTS and Docker. So far we have covered following topics

• Part 1 – Overview of different tools and technologies we will be using during this series
• Part 2 – Setup VSTS Continuous Integration (CI) build for DotNet Core 2.1 multi-container app
• Part 3 – Integrate DockerHub container registry with VSTS CI build to publish Docker images
• Part 4 – Provision Docker Swarm cluster in Azure using ACS Engine
• Part 5 – Create SSH Endpoint to Docker Swarm running in Azure

In this post we will see how to define a release pipeline using VSTS. Release pipeline will build upon the artifacts which are produced from the earlier stages of Continuous Integration (CI) build.

As part of this setup we will be performing following steps

1. Publish build artifacts from CI build
2. Create new Release Pipeline
3. Copy Docker Compose file to Swarm cluster

Publish build Artifacts from CI build

In a traditional CI build, the Artifacts are some sort of installer like an EXE or a JAR / WAR file depending on the type of programming language. As we saw in Part 3, the output of our build is already packaged as a self contained container image and already published to the container registry like DockerHub. What can be an Artifact we can publish as part of CI build for containerized app?

We don’t need to publish any artifact like EXE in this case. We need to adjust our publishing part to accommodate the changes due to containerization. Think of it from a deployment point of view. The release pipeline takes the output of the build pipeline and deploys it to the target environment. Do you remember the unit of deployment in Docker world? As you might have guessed it is a Docker image. When we are dealing with multi-container apps, the deployable is the docker-compose file which defines the relationship between different services.

As part of DevOps practices it is advisable to have all the code related to build and release process stored within the source control. I have created a directory named VSTS-Deploy in the project structure. This contains the files related to deployment aspects. This is usually referred to as infrastructure as code. We start off with the docker-stack-swarm.yml which describes the services similar to the one we had used during the docker-compose part.

The difference here is that we are describing the desired state configuration in the target environment instead of defining the build process. The images are already available in DockerHub registry and will be pulled down when the Docker Swarm orchestrator creates instances of the containers.

There are couple of shell scripts in the same folder. We will use them in later part of this series. For now imagine that we need all these contents of the VSTS-Deploy folder to be published as part of the artifact. Lets start with modifying the CI build definition and add a task for publishing the Artifact as the last task

Select the path to publish and set it to DotNet2017/VSTS-Deploy as the source. Provide the artifact name. Coming from Team Foundation Server background drop is a convention. You can name it whatever you like. Save the changes and queue the new build. Verify that the drop folder is created by checking the build log. We now have all the elements to start our release pipeline.

Create new Release Pipeline

Hover on top of the Build & Release menu from the top navigation bar. Click on Releases and create a new release pipeline. VSTS provides various predefined templates for us to choose from. We will start with a Empty process template.

This is the place where we define different steps to deploy our package to multiple environments. As per DevOps best practices same package should be deployed to all the environments. What changes between environments is the environment specific settings like number of machines, the firepower of machines etc. For us Docker solves this problem by packaging the application and all its dependencies into a single image. Lets create an environment and call it Production.

Lets link the Artifacts from build output to the release pipeline. Click on Add artifact button to select the build which should act as the source for this release

We could choose from multiple sources like Git, GitHub Team Foundation Server, Jenkins, Azure Container Registry or Docker. We will stick with Build in our case. Select Docker-CI as the source. We now have the link established between the CI build published artifact and the release build. Next step is to define the various steps of the release process. Click on the link which says 1 phase, 0 tasks in Production environment.

Copy Docker-compose file to Swarm Cluster

In order to copy the docker-compose file named docker-stack-swarm.yml, we need to add a Copy files over SSH task. Search for the task and add it to release definition

You might be able to relate to creating the SSH endpoint connection in the previous post. We use that endpoint now to specify where to copy the file. We also specify the source folder, docker-compose filename and target folder as shown with numbers 2, 3 and 4 below. When the secure copy task is completed, we will have the file copied to deploy folder in the master node of the swarm cluster.

Save all the changes and hold on to your seatbelts. We are about to send some file into the cloud with a push of a button. Go and click on the Release button and Create release. Select environment as Production

The moment of glory is about to come when you click the create button. Unfortunately there are still some clicks required to send the release into cloud. The create process just creates a release which is ready for deployment. We need to go and trigger the deployment by clicking on the deploy button.

Once again select Production as the target environment and finally the deployment process starts. You can monitor the log by navigating to the Logs link. If everything goes fine you should see the log messages indicating that the drop folder is created on the agent node and also the file is securely copied over to the target machine.

Conclusion

We were able to successfully connect the output of our CI build process and initiate the release pipeline. If you have experience in developing CI CD pipelines you would realize the difference in the way we publish the artifacts with containerized build. Instead of publishing exe or jar files we publish the docker-compose file which will make use of the images stored in the container registry.

VSTS is a great tool which makes this possible with minimal effort. Although the process looks a bit tedious for the moment to setup the release pipeline, we will see in the next post how all this can be automated. We will also deploy the containers in the swarm cluster in the next post. Stay tuned for the real fun.  until next time code with passion and strive for excellence.

Share: