What is a Pipeline?
A pipeline is a tool, and a mentality: It is a way of working and thinking.
I spent more than a decade programming in PHP before I came anywhere near the DevOps methodology. When you need to "deploy" something in PHP, it's as simple as uploading a bunch of files to a server and voila, you're done. Using automated synchronisation tools may help with this process, but essentially, there is little control, one is simply replacing files in a production environment with those from another environment, e.g. from a local development machine.
When it comes to .NET development in Azure, things get a little more sophisticated.
A Pipeline allows you to build your .NET code automatically on a trigger (for example a check-in to the master branch) or, if you wish, manually.
You can also run tests from your code assemblies after building, but before publishing. This helps ensure that any bugs that might have been missed during the pull-request never reach production. Of course, it depends on how good you are, as developers, at writing your unit tests....
After code is published to a directory, you can use Releases to automatically deploy this built code to a resource, for example, to a resource in Azure like a Function App or an App Service.
Azure DevOps provides an easy way to create Pipelines and Releases, so let's take a look at this now.
How to build your first pipeline
Let's get Azure DevOps opened up in our browser: dev.azure.com
NOTE: If your organisation has DevOps projects on another subdomain, you'll need to go there instead.
Within your project, go to Pipelines in the menu on the left hand side.
Click the New pipeline button.
The wizard will ask you: Where is your code?
The logical choice, if you are using Azure DevOps to host your repositories, is Azure Repos Git.
Select your repository and go to the next step.
DevOps here analyses your code and makes suggestions about what type of configuration to use. Select the one most appropriate to the project (within the solution) that you want to deploy.
NOTE: Each "project" from here on is equivalent to a *.csproj project within a *.sln solution.
On the popup, you will need to select your desired subscription in Azure.
Next, select the correct resource. A list of resources which match your deployment configuration type will be shown, according to your permissions within that subscription.
Leave the Working Directory as it is (usually the default populated variable is sufficient) and click Validate and configure.
Follow any remaining steps, if applicable, to create your pipeline. Once created, it's important to edit your pipeline to get it just right.
Editing your pipeline
Back on the list of Pipelines, find your pipeline and click on it. Then click the Edit button in the top right.
There are several tabs here: Tasks, Variables, Triggers, Options, Retention and History.
The Tasks tab is the most important when configuring a pipeline, as it defines the tasks to run in a specific order.
Jobs - Agent v Agentless
There are two types of job - Agent Jobs and Agentless Jobs.
Most of the time you're going to want to use Agent job 1 (the default job), but you do have the option to add more later.
Read up about jobs here:
docs.microsoft.com
Frequently used tasks
Adding a task to your job is easy. Just click the big blue + (plus) icon on the row Agent job 1 and you will see a menu "Add tasks" on the right hand side.
I'll now list some of the most common tasks you'll want to use in your Build Pipeline.
NuGet package restore
Task type: .NET Core
Display name: dotnet restore
Command: restore
Enable Use packages from NuGet.org
This step enables to you to build packages that you've used from NuGet.
NOTE: Forgetting to add this step is a common reason why Build Pipelines fail!
Build integration tests and unit tests
Task type: .NET Core
Display name: Build Tests
Command: build
Path to projects: **/*Tests*.csproj
Arguments: --configuration Release
You should have integration tests and/or unit tests in your CI/CD pipeline, so you will need to build them here before they can be run.
Build your project
Task type: .NET Core
Command: build
Path to projects: **/MyProjectNameHere.csproj
Arguments: --output publish_output --configuration Release
You need to be specific about the path to the exact project you want to release. The publish_output flag tells DevOps that it is these DLLs or whatever it is you're publishing should be packaged and ready to 'drop'.
Run integration tests and unit tests
Task type: Visual Studio Test
Display name: Test Assemblies
Test files:
**\*Tests*.dll
**\bin\$(BuildConfiguration)\**\*test*.dll
!**\*TestAdapter.dll
!**\obj\**
Reporting options:
Enable Fail the task if a minimum number of tests are not run.
Minimum # of tests: 1 (or as desired)
It is important to run your tests after building your tests!
Archive published output
Task type: Archive files
Root folder or file to archive: publish_output/
Archive type: zip
Archive file to create: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
This step prepares your ZIP file for publishing the Artifact.
Publish the Artifact
Task type: Publish build artifacts
Display name: Publish Artifact: drop
Path to publish: $(Build.ArtifactStagingDirectory)
Artifact name: drop
Artifact publish location: Azure Pipelines
This step drops the output ZIP into a drop location so you can release (deploy) to your resource in Azure.
Enable continuous integration
Click the tab called Triggers.
There should be a checkbox, "Enable continuous integration". Check this.
Then, make sure to add a Branch filter to Include your master branch.
Run the Build Pipeline
When you're done editing, make sure to hit Save (it's hiding under Save & queue).
Go back to your Pipeline again, and click the blue button Run pipeline
Select an appropriate Agent pool (Azure Pipelines), Agent Specification and branch (usually master).
Under the Advanced options, check the Variables.
Make sure that system.debug is set to false if you are doing a production deployment.
When you're ready, click Run. It may take some time, but when it's complete, you can check if the build was successful in the Runs tab of your pipeline.
In order to perform a Release, you must have a successful build! If it's Failing, you need to investigate why that is before proceeding to the next step.
Releases
Under the Pipelines menu in DevOps, click on Releases.
Then click + New and select New release pipeline.
Choose the right template
The wizard gives you option to create a release pipeline based on a template.
Try to find the one which is most suitable to your project.
If you are deploying a .NET Core Web API, then you need to choose Azure App Service deployment.
There are other options, such as for Azure Functions or Kubernetes clusters.
When you've found the right one, click Apply.
There are two main sections to the pipeline - Artifacts and Stages.
Artifacts
Inside the section called Artifacts, click + Add
To the right, you'll see Add an artifact.
Since we want to use our Build from earlier, let's choose Source type Build.
Use the dropdown menu for Source (build pipeline) to select your pipeline. Obviously...
Then click the Add button.
There should be a small circular 'lightning' icon ⚡ inside the Artifacts box now. Click this to get the menu for Continuous deployment trigger.
If you want to enable it, it will create a new release everytime a new Build is successfully run. This is the "CD" part of "CI/CD".
Remember to include the Build branch filter for the master branch.
Stages
Next, we need to configure Stage 1. You don't need to click on the Stage itself, but rather on the text that shows ❗ 1 job, 1 task to open the Tasks configuration.
Inside the Tasks tab, you should see ❗ Some settings need attention
Click this text underneath Stage 1 and choose your Azure subscription. You may need click the Authorize button to create a service principal in Azure which is what DevOps uses to carry out the deployment.
Fill in all the rest of the settings - they are fairly self-explanatory.
The Package or folder should be: $(System.DefaultWorkingDirectory)/**/*.zip
Remember to hit Save to save your Release pipeline.
When you're done configuring, click on View releases.
Deploy the build
Now it's time to create a release. Click the big blue button Create release.
In the popup dialogue, Create a new release, select the stages for a trigger change from automated to manual (if you have selected the automatic deployment).
For the Artifacts, ensure that the correct Source alias is visible, and you have the right Version number.
Write in a brief Release description that you can refer to later, and click Create.
At this point, the Release should be queued and executed by DevOps. You can follow the progress by clicking on the Release. It may take a while, but be patient!
And that's it! When completed, your code will have been deployed to your resource in Azure.
Summary
Use Build Pipelines to restore NuGet packages, build your project, build tests, run tests, create a ZIP file and publish the artifact (drop). Use Release Pipelines to deploy your code to the appropriate Azure resource.
Use Triggers to enable Continuous Integration and Continuous Deployment.
If you found this article helpful, please share it and leave a comment! Thanks for reading!