API Versioning with Swagger, Azure API Management Services and ASP.NET Core - A Frictionless DevOps Flow
In the second and last part of this series, I'll talk about a frictionless developer experience for versioning .NET APIs and automatically publishing those versions to Azure API Management Services.
Hi there. This post is a follow up to the introduction that I published earlier, where I describe API versioning concepts in general and introduce each technology that is listed on the title of this blog post.
In the second and last part of this series, I'll talk about a frictionless developer experience for versioning .NET APIs and automatically publishing those versions to Azure API Management Services.
I've recently been working with a number of API's that are deployed on Azure behind the Azure API Management Services. These APIs are used by client-side applications such as Angular & React SPAs, iOS & Android apps as well as a number of other back-end services. So it's important that they run with high availability and their contracts are never broken overnight.
When delivering frequent updates to an API, versioning comes into play as soon as evolving an endpoint is not a feasible option anymore. So an API team has to have a process around versioning. One that is preferably less painful and more automated.
"Vertical responsibility" is the term I read and think about a lot. As teams, we want to have vertical responsibility in delivering our services to our users. We build it, deploy it, release it, monitor it - take 100% responsibility and autonomy in making our software & services a success (or a failure). Can't explain it better than Martin Fowler does in his talk "What Does Tech Excellence Look Like?".
Using the same philosophy, we want to own and design our versioning process and we want it to be as smooth and automated as possible. A few good properties we could look for in such a process are:
- Developer friendly. Code-first versioning.
- Continuous Deployment friendly. "git push" and everything flows from there.
- Azure API Manager friendly (if relevant for your case).
Let's look at these in a bit more detail.
Code-first versioning
I mentioned aspnet-api-versioning repo before. A library that makes it easy to add versioning support to an API in an elegant way.
In its simplest form all that's necessary to enable versioning for an API is:
By setting ReportApiVersions to true, we're configuring the library to add a standard HTTP response header indicating all the version numbers supporting a particular API method.
As a developer you have the freedom to choose how much you want to separate your code across different versions. On one end of the spectrum you can completely separate your models and controllers under different version folders:
On the opposite end, you can choose to keep them side by side and even version on the action level:
.NET Core Tutorials article on API Versioning in ASP.net Core is a great way to get started with this library.
Whatever your favourite style is, I strongly recommend to keep your versioning code as simple as possible. Last thing you want is to add unnecessary complexity to your codebase because of versioning.
Continuous Zero Downtime Deployments
I've seen teams working successfully without introducing any sort of versioning concept into their API codebases. Here's what they do when they're introducing a breaking change:
Let's say the application consists of a SPA front-end, a REST API and Azure API Management Service in between. And let's say the team is introducing a breaking change between V1 and V2. Note that this is an example where the team controls both API and client at the same time - which means no long term support for older versions of API is necessary.
- Team does not touch the V1 entry in Azure API Manager, nor the existing App Service that hosts the V1 API internally.
- They deploy V2 API as a separate application under a different internal URL. Think of it as a completely new Azure App Service. VSTS release pipelines make it relatively easy to achieve this by changing a few configuration variable in the release pipeline.
- Then they create the new V2 entry in Azure API Manager and point it to the new Azure App Service internally (where API V2 resides).
- Then they deploy V2 of the SPA on top of V1 - effectively replacing it. Not literally though, you should always use deployment slots and swapping during your deployments.
- Now V1 of the API is running side by side with V2 but new users are always downloading the V2 of the SPA. Only existing users who were already working with V1 SPA during the deployment continue doing so without any interruption.
- Team awaits until all V1 users are gone - effectively draining the V1 system. Once it's quite in there, they drop V1 from Azure Api Management Service and remove the V1 App Service deployment.
This works.. but it requires a bunch of manual steps, checks and controls every time a breaking change is introduced. A better way would be to introduce versioning inside the codebase as a first class citizen. And then have a deployment step that can create and drop version entries inside Azure API Manager by automatically detecting the versions in the API. Then a new version release becomes as easy as merging to the main branch and kicking off the deployment pipeline.
Combining Swagger with Versioning
If you don't know what Swagger and Swashbuckle is please take a quick look at the relevant section in my previous post.
OK let's say we've now introduced versioning into our API codebase. For API documentation and automation purposes we also want to integrate this versioning information with Swagger docs.
By bringing the following three packages together in Startup.cs it's possible to glue Swagger docs and versioning constructs together.
Check this pseudo Startup.cs file below to see how Api Versioning and Swagger Docs are configured and then stitched together. If you want to see the full version of this Startup.cs click here.
Taming Azure API Management Services for Continuous Deployments
I spent a good amount of time trying to figure out how to automate the process of importing APIs into Azure API Management Services with versioning support.
Import-AzureRmApiManagementApi Powershell cmdlet let's me import an API into Azure API Manager using the Swagger Open API format. But if I want to tie that to a VersionSet that I created earlier, it seems to be not possible.. or I've been stupid enough to miss the connection there. Azure PowerShell documentation doesn't contain any hints and I couldn't find any examples of it on the internet so far.
While searching for a solution, I came across this post from Ronald Wildenberg where he uses direct REST API calls in his Powershell script, instead of cmdlets, and that's how he manages to create a versioned API inside Azure API Manager.
At that point I decided to follow a similar route and use REST API calls whenever I fail to accomplish a step using Powershell cmdlets alone. The resulting script which you can find at the end of this post is a combination of REST API calls and Powershell cmdlets. This script can run as a release step inside VSTS and it goes through the following steps:
- Authenticates through Azure AD App Registration and gets an AccessToken to be used in REST API calls. Check this blog post to learn more details.
- Downloads the main Swagger HTML file and using that, it detects all versions that an API contains and paths to their Swagger JSON files.
- One by one, imports all versions of the API into Azure API Management Service using the Swagger JSON files discovered earlier in the previous step.
- Cleans up the version definitions from Azure API Manager which no longer exists in the actual API itself. Keep in mind that we should only drop a version after keeping it in a deprecated state for a while - that's the whole point of versioning anyway.
Effectively what happens is that at each new release, all versions of the API gets fully synced to Azure API Management Service, removing the need for any manual operations.
Here's a screenshot of the Swagger docs of a multi-version API and then how it looks in Azure API Management Services after the automated import.
In this specific case, the output of the API Management release step was as follows:
Swagger HTML file is downloaded...
Swagger JSON paths are discovered...
*** Starting to process '/swagger/v1/swagger.json' ***
Swagger JSON file is downloaded...
Swagger JSON file is updated...
Api version set is created...
A versioned Api is created based on the version set...
A Swagger definition is imported into the versioned Api...
Versioned Api is updated to its final state...
*** Finished processing '/swagger/v1/swagger.json' ***
*** Starting to process '/swagger/v2/swagger.json' ***
Swagger JSON file is downloaded...
Swagger JSON file is updated...
Api version set is created...
A versioned Api is created based on the version set...
A Swagger definition is imported into the versioned Api...
Versioned Api is updated to its final state...
*** Finished processing '/swagger/v2/swagger.json' ***
*** Starting to process '/swagger/v3/swagger.json' ***
Swagger JSON file is downloaded...
Swagger JSON file is updated...
Api version set is created...
A versioned Api is created based on the version set...
A Swagger definition is imported into the versioned Api...
Versioned Api is updated to its final state...
*** Finished processing '/swagger/v3/swagger.json' ***
##[section]Finishing: Inline Azure Powershell
And last but not least, below you can find the full powershell release step import-api-to-azure-api-management-with-versioning.ps1. This script can be run inside VSTS using the Inline Powershell VSTS task.
I recently noticed that now there's also a VSTS extension called Azure Management Suite which enables creating or updating a versioned API inside a release pipeline. But if you're looking for a fully automated solution where API versions are automatically detected, imported and even dropped when necessary, you still need to use a custom script like the one I shared above.
Thank you for reading! I hope you found it useful.