Azure KeyVault strategies

In web application development, one of the key security practice to take care of is not to store your secrets in safe place away from source code. So then, what we do is usually put these secrets into a KeyVault and access them on demand. But then, what happens when we have many applications each having a couple of secrets to store? What happens when each of those applications are setup in different environments? We can easily find ourselves in a mess. In this article I will discuss some Azure KeyVault strategies on how to organize them and better manage application secrets.

How to organize the KeyVaults?

As soon as we decide to use a key vault, we’ll ask ourselves, how do we organize them. Do we put everything into one vault, separate vaults, what are the recommended best practices.

There are actually many potential strategies to solve this problem. The strategy you’ll choose will be highly influenced by your organization and team setup in my opinion.

Let us discuss some potential strategies to organize your Azure KeyVaults.

One KeyVault per application/environment

One of the most recommended strategies is to use one KeyVault per application environment. This would allow us to segregate the privileges on who can see which secrets. For example, maybe the developers are allowed to see Dev environment secrets, but only the seniors can see the Quality environment secrets and only the operations team can see the Production environment secrets.

With this strategy, access to secrets can be kept to minimum and fine grained. But, this approach also comes with a downside. If you have a DevOps team responsible for all environments and several applications, this does not scale well, as you’ll end up giving access everyone to every KeyVault and this will increase the proper management overhead.

On the other side, this approach works well though if the teams managing applications environments are different, e.g. if you have a dedicated operations department only for Production environment.

One KeyVault per Application

Another possible approach is to put all application secrets for all environments into one KeyVault. This would bring down the number of necessary vaults but it would also allow everyone to see all secrets. Also, you’ll need to come up with a proper naming structure to easily differentiate environment specific secrets.

If this is not clashing with your security policies, this could work well if you have DevOps teams responsible for all environments. One big downside of this approach to mention is, it is susceptible of unintentional accidental changes, e.g. when someone was intending to change the Dev environment connection string but accidentally changes the Production environment secret.

One KeyVault per team

While this is maybe the approach with the least KeyVault access management effort requirements, I think this works contrary for secret management. Having all secrets in one KeyVault will make secret management very difficult if the team is managing more than one application. It also makes it very easy for a team member to delete or change an unintended secrets and have unintended side effects on other applications.

I would recommend to avoid this strategy if possible.

Conclusion

When evaluating the KeyVault strategy, I would take into consideration these factors:

  • Security requirements
  • Organizational requirements
  • Access management effort

Based on these criteria, you should determine which strategy will give you a good enough security level (although, there is never enough security ;)) and management access management overhead. I hope these Azure KeyVault strategies give you some hints in deciding what works well for your case.

Are Azure Certifications Valuable?

It has been years since I stopped taking certification exams and lately I decided to get back to that adventure. In September 2020 I passed exams to be recognised as Microsoft Certified Azure Solutions Architect Expert.

Arian Celina - Certified Azure Solutions Architect Expert

Before and after passing the exam, I asked myself the question which I have seen many people ask as well; Are Azure Certifications valuable? I believe the question is applicable not only for Azure Certifications, but in general for any industry certification. Are they valued by the community and employers? Do they make a difference in your career? I have also written about this topic in the past, and yet it seems the same questions are still coming up. Let me share my updated opinion with you.

From the motivational perspective, it is very important to clarify the answer to those questions so we know why we are putting all that effort to learn and prepare for the exams. I also think these questions should be rephrased with a reversed perspective. Before we modify question, let us first discuss about the value itself.

What is the value and who defines it? Who can tell if a certification is valuable? I think think there is no single source of truth for that. From experience, I have encountered employers and peers who value a certified professional more than someone who is not certified. I have also seen opposite. As someone who have interviewed more than 100 candidates so far, I can say both can be right and wrong.

I have met certified candidates who have not been able to defend their title with knowledge. So I have met people who were not certified on a topic but were more knowledgeable than their certified peers. I have also met certified people who were subject matter experts and knew their topic in detail, kind of detail that we do not easily learn by randomly playing with the technology.

When I reflect on my late Azure certification journey but also on my previous certifications for .NET and Java, what I remember is that while preparing for the exams I have often learned hidden details about the topics which I have not encountered during the daily work. Those hidden details then later have quite often have saved me time or effort and enabled me to bring better solutions into life. This in itself is a value for me.

Considering this, the revised question I think we need to ask ourselves would be:

What do we gain from this certification?


In my opinion, we should not get certified so other people value us more because we hold that title. We should do certifications to learn better the technology we like. Of course one can do that without taking the exams and without getting certified. It’s just that taking the exams pushes you to follow a certain curricula which is reviewed by experts and that often gives more structure to the learning.

I have been using Azure for years and have quite some experience deploying and running web applications on various forms of workloads, containerised and non-containerised. Yet, when I took the late exams for Azure Cloud Solutions Architect Expert certification, I learned a lot about some not so familiar topics to me, like migrations of Virtual Machines from on-premise datacenter to Azure, about backing up Virtual Machines, or about Express Routes. Taking those exams pushed me to learn more about those topics which probably I would not in my daily work.

As a conclusion, in my opinion industry certifications are valuable as they push us to get better at that topic and this inherently will make us a more valuable contributor to our team, company and community. When we do that, whether the certification is valued by our potential future employers plays a smaller role in answering this question.

Automated resource creation in Azure

When creating single resources, e.g. a Virtual Machine or a Web App Service, it is very convenient to do it from Azure Portal. It is so convenient, that quite often we rush into creating a whole setup immediately from the portal. It is as soon as we need to copy the setup when we see that this does not scale. In this article, I will elaborate our current options for automated resource creation in Azure and their advantages and disadvantages based on my personal experiences.

When is automated resource creation necessary?

Before digging into technical details, let us first understand, when do we need to create resources automatically. This approach is my personal choice anytime I am setting up something that will stay longer than a day or two. In other words, if I am testing an idea, need a quick web app service or a storage, etc…, I go ahead and create it from the portal. If I am setting up a solution to deploy an application, then I know that eventually I will need to recreate that setup again. This could be that we will need to replicate the environment, e.g. create a staging environment. It could also be that we need to replicate the same setup in another region. In such scenarios, automating the resource creation will save you great deal of effort later.

Another reason to have resource creation automated is security. One has to be prepared for a catastrophic situation, e.g. your existing setup is not accessible anymore, then having it automated will enable you to recreate it fast and lowering your downtime.

Note: although this post is focused on options for Azure, equivalent alternative options exist for AWS Cloud as well.

Let us discuss what are our automation options?

ARM Templates

Azure Resource Manager (ARM) Templates is the Microsoft’s recommended way of automation. These templates essentially are JSON files with a “simple” structure. They can be written using any text editor, but there is particularly good support in VS Code editor for these templates.

Advantages

There are certain advantages in using ARM templates, the most important one in my opinion being the support given inside Azure. They can be very well integrated into Azure DevOps to automate infrastructure creation. I can also download the template of every manually created resource (mostly accurate), put it in source control and connect to a pipeline. Voila, from manual to automated resource creation in a few simple steps.

These templates can also be used to make sure that the infrastructure has not changed. A valid scenario is, if you run the infrastructure pipeline regularly, and if someone has changed a setting or removed a resource manually, the pipeline will correct the change and bring it to a desired state.

Also, its good integration with Azure KeyVault allows us to store secrets safely in the vault and access them easily from the pipeline, a particularly useful feature.

Disadvantages

There are certain disadvantages about ARM templates too. They are an Azure feature and this knowledge is not reusable in other cloud platforms. Also, writing them is not super easy (though this is improving every day with better support in VS Code).

In my opinion, I would also classify the documentation about ARM templates as a disadvantage. Microsoft is improving this continuously by providing more samples and tutorials, but at the time of this writing, if you need something beyond a simple setup, finding your way will not be super easy.

Terraform

Terraform is a cloud automation tool created by HashiCorp. Its support for Azure is pretty well and mature. Having multi cloud support also makes Terraform an appealing tool.

Advantages

One of the main advantages of Terraform is it having multi cloud support. Even if you are not using different cloud platform, you can reuse this knowledge later when you need to use another cloud provider. The documentation is pretty decent and you can find a lot of blogs and samples on the internet. Another very big advantage is possibility to preview my changes. I can preview all the potential changes my script is going to make if I run giving me a possibility to prevent an endangering operation. This gives me confidence before executing any scripts especially against productive environments

Another favourite feature of Terraform is the workspaces. Workspaces makes it super easy to manage different environments of the same infrastructure setup.

Disadvantages

Although terraform is advertised as a multi cloud solution, the abstraction of cloud providers are not at the desired level. Meaning, the building blocs of terraform files are cloud specific. When you create an Azure storage, it’s not the same code used to create an AWS S3 bucket.

Another big problem that can arise with Terraform is the Terraform state file. The state file contains the current state and if it gets corrupted, then one cannot execute any changes against the cloud environment. Therefore, it’s important to state it in a shared drive where all the users of the script have access to. Also, as the state file contains all the secrets applied, it itself becomes a secret you want to protect well.

Azure SDKs

Azure offers SDKs for different programming languages. They offer another programmatic alternative to automated resource creation in Azure. This could be very useful in some specific scenarios, e.g. if you want to create a custom one-stop-shop resource creation in your organisation where you can enforce your standards as well as hook resource creation to your specific organisational workflows and needs.

My experience so far in using Azure SDKs has been very limited, though I have not developed a very pleasant opinion about them. One big disadvantage that I have faced so far was that the SDK itself was not up to date with Azure. One such scenario was when creating an App Service, available options for selecting the tech stack and some other settings offered by SDK was not lacking the options available in the portal.

Conclusion

Overall I would say, we are lucky having many options we can choose from. I believe often which one one will choose depends on the circumstances, but I hope this list of advantages and disadvantages will help you shape your decision. In my case, I try to use Terraform whenever I can, if not then I usually fall back to ARM templates.