Designing Azure Function App for production

This post is part of a series about Azure Functions and PowerShell. Check out the list of other posts in the series!

Intro

Creating an Azure Function in the Azure Portal is super easy, just hit Add, select “Function App”, hit Create and fill in the form. This works for a demo, but it only gives us a simple Function App without any customization. There are a few things I always tend to customize when I want to run a Function App in production (or production-like) environments.

Let’s first have a short dive into the Function App, some features, and its dependencies to get a good view of the architecture we want to create. For the two first posts I will focus on design and not go into how to script anything. If you want to deploy a Function App using an ARM template, there will be a post on that, check my first post in the series for an updated index.

Standard Architecture

In its simplest form, a Function App relies on an App Service and a storage account. The App Service provides compute (essentially CPU and RAM) and the storage account is used to store the code it is supposed to run. When using the default configuration from the Azure portal, the storage account is also used to store the API-keys used to invoke my functions. This works, but now when we’re talking about production usage, there are a few things I want to customize. This post will cover the generic things I usually customize on any Function App regardless of language I intend to use later on.

Standard Functions Architecture

Assigning a Managed Identity

Lots of services in Azure supports Managed Identities and Functions is no exception. A Managed Identity will provide your code with an identity in Azure Active Directory, an identity that can be used to authenticate to other Azure resources like Key Vault, or a why not a database? Another common scenario is to use the Managed Identity to authenticate to the Azure Resource Manager API to automate management of Azure Resources.

Managed Identities could either be user assigned or system assigned. I prefer using Managed Identities assigned by the system whenever possible. A system assigned Managed Identity is created with my Function App and is automatically cleaned up when my app is removed. This has a cleanliness that appeals to me and it helps me make sure that I don’t re-use or share identities between more than one app.

You can read more about Managed Identities and Functions in this article on docs: How to use managed identities for App Service and Azure Functions

Storing secrets in Key Vault

Secrets should be stored in a Key Vault whenever possible. Azure App Services, which is the service Azure Functions is built upon, has great integration with Azure Key Vault. By creating a Key Vault and assigning the Managed Identity mentioned above access to the secrets, the integration works seamlessly!

First I want to store the connection string to the storage account in KeyVault. After creating a secret containing the connection string and giving the Managed Identity access to GET secrets from the vault, all I need to do is entering a Key Vault reference instead of the actual connection string in the application setting “AzureWebJobsStorage”. The reference can be any of the two examples below. I usually use the SecretUri variant.

@Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931)

@Microsoft.KeyVault(VaultName=myvault;SecretName=mysecret;SecretVersion=ec96f02080254f109c51a1f14cdb1931)

If you want you can read more on how to use Key Vault references for App Service and Azure Functions on docs.

But we’re not done there! Do you remember the API keys I mentioned before? The ones that are used to call a function. They are by default stored in a blob container called “azure-webjobs-secrets”. Anyone with read access to the storage account could extract them, that won’t do.

Storing function keys in Key Vault

Guess what? We can of course store them in Key Vault! For this to work, the Managed Identity needs not only GET access to secrets in the vault, but also LIST and SET access. Once that is taken care of, I need to set three applications settings, one that tells the Function App to use the AzureWebJobSecretStorageType of “keyvault” and one that sets AzureWebJobsSecretStorageKeyVaultName to the name of my Key Vault. There is also a third setting called AzureWebJobsSecretStorageKeyVaultConnectionString which isn’t needed when we are using a Managed Identity. I set this one to an empty string just to make sure I have it.

For example:

"AzureWebJobSecretStorageType": "keyvault",
"AzureWebJobsSecretStorageKeyVaultName": "my-keyvault",
"AzureWebJobsSecretStorageKeyVaultConnectionString": ""

I haven’t found this documented somewhere but Azure Functions is Open Source and the code implementing this feature can be found on GitHub

Securing access

By default when setting up a Function App, code can be deployed by using FTP and all versions of TLS are accepted.

To secure this I usually disable FTP access since I’m not going to use it and I usually configure my app to require TLS 1.2.

FTP state can be set to Disabled under Configuration -> General Settings.

TLS 1.2 can be required under TLS/SSL settings.

Monitoring and logs

Log to Applicatin Insights

For production, I also want to send all logs and metrics to Application Insights. This is an option that can be enabled when creating a Function App in the portal, or by adding the instrumentation key for an Application Insights resource to an application setting called APPINSIGHTS_INSTRUMENTATIONKEY. As far as I know, storing this key in a Key Vault does not work.

Audit Key Vault access

Other than that, I also want some auditing on my Key Vault so I know who accesses my keys. In my following posts I’ll solve that by sending audit events from Key Vault to my Azure Storage Account as well as optionally sending audit logs to an existing Log Analytics workspace using diagnostic settings on my Key Vault.

Log to Log Analytics

There is also an option, currently in preview, to send diagnostic logs from a Function App to Log Analytics. I haven’t found that I get any added benefit from that compared to Application Insights so that’s not something I’ve used. Please tell me in the comments how you are using it!

Configuring the Function App for PowerShell

Select PowerShell as runtime

To set PowerShell as the chosen language runtime, we simply set the application setting FUNCTIONS_WORKER_RUNTIME = "powershell". Simple as that!

Set PowerShell version

To run PowerShell 7, we need to be running version 3 of the function runtime. This is simply configured by setting two applications settings FUNCTIONS_EXTENSION_VERSION = "~3" and FUNCTIONS_WORKER_RUNTIME_VERSION = "~7".

Configure concurrency

There are two settings we need to consider for concurrency. The PowerShell runtime can only process one invocation of a function at a time. If I want to send a large numer of requests at once or I want my functions to call each other, this just won’t do!

There are two ways of increasing the concurrency in a Function App running PowerShell. It can run multiple instances of PowerShell and it can run several runspaces in each instance of PowerShell. Since my functions usually aren’t very CPU intensive I always set the function to run up to 10 instances of PowerShell in each Function App instance. This is done by setting the application setting FUNCTIONS_WORKER_PROCESS_COUNT = 10.

If I have functions that really need to process a lot of requests at once, I can also run several runspaces in each PowerShell process by setting to application setting PSWorkerInProcConcurrencyUpperBound to a value higher than 1. This is great for example when you need to call the ARM api several times to start/stop resources like Virtual Machines or do large scaling exercices. Basically any time my code mostly is idle waiting for a response from for example an API, concurrency is great! However! There are some shared state between the runspaces that the Function runtime uses, if you get wierd errors, try setting PSWorkerInProcConcurrencyUpperBound back to 1 to troubleshoot.

That’s it! We have a Function App ready to run PowerShell!

Cosmetica

For that extra crisp look and feel, I also like to add the application setting called AzureWebJobsDisableHomepage which will disable the status page telling users that “Your Functions 3.0 app is up and running”

"AzureWebJobsDisableHomepage": true

To prevent that anyoen edits the code of my Function App from the Portal I also add the application setting FUNCTIONS_APP_EDIT_MODE. According to the documentation, this setting dictates whether editing in the Azure portal is enabled. Valid values are “readwrite” and “readonly”.

"FUNCTIONS_APP_EDIT_MODE": "readonly"

Summary

Now that our Function App has a Managed Identity, all secrets are stored in Key Vault and logs are being sent to Application Insights, things are starting to look quite good!

The last thing I might want to add to my Function App is authentication and authorization. For example, I can require any user calling my Funcion App to have a valid access token from Azure AD. Since this is a rather large topic, I’ll discuss that in a separate post about Azure Functions and Azure AD authentication.

Azure Functions architecture for production use.

Stay tuned for the next part where I’ll cover what to customize when writing Functions in PowerShell!