Identity and secrets on Azure

An agent built your survey pipeline a month ago. It reads documents from Blob Storage, writes rows to PostgreSQL, calls an LLM, and serves a React dashboard from FastAPI. Somewhere in there are a storage connection string, a database password and an API key, and you are not entirely sure where each one lives or who can read it. This article is about making that picture legible: what the pieces are, which arrangement is actually safe, and what to do at 11pm when a key turns up in a public repo.

The three places a secret can live

There are only three realistic homes for a credential in an Azure app, and they are not equally good.

In the code. A string literal, or a default value in a getenv call. This is the one agents reach for when they cannot see your environment, and it is the one you are grepping for later in this article.

In app settings. On App Service and Container Apps, app settings are key-value pairs you configure on the resource; the platform injects them into your process as environment variables. They are encrypted at rest and invisible in your repo. This is a real improvement, but anyone with Contributor on the app can read them in the portal, and there is no rotation story.

In Key Vault. Azure Key Vault is a dedicated secret store with its own access control, audit log, versioning, soft delete and expiry events. Your app reads from it at runtime, or App Service resolves it for you.

The best arrangement, though, is a fourth option: have no secret at all.

Managed identity, and why it wins

A managed identity is an identity in Microsoft Entra ID that Azure attaches to a compute resource and manages the credentials for. Your code asks the platform for a short-lived access token and presents that token to Blob Storage or Key Vault. There is no password anywhere — not in code, not in app settings, not in the vault.

Microsoft’s own guidance is blunt about the comparison. A connection string grants full rights to the resource and works for anyone who holds it. A token is issued only to the specific identity you enabled, carries only the permissions you granted that identity, and is attributable in sign-in logs. And there is nothing to rotate, because Azure rotates it for you. Managed identities are free.

There are two kinds:

Microsoft names user-assigned as the recommended type, largely because permissions survive the app being recreated.

DefaultAzureCredential in Python

The Azure SDK for Python takes a credential object. Agents nearly always reach for DefaultAzureCredential, and for once they are mostly right.

# Works on your laptop (az login) AND on App Service (managed identity).
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient

credential = DefaultAzureCredential()
blobs = BlobServiceClient(
    account_url="https://iffresearch.blob.core.windows.net",  # no key in the URL
    credential=credential,
)

DefaultAzureCredential is an ordered chain. It tries each link and the first one to return a token wins. In the Python library the order is: environment variables, Workload Identity, Managed Identity, the Windows shared token cache, Visual Studio Code, Azure CLI, Azure PowerShell, Azure Developer CLI, interactive browser (off by default), then the OS broker.

That is why identical code works in both places. Locally, the first three links fail and AzureCliCredential picks up your az login. In Azure, Managed Identity answers.

The catch is that the chain is opaque. Microsoft’s own docs recommend replacing it with a specific credential once you are deployed, because failures are hard to attribute, each failed link costs latency, and a stray environment variable on the host silently changes which identity is used.

# Better in production: say exactly what you mean.
import os
from azure.identity import ManagedIdentityCredential, AzureCliCredential

if os.getenv("WEBSITE_HOSTNAME"):          # set by App Service
    credential = ManagedIdentityCredential(
        client_id=os.environ["UAMI_CLIENT_ID"],   # needed for user-assigned
    )
else:
    credential = AzureCliCredential()

If you would rather keep one code path, recent azure-identity versions honour an environment variable: AZURE_TOKEN_CREDENTIALS=prod drops all the developer-tool links, and setting it to a credential name such as ManagedIdentityCredential narrows the chain to one.

When it does fail, turn the logging on rather than guessing:

import logging, sys
logger = logging.getLogger("azure.identity")   # the chain logs each attempt here
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(stream=sys.stdout))

You will get lines like ManagedIdentityCredential.get_token failed: ... no response from the IMDS endpoint, which tells you the identity is not enabled, versus a 403 from the service, which tells you the role assignment is missing. Keep DEBUG off in production: it prints tenant and client IDs.

Key Vault, without code changes

Sometimes there is genuinely a secret — a third-party API key, say. Put it in Key Vault and let App Service fetch it:

# Create the secret, then let the app's identity read it.
az keyvault secret set --vault-name iff-kv --name openai-key --value "<value>"

az webapp identity assign --name iff-api --resource-group iff-rg

az role assignment create --role "Key Vault Secrets User" \
  --assignee-object-id "$(az webapp identity show -n iff-api -g iff-rg --query principalId -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --scope "$(az keyvault show -n iff-kv --query id -o tsv)"

Then set the app setting’s value to a Key Vault reference:

az webapp config appsettings set -n iff-api -g iff-rg --settings \
  OPENAI_KEY='@Microsoft.KeyVault(SecretUri=https://iff-kv.vault.azure.net/secrets/openai-key)'

Your code keeps reading os.environ["OPENAI_KEY"]. Two behaviours will bite you:

• The 24-hour cache. App Service caches resolved references and refetches them every 24 hours, so a rotated secret can lag by a day. Any configuration change restarts the app and refetches immediately, which is the practical way to force it.
• Silent failure. If the reference cannot be resolved, the app receives the literal string @Microsoft.KeyVault(...) as the value. You will not see an auth error; you will see your HTTP client complain about a malformed API key. If a secret looks nonsensical in a log, check whether it is actually an unresolved reference.

RBAC and least privilege, concretely

Azure RBAC (role-based access control) has two halves people conflate. The control plane manages resources; the data plane reads and writes what is inside them. Owner and Contributor are control-plane roles. They let you regenerate a storage account’s keys and never let you read a blob.

That surprise 403 as subscription Owner is this distinction, not a bug. You need a data role, assigned explicitly:

Note what is missing: Key Vault Contributor is a trap. It manages the vault resource, grants no data access, and yet lets its holder grant themselves data access by editing an access policy. Least privilege means narrowing the role and the scope. Subscription-scope Contributor on an app identity is the commonest over-grant in agent-written Bicep.

// Agent version: works, and lets the app delete your production database.
resource tooMuch 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(subscription().id, api.id, 'contributor')
  scope: subscription()                    // <- entire subscription
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions',
      'b24988ac-6180-42a0-ab88-20f7382dd24c')   // Contributor
    principalId: api.identity.principalId
    principalType: 'ServicePrincipal'
  }
}

// Better: one role, one vault.
resource secretsUser 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(vault.id, api.id, 'kv-secrets-user')
  scope: vault                             // <- the vault only
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions',
      '4633458b-17de-408a-b874-0445c86b69e6')   // Key Vault Secrets User
    principalId: api.identity.principalId
    principalType: 'ServicePrincipal'
  }
}

One more practical point: role assignment changes take up to 10 minutes to propagate. A 403 immediately after an assignment is not evidence the role is wrong. Wait, then retest.

Service principals, and rotation when you cannot avoid a secret

A service principal is an identity for an application, authenticated with a client ID plus a client secret or certificate. It is what EnvironmentCredential uses, via AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_CLIENT_SECRET. Because that secret is a real password with a real expiry, treat service principals as the fallback for workloads genuinely outside Azure. Seeing AZURE_CLIENT_SECRET on an App Service is a smell: that app could have used a managed identity.

When a secret is unavoidable, rotate it on a schedule rather than after an incident. The Azure pattern for services with two equivalent keys works like this: Key Vault holds the live key as the current secret version and raises a SecretNearExpiry event 30 days before expiry; an automation regenerates the other key and stores it as the new version. Because two keys are always valid, nothing goes down mid-rotation. One consequence to remember: regenerating a storage account key invalidates every account-level SAS signed with it, so reissue those too.

When a secret leaks into git

The instinct is to rewrite history. That is the wrong first move. Once a value is pushed to a remote, assume it is public — scrapers find keys in minutes, and a force-push does not un-clone anyone’s copy.

The order that matters:

1. Kill the credential at its source. Regenerate the storage key, change the database password, revoke the API key. Only this actually stops access.
2. Update Key Vault and redeploy. Add the new value as a new secret version; restart so references refetch.
3. Look for use. Check the resource’s diagnostic logs and Entra sign-in logs for access in the exposure window, from addresses you do not recognise.
4. Reissue dependants. Any SAS URL signed with a regenerated key is now dead; anything using the old password needs updating.
5. Then clean the repository history, and add the file to .gitignore.
6. Remove the need. If a managed identity could have done this job, migrate now while the incident is fresh.

The best defence is structural: one vault per application per environment, so a compromised development secret cannot reach production; soft delete and purge protection on, so a deleted secret is recoverable for 7 to 90 days; and a pre-commit secret scanner, because agents are tireless and will do this again.

What to check in your own code

• Grep for AccountKey=, DefaultEndpointsProtocol=, sig=, password=, api_key= and AZURE_CLIENT_SECRET across the repo, including Bicep, ARM parameter files, GitHub Actions workflows, Dockerfiles and notebooks.
• Grep for getenv and environ.get calls with a second argument — check that the default is a placeholder, not the real value.
• Check any front-end .env: anything prefixed VITE_ or NEXT_PUBLIC_ is shipped to the browser and is not a secret.
• In the portal, open the app’s Configuration blade: are there raw secrets sitting in app settings that a Key Vault reference (or a managed identity) could replace?
• In the portal, open Identity on the app: is a managed identity on? If not, nothing else in this article is available to you.
• In the portal, open Access control (IAM) on the subscription and the resource group: is any app identity holding Contributor or Owner? Note it and narrow it.
• Check Key Vault’s Properties for soft delete and purge protection, and its Access configuration for Azure RBAC rather than legacy access policies.
• Search logs for the literal string @Microsoft.KeyVault — if it appears as a value, a reference failed to resolve.