Data Access Auditing In Azure Cosmos DB

In this blog we will talk about how to set up auditing and monitor access to data stored in Azure Cosmos DB.

The need to audit access to sensitive data is a common request in modern IT world. A lot of industries operate under the premise of government regulations (like HIPAA) that obligate them to have instrumentation in place that can answer questions such as who accesses sensitive data and what data that user was able to see.

In Cosmos DB, the functionality for data access auditing is implemented with the help of the component called "Diagnostics Settings." At the time of writing of this article, the functionality was only supported for Core(SQL) API, however, Microsoft acknowledged to me that they were working to extend the approach to also cover other API offerings in Azure Cosmos DB.  

Before going through the example, I'd like to point out that several conditions need to align in order for the approach to work. The following conditions need to be met to collect the audit data and be able to link the queries run against the container with the users who executed them:

• DataPlaneRequests and QueryRuntimeStatistics diagnostics settings are enabled for the Core API account
• Either RBAC or Resource token authentication is used to authenticate into the Cosmos DB account
• The user authenticating via RBAC doesn't also have access to the account access keys, or disableLocalAuth account setting is set to true.

In the example below, we will be relying on RBAC authentication to demonstrate the approach.

1. Navigate to the account level Diagnostics settings and enable the following two settings:

• DataPlaneRequests: the setting is responsible for tracking user information associated with account activities. In our case, we are interested in the activity of querying data inside the containers,
• QueryRuntimeStatistics: the setting is responsible for logging queries executed against the container data. The verbosity level of the logging is controlled by the enableFullTextQuery setting (more on it later in this article):

 

2. Run a sample query against the container. Take a note of the highlighted Data Explorer RBAC enabled endpoint used for authentication:

3. Navigate to the logs and issue the following KQL query:

AzureDiagnostics
| where Category == "DataPlaneRequests"
| project activityId_g, aadPrincipalId_g
| join kind = rightouter (AzureDiagnostics

| where Category == "QueryRuntimeStatistics"

| project activityId_g, TimeGenerated, databasename_s, collectionname_s, querytext_s)

on $left.activityId_g == $right.activityId_g

| project aadPrincipalId_g, TimeGenerated, databasename_s, collectionname_s, querytext_s

Your output will depend on the state of the enableFullTextQuery setting:

When enableFullTextQuery = false:

    When enableFullTextQuery = true:

 

aadPriciplalId_g is the value of the PrincipleID associated with the user who executed the query and can be looked up in the Azure Active Directory

* Enabling enableFullTextQuery may seem like an obvious thing to do, however, keep in mind that after enabling it, the parameters of the query also get stored in the Log Analytics database used to store and analyze the logs. The parameters may contain sensitive data. In that case, the log analytics platform itself also would need to adhere to the very same concept of sensitive data access auditing that we are using this platform for - "Catch 22 warning."