Scaling Indicium

Introduction

You can configure Indicium to use an Azure Storage Account when scaling is required. Indicium supports the Blob container configuration option of an Azure Storage account.

When configuring a Blob container for Indicium, it is important to limit access to the storage account so only authorized applications can use it. This topic lists some best practices.

• Scaling Indicium on Azure
• Scaling Indicium on-premise
• Add a Blob Lifecycle policy

note

When using Indicium on AWS instead of Azure, encryption keys must be saved in the AWS Secrets Manager. For more information, see: Store encryption keys on AWS.

Scaling Indicium on Azure

You can use an Azure Blob container with Azure App Service. This consists of:

• Create a storage account in the Azure Portal.
• Create a Blob container and configure access to it.
• Configure some settings in the appsettings.json configuration file to use the newly created Azure Blob container.

Create a storage account in the Azure Portal

To enable Azure storage account access to an App Service, you must first create a storage account using the Azure Portal:

1. Start the Storage accounts creation wizard.

2. Select the Advanced tab.

3. Clear the Enable blob public access checkbox to disable public blob access.

4. Optionally, if you plan to use this storage account for caching only, clear the Enable storage account key access checkbox since Indicium will not use this key access.

5. In the Firewalls and virtual networks tab, under Public network access, select Enabled from selected virtual networks and IP addresses to configure networking on the storage account.

6. Click Save. Now, the Virtual networks section appears.

7. Select a virtual network. If there is no virtual network, you must first create one using the Azure portal.

Create a Blob container and configure access to it

1. After creating the Storage account, locate this account in the Azure portal to create the Blob container:

2. Select the Containers tab.

3. Click + Container and provide a name for it. This name will be used later when configuring Indicium.

Configure access to the blob container:

1. Open the newly created container.

2. On the left-hand side, click Access Control (IAM).

3. Click + Add, and then Add role assignment.

4. On this screen, click Storage Blob Data Contributor and click Next.

5. Change the option Assign access to to Managed Identity.

6. Click +Select members.

7. On the right-hand side, change Managed identity to App Service and select the Azure App Service. If the Azure App Service is not on the list, make sure that your App Service has System assigned selected. You can find this setting in the Azure Portal, on the Identity tab of your Azure App Service.

8. Click Save to save the role assignment. Now, in the Azure Portal, add the Virtual network to the Indicium app service:

9. In the Networking tab of the app service, on the right-hand side, click VNet integration. A new VNet Integration tab opens.

10. In this screen, click Add VNet, and select the Virtual network just created.

Configure settings in appsettings.json

To use the newly created Azure Blob container, add the following section to the appsettings.json configuration file, and specify the values for Type, AzureStorageAccountName, and StoragePath.

The Storage path must be the name of the Blob container just created.

"FileCache": {
      "Type": "AzureBlob",
      "AzureStorageAccountName" : "<name of your Azure Storage account>",
      "StoragePath": "indicium-cache"
  }

Alternatively, you can add the settings by adding Application settings to the Indicium App Service:

• Name: FileCache:Type, Value: AzureBlob
• Name: FileCache:AzureStorageAccountName, Value: <storage account name>
• Name: FileCache:StoragePath, Value: indicium-cache (the name of file blob container)

After changing the settings, Indicium uses use the blob container to store the files.

Redis cache

When scaling Indicium, you must configure a Redis cache. Indicium will use this cache to store the state that all Indicium instances can access. The size of the cache depends on your usage scenario, the number of users, and the application model.

We cannot specify a general cache size. We advise to monitor for “evicted” keys and the amount of memory the cache uses, so you can scale up when required.

Configure an Azure Cache for Redis

In the Azure Portal, you can use the Azure Cache for Redis wizard to configure a Redis cache to store data. We advise to use private endpoints to limit access to your Redis instance. For this, your Indicium App Service must be in a Virtual Network.

To create and configure an Azure Cache for Redis:

1. If you do not have a virtual network, create one in the Azure Portal.

2. This virtual network must have two subnets, one for the Indicium App Service and one for the Redis private endpoint.
   By default, there is one default subnet with address range: 10.1.0.0/24.

3. Provide a name for the other subnet, for example, “Redis”, with address range: 10.2.0.0/24.

4. In the Azure Portal, add the virtual network to the Indicium App Service.

5. In the Networking tab of the app service, on the right-hand side, click VNet integration.
   A new VNet Integration tab opens.

6. In this screen, click Add VNet, and select the virtual network just created.

7. In the Azure cache for Redis wizard, select the Private endpoint tab.

8. Click +Private Endpoint.

9. Select the correct subscription and resource group and provide a name for the private endpoint.

10. Select the Resource tab.

11. In the Resource type field, select Microsoft.Cache/Redis.

12. In the Resource field, select the Redis instance.

13. Click Next.

14. In the Virtual networks section, select the correct Virtual Network. This must be the same network as the Indicium App Service uses.

15. Select the subnet created earlier.

16. Finish the wizard to create the private endpoint.

To use the newly created Redis cache, add the following section to the appsettings.json configuration file, and specify the ConnectionString.

"RedisSettings": {
    "ConnectionString": "<connection string>"
    }

Alternatively, you can add an Application setting to the Indicium App Service:

• Name: RedisSettings:ConnectionString
• Value: @Microsoft.KeyVault(VaultName=<key-vault-name>;SecretName=<secret-name>)

When adding this application setting, we advise saving the ConnectionString in an Azure Key vault. By using the above notation, the secret is stored in a safe location in Azure and referenced by the App Service.

warning

Do not use the private link name to connect when using private endpoints.
Instead, always connect to the <cache_name>.redis.cache.windows.net endpoint. You can find the host name in the Overview tab of the Azure Cache for Redis wizard.

Scaling Indicium on-premise

You can also use an Azure blob container on-premise. This consists of:

• Create a storage account in the Azure Portal.
• Create a Blob container.
• Create a new App registration in the Azure portal.
• Configure access to the Blob container.
• Configure some settings in the appsettings.json configuration file to use the newly created Azure Blob container.

note

You can also make a “private endpoint connection” to your on-premise server. This is beyond the scope of this documentation.

Create a storage account in the Azure Portal (on-premise)

To enable Azure storage account access to an App Service, you must first create a storage account using the Azure Portal:

1. Start the Storage account creation wizard.

2. Select the Advanced tab.

3. Clear the Enable blob public access checkbox to disable public blob access.

4. Optionally, if you plan to use this storage account only for caching, clear the Enable storage account key access checkbox since Indicium will not use this key access.

5. In the Firewalls and virtual networks tab, under Public network access, select Enabled from selected virtual networks and IP addresses to configure networking on the storage account.

6. Click Save.

Now, the Virtual networks section appears. This section does not need changes. We will come back to this page after creating the Storage account to change the configuration.

1. Select the Networking tab.
2. In the Firewall section, add the IP address (or range) of your on-premise network.
3. Click Save to save the newly created configuration.

After creating the Storage account, locate this account in the Azure portal to create the Blob container:

1. Select the Containers tab.
2. Click +Container and provide a name for it. This name will be used later when configuring Indicium.

Create a new App registration in the Azure portal

You must create an App Registration in the Azure portal, to be able to connect to the Blob container using a client/secret:

1. In Azure Active Directory, select App registrations to create the registration.

After creating the registration, you can find the Application (client) ID and the Tenant id in the Overview menu. These IDs will be used later.

1. From the Certificates & secrets menu, add a new "Client secret".

2. Copy or make a note of the secret value (not the secret id). This value is used later, when configuring settings in appsettings.json.

Configure access to the Blob container

To configure access to the blob container:

1. Open the newly created container.

2. On the left-hand side, click Access Control (IAM).

3. Click Storage Blob Data Contributor, and click Next.

4. Set the option Assign access to to: User, group, or service principal.

5. Click +Select members.

6. On the right-hand side, search for the App Registration created earlier, and click Save to save the role assignment.

Configure settings in appsettings.json (on-premise)

To use the newly created Azure Blob container, add the following section to the appsettings.json configuration file.

The AzureTenantID, AzureClientID, and AzureClientSecret are the values copied earlier from the App Registration. The StoragePath is the name of the Blob container just created.

"FileCache": {
      "Type": "AzureBlob",
      "AzureStorageAccountName" : "<name of your Azure Storage account>",
      "AzureTenantId": "<tenant id>",
      "AzureClientId": "<client id>",
      "AzureClientSecret": "<tenant id value>",
      "StoragePath": "indicium-cache"
  }

Use Redis on-premise

On a Windows platform, Redis is not officially supported. However, you can use Redis for instance with Docker on Windows or install it on a Linux server. The Redis website provides an installation guide.

To enable the eviction policy, change the following settings in a redis.conf file. This file will be used later when starting the container.

maxmemory-policy volatile-lru
maxmemory 256M

When running Docker on Windows, execute the following command to use the configuration:

docker run -it -p 6379:6379 -v ${pwd}\redis.conf:/usr/local/etc/redis/redis.conf redis redis-server /usr/local/etc/redis/redis.conf

After installation, to use the Redis cache, add the following section to the appsettings.json configuration file, and specify the ConnectionString. The connection string must have format: <host>:6379.

By default, Redis does not enable authentication. To enable authentication, please refer to the Redis documentation.

"RedisSettings": {
    "ConnectionString": "<connection string>"
    }

Add a Blob Lifecycle policy

The blob container can grow in size and in some cases might contain stale files. This can happen when a user uploads a file but closes the browser without saving the record.

Azure provides an automatic way to delete files from the container:

1. Open the Azure Storage account.
2. In the menu on the left-hand side, click Lifecycle management.
3. Click +Add a rule to create a new rule and provide a name for it.

For example, you can create a condition to automatically delete files that were last modified 7 days ago to keep the cache clean. Also, this will keep your blob storage cost low.