Image Analysis with Azure AI Vision

Welcome to the Blazor Simple AI Single Page App, Part 2 of the Microsoft AI services journey, which now includes image analysis utilising Microsoft Azure AI Vision. The Vision Read API is used to extract the text from an image. This document explains the project in my GitHub repository which is available here: https://github.com/tejinderrai/public/tree/main/BlazorSimpleAI.

If you would like to download both part 1 and part 2 as a PDF document, you can download the PDF here.

Since part 1, the following nuget packages have been added to the project.

Azure AI Vision Image Analysis – for reading text and metadata from images.

Radzen Blazor – for providing an amazing UI experience.

Azure Storage Blob – for handling interactions with Azure Blob Storage.

Visual Changes

I have made some appealing improvements from the basic Blazor template and styled the UI based on a project from Martin Mogusu available here: GitHub – martinmogusu/blazor-top-navbar: A top navbar example created in blazor. This saved me a lot of time and all I had to do was apply my own visual styles after the top navigation was applied to the project in shared/NavMenu.razor. In addition, I had added a pre-built model for interactive Invoice Analysis and processing, which I will leave the full explanation until Part 3 of this post.

Components

Three components have been developed for the image analysis. These are as follows:

1. Vision.razor – The Image Analysis page
2. VisionBlobLoader.razor– This includes the capability to upload files to Azure blob storage, which also sets the content type for the blob file.
3. VisionBlobFileList.razor – This is a child component embedded into the VisionBlobLoader component, which lists the image files that have been uploaded to Azure blob storage.

Learn about Microsoft AI Vision

To learn more about the capabilities of Microsoft AI Vision, see What is Azure AI Vision? – Azure AI services | Microsoft Learn. Azure AI Vision includes more analysis capabilities, not just specifically  image files.

Configuration Settings Changes

The following configuration settings were added to appsettings.json.

  “AzureVsionConfig”: {

    “AzureAIVisionEndpoint”: “https://%5BYour AI Vision Service].cognitiveservices.azure.com/”,

    “AzureAIVisionKeyCredential”: “[AI Vision Service Key]”

  },

  “AzureStorageConfig”: {

    “AzureStorageConnectionString”: “[Your Storage Account Connection String”,

    “AzureStorageContainer”: “[Your Storage Account Container]”,

    “AzureStorageAccountName”: “[Your Storage Account Name]”,

    “AzureStorageAccountKey”: “Your Storage Account Key”

  },

Note: Whist this project utilises the service key, in an enterprise environment, you must consider using token based access to the service secured by Microsoft Entra ID, or if you wish to utilise the service key for any reason, utilise Azure Key Vault to protect the key used by the application with a managed identity for the application to access the service key stored in Azure Key Vault.

Components

File Upload Component (VisionBlobLoader)

The file upload component utilises Blazor InputFile for the user to select the file to upload in the application. The component reads the Azure Storage connection string from the configuration, including the container, then uploads the file to the container and also adds a blob http header for the file content type taken from the file properties. The Radzen notification service is used to notify the user of the application activities. I also included a basic spinner as part of the interaction for the upload process.

Blob List Component (VisionBlobFileList.razor)

This component reads the Azure Storage connection string from the configuration, including the container, then displays the blob file names in a Radzen DataGrid. A button is added to Analyse the image, which then calls the Radzen notification service to display the activities being taken by the application.

Data Classes

Two data classes have been created as follows:

• AzureBlobFile.cs – Azure blob file properties
• ImageDetails.cs – Image details for extraction from the AI Vision Analysis

The UI

The UI is as follows. Notice the menu control has now changed since Part 1. Invoice Analysis will be formed in Part 3, at the time of writing this blog post, I had already uploaded the code to my GitHub repo.

Home page (Chat)

Image Analysis

Upload File Control

Upload Action Spinner

Radzen Blazor File Uploaded Notification

Process Button

The process button read the application configuration for the Azure AI Vision endpoint and service key, then retrieves a SAS token from Azure for the blob being processed and a URL is generated with the generated SAS token, then this is submitted to Azure AI Vision with the generated URL. The SAS token is generated by the async method CreateServiceSASBlob(string BlobName) in the component class. Whilst the method can be defined as a utility class, I have composed this for easier reading of code.

Image Analysis Dialog

When the image processing has completed, a Radzen notification is displayed to the user, with a Radzen dialog popping up to show basic metadata (height and width) of the image, including the text the AI Vision service has extracted as well as the image itself.

That is AI Vision and Image Analysis wrapped up.

Part 3 will focus on processing invoices using the pre-built AI model “prebuilt-invoice” part of Microsoft Azure AI Document Intelligence and creating output files for further processing.