Welcome to JARVIS, the AI document redaction processor. At the time of publishing this version of his document, I was still developing Jarvis as a fully working product. The current version supports word documents and PDFs. With the development of Blazor Simple AI, I can also utilise the image analysis component to redact PII information from images..
If you would like to download a PDF of this post, you can download it here.
Jarvis is made up of the following technologies:
Microsoft .NET Blazor (.NET 6.0 LTS release)
Microsoft Azure Cognitive Services (Text Analytics Service)
Microsoft Azure Cosmos DB (for maintaining document and redaction processor metadata)
Azure Web App (hosting the JARVIS AI Web App)
Azure Storage (source document upload and redaction storage)
Microsoft Azure Function App (for APIs that process PII data and perform redaction processing)
Radzen Blazor components (for an amazing UI experience)
A document named “IPAddressandNamesx10Pages.docx” contains the following information, repeated within 10 pages.
——————————————————————————————————————————-
The IP Address if the legacy system is 10.254.1.1.
The company that owns the legacy system is Microsoft.
The original founders of the company Microsoft are Bill Gates and Paul Allen.
——————————————————————————————————————————-
The document is uploaded to Jarvis, the AI Document redaction processor.
The user clicks “Process” to determine the PII and confidential data held in the document.
A notification is sent to the user to advise the document has been submitted for processing.
About 3 seconds later Jarvis has identified the PII and confidential data in the document and provides a notification to the user.
The user can then click “View” and then select which data needs to be redacted from the document.
The user then clicks “Save choices”. This will save the collection of choices, the metadata, to Azure Cosmos DB.
The user then clicks “Redact” and the user is notified of the submission and completion.
The user clicks the download button which is available after the redaction process has completed. The document is displayed with the information redacted using Microsoft Office apps (this can be downloaded to the machine directly also).
The process is going to be made simpler by a set of walkthroughs in the UI which will be a set of steps with instructions, including a preview document component.
Welcome to the Blazor Simple AI Single Page App, Part 2 of the Microsoft AI services journey, which now includes image analysis utilising Microsoft Azure AI Vision. The Vision Read API is used to extract the text from an image. This document explains the project in my GitHub repository which is available here: https://github.com/tejinderrai/public/tree/main/BlazorSimpleAI.
If you would like to download both part 1 and part 2 as a PDF document, you can download the PDF here.
Since part 1, the following nuget packages have been added to the project.
Azure AI Vision Image Analysis – for reading text and metadata from images.
Radzen Blazor – for providing an amazing UI experience.
Azure Storage Blob – for handling interactions with Azure Blob Storage.
Visual Changes
I have made some appealing improvements from the basic Blazor template and styled the UI based on a project from Martin Mogusu available here: GitHub – martinmogusu/blazor-top-navbar: A top navbar example created in blazor. This saved me a lot of time and all I had to do was apply my own visual styles after the top navigation was applied to the project in shared/NavMenu.razor. In addition, I had added a pre-built model for interactive Invoice Analysis and processing, which I will leave the full explanation until Part 3 of this post.
Components
Three components have been developed for the image analysis. These are as follows:
Vision.razor – The Image Analysis page
VisionBlobLoader.razor– This includes the capability to upload files to Azure blob storage, which also sets the content type for the blob file.
VisionBlobFileList.razor – This is a child component embedded into the VisionBlobLoader component, which lists the image files that have been uploaded to Azure blob storage.
Note: Whist this project utilises the service key, in an enterprise environment, you must consider using token based access to the service secured by Microsoft Entra ID, or if you wish to utilise the service key for any reason, utilise Azure Key Vault to protect the key used by the application with a managed identity for the application to access the service key stored in Azure Key Vault.
Components
File Upload Component (VisionBlobLoader)
The file upload component utilises Blazor InputFile for the user to select the file to upload in the application. The component reads the Azure Storage connection string from the configuration, including the container, then uploads the file to the container and also adds a blob http header for the file content type taken from the file properties. The Radzen notification service is used to notify the user of the application activities. I also included a basic spinner as part of the interaction for the upload process.
Blob List Component (VisionBlobFileList.razor)
This component reads the Azure Storage connection string from the configuration, including the container, then displays the blob file names in a Radzen DataGrid. A button is added to Analyse the image, which then calls the Radzen notification service to display the activities being taken by the application.
Data Classes
Two data classes have been created as follows:
AzureBlobFile.cs – Azure blob file properties
ImageDetails.cs – Image details for extraction from the AI Vision Analysis
The UI
The UI is as follows. Notice the menu control has now changed since Part 1. Invoice Analysis will be formed in Part 3, at the time of writing this blog post, I had already uploaded the code to my GitHub repo.
Home page (Chat)
Image Analysis
Upload File Control
Upload Action Spinner
Radzen Blazor File Uploaded Notification
Process Button
The process button read the application configuration for the Azure AI Vision endpoint and service key, then retrieves a SAS token from Azure for the blob being processed and a URL is generated with the generated SAS token, then this is submitted to Azure AI Vision with the generated URL. The SAS token is generated by the async method CreateServiceSASBlob(string BlobName) in the component class. Whilst the method can be defined as a utility class, I have composed this for easier reading of code.
Image Analysis Dialog
When the image processing has completed, a Radzen notification is displayed to the user, with a Radzen dialog popping up to show basic metadata (height and width) of the image, including the text the AI Vision service has extracted as well as the image itself.
That is AI Vision and Image Analysis wrapped up.
Part 3 will focus on processing invoices using the pre-built AI model “prebuilt-invoice” part of Microsoft Azure AI Document Intelligence and creating output files for further processing.
Welcome to the Blazor Simple AI Single Page App, the AI App that responds to questions instantly using Microsoft Azure OpenAI Services. This document explains the .NET project I developed which I have pushed to my public Github repository which is available here: https://github.com/tejinderrai/public/tree/main/BlazorSimpleAI.
If you wish you to download the PDF version of this blog post, it is available here.
Technologies
Blazor Simple AI is made up of the following technologies:
Microsoft .NET Blazor (.NET 6.0 LTS release)
Microsoft Azure.AI.OpenAI .NET Library
Microsoft Azure AI Services – OpenAI
It’s that simple!
Why Blazor?
Blazor is simply amazing, I have been developing Blazor projects for over four years. There has been great demand for Blazor over the past few years and as a component framework and use of C# this is exactly what I need to develop solutions and concepts super fast!
What Blazor Simple AI Does?
Blazor Simple AI is a Blazor server side single page app which has a single page and a single component. The razor page has two basic user interface controls, a textbox and a submit button for a user to enter the question for Azure OpenAI. The component “AzureOpenAIChat.razor”, has a single parameter which receives the question from the main index page. When the parameter is received by the child component, the component has OnParametersSetAsync() method which then retrieves the appsettings.json values in relation to the Azure OpenAI service AI endpoint, Azure OpenAI key and the deployment name which has the associated model, which was deployed with Azure AI Studio, then send the text to the Azure OpenAI service and retrieves and displays the response.
Core Blazor Template Changes
There have been some basic changes to the basic Blazor layout to accommodate the project. These are as follows:
The sidebar has been removed from the MainLayout.razor page
A new Index.razor.css style sheet has been added to centre the UI components on the page
A new Components folder has been added to the project
A new component named AzureOpenAIChat.razor has been added into the Components folder
A new configuration section has been added to appsettings.json to include the configuration required for the project to interact with the Azure OpenAI service.
The title and main element have had text changes to represent the project name and description
Manage Deployments in Azure AI Studio and create a deployment using the gpt-35-turbo model
Update the appsettings.json with the settings
"AzureAIConfig": {
"OpenAIEndpoint": "https://[You Azure OpenAI Service].openai.azure.com/",
"OpenAIKeyCredential": "[Your Azure Open AI Key]",
"OpenAIDeploymentName": "[Your Azure Open AI Deployment Name]"
"RetroResponse": "true or false"
}
Build the project and ask Azure OpenAI anything you like.
The UI
The landing page.
Sample Questions and Responses
Question 1
Who founded Microsoft?
Question 2
Who developed OpenAI?
Question 3
How can I develop a Blazor App?
Basic CSS
The AzureOpenAIChat.razor component has a basic CSS style sheet which allows the deployment to have a retro style response or a basic response text visualization option. If the app setting below is set to true, you will get the retro response as per the sample above. For a standard non-retro style response, you can set the value to false, example below.
Following on from Part 6, where I provided the views of the components I developed to enter, display and edit data, I am soon going to publish the source code in my GitHub repo.
What have I been up to recently….
I’ve been working on a number of projects recently, from event driven architecture application design, integrations with Oracle e-Business suite, Azure Event Hubs, Azure Function Apps, API Apps, Microsoft Anti-Malware for Blob storage, event receivers and processors, Cosmos DB as a store for events, integration with Adobe Cloud and multi-tenant auth provider Blazor projects.
What’s coming next…
It’s also time for me to refresh my Xamarin Cloud Release project which I published to Google play a few years ago with a complete refresh using a .NET MAUI Blazor Hybrid app. This will also mean migrating and updating the MySQL datastore included in the project.
I believe it’s also time to refresh my blog post format, so there is a standard format moving forward for every blog post, so I have recently created a template for this exact purpose. In addition, I believe it is also time to move away from articles describing coded solutions for Blazor projects, to also include general Microsoft Azure architecture and recent updates in smaller short focused pieces more frequently. After all, I am a Microsoft Cloud Solution Architect!
I’ve been quite busy with all those projects and now I am going to ensure that some of the project learnings and configuration, including the architecture, is going to be part of the next set of blog posts. In addition, I have an exciting new project which I had shelved previously that I think it’s now time that It was code complete and then hosted in Microsoft Azure, so watch out for my announcement in the coming months.
Following on from Part 5, where I provided the links to the CarShop C# project which defines the data model classes for the CarShop database tables and views, I have recently been working on the core components which interact with the entity framework to add data to the Azure SQL CarShop database. There are a number of components which are being developed and I focus on these so that I can start to build out the forms and the interactions from a user experience including the layout of the edit forms and controls. I haven’t yet published the source code onto my github repo, but I expect to do this in the next few months. For now, I’ve have provided a preview of the form layouts and basic capabilities whilst I complete the forms and capabilities, including the completion of testing the components.
There are a few components I’ve built, which I have briefly described below that query/display the data loaded in the previous data seeding scripts. There are components with parameters that dynamically update child components, another which uses a NavigationManager to re-direct edit pages and others that utilise lists and display the entity model data in DevExpress UI components, including ASP.Net Blazor EditForm controls.
Hint: Click on each image and the browser will expand the image to the full size of the screen. This will give you the same view as seen in the my development environment.
Fuel Types – two components with a dynamic fuel type view when new fuel types are added.
Car Manufacturer List
Car Fuel Types with edit functionality
Car Colours
Car Models
New Car Model
Customer List – inline Edit Capability (DevExpress UI Component)
Following on from Part 4, where I provided the links to the CarShop views SQL, I have now published the C# project which defines the data model classes for the CarShop database tables and views. The data classes project has been separated into it’s own project as this will be shared as a dependency with the CarShop Blazor Server project and in future, I will develop the API’s in a separate project.
The project has been published on my GitHub repo here.
You can also follow the guidance from Microsoft to create model classes with the Entity Framework, documented here.
Following on from Part 3, where I provided the links to the CarShop database seeding SQL, I have now published the database view SQL scripts in my public GitHub repo. At present there are two database views.
The CarShop database view SQL scripts can be found in my GitHub repohere.
It is likely that over the course of developing the project, the number of views will expand, hence the reason to separate out the location of the view SQL scripts as apposed to the database mode creation or data seeding SQL scripts.
Following on from Part 2, where I provided the link to the CarShop schema deployment SQL, I have now published the database seeding scripts in my public GitHub repo, for the initial supporting data. At present there are six tables in the database which have data seeding scripts.
The data seeding scripts are listed below.
1 – EngineSize.sql – Inserts a list of engine sizes into the EngineSize table. 2 – Salutation.sql – Inserts a list of salutations for customers into the Salutation table (only basic salutations are loaded) 3 – CarFuelType.sql – Inserts the car fuel types for vehicles into the CarFuelTypes table 4 – CarColour.sql – Inserts a basic list of car colours into the CarColours table 5 – CarManufacturers.sql – Inserts a basic list of car manufacturers into the CarManufacturers table 6 – VehicleStatus.sql – Inserts a list of Vehicle status flags into the VehicleStatus table.
The CarShop data seeding SQL scripts can be found in my GitHub repohere.
Following on from Part 1, where I introduced the CarShop project, I have now published the database scripts in my public GitHub repo, for the data model. At present there are nine tables in the database schema, which need to be created in a specific order to maintain relationships. The tables are listed below.
CarManufacturers – Table to hold all car manufacturer details
CarModels – Table to hold all car models
VehicleStatus – Table to hold the vehicle status e.g. “For Sale”
CarFuelTypes – Table to hold the fuel type for a vehicle
CarColours – Table to hold the car colours
Engine Size – Table to hold all car engine sizes
Vehicles – Table to hold all car vehicle details which has relationships to the above tables as per the Part 1 CarShop project blog post showing the schema diagram.
Salutations – Table to hold all customer salutations
Customers – Table to hold all customer details which has a relationship with Salutations
Some of the tables hold default e.g. Vehicles previous owners has a default of 1 since the vehicles are not new at the Carshop.
The CarShop model SQL scripts can be found in my GitHub repohere.
Over the past few months I have been working on a sample project, which will eventually be published to my GitHub repo. The project specifically focusses on Microsoft Entity Framework with an Azure SQL database and also utilises Blazor as the underlying UI and logic layer, including DevExpress for Blazor UI components. This is an initial post which describes the project and it’s capabilities.
The CarShop project was envisioned from wanting to build something new and then work on multiple articles rather than only a few for a project, for part of this year. This will enable me to provide updates at various intervals and at some stage, including the publishing of the code.
Why a CarShop?
Since I am car fan and have been for many years, I thought this would be an exciting project to work on this year. With Blazor + .NET and DevExpress being some of my favorite development frameworks, I thought this would be ideal as a project.
The database schema
Since I am using the Microsoft Entity Framework in the Blazor Visual Studio project, the schema was exactly where I wanted to start. As the iterations developed, I decided to go straight into Azure SQL to provision my tables, entities and relationships, primary keys and foreign keys etc. The project needs to store car details (at a basic level), customer details, car manufacturers, car models, fuel types, engine sizes etc. Whilst it is a simple model to start with, it’s relatively simple to expand the schema as I see fit, both from the SQL backend as well as the coded elements and data classes.
CarShop Schema – Developed using dbForgeStudio 2022 for SQL Server
Since this is a relatively simple sample project, the data is held in a single Azure SQL database. As you can see, the Vehicles table has the most relationships with car fuel types, vehicle status, engine size, colours, models and manufacturers. For the customers table, I’ve kept the design simple for now although I intend to expand this into a scenario where there may need to be some data quality checks and periodic checks around when the customer data was last updated, for reasons I will include in a future post.
Part 2 will focus on the Transact-SQL, so that the schema can be provisioned.
Welcome to Tejinder Rai's blog, Cloud Solutions Architect 