Office365-Azure AI hybrid: Get more out from your SharePoint Forms using Sentiment Analysis

SharePoint Forms are a great way to capture feedback from users who are filling data for an application or process. And not only feedback, there are many requirements (see conclusion for some ideas) where we can use SharePoint forms for capturing valuable information for businesses inputs. However, most the time, because of the size and amount of such feedback the captured information just sits in a list to be forgotten with time. So, what is an easy way to work with cumulative data without looking or going through the large chunks of them.

One answer for this scenario is to use big data or similar but the challenge with this approach is that they are economically costly, so not many smaller applications could use this. One of the clever approaches would be to make the data relevant for the reviewer so that there could be a selective review of it.

In this blog, we will look at a similar approach to use AI Sentiment analysis to flag the relevant content for the users to look at.

Note: The Sentiment analysis process is not a perfect but a nearly accurate representation of the commentary. The analysis provides us a quick score which could then be used to filter the relevant information. Then a final human evaluation of the highlighted content is still recommended

Architecture

The architecture is simple but needs an Azure Subscription to host the AI services for Text Analytics. Briefly about Text Analytics, it is one of the AI services provided by Azure AI for analysing text data at rest or in transit. The Text analytics uses web api hosting api in Azure for each region, but the content is stored in the client tenant itself. To have a look at the privacy policy, please check here.

The interaction between Text Analytics and SharePoint data could be done using a SPFx web part, Flow or PowerShell. In this blog, we will look at steps for flow to read feedback and populate a column in SharePoint list, which can then data to a SharePoint web part.

Implementation

Now lets’ take a look at the steps below:

1. For creating a Sentiment Analysis endpoint, we will create a Text Analytics cognitive service on Azure. And that’s it!! we are ready to the Text Analytics service for Sentiment Analysis. Easy right!! every Text Analytics service comes with an api endpoint which allows it to be invoked from any application.

Note: Sometimes businesses, especially the smaller firms, are worried about cost and maintenance of the AI Services. However, I would recommend that the cost is a later thought after validating the use of the service. For trial, there are various free tier and cost-effective Pricing tiers, for e.g. S0 for Sentiment Analytics provides 1000 calls per minute at a minuscule rate (0.70 cents – this could vary as per your license)

2. Next, we set up flow to send data to the Azure AI service to get the Sentiment analysis on the feedback. Now you might ask why Flow? There are many reasons:

• Scheduling of workloads to control the use of the pricing tier effectively
• Batch processing allows data to the translated in buckets thus saving and making effective use of AI Pricing
• Effective for data that seldom changes

3. Now, in flow we will call this endpoint using the HTTP Service. For calling the HTTP endpoint let’s see the request payload that needs to be provided for the service. The calling syntax and request body is below. For more information, about the request body please check here.

Please note that this call could be done from any application that supports web service calls and can handle response json that is returned.

Web API
https://australiaeast.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment
Request Headers
{
  "Content-Type": "application/json",
  "Ocp-Apim-Subscription-Key": "0ee3f5c24e6043789a6e3d37f86f80a3"
}

Sample Request Body
{
  "documents": [
    {
      "language": "en",
      "id": "1",
      "text": "Hello world. This is some input text that I love."
    },
    {
      "language": "fr",
      "id": "2",
      "text": "Bonjour tout le monde"
    },
    {
      "language": "es",
      "id": "3",
      "text": "La carretera estaba atascada. Había mucho tráfico el día de ayer."
    }
  ]
}

4. After the call is made we get the response in json with a sentiment score for the text provided, which can then be saved in a SharePoint column for further use for eg. we could use column formatting and show a formatted color type for the sentiment. Below is any example reponse. More information about the the reponse body format can be found here

A quick sample response body is below.

{
  "documents": [
    {
      "id": "1",
      "score": 0.92
    },
    {
      "id": "2",
      "score": 0.85
    },
    {
      "id": "3",
      "score": 0.34
    }
  ],
  "errors": []
}

Key highlights

1. Text Analytics request body expects some text, which could be empty but not null. So, handle any null conditions while building the request.

2. During the api call, we could provide any text blocks for analysis during a single request. For that we will need to create a request body with incrementing ID number and add text for each block. So, as long you could create a dynamic JSON, we could use a single request to get the analysis score for multiple text data.

3. As mentioned above, sentiment scores received from the API are just indicative. The reason being that sometime the provided text could have trigger words which might get the sentiment score too high or too low, which mayn’t be the intent of the statement. So, the score should be used an indicator and not as a final output until verified by the stakeholders.

Conclusion

In this blog, we saw how we could use Azure Text Analysis API for making feedback data or any such review information relevant. This use case could be used for various purposes, such as:

1. Customer compliants or feedback
2. Product feedback
3. Surveys
4. Inspection data