SENTIMENT ANALYSIS OF YAMMER POSTS - MICROSOFT FLOW & AZURE COGNITIVE SERVICES

As an Office 365 product manager and corporate evangelist, I’m responsible for engaging users and driving adoption of Microsoft Collaboration tools. Measuring the saturation and use of Office 365 is a key part of my role. Yes, I regularly review Office 365 usage metrics for high-level trending. But metrics alone don’t tell the story of user satisfaction and adoption. In order to build better training and adoption programs, I need to understand why my dedicated users love the tools and why others remain resistant.

Many companies rely on surveys to gather end-user feedback. While surveys are useful for gathering specific types of quantitative data, surveys are one-dimensional. You can’t dynamically ask follow-up questions to learn more about specific survey responses, and you can only capture a limited set of data points. Innovation games enable you to gather a much broader set of quantitative and qualitative user data, but require an investment of time to facilitate games and distill the results. For best results, I recommend a multidisciplinary approach that leverages Office 365 usage statistics, user survey responses, innovation games data, user testimonials, etc. to measure user satisfaction. 

With the release of artificial intelligence (AI) and machine-learning algorithms, we also have the ability to gather user sentiments automatically. If your organization uses Yammer to drive employee engagement and empower open dialogue, you have a wealth of user data that can be analyzed. With Azure Cognitive Services and Microsoft Flow, you can perform automated sentiment analysis of your Yammer group posts. Sentiment scores for each Yammer message can be stored in SharePoint and visualized for trending analysis via Power BI. You can even send push email notifications to your Office 365 administrators or Corporate Communications team when strong positive or negative messages are posted in Yammer.

Chris Bortlik, Principal Technical Architect for Microsoft, recently shared a blog post on Yammer sentiment analysis. I used Chris’ model, with a few modifications, to gather and report on Office 365 user sentiment.

The scenario:
My organization leverages a Microsoft Flow Yammer group to foster employee conversations and questions/answers about flow. We want to monitor the Microsoft Flow Yammer group using sentiment analysis so we can:

• Identify negative flow Yammer posts that require follow-up
• Identify positive Yammer posts that can serve as user testimonials or references
• Define trends in our Microsoft Flow Yammer posts (e.g. daily/weekly/monthly positive and negative trends, overall positive or negative sentiments for flow, etc.)
• Validate the success of our Microsoft Flow education and adoption program (e.g. confirm we’re seeing growth in the volume of positive flow Yammer posts over time)

The setup:
Follow the steps outlined below to set up automated Yammer sentiment analysis.

Step 1: Confirm you have a Cognitive Services Text Analytics Account. In order to set up this solution, you will need a Cognitive Services account key and a root site URL.

Step 2: Create a SharePoint list to store your Yammer sentiment analysis scores. Flow will create a new item in your list for each Yammer message it analyzes. Here’s a list of the custom columns I added to my list:

• Score – Number column; stores the sentiment rating for each Yammer message
• Message link – Hyperlink column; stores a link to the rated Yammer message
• Posted by – Person/Group column; stores the name of the person that posted the Yammer message
• Thread ID – Single line of text column; stores the Yammer thread ID for the message. Enables you to sort, filter, and group sentiment scores for a given Yammer thread (including original message and replies).

Azure Cognitive Services will provide a numeric sentiment score between 0 and 1 for each Yammer message it analyzes. The more negative a Yammer message is, the closer to 0 its score will be. More positive messages will receive a rating closer to 1.

Here’s a screen shot of my SharePoint list. Each list item represents a rated Yammer message:

Step 3: Identify the Yammer group you want to perform sentiment analysis on. You can set up sentiment analysis for multiple Yammer groups, but each will require a separate flow process. I also recommend setting up a different SharePoint list to hold sentiment scores for each of your Yammer groups. (Having different SharePoint lists enables you to set up different trending reports on Yammer group sentiment.)

Step 4: Create your Microsoft Flow. I created my flow from scratch (not using a template). Here’s a quick breakdown of the flow conditions and actions:

• When there is a new message in a group – Detects when a new Yammer message is posted in my Yammer group
• Get user details – Pulls Yammer user profile details. (Enables us to capture the full name and email address for the person posting the Yammer message.)
• Detect Sentiment – Calls the Azure Cognitive Services API so it can calculate a sentiment score for the Yammer message
• Create item – Creates a SharePoint list item for the Yammer message being analyzed
• If the comment is negative – Sends an email to my Office 365 admin team if the sentiment score for a Yammer message is ≤0.3.
• If the comment is positive – Sends an email to my Office 365 admin team if the sentiment score for a Yammer message is ≥0.7.

Step 5: Create Power BI report(s) to visualize your Yammer sentiment scores. Published reports can be rendered in your SharePoint Online Communications or Team sites using the Power BI web part. For help in setting up sentiment analysis slicers, check out this DataChant blog post.

Here’s a sample dashboard that shows Yammer sentiment data for my Microsoft Flow Yammer group:

Step 6: Distill the results. Once you start calculating Yammer sentiment and have reports to visualize the data, you can analyze the results and follow up where needed. Here are a few ideas to get you started:

• Break down negative Yammer posts (e.g. posts with a score ≤0.3) by user. Schedule follow-up meetings with Office 365 end-users that consistently post negative messages. The goal is to ask questions and understand the pain points the users are facing. Perhaps they have hardware or network issues that impact their productivity. Or maybe they’re having issues with Microsoft Flow and need a coach/mentor to spur their learning. Having one-on-one dialogues provides the opportunity for candid feedback and enables you to make a difference in the user’s productivity and technology experience.
• Identify Office 365 enthusiasts. Break down Yammer posts by volume or by high sentiment average in order to find power users across your organization. Set up meetings with these enthusiasts to understand how they leverage Office 365. Incorporate them into your user group or internal community and support them in their growth. These enthusiasts can become your Office 365 evangelists!
• Monitor the volume of Yammer posts in your group. Build a gauge that shows your number of Yammer messages month-to-date and identifies progress towards your monthly Yammer message goal. Keeping an eye on your total posts month-to-date and year-to-date will help you monitor use over time and highlight areas you may need to invest additional time and adoption efforts in.

• Optimize your communications. If one of your Office 365 admin resources has consistent negative Yammer sentiment scores, have them evaluate the verbiage they’re using. Slight wording changes can change the tone of their messages, increasing Yammer sentiment scores and better engaging with end-users.
• Take a health pulse. Build trending visuals that show average post volumes and sentiment averages by week or month. If you start seeing spikes on volume of posts and/or significant changes in your sentiment averages, it’s time to dig deeper. Perhaps you’re seeing a spike in interest in PowerApps after a compelling user group presentation or have network bandwidth issues that are causing issues. Either way, Yammer sentiment analysis can be your early warning indicator that something has changed.

https://blog.splibrarian.com/2019/01/14/using-microsoft-flow-azure-cognitive-services-to-automate-sentiment-analysis-of-yammer-posts/