The problem
Sometimes I get frustrated that tools don't exist to help people understand language. There are any amount of AI tools that will generate boilerplate text for you, but very little thinking about how to deliver insights into language usage at a natural level. With this project, I aimed to solve that.
We have no budget for tooling, so anything would have to be custom coded and fully designed by me. We'd use publicly accessible data, removing the need to tap into expensive APIs. I'd only use data that was from the user's own mouth. We didn't want our interpretation of their words, but a tool to discover raw insights.
What I wanted to learn
This was a tool by a writer, for writers. I already had a good idea of what would be useful for me, but I evaluated this by running small, unmoderated testing with folks from professional groups that I'm a member of. This was ultimately a tool for my team to gather insights from, so we were the ultimate proof of it being worth the effort to adopt.
What we found
- The only dataset that was easily available was our Google Play Reviews.
- This dataset went back to 2018, and represented over 39,000 reviews. Manually going through it would be a job in itself.
- Sentiment was key to understanding how language was being applied. Someone 'angry' with us tended to use a lot more words to explain their problems.
- We couldn't use standard dictionary searches, as language evolves so quickly with Gen Z and our audience that we had to know how they were using it before we could search. It became a bit of a chicken-and-egg situation.
What I did
- Tech: I created a Python-based web crawler to create our initial dataset.
- I created this adaptable, so we could look at any company in the same way if we wanted to.
- I did an initial analysis of the data using Google Sheets.
- To improve on this data, I created a separate Python script using the Natural Language Toolkit, which helped pick up on some initial trends.
- Ultimately, Google Sheets became too much to maintain and a chore to search through. We needed something we could search for a term, and get a beat on how users were using it, or if language we'd introduced has started to pull through to users.
- Content and final design: You can search for individual terms or words. This is handy for finding connections between words. 'Do users who talk about X also talk about Y?'
- 'Context' data is front and centre. Knowing the average number of words in a review gives an idea of how engaged a user is when using a specific term, etc.
- A further context setting panel includes both the ability to view reviews from a specific date and the frequency of words used around the search term. This helps to understand what you're looking at, and nudges you towards any surface insights.
- The second panel essentially serves as an extra filter, split by emojis used within the review. To their original purpose, emojis = emotion, and they serve double duty here to restrict the data to particular emotional states that a user may have felt while writing their review. For example, if I'm trying to learn if 'Assets' is a term users resonate with, and I see a high count of the enraged emoji, 😡, it sets the context for a deeper dive.
Every string on the screen was a decision
Open any one of these and I will tell you why it is worded that way.
Allows discovery of connections between words and how users use language in context.
What happened
I use this tool almost daily to check in on terms I'm introducing into the product. It's a limited dataset, and not exhaustive by any means, but it serves as an initial guide to build off. I share these links frequently with our C-level team to justify changes, and it has become one of many tools I use to build user insight.