We’re very lucky to be a part of a fast-growing SaaS B2B2C business - the work is interesting, exciting, challenging, and rewarding. As product managers, we’re smack bang in the middle of everything and we play a central role in driving the business’ growth. But with rapid growth comes growing pains.
One area we’ve recently noticed has been under strain is how we communicate changes to our product to the wider business.
Every quarter we dedicate 10% of our objectives to improving how we work, so for the first quarter of 2021, we decided it was time to tackle our communication process for product releases.
We use Kanban to manage our workflow and practice continuous development, releasing new features and updates daily. Some of these releases go pretty much unnoticed, but a fairly large chunk will change a part of the user experience.
We have a company-wide Slack channel called #productupdates where we post about each release, so people could know when a change was live.
We also sent out a weekly Product Newsletter on Monday mornings with a roundup of the features released the previous week.
For bigger or more complex releases, we would hold training sessions for the teams that would be impacted by these changes.
With the volume of changes we were making to the product each week, we started noticing a few problems:
We wanted to understand and address what was causing these problems, and understand how to better communicate product changes, so our colleagues could self-serve and so that we could reclaim some of our own time.
As always with product management, we started by trying to understand the problem. We spoke to a bunch of our colleagues from all areas of the business about what they want to know when we release an update to the product.
We learned that people were contacting us directly because:
From this feedback we formed the following hypotheses:
To get the ball rolling, we created a centralised place where all the information about releases could go, so that people could self-serve right straight away.
When we discussed what we should use as our centralised place all sorts of ideas were thrown out, including tools like Notion or internal wikis, but we wanted to test a hypothesis and the fastest way to do that was to start simple: we created a spreadsheet.
In it, we categorised and added links to any release notes we created in the previous quarter (regardless of quality), and shared it in our #productupdates Slack channel.
Next, we looked at improving the quality of our release notes to ensure that our colleagues had the answers to their questions in the first place.
We felt it was important to do this in a standardised way so that the information wouldn't be impacted by the different styles of each product manager and which would also make it harder to measure the impact of changing our process.
During our earlier discovery, we asked our colleagues exactly what information they wanted about a release. Based on this feedback, we created a release notes template on a Google doc which acted as a checklist for the information we needed to include and committed to using this template for every new release.
The new release notes template contained the following:
Our new release communication process now looked like this:
After a few weeks of following this process we did see a slight drop in people contacting us with simple questions, but not as significant as we had hoped.
We went back to our colleagues to find out more. The response was mixed.
Out of 18 people who answered the question: ‘Would you know where to find the product release notes?’
Out of 25 people who answered the question: ‘Do you use the release notes?’
So far, so-so, but not the impact we were aiming for. We were encouraged by the number of people who used the release notes, particularly as the people who said they didn’t, tended to be more senior members of staff who don’t need to know the intricacies of each new release (as they’re not on the phones to customers).
However, we were worried that people still weren’t aware of the centralised spreadsheet and were still struggling to find past release notes, because they were still coming directly to us with questions.
We also received feedback that, while the new template was good, there was information people wanted that was still missing.
Armed with more data and a better understanding of what our colleagues were looking for, we made a few adjustments:
We’re tentatively calling our new process a success as we’re still measuring its impact, but we have all seen a substantial drop in questions about recent updates to the product and we’ve received some positive feedback from the people across the business who have been using our new release notes.
The Google Docs activity dashboard has provided us with visibility about who has viewed our release notes so we can see that they’re being used and by whom - and also who isn’t using them - so we are able to get more targeted feedback.
Now that we have validated our hypotheses, the next step is to invest in making the process better.
For example, building an internal wiki which might make categorisation and searching for specific release notes easier, particularly with the high volume of releases.
Going forward we will continue to track engagement, collect feedback, and ensure our team members have the information they need at their fingertips. This process has been a great way for us to apply lean development principles and shows that you don’t have to be building a new product to implement them yourselves.
Want to advance in your tech career? Check out the jobs we have available.