Employee feedback has been central to making Azavea what it is today. Our 10% for R&D and Learning program, profit-sharing, bicycle, office layout, monthly company updates, and a broad range of operational features of the company have come about through employee suggestions and feedback. This feedback happens in a variety of ways, including performance reviews, monthly office hours, the scrum process, CultureAmp surveys, and one-on-ones with managers. In this article, I’d like to outline one of the feedback mechanisms, called Start-Stop-Continue.

How it works

I first read about the idea in 2010, and we held our first one in March 2011. The Start-Stop-Continue (SSC) process has a few key features. Colleagues respond to an anonymous survey that asks three questions:

• What is the company not doing that it should be doing (Start)?
• What is the company doing that it should not be doing (Stop)?
• What is the company doing well that we need to preserve?  (Continue)?

Having collected this input, I address the aggregated feedback in a company-wide forum. This concept isn’t unique to Azavea, and many organizations utilize this process (companies, universities, design teams, etc.) in order to increase communication, transparency, and trust.

Our first SSC turned up a lot of feedback, and I held a second one six months later and then at ad hoc times over the next couple of years. Since 2014 I have aligned Start-Stop-Continue with our Annual Meeting in February or March.

Why it works

As described above, Start-Stop-Continue is pretty simple. It’s a clear cut way for colleagues to share their feedback. There are only three questions and you don’t have to answer them all. It’s also anonymous, which enables people who may not feel comfortable raising an issue with the CEO directly to do so. We have other opportunities for sharing feedback, but I’ve learned over the years that a range of feedback mechanisms is important. And the candor of the anonymous SSC plays an important role.

I want to emphasize that it doesn’t always work, and it doesn’t work for all company issues. In the first few years, we had very long discussions about fairly contentious topics. Over time, it became clear that complex topics that require extensive discussion were not good candidates for this process. We learned the hard way that SSC raises such a diversity of topics that it’s best to time-box discussion on any one topic, and that it’s a quicker process when I remove items discussed in previous rounds. We’ve edited and refined the system over the years, but have ultimately determined that the SSC is useful, important, and effective.

Changes made

Since we started doing Start-Stop-Continue, we’ve made several changes in company policies and how we run the business. These changes range from the substantial (how vacation time is handled) to the trivial (refilling the snacks jars two times a day instead of one and purchasing Dance-Dance-Revolution for the office). A few highlights:

Office

• Bought a ping-pong table
• Installed pullup bars when we moved to a new office
• Bought a pool table (one year)
• Limited pool table hours to after 5 pm (the next year)
• Added standing desks, initially shared, and later for anyone that asked
• Added healthy snacks (weekly order of fruit and vegetables)
• Raised the AC temperature by 2 degrees in the summer months
• Offered shared bikes (we now provide three shared Indego keys and accounts)

Benefits

• Increased PTO for jury duty from 3 to 5 days
• Converted half of our observed federal holidays to vacation days in order to give employees more flexibility in when they take time off
• Began running annual flu vaccine clinics free of charge to any employee who wants one
• Allowed employees to convert unused personal days to vacation days at the end of the year (to avoid a rush of personal days in December)

Technology

• Introduced solid-state drives on every workstation or laptop; this started as a trial run following one year’s Start-Stop-Continue, then it became the standard
• Placed computers in every conference room
• Implemented mandatory 2-factor authentication

Transparency

• I started holding office hours — a monthly opportunity for any employee to have a 30-minute conversation about anything with the CEO
• Converted our monthly R&D standup to a monthly company update from the CEO
• Made our contract balance sheet public within the company

Company

• Integrated phone screens into our interview process
• Organized company ping-pong and chess tournaments
• Began a staff rotation for cleanup after company meetings where there’s food involved
• Started composting
• Gave out more company swag (last year’s backpacks were a big hit!)
• Subscribed to print news in an effort to support media, following the 2016 election

It’s not perfect

As I mentioned above, it’s not an ideal process for everything. A few of the shortcomings are outlined below.

Repeats

Despite publishing the notes every year and asking people to review them, some topics inevitably pop up year after year:

• “Stop leaving dishes in the sink”
• “Start allowing dogs in the office”
• “Start paying for 30 min lunch”
• “Stop developing SaaS products”
• “Stop pursuing R&D grants”
• “It’s too cold in this office and I’ve run out of outerwear!”

The repeat topics became enough of a joke that one year a colleague created a Start-Stop-Continue BINGO. Repeats are inevitable, both because there are always new colleagues that haven’t been part of previous discussions and because there will be disagreement (see below) or no clear way to resolve an issue. We won’t be able to respond to every suggestion, some suggestions may not be feasible, and we won’t be able to come to a consensus on every issue. Whether or not to include a repeat item is at my discretion. Sometimes I think it’s worth additional discussion and sometimes I’ll put it in the “Asked and Answered” list.

Bias toward those that respond

Not everybody responds to the survey sent out ahead of the meeting. In the last couple of years, less than half of the company responded. This is probably inevitable as we get larger, but it also risks highlighting the people that have something to say at the expense of others. These others can represent a silent majority that either feels like things are fine or feels like even if they said something, nothing would change. We’re working on making sure all of our employees feel heard.

We can’t change that

This format of feedback points out that there are things we are (mostly) powerless to change. It’s been suggested several times that we look into alternative energy providers and understand our energy sourcing. At our old office that was legitimately impossible as we were locked into what the building offered. When we signed a lease at our new office, we made sure energy sourcing was negotiable. Over the years, one of the most popular suggestions is that we stop requiring timesheets, but while we all would prefer that we didn’t have to track our time, most of our revenue derives from professional services, and we won’t be in business very long if we don’t accurately track our time.

We can’t change that in this forum

There are some things that we can’t solve in this meeting, but discussion here can lead to longer-term changes in other venues. We have had consistent feedback over time about improving the diversity of the company. This has led to a company objective in 2018 to develop a Diversity & Inclusion Guidelines document as well as several initiatives to improve our practices. This year we’ve formed a D&I working group, which is similarly charged with suggesting additional changes.

In the past, SSC was a platform for folks to share their concerns about working with certain companies or on certain projects. These comments, in addition to several conversations in my office hours and in project debriefs, led me to write Azavea’s Project Selection Guidelines. I’m very proud of this document and I’m glad we have it. This change didn’t happen at SSC, but it was part of the process of codifying how we address this aspect of the business.

Disagreement

As with any company-wide discussion, there is plenty of disagreement. Often when one employee wants to start something, another wants to stop it. We’ve had disagreements in a number of areas. Here’s a sample:

• The coffee is amazing vs. the coffee is too bitter/sweet/sour/etc.
• Adopt Slack vs. abolish Slack (I’m personally in the “Slack is a plague on productivity” camp, but few of my colleagues agree with me)
• Continue SSC vs. abolish SSC
• The office is too loud vs. the office is too quiet – people should talk more
• Buy more unhealthy snacks (nuts and candy) vs. get rid of unhealthy snacks

When in doubt, we default to being generous in our benefits (we believe that providing “radical” benefits shouldn’t be that radical). Majority opinion is given some preference but isn’t necessarily the rule, and if things are going well, we don’t want to mess with what’s working. For instance, we won’t change coffees or stop using Slack if most people enjoy these methods and if the alternative would be a major process overhaul. But minority opinion also matters, particularly when the subject affects the welfare of a few people. For example, many people are in favor of having office dogs – I’d personally welcome it, and we negotiated the option into our lease, but a few people have severe allergies to dogs, so we’re not going to be having dogs.

Flip-flops

We find that some years we implement a change, only to have requests to reverse that change the next year. One example of this was when we traded some observed federal holidays for vacation days, in order to make time off more flexible for folks. The next year a small group of people wanted their federal holidays back. For some issues, there is rarely a permanent solution. With a changing and growing employee base, we have to listen anew every year.

One year we introduced a pool table after an SSC. The next year we dealt with the varying expectations and pain points around that pool table. Some colleagues were concerned that clients would think we were spending all of their money and our time playing pool. Others were annoyed at the loud noises it incurred at all hours. After some discussion, the pool fans agreed to limit their play to after 5 pm.

SSC is not perfect as a format for discussing feedback or solving problems. As we do things that seem to solve significant pain points one year, we see complaints about the fix the next year. Also as we grow in size, this format is less useful in curating good discussion and problem-solving than it was when we were 25 people.

Some things I’ve learned

Timeline: I have tried different timelines, and I have found that the response rate does not change, whether I send out the questionnaire one month, two weeks, or 5 days ahead of time. In general, I try to send this out about two weeks before the meeting and give colleagues one week to complete.

Response rate: As the company has grown the response rate has declined somewhat, but in most years, about one-third to one-half of my colleagues respond to the survey.

Group the items into themes: Sometimes people submit one or two items in each of the three questions, and sometimes it’s a long list of items. I try to organize items into thematic groups, and at the actual meeting, I present slides organized into these themes. In recent years, these themes have settled into: how the physical office space functions, the technology we use, benefits and people operations, and a grab bag everything else. The majority of issues brought up at our SSC tend to be culture-focused. We think people’s lives here are pretty important, and my colleagues have a lot of ideas about how to both maintain what we have and improve on it.

Keep the pacing brisk: At the beginning of the meeting, I give an overview of the process, highlight some updates from the previous year’s suggestions, and highlight a few items as “Asked and Answered” for items that we have previously discussed at length and will not be re-hashed this year. For each theme, we briefly highlight the things people want to continue but spend most of our time on the things people want to start or stop. I usually give a preamble and try to interpret the suggestion (sometimes I have to guess at some cryptic statements). Then I open up the floor for discussion for each issue, with a 5-minute time box enforced by a time-keeper. If the sentiment isn’t clear, I will often ask for a show of hands for different options. If there is no consensus, or if it is a matter that needs to be researched further before a decision is made, it is kicked to me or the appropriate VP or manager to further explore and report back on a later date. Alternatively, I may ask for a volunteer that wants to take on the task.

Follow through is important: There is a designated note-taker, and we both publish the notes and try to follow up on each discussion.

Why it’s important

It’s important that we have a range of mechanisms for feedback. The SSC forum acts as a method of public accountability for actually doing things that we say we are going to do. We also have an obligation to say why we are not going to implement a suggestion.

Start-Stop-Continue has also been a way to clarify things for people. At a company our size, people often have a misconception of what a policy is. It’s easy for a misconception to be passed around by word of mouth. SSC provides a mechanism to clarify misconceptions and assure people that we’re doing the right thing.

Finally, it’s important for people to have the opportunity to express opinions in an anonymous way. Anonymity gives people permission for a kind of candor that we’re not always used to. At Azavea, we place a high value around transparency and putting people first, with respect, kindness, and humility. As CEO, I’m wrong a lot of the time, and I need to have a forum through which to hear that feedback. For me, I find this process both extremely helpful and also one that generates a lot of anxiety. Sometimes the feedback is heart-warming and encouraging. Here are some examples from the past few years:

• Continue: Running the company with moral integrity
• Continue: Interesting, impactful work that makes the world a better place
• Continue: Caring deeply about Azavea employees and their families
• Continue: Being the best place to build software in Philadelphia

But it can also be stark and unvarnished criticism; anonymity gives people permission for a kind of candor that isn’t always easy to hear. Like most people I find the negative feedback harder to take than the positive and being confronted with all of the things people want better can be tough. Nonetheless, the general consensus has been that Start-Stop-Continue belongs alongside the other ways that we invite suggestions for how to improve the organization. From the start, I’ve wanted to build a great organization in which people carry out fulfilling work and have positive relationships with their colleagues. The only way to actually accomplish that is to do so together and this type of feedback and discussion is critical to fulfilling that vision.