Becoming a Manager: Learning to Rely on Data

Having been a manager* for a while now, I’ve learned more about what it means and what changes it requires in thinking.  This installment of the “Becoming a Manager” series covers the increasing reliance on abstract data that is required as you move up the ranks.  Everyone who is an IC knows that upper management demands lots of charts and data.  Sometimes this makes sense.  Other times we know it distorts the reality which is apparent on the ground.

When I was a lead I managed by walking around.  I didn’t pay much attention to statistics.  Instead, I would regularly touch each of my team members.  I would have 1:1s, regular scrum meetings, and hallway conversations.  This allowed me to have a strong sense for what was going on in my team.  This worked great with 6 reports.  When I became a manager I tried to continue this same method of staying on top of what my team was up to.  The problem was, however, that I now had 20 reports.  It’s not possible to touch all of them regularly enough.  It is also a lot harder to keep the issues at play in 20 peoples’ daily work in one’s head.  Finally, most of these 20 had leads which stand in the org chart between myself and them.  Trying to manage by walking around undercuts these leads because now their reports have 2 managers, not one.  This is confusing for all involved.

What about just touching my direct reports and getting a sense for the product from them?  This would seem to work, but is not terribly effective.  Each person reports differently and normalizing the information coming from each is difficult.  It becomes worse when people use the same words to mean different things.  When I try this technique, I almost always later find out that while I’m getting the same information from each, the results on the ground in each team differ greatly.  How then to get a normalized view of what is going on at the IC level in each team without going to each person to ask?

The answer lies in gathering data across the team.  I’ve come to rely on the dreaded charts for much of my knowledge.  I have learned to take advantage of tools to track how the team is progressing in getting its work items done, how many bugs are active, what our pass rate is, etc.  This allows me to get a view at a glance of how we are progressing across several key metrics.  As long as I am gathering the right data, I can have an accurate view of how the team is doing.  Based on this data, I can know which areas are doing well and which ones need more personal attention.

The key here is choosing the right metrics to measure.  The team will optimize for whatever it is I am measuring.  A wrong metric will distort behavior in undesirable ways.  I have found it is important to track only a few items.  A small number of items can be understood by all.  These do not describe everything about the team, but they can act as the proverbial canaries which point out trouble early.  I make sure everyone knows exactly what I am tracking.  Every one of my leads knows the queries I use to monitor things.  This allows them to point out flaws in my methodology and gives them transparency so there are no surprises.  It is also important to keep the metrics stable over time.  A different chart each week gives people whiplash.

One point about using data to monitor a team:  you need to stay flexible in the use of the data.  The data is a rough facsimile of a real thing that needs to be done.  The data itself is not the goal.  I have seen too many managers confuse the data with reality.  This causes them to push for clean metrics even when this causes undesirable distortions in behavior.  If the data is showing a different state than reality, fix the metric, not the behavior.  Align the data with the team’s actions, not the other way around.  It is important to note that data hides a lot of things.  Relying solely on data is a surefire way to fail.  I still get out and walk around to validate the conclusion I’m drawing from the charts.  The data merely helps me know which areas to spend my limited time digging into deeper.

It takes a while as a manager to become accustomed to viewing the world through the lens of spreadsheets and charts.  Just as it is difficult to learn to trust others, it is difficult to learn to trust data.  It is even more difficult to learn when not to trust the same data.

 

* Manager means having leads report to you.  A lead is someone who has only individual contributors reporting to you.