Our Vice President came to us asking if we could build a dashboard to measure team velocity. We came back to him with a list of metrics we would rather measure, instead.

We wanted to improve DevOps practices and behaviors across multiple squads that are building and operating a global marketplace. There are hundreds of things we could measure. We seriously considered about 50 metrics that we could track (many from DevOps Enterprise Summit talks, or Accelerate) and chose to surface a dozen or so that would give us the most insights into agile delivery, the flow of work, availability, and code health.

This talk is about the metrics.

Assumption: you have a culture that will visualize metrics without fear.

How do we justify the value of each metric when challenged?

How do we set our thresholds for "good", "acceptable", and "poor" scores?

Which metrics are related and grouped together on the dashboard?

Which metrics do we intentionally not visualize and why?

Which metrics do we wish we had, that we don't?

Why we made it difficult to compare data across teams.