9 min read

Lessons Learned from 5 years using Pivotal Tracker

Everything is made up, and the points don't matter
Lessons Learned from 5 years using Pivotal Tracker

I spent about five years using Pivotal Tracker on a variety of teams across Cloud Foundry. Saw a lot a lot of the same problems come up over and over again, and a few solutions to them. Some of those problems are probably specific to the particular time and place that created them, but if you use Tracker, you might have had a few of these yourself, and you might find some of these lessons useful.

Story acceptance responsibility rests with product authority

On Pivotal Cloud Foundry we called the "chief story writer and accepter" role "Product Manager," so that's how I'll refer to this role in this piece, but I'm told this role doesn't have much in common with Product Management at most companies. This is the person who holds the standard for "done" for the team, who is responsible for its behavior and its impact on the problem it solves.

For this person, story acceptance is a key tool for their core job: Defining the product. If anyone else does this work regularly – engineers, QA, or junior PMs – it becomes tedious busywork.

If you don't have a PM, don't fuss with trying to rotate acceptance duties amongst the engineers. Use a different tool, one that doesn't have as strict a separation between designing, delivering, and evaluating.

If you're tempted to off load story acceptance because it takes so much of the PM's time, make story acceptance faster. Removing the product manager from the daily work of acceptance and direct contact with the product might have its place, but that place isn't in a Pivotal Tracker backlog.

Acceptance must be fast and easy

If you take away one thing from this post, let it be this: Acceptance for most stories should take a few minutes. An hour at most.

If acceptance takes longer – if accepting a single story regularly takes hours – two things are going to happen.

One, product managers are going to tempted to off load story acceptance. They'll try to hand it off to "junior" or "tactical" PMs, or to QA, or to engineering managers, or whatever other chump is handy. And then: Tedious busywork.

Two, engineers are going to get frustrated. They may not be able to articulate why. Fast feedback – as Elisabeth Hendrickson explains better than I could – creates a sense of momentum, and momentum is like oxygen for an engineering team. If you don't have quite enough of it, you get slow tissue death.

Speeding up acceptance can be a tricky problem. The organization may need to take a hard look at the complexity of the product, and how it sources, hires, onboards, and promotes product managers. These are tough, unforgiving people problems: Slow acceptance across many teams may be a sign that, as great as your senior product leaders may be at defining a vision for the product, they're failing at leading the product organization.

One thing that won't work is berating your Product Managers about how much acceptance they have left to do. At best, you'll get Product Managers working overtime – people who are stressed and scared don't solve problems creatively, so don't stress or scare people you want to solve problems creatively.

"Finished" Is Not "Done"

One of the insidious things about Tracker (and a lot of Extreme Programming practices) is that it can lure you into thinking that substitutes for customer feedback are just as good as the real thing.

"The feature's finished, I clicked 'deliver' and the PM accepted it." Nope.

The feature isn't done until you hear back from a customer about it.

Most conversations about points are procrastination

I used to introduce new people to IPM (the "iteration planning meeting") with this quote from Whose Line Is It Anyway: "Everything is made up, and the points don't matter."

Pointing, as any Pivot will tell you, can be useful. It can surface differences in what people know about the system, and give feedback to the PM that a given story is bigger than they thought.

Beyond that, though, there's very little actual value in the various calculations that Tracker offers you. They can be fun to play around with, and they can be less-bad than a lot of the numbers that client teams are used to working with, but pay as little attention to them as you can.

There are only three sizes of story

No matter what pointing system my team used, we always ended up using the same three-position scale.

"Easy"

A lot of valuable changes to a given piece of software are more-or-less trivial to implement. Change a string. Adjust an existing "if" condition.

"About Right"

This is a story that the team is comfortable working. They've probably done similar work before. They understand roughly the work involved, and they don't expect many surprises.

"Too Big"

This a story the team isn't comfortable working. It might be a couple of "about right" stories stuck together, but more likely there's just something about it that makes it harder to estimate. The team hasn't done work like it before, or it touches a subsystem that they're afraid of.

In theory there's also a "way too big, send it back" size, but I don't think I ever saw one in the wild.

The Unit of Work is not the Unit of Value

There's a lot to like about small stories. They're easier to work on. They're faster to accept. Misunderstandings are less likely. The customer gets some value faster, and gets to use it while they wait for the rest of your work.

Ideally, the "right size" of story for a team to work also represents a meaningful new capability for the user. The "unit of work" is also the same as a the "unit of value." Tracker works best when this is usually the case.

But sometimes, a story can be made easier to work on by making it smaller than matters to the customer. Sometimes, a story at the size that the product manager and the designer and the customer think about is too big for an engineering team to handle effectively.

If you find yourselves talking a lot about the difference between a chore and a feature a lot, about how to break down stories ("should we do a spike first?"), or about how to write acceptance criteria for "technical" stories ("as an API client...") you're probably in a domain where the "unit of value" is much larger than the "unit of work." You can use Tracker in this situation, but it's not designed for it. If you can, find ways to bring these scales back in line.

Users should always be humans

Don't write "as an API client" stories. They might solve a problem in the moment, but they're probably hiding signal about problems with your team structure and your product architecture. They're a little bit like slowing down your release cadence because releases are risky – avoids short term pain now, makes everything harder in the long run.

"Chore vs. Feature?" doesn't matter

Teams can spend a lot of time discussing whether a particular story is a chore or a feature. Most of this discussion is waste, especially any discussion that hinges on "value to the customer."

A feature is a story that requires acceptance, and that a PM might not choose to prioritize if it takes too much time. A chore is a story that doesn't require acceptance, and that the engineers decide whether the team works on.

If a team is spending a lot of time talking about "chore versus feature," what they're actually talking about is, "Who gets to decide what we work on, and when?" and there's some sticky social reason that question is hard to answer. Software teams only function when the answer to that question is clear, so get to work on that, and the "chore vs. feature" question will take care of itself.

If a team doesn't work on chores, it's because of the engineers

Another of the insidious little things about Tracker is that it makes working on features rewarding and easy. The value is clear. The priority order is handled. It's obvious when they're done.

Chores, user research, design, story shaping – these are all relatively hard. You have to decide heuristically when you're done. In the case of chores, you have to decide priority yourself.

Engineers like to complain that PMs or managers won't "let them" do chores, and I'm sure this is true in some places, but I have personally never seen this to be the case. Always, always, when I encountered a team that was waste-deep in tech debt and investigated why, the product manager was the main person wishing engineers would work on that tech debt more – so that stories could be delivered faster. It was the engineers convincing themselves that it wasn't worth it, or stuck arguing about what to prioritize first.

Never put chores in the icebox

They won't get done. Engineers hate and fear prioritizing things. If they don't prioritize a story the moment they write it, it'll never happen.

Spend at least 25% of the team's time on chores

This is a rough heuristic, but any less than that and the team is probably buying today's productivity at the cost of tomorrow's.

A number of Pivots I worked with were uncomfortable with this number; they felt the team was "wasting" time on things that didn't provide "value." My (probably unfair) opinion was that these are folks who were used to leaving projects before the consequences of their decisions caught up to them.

The reason this heuristic works is a lot like why releasing more makes releasing easier and a product more stable. Tackling tech debt is hard, and requires skills that engineers may not otherwise practice much. The only way to get better at hard things is to do them more often.

Put chores beneath the next release marker

This is a "one weird trick" kind of thing but it works. Release Markers are one of the best features that most teams I was on didn't use. The teams that did use release markers were much more focused and effective, because the team always knew where the next event the PM cared about was – the next moment when they expect the software to be able to deliver meaningful new value to a customer. (This should never be more than a week or two out.)

A typical release marker won't be more than a few weeks out.

New stories go in the bottom of the backlog

Most of the teams I worked on had a bunch of stories at the bottom of the backlog that weren't ever going to get worked. New stories somehow always ended up being more important – the PM would habitually insert new stories somewhere in the middle of the backlog. This created an algal mat of dead stories, never to be worked on.

Occasionally – very occasionally – it's appropriate to put a story up in the next iteration or so. But that shouldn't be the standard position. New stories go at the bottom of the backlog. If a story is never important enough to work on before new stuff, delete it.

Delete your icebox

The icebox is the worst feature of Tracker. In theory it's where the PM puts stories that aren't ready to be worked. In practice, it's where stories go to die. Anything that's been in there longer than a couple of weeks is never going to get done. If you do ever work on it, so much context will have likely changed that you'll need to rewrite it anyway.

You're not saving yourself any time by keeping those stories. You're a grocer with a pile of rotting fruit in your back room, hoarding it because you're afraid to admit that you ordered too much from the wholesaler.

Discard your rotting fruit. If it's important, it'll come back.

Keep a "Won't Do" List

One thing "a place stories go to die" is appropriate for storing is stories that you won't do. Feature requests that the Product Manager has decided aren't worth the complexity they'd introduce to the product can live happily in the icebox, along with the reasoning of why they won't be prioritized.

Links, etc.

Nobody gives a hoot about profit – Nobody gives a hoot about team effectiveness, either. There's probably a longer piece I'm going to write about this but the short version is, it's seriously okay to prioritize working a particular way because you like it, because it's good for you. I spent way too much time trying to Maximize Team Effectiveness at the expense of my own mental health.

As an industry, we spend way too much time arguing about whether practices are "good." It seriously does not matter, use that time to do something life affirming.

I'm mildly obsessed with Slate's coverage of Olympic gymnastics. A 1500 word rant about how ugly and cursed something called the "wolf turn" is? Yes. I love good writing about super niche hatreds.

This interview going into detail about why Simone Biles withdrew from competition also resonated with me – it's a little different from the way we usually talk about mental health impacting physical health.

It feels the world stops, like a record scratch while you’re in the air and you think you’ve stopped, because the world’s stopped and then suddenly the apparatus is on your head and everything is upside down—because most gymnasts get so good at their respective tricks that when they compete, it’s all muscle memory. You basically go into a blackout state when you’re doing it. Sometimes you can accidentally snap to in the middle of it, like, where am I? And then the awareness goes away and you’re on your head.