Measuring the success of a development project is tricky. It can be easy to put too much emphasis on arbitrary metrics, but success is largely not defined by metrics. If you take a step back and look at the bigger picture, success is not as clear cut as “a velocity of 10 points per week”. Let’s look at the ways we can measure if your software development project is successful.
Mandatory: Are You Delivering Value?
Value could be for your users in the form of features, for your stakeholders in terms of admin tools, for your developers in terms of paying down technical debt, or anything else that’s bringing value to your company. Try to think of all involved parties and what your priorities are.
A development team can be working through 100 points per week and finish way under budget, but if they’re not delivering any value to the product they’re wasting time and money. On the other hand, your project could ship months late, be $100,000 over budget, and still be a raging success because your customers loved it and your sales skyrocketed.
Are Your Customers Happy?
Your project’s customers could be actual paying customers or it could be Larry over in accounting. Either way, if you’re not keeping your customers happy, then your project is probably going downhill. Happiness is subjective, but this metric doesn’t have to be terribly hard to measure. Here at Quick Left, we have weekly surveys that our customers fill out. It’s simple, just a 1-10 rating score and a comment box. This is easy for our customers to fill out and gives us direct feedback on how we’re doing. Even better, we can plot this over time to see how we’re trending. If we get a survey that has a low score, then we can immediately bring it up and take action to correct it.
This same metric can be done with customers on a product, you just have to be a little more creative on how you solicit that feedback. You could start by sending a subset of users an email saying, “Hey, we’ve been making some changes lately. How are we doing?”
How Healthy Is Your Code Base?
Ahh, technical debt. This metric is usually a huge deal for developers and an oversight for managers. In reality, your technical debt can directly correlate with how fast you are able to respond to market changes. One extreme of this is that the project has been hacked together so much that it has to be rewritten in order to make future changes. It usually doesn’t get that bad though. A common scenario would be after 6 months of development, your team needs to take a month to refactor before it can move on to bigger changes.
It’s useful to think of technical debt as exactly that: debt. You’re trading quality code for a faster implementation time. You also accumulate interest on your debt, increasing the time it takes to implement features. If you take on a large enough debt it could easily double the amount of time features take to complete. Keeping this in mind, it’s always important to focus on writing quality code, not just hacking things together.
The amount of technical debt you accumulate is in large part determined by the quality of your developers and project managers. However, you could also accumulate large amounts of technical debt due to a short timeline. In general, the tighter your timeline is, the more likely you’ll increase technical debt. Trying to do way too much work in way too short of a timeframe is otherwise known as the “death march”, which no one wants to be a part of.
Are You On Budget?
When I say budget, I mean both time and money. Your project may not have a budget for either if you’re developing in-house, but most projects have at least one. If you have a money budget, that translates into a time budget because your employees or contractors have an hourly rate. So almost always we’re developing on a time budget.
You can use a project tracker like Pivotal Tracker, Sprintly, JIRA, or even sticky notes to help track your progress. The general idea here is to estimate the work you have to do and track how much work you’re doing per unit of time. Then you can use your velocity (amount of work / time) to make sure you’re on track to hit your milestones. For instance, if you’re using Pivotal Tracker you’d assign points to user stories and track your velocity in points completed per week. Then you can use your weekly velocity to extrapolate into the future to see if you’ll get the allotted work done on time. The same thing would work with sticky notes per week.
The catch here is that this metric can be entirely arbitrary based on your initial budget. If you budget a six month project to be done in two months, then you will always be under velocity. This doesn’t mean the project is a failure though. It actually might be a big success in every other regard. The initial estimate was off, not the project. The best you can do is to constantly monitor your expectations and make sure they are reasonable.
In Conclusion: Adapt To Your Situation
Not all of these are equally important all the time. It varies depending on the situation. Sometimes the deadline is the most important thing. Sometimes code quality is the most important thing. Usually keeping your users happy is very important, but it could also be that no matter what you do Larry won’t like the accounting software you’re building for him. Above all, you need to ensure you’re delivering value because if you’re not doing that, then what are you doing?