Software metrics are one of those topics that come around over and over. Most recently, Irving Reid got the ball rolling the other day with this post. I responded with a couple links of my own through the backchannel which meant I didn’t trackback, so here they are in the open (with the trackback).

  • Michael’s presentation entitled [Cem’s](http://www.developsense.com/presentations/metricsminefieldtassq.pdfThe Metrics Minefield</a></li>
  • <a href=>) paper on [Software Engineering Metrics: What Do They Measure and How Do We Know?](http://www.kaner.com/pdfs/metrics2004.pdf)
  • A gem from a mailing list conversation recently; metrics answer the question of ‘where are we now?’ and provide context to the question of ‘where do we want to be?’ (or something like that)
  • Sandy Kemsley blogged yesterday from IQPC BPM Summit and one session had lots of great advice about how to — and how not to — develop metrics that work for your company. Sure, this is in the context of BPM, but likely also applies to how we build and test software
  • Some points on metrics I’ve had from a long thread on metrics on the CMMi mailing list
    • Eliyahu Goldratt, creator of TOC (Theory of Constraints), has some famous sayings:
      “Tell me how you measure me, and I’ll tell you how I will behavior.”
      “Measure me in an illogical way, then don’t complain of my illogical behavior!”
      His point is that policies and measurements are the main drivers behind human behavior. Think about it and you’ll see that’s almost always the case, at work, at home, at church, at school, wherever… Unfortunately, the prevailing practice is to establish local optima rules (do what is best for some part of the system – individuals, departments, functions), which are often in conflict with the global optimum (do what will be best for the overall system – the organization).
      This knowledge must be, thus, used to induce the parts of the system to do what’s best for the system as a whole (system thinking), i.e., to achieve the goal of the system. For that, the goal must be clearly stated and be known to everyone in the organization. Then you must develop policies and measurements that will drive people to take the needed actions in order to achieve the intermediate objectives, which will lead to the goal.
      TOC provides simple, yet powerful, tools and processes to help developing such policies and measurements.
      This is a “natural law”: measurements are tied to behavior and performance of people. You may ignore it at your own risk… As you may try to ignore gravity… 😉
      So we better learn how to use this law for our advantage…
    • So, what does TOC suggest as measurement categories on “What is not done properly”?
      1. Things that should have been done and were not
        • If we didn’t do the things we should have done, our clients (whoever they are) got disappointed with us
        • This is a measure of our RELIABILITY
      2. Things that should not have been done but, nevertheless, were done
      • If we did things that were not needed, we have overloaded the system with junk! We weren’t effective.
      • This is a measure of our EFFECTIVENESS. What is the beauty of having only these two measurements categories? First, they don’t overlap! No ambiguity!
        Second, they are clearly focused on the goal of the system: being effective (do what it’s supposed to do) and reliable (we can trust it delivers what it promisses). Isn’t it what CMMI is all about? 🙂
  • For those interested on knowing more about these measurement “philosophy”, I (not the me “I”, but the original poster’s “I”) recommend the audio-book “Beyond the Goal”, by Eliyahu Goldratt. It’s a set of 8 CDs, accompanied by the respective PPTs. There is a downloadable version (with no PPTs) at www.audible.com . Of course, there is much more being discussed in this book, all of it is very relevant for the IT industry, particularly for ERP customers and suppliers.
  • When metrics are being developed they should not be attached to employee performance, or indicate how an employee is performing etc, but rather how a process is performing or such.
  • It is totally appropriate to use measures to evaluate individual performance. However, I believe that those measures are different than the ones that we are using to manage process execution.
  • It’s been said that “what gets measured gets done.” I believe that “what gets measured gets manipulated” – especially if my future pay (or even job security) rests on those measures! Tell me the numbers I need to get my raise and I’ll make sure you get them, whether they reflect reality or not. This is certainly NOT the kind of measurement capture, analysis, and reporting that we can tolerate when trying to manage process performance.