| Books We Recommend |
 |
 |
 |
 |
How Do You Measure Up
What is the test of a good measure? Actually there are six.
1. It helps you to understand and improve performance.
2. It is related to the purpose of the system and what matters to customers.
3. It takes an end to end measure of the system
4. It shows the extent of variation in performance
5. It is visual and in the hands of those who do the work
6. It shows the relationship between leading (throughput, service) and lagging (operating expense) measures.
How many did you check?
P.S. Just to be clear, targets do not pass the test of a good measure.
As ever, I would love to hear what you thought about this blog. Click here to leave your comment now
8 Responses to “How Do You Measure Up”
Leave a Reply
March 4th, 2009 at 3:03 pm
Succinct and straightforward. This is now on the wall in my office. You never know, it might generate some thinking, even discussion.
March 4th, 2009 at 3:06 pm
I assume that “takes an end to end measure of the system” is to ensure that any identified improvements will improve the overall system performance?
March 4th, 2009 at 3:13 pm
I agree entirely with these characteristics of a good measures. For example, I have recently reviewed KPI’s for an IT call centre which showed significant improvement in performance. The problem was that the KPI measured number of calls dealt with in a month - unimportant to the client. The review showed an overall increase in the number of calls, extended resolution times, increased failure demand and reduced first time resolutions to problems. To deliver the above KPI there had been an accompanying increase in staff and therefore cost. No-one had looked at the design and performance of the actual system.
March 6th, 2009 at 12:49 pm
Recently I saw news pictures of 8,000 Honda cars mothballed in an export park and it made me think of a Council planning department. Each year planning departments are required to report to Audit Scotland the percentage of planning applications processed within 8 weeks. It’s a Statutory Performance Indicator and is one of the figures used to judge the effectiveness of your local Council. This is the local government approach to performance management, something that the car industry have been perfecting for decades.
The car industry however is different from a Planning department. Car manufacturers can decide to stop making cars if they decide they’re making too many. When they are producing cars they can do it at a known rate. Such is the history of measurement, process improvement and reliabilty engineering a car manufacturer can say with confidence how many cars they can produce in a day. No doubt they have data on recent customer preferences so they can make cars in the the right mix of colours, engine sizes etc. to anticipate short term demand. Of course even the best car manufacturers fail to predict some events. Hence the 8,000 mothballed Hondas.
Planning departments are different. A planning officers job begins when a customer presents him with an application. The customer effectively brings some raw material to the factory gate. Some customers bring enough for a dormer window in their loft. Others bring enough for 100 new houses or a new retail park. In some years the customers will be beating a path to the factory gate. In others a credit crunch will frighten them away. In each case the target set by central government for processing is 8 weeks. And yet a Planning department is generally a fixed resource. There is an established number of planning officers and, with the exception of some overtime working perhaps, little scope for making short term adjustments. And yet we continue to measure processing within an 8 week period as if it’s a production line.. It’s less an indicator of performance and more an indicator of the financial health of the economy and the availability of land for development.
How would a car manufacturer manage his business if they relied on customers arriving unannounced with varying a load of components to be turned into vehicles within 8 weeks? Probably very differently than they do today. Which is why you should be very careful before you describe to Planning officers how they should be practising performance management like Toyota or Honda. It only annoys them.
March 10th, 2009 at 8:35 pm
I have read Andy’s reply with interest. I think you have missed the point Andy no systems thinkers are saying (well I don’t know any!) that you can run a service organization like Toyota or Honda. Manufacturing is very different to Service as you rightly say. The point of transaction in a service organisation is when you as a customer try to pull value from the organisation.
For example putting in a planning application:
how easy is it for the customer to do this?
do the Planners make it easy or hard?
why is the target 8 weeks?
8 weeks is an arbitrary target why can’t planners do it quicker?
how many times do the Planners not have all the information to make the decision?
plus numerous other questions I won’t bore the reader’s with here.
Until you understand the purpose of the system from the customer perspective and how the current system performs against this purpose then arbitrary targets are meaningless. I wouldn’t mind betting that 8 weeks would be easily met within the current headcount and at lower cost even at the busiest times if the system allowed the front line staff to do what they are paid to do - pass plans.
Perhaps you should invest in a 5 day scoping (commission please Stuart :)) from Vanguard this will tell you just how your current system is performing.
On the Toyota analogy if you delivered to the factory gate all of the parts in the correct order then they would build your car within 24 hours not 8 weeks
could Planning do that?
March 19th, 2009 at 2:03 pm
Interesting debate,I think the media, pundit and ministerial commentary on the latest NHS (Staffordshire) problems are exactly as we could predict. It is not that targets which cause these problems it is just the wrong targets. It should be 95% of patients seen in Casualty within 4 hours not 97%. No one seems to have a voice that is brave enough to say that the targets drive practice that fails the patient and that they should be scrapped completely.
I actually agree that targets always make things worse but I do seem to have problems getting most people especially at work to agree with me.
I assume that the answer is to show them the wrong headed nature of targets by getting them involved in the work, but for some of the people I need to influence this is really difficult to do.
Any tips or a simple explanation of the folly of targeting (the wrong thing) that if delivered on the Today programme during the next NHS Targets debate would convince the Pundits and the Bureaucrats that targets are wrong
A
March 23rd, 2009 at 9:52 pm
Hi Andy, you are right, you can’t convince people unless they are willing to see it for themselves.
Something weve found effective is to take some data that shows that KPIs look good, and then run the data from the customers’ perspective in a control chart. You will often see that the two numbers look very different.
Or if you have brave staff get them to tell you all the cheats they use to hit the targets. Can you let me know more about the service you run and I can be a bit more specific.
On the second point the question to ask is “what method was used to set the target”? and “what is known about how it has, actually, affected performance”? Hopefully it might get a bright manager curious.
A good book on the subject is “punished by rewards” by Alfie Kohn.
Stuart.
April 8th, 2009 at 3:48 pm
Andy you are so not alone. If I get into a debate with a target setter and tell them their methods make life worse they look at me as if I have two heads. It is so “obvious” that setting targets drives improvement that they cannot see what actually happens. Splitting an organisation into functions and then managing each is the only way most people can make something simple enough that they can manage it. To see the organisation as a complex system and to make it function in an optimal way for its purpose seems much harder to do. I think the problem is that if you dont set “stretch” targets then somehow you aren’t tough enough to be a manager.