"Data professionals hate this. We’re trained to believe that good analysis leads to good decisions, that facts win arguments, that rationality prevails."
For a man to get a business case approved, just like any sale, the number one influencing factor is the credibility of the man making the sale.
Board members do not buy from men they don't trust.
The fact is that "data professionals" can not be trusted.
Trust is EARNED. It takes years to for a man to establish his reputation as an honest man of honour and integrity. It takes one lie to destroy it.
Men in the "data" area have destroyed their own credibility. Indeed, they have destroyed the credibility of honest men of honour and integrity like me. I have been tarred with the same brush as the snake oil salesmen.
In 2000 our airline, QANTAS, had their cargo division have a THIRD failed data warehousing project. The project manager was fired and the FOURTH project was given to a man with 20+ years in QANTAS Cargo. He was told if he failed he would be fired too.
So he called about 10 of his pals in the industry and explained his job was on the line with the new QANTAS Cargo Data Warehouse project. He asked his pals to please recommend who he should talk to for the project when it was his job on the line.
He was given my name FOUR TIMES and no other man was mentioned twice.
I eventually got the project. Why? The CEO and the senior managers trusted me because I EARNED their trust during the 8 week prototype phase. It did not hurt that 4 trusted advisors gave my name and said that if the project absolutely HAD to be successful they should hire me.
I was not surprised the prior projects failed. There were issues that were VERY complex. It was the toughest technical project I had done to that date. We actually had to invent new techniques to make it work.
It went on to be called "The most successful IT project QANTAS had ever run."
Indeed, during the interview process I gave the CFO an idea that he said was worth a couple of million dollars profit.
One thing that helped me earn credibility and trust was by giving them an idea saving them a couple of million dollars a year in the first two weeks.
When we did the demo at 8 weeks the CEO commented that I looked like I had been working there for 10 years. The CEO and senior managers were SHOCKED at my level of business knowledge of how the Cargo division worked in just 8 weeks. They could not believe my level of understanding of their business.
In short, when I asked for the $A1.7M budget I had EARNED their respect and their trust.
There is more to that story and I have published that.
The bottom line is that to get a million dollar plus "data project" budgeted the man presenting the business case has to be believed by those he is selling to. He has to be respected and trusted by the men voting on the decision.
Sadly, in our industry segment, men have focused on becoming better liars than becoming better at our profession.
After all? It is FAR easier to become a better liar than it is to become better at the "data" profession as it is so falsely called now.
Our industry segment is in a very sad state. Even my good friend Bill Imon has commented on this recently.
I can’t speak for James (though I suspect he’ll more or less agree with you too), but I tend to agree with you. There’s a huge % of data and IT people that think their job is to just build tech, and refuse to meet their business stakeholders where they’re at (let alone learn about the business)
James’ article (and most of my own writing) presupposes that our readers have already made that first leap, or are working on it. We’re not speaking to the arrogant (and frankly helpless) IT crowd that keep blaming the business for their own failings.
Thanks for your comment and the restack Peter! It’s a harsh message, but one more folks need to hear. And also welcome to the DPM community, I saw you just joined us 😊
I am 62 now. So I will not be around to see our profession become a "profession" if it ever does. Personally, I think our "profession" is doomed and will never hold any level of professional respect ever again. Today we are less than plumbers professionally.
Back in the early 80s many of us sincerely believed that the profession of building large computer systems might, one day, be on a par with the profession of building large buildings and similar things.
One of my uncles was a builder and I used to labour for him on his houses. I was first allow to stand out side the building sites when I was 6. And I was first allowed to actually step on site when I was 7.
As a little boy was I totally fascinated how it was possible to build houses like my uncle did. I had a hard enough time building anything with my building blocks. So you can imagine a 5 year old boy wanting to "help" his uncle build houses.
It was along slow process. I had to learn the name of every tool. I had to learn what it was used for. I had to learn where each tool was in each builders truck. They would call out from the building site and my cousin and I had the job of fetching the right tool from the right truck when we were 7.
Finally we were allowed to watch closely as the builders went about their work. And when they needed a tool they had in their truck we were to fetch it and watch how it was being used. We were taught about blueprints and plans. We sat with my uncle, his father, as the drafted up plans at his amazing desk.
We saw the attention to detail that went into building a house close up and personal. My uncle built 7 houses in 7 years and I laboured on them all. Of course I was much more useful when I was 14 than when I was 7. I also worked at the "mobile home" factory her worked at. They build homes and then delivered them on site. So I saw many homes built at the factory. Our job was to make the concrete blocks the homes sat on.
So I spent from 7 to 16 years watching homes being built. From descriptions of the home, to the first sketches, to the plans and blue prints, to the turning of the first sod, to the finished product.
When I started my first programming job in 1982 it very quickly became obvious to me what parallels existed between building a house and building a computer system. Indeed, Barry Boehm release a book called "Software Engineering Economics" in which he compared building software systems to building buildings.
The first system I worked on from 1982-85 was very badly built and very buggy. In 1983-4 I was the sole support guy for the 750KLOC system, the largest in the country, and I was getting 50-60 calls a day for problems I had to solve. I was working 60 hours a week plus doing a full time univeristy load in 1984. In 1984 there was not a night I got more than 4 hours sleep and there were plenty of sleepless nights too.
When I joined IBM in 1986 I joined the international software development lab in Australia. I joined a project called COBRA. The Common Billing and Reconciliation Application. As the name implies, it produced invoices. And it was to be installed in 26 countries including Canada, Japan, Australia as well as all south east asian countries and all south american countries.
It was the first international software development project in IBM Australia and our reputations were riding on it. I was blessed to be given a trainee position on that project. My manager, Terry, gave me the break of my life letting me work on that project.
Anyway, in my first week my manager asked me what I would like to work on as part of deciding what work I should be assigned to. I told him I wanted to work on testing and improving the quality of the system we produced because high quality software was important.
He actually laughed at me and said that I would not have anyone argue with me if I wanted to do testing. In the end he said "If you want to do testing? Knock yourself out." It was the least popular job in software development. I was introduced to the Business Analyst who ran the testing team. His name was George. Terry said: "He's all yours, do with him whatever you will, he actually wants to do testing."
George was of the same mind as me, that quality in software mattered. I then worked from 9am to past midnight all year. There was almost no day I went home earlier than midnight even when my girlfriend moved into my house. She often asked why she bothered since the only thing I did there was sleep.
We also often worked weekends. I was, of course, a first year trainee at the time and no one took much notice of me. But I worked my arse off testing the system as it was being developed. I had a photographic memory so I read all 300KLOC and when I found a bug I would write out what the code should look like and give it back to the development team.
Over time I earned respect as "this crazy guy who knows every line of code and works insane hours in the hope of finding more bugs to fix." Most of the team thought I was quite insane. Only the other guys in the testing team were actually even friendly. I was "crazy" as far as all the "normal" people were concerned.
But do you know what happened next? We committed COBRA 1.0 to the release tapes in January 1987, on schedule, under budget, and we installed it into 7 countries including Canada, Hong Kong / China, Singapore, New Zealand. And not one country reported one bug. This was a 1.0 BILLING SYSTEM that was 300KLOC and no one who worked for any of the IBM countries found a bug in it.
The system ran for over three years in IBM New Zealand before the upgrade and not one problem report was created. Not even one.
This, of course, was unprecedented. Everyone wanted to know "how could this be possible?". And the answer was that the man most responsible was George who was even more crazy than me. We got along like a house on fire. LOL!
After the success of Cobra 1.0 the next release, 2.0, was a disaster of unmitigated proportions. The IBM Manager lied to us about how it was going, the project manager lied to us about how it was going. And I wound up in the office of the Billing Manager for IBM Malaysia with the CFO and the project manager being grilled as to the results of what was happening in the Philippines install...that I had been told was going perfectly ok.
I was able to rescue that install and when I got back to Australia I wrote a VERY damning report on COBRA 2.0 and presented it to three levels of management. They were NOT HAPPY at my report and presentation. My position was that the entire labs future was on the line and there was no reason why IBM Corp would give us funding if we produced crap software.
So I campaigned inside IBM for software development methodologies to be more akin to building buildings. I campaigned very hard inside IBM to improve our development methods to be more like we had done on 1.0 and less like we had done on 2.0.
As part of my campaigning I was dragged into IBM HR numerous times and scolded and told I would be fired if I did not "ease up on the women" because I was "creating a toxic work environment". In the end, in December 1989, I could see that our development lab was going to be shut down. I told Terry I was leaving because I believed the lab would be shut down inside 5 years. I was wrong. It took 6 years before it was shut down.
More than 300 good paying software development jobs were sent to Malaysia because you can develop crappy buggy software anywhere with cheap people. That lab was later relocated again to India and finally IBM gave up on developing it's own software and went with SAP because the software being developed was crap.
The same happened across IBM labs. I worked in the DB2 lab and I had many friends in the DB2 lab. They all complained of the same thing. Lowering software development standards.
When I moved to IBM Marketing in 1990 I then promoted high quality software development processes to our customers...a bit hypocritical I know given what we were doing internally. And I continued to promote high quality software development until 2002. This included the area of data warehousing that I moved into by complete accident in 1991.
As you know, most development projects are fixated on an end date. But my mentor on COBRA 1.0 taught me "The users complain about the bugs in a system years after they have forgotten the implementation date."
In the end the model of "develop crappy buggy software using cheap people where ever they live and sell that to the marketplace" won the day. That's what we have today. We have cheap, crappy, buggy software that was more likely than not developed by poor underpaid programmers in India that crashes more often than I have a good nights sleep.
After 44 years in this industry? If we built houses like we build software our houses would fall down by the end of the first week.
If we built aeroplanes like we build software all transatlantic flights would have to land in Greenland to reboot the operating system for the next portion of the flight.
And the area of "data" has not been immune from this same "cheap and buggy" approach.
Yes, these are harsh words. But if our industry segment is not willing to listen to men like Bill, Ralph, Sean Kelly and I? Then harsh words are warranted.
The issue with "data projects" not being approved is not the above, though the above is all true.
The issue with "data projects" not being approved is that IT people in general and "data people" in general have zero credibility on the business side of the house.
Nor should they have any credibility.
We have managed our profession like a joke and so now "data professionals" are a joke on the business side of the house.
Not that long ago I was in a meeting with a CEO of a smallish company. She could see I was 60 years old and had long term experience. She was about my age. She asked me how it was possible for IT people to be so ignorant and so arrogant at the same time. She pointed out that most of her IT consultants would not know a debit from a credit.
I told her that 40 years of IT people telling themselves how smart they were while actually being very ignorant has led to a level of arrogance that was, frankly, embarrassing.
I told her that in the 90s I had tried very hard to promote honesty and hard work in my profession.
I told her that I tried very hard to have men take the position that we should name and shame the snake oil salesmen. I told her I failed and by 2002 I had given up. I just went about my business as best I could while snake oil salesmen got the majority of the attention in our profession.
And now? Every board member of every large company, every senior manager of every large company, knows full well that people in the "data area" are not to be trusted.
They know men like me exist in our segment, honest men, but they also know men like me are as rare as hens teeth and not even our own industry segment respects honest men of honour and integrity.
So they are not about to trust anyone in the "IT" area in general or the "data area" specifically, which is my area.
When the men in "data" want to be taken seriously by board members? They will take my advice that I have freely offered since the 80s.
Step one is "Be honest".
If a man in our segment is not even willing to be honest? Then I don't want to talk to him. I only want to talk to the very few honest men in our segment. Maybe they will take my advice, maybe they will not.
But any man presenting a business case for a "data project" should know that he is considered to be a "snake oil salesman" today by the men he is presenting to. And justly so.
Look at the failure rates of "data projects". Still over 50%. And we don't name the culprits. We let them go and commit other failures on other projects. We have no professional standards. We have less professional standards than plumbers.
Hi James and other readers.
"Data professionals hate this. We’re trained to believe that good analysis leads to good decisions, that facts win arguments, that rationality prevails."
For a man to get a business case approved, just like any sale, the number one influencing factor is the credibility of the man making the sale.
Board members do not buy from men they don't trust.
The fact is that "data professionals" can not be trusted.
Trust is EARNED. It takes years to for a man to establish his reputation as an honest man of honour and integrity. It takes one lie to destroy it.
Men in the "data" area have destroyed their own credibility. Indeed, they have destroyed the credibility of honest men of honour and integrity like me. I have been tarred with the same brush as the snake oil salesmen.
In 2000 our airline, QANTAS, had their cargo division have a THIRD failed data warehousing project. The project manager was fired and the FOURTH project was given to a man with 20+ years in QANTAS Cargo. He was told if he failed he would be fired too.
So he called about 10 of his pals in the industry and explained his job was on the line with the new QANTAS Cargo Data Warehouse project. He asked his pals to please recommend who he should talk to for the project when it was his job on the line.
He was given my name FOUR TIMES and no other man was mentioned twice.
I eventually got the project. Why? The CEO and the senior managers trusted me because I EARNED their trust during the 8 week prototype phase. It did not hurt that 4 trusted advisors gave my name and said that if the project absolutely HAD to be successful they should hire me.
I was not surprised the prior projects failed. There were issues that were VERY complex. It was the toughest technical project I had done to that date. We actually had to invent new techniques to make it work.
It went on to be called "The most successful IT project QANTAS had ever run."
Indeed, during the interview process I gave the CFO an idea that he said was worth a couple of million dollars profit.
One thing that helped me earn credibility and trust was by giving them an idea saving them a couple of million dollars a year in the first two weeks.
When we did the demo at 8 weeks the CEO commented that I looked like I had been working there for 10 years. The CEO and senior managers were SHOCKED at my level of business knowledge of how the Cargo division worked in just 8 weeks. They could not believe my level of understanding of their business.
In short, when I asked for the $A1.7M budget I had EARNED their respect and their trust.
There is more to that story and I have published that.
The bottom line is that to get a million dollar plus "data project" budgeted the man presenting the business case has to be believed by those he is selling to. He has to be respected and trusted by the men voting on the decision.
Sadly, in our industry segment, men have focused on becoming better liars than becoming better at our profession.
After all? It is FAR easier to become a better liar than it is to become better at the "data" profession as it is so falsely called now.
Our industry segment is in a very sad state. Even my good friend Bill Imon has commented on this recently.
I can’t speak for James (though I suspect he’ll more or less agree with you too), but I tend to agree with you. There’s a huge % of data and IT people that think their job is to just build tech, and refuse to meet their business stakeholders where they’re at (let alone learn about the business)
James’ article (and most of my own writing) presupposes that our readers have already made that first leap, or are working on it. We’re not speaking to the arrogant (and frankly helpless) IT crowd that keep blaming the business for their own failings.
Thanks for your comment and the restack Peter! It’s a harsh message, but one more folks need to hear. And also welcome to the DPM community, I saw you just joined us 😊
Hello Nick,
I am 62 now. So I will not be around to see our profession become a "profession" if it ever does. Personally, I think our "profession" is doomed and will never hold any level of professional respect ever again. Today we are less than plumbers professionally.
Back in the early 80s many of us sincerely believed that the profession of building large computer systems might, one day, be on a par with the profession of building large buildings and similar things.
One of my uncles was a builder and I used to labour for him on his houses. I was first allow to stand out side the building sites when I was 6. And I was first allowed to actually step on site when I was 7.
As a little boy was I totally fascinated how it was possible to build houses like my uncle did. I had a hard enough time building anything with my building blocks. So you can imagine a 5 year old boy wanting to "help" his uncle build houses.
It was along slow process. I had to learn the name of every tool. I had to learn what it was used for. I had to learn where each tool was in each builders truck. They would call out from the building site and my cousin and I had the job of fetching the right tool from the right truck when we were 7.
Finally we were allowed to watch closely as the builders went about their work. And when they needed a tool they had in their truck we were to fetch it and watch how it was being used. We were taught about blueprints and plans. We sat with my uncle, his father, as the drafted up plans at his amazing desk.
We saw the attention to detail that went into building a house close up and personal. My uncle built 7 houses in 7 years and I laboured on them all. Of course I was much more useful when I was 14 than when I was 7. I also worked at the "mobile home" factory her worked at. They build homes and then delivered them on site. So I saw many homes built at the factory. Our job was to make the concrete blocks the homes sat on.
So I spent from 7 to 16 years watching homes being built. From descriptions of the home, to the first sketches, to the plans and blue prints, to the turning of the first sod, to the finished product.
When I started my first programming job in 1982 it very quickly became obvious to me what parallels existed between building a house and building a computer system. Indeed, Barry Boehm release a book called "Software Engineering Economics" in which he compared building software systems to building buildings.
The first system I worked on from 1982-85 was very badly built and very buggy. In 1983-4 I was the sole support guy for the 750KLOC system, the largest in the country, and I was getting 50-60 calls a day for problems I had to solve. I was working 60 hours a week plus doing a full time univeristy load in 1984. In 1984 there was not a night I got more than 4 hours sleep and there were plenty of sleepless nights too.
When I joined IBM in 1986 I joined the international software development lab in Australia. I joined a project called COBRA. The Common Billing and Reconciliation Application. As the name implies, it produced invoices. And it was to be installed in 26 countries including Canada, Japan, Australia as well as all south east asian countries and all south american countries.
It was the first international software development project in IBM Australia and our reputations were riding on it. I was blessed to be given a trainee position on that project. My manager, Terry, gave me the break of my life letting me work on that project.
Anyway, in my first week my manager asked me what I would like to work on as part of deciding what work I should be assigned to. I told him I wanted to work on testing and improving the quality of the system we produced because high quality software was important.
He actually laughed at me and said that I would not have anyone argue with me if I wanted to do testing. In the end he said "If you want to do testing? Knock yourself out." It was the least popular job in software development. I was introduced to the Business Analyst who ran the testing team. His name was George. Terry said: "He's all yours, do with him whatever you will, he actually wants to do testing."
George was of the same mind as me, that quality in software mattered. I then worked from 9am to past midnight all year. There was almost no day I went home earlier than midnight even when my girlfriend moved into my house. She often asked why she bothered since the only thing I did there was sleep.
We also often worked weekends. I was, of course, a first year trainee at the time and no one took much notice of me. But I worked my arse off testing the system as it was being developed. I had a photographic memory so I read all 300KLOC and when I found a bug I would write out what the code should look like and give it back to the development team.
Over time I earned respect as "this crazy guy who knows every line of code and works insane hours in the hope of finding more bugs to fix." Most of the team thought I was quite insane. Only the other guys in the testing team were actually even friendly. I was "crazy" as far as all the "normal" people were concerned.
But do you know what happened next? We committed COBRA 1.0 to the release tapes in January 1987, on schedule, under budget, and we installed it into 7 countries including Canada, Hong Kong / China, Singapore, New Zealand. And not one country reported one bug. This was a 1.0 BILLING SYSTEM that was 300KLOC and no one who worked for any of the IBM countries found a bug in it.
The system ran for over three years in IBM New Zealand before the upgrade and not one problem report was created. Not even one.
This, of course, was unprecedented. Everyone wanted to know "how could this be possible?". And the answer was that the man most responsible was George who was even more crazy than me. We got along like a house on fire. LOL!
After the success of Cobra 1.0 the next release, 2.0, was a disaster of unmitigated proportions. The IBM Manager lied to us about how it was going, the project manager lied to us about how it was going. And I wound up in the office of the Billing Manager for IBM Malaysia with the CFO and the project manager being grilled as to the results of what was happening in the Philippines install...that I had been told was going perfectly ok.
I was able to rescue that install and when I got back to Australia I wrote a VERY damning report on COBRA 2.0 and presented it to three levels of management. They were NOT HAPPY at my report and presentation. My position was that the entire labs future was on the line and there was no reason why IBM Corp would give us funding if we produced crap software.
So I campaigned inside IBM for software development methodologies to be more akin to building buildings. I campaigned very hard inside IBM to improve our development methods to be more like we had done on 1.0 and less like we had done on 2.0.
As part of my campaigning I was dragged into IBM HR numerous times and scolded and told I would be fired if I did not "ease up on the women" because I was "creating a toxic work environment". In the end, in December 1989, I could see that our development lab was going to be shut down. I told Terry I was leaving because I believed the lab would be shut down inside 5 years. I was wrong. It took 6 years before it was shut down.
More than 300 good paying software development jobs were sent to Malaysia because you can develop crappy buggy software anywhere with cheap people. That lab was later relocated again to India and finally IBM gave up on developing it's own software and went with SAP because the software being developed was crap.
The same happened across IBM labs. I worked in the DB2 lab and I had many friends in the DB2 lab. They all complained of the same thing. Lowering software development standards.
When I moved to IBM Marketing in 1990 I then promoted high quality software development processes to our customers...a bit hypocritical I know given what we were doing internally. And I continued to promote high quality software development until 2002. This included the area of data warehousing that I moved into by complete accident in 1991.
As you know, most development projects are fixated on an end date. But my mentor on COBRA 1.0 taught me "The users complain about the bugs in a system years after they have forgotten the implementation date."
In the end the model of "develop crappy buggy software using cheap people where ever they live and sell that to the marketplace" won the day. That's what we have today. We have cheap, crappy, buggy software that was more likely than not developed by poor underpaid programmers in India that crashes more often than I have a good nights sleep.
After 44 years in this industry? If we built houses like we build software our houses would fall down by the end of the first week.
If we built aeroplanes like we build software all transatlantic flights would have to land in Greenland to reboot the operating system for the next portion of the flight.
And the area of "data" has not been immune from this same "cheap and buggy" approach.
Yes, these are harsh words. But if our industry segment is not willing to listen to men like Bill, Ralph, Sean Kelly and I? Then harsh words are warranted.
Hi James, and other readers.
The issue with "data projects" not being approved is not the above, though the above is all true.
The issue with "data projects" not being approved is that IT people in general and "data people" in general have zero credibility on the business side of the house.
Nor should they have any credibility.
We have managed our profession like a joke and so now "data professionals" are a joke on the business side of the house.
Not that long ago I was in a meeting with a CEO of a smallish company. She could see I was 60 years old and had long term experience. She was about my age. She asked me how it was possible for IT people to be so ignorant and so arrogant at the same time. She pointed out that most of her IT consultants would not know a debit from a credit.
I told her that 40 years of IT people telling themselves how smart they were while actually being very ignorant has led to a level of arrogance that was, frankly, embarrassing.
I told her that in the 90s I had tried very hard to promote honesty and hard work in my profession.
I told her that I tried very hard to have men take the position that we should name and shame the snake oil salesmen. I told her I failed and by 2002 I had given up. I just went about my business as best I could while snake oil salesmen got the majority of the attention in our profession.
And now? Every board member of every large company, every senior manager of every large company, knows full well that people in the "data area" are not to be trusted.
They know men like me exist in our segment, honest men, but they also know men like me are as rare as hens teeth and not even our own industry segment respects honest men of honour and integrity.
So they are not about to trust anyone in the "IT" area in general or the "data area" specifically, which is my area.
When the men in "data" want to be taken seriously by board members? They will take my advice that I have freely offered since the 80s.
Step one is "Be honest".
If a man in our segment is not even willing to be honest? Then I don't want to talk to him. I only want to talk to the very few honest men in our segment. Maybe they will take my advice, maybe they will not.
But any man presenting a business case for a "data project" should know that he is considered to be a "snake oil salesman" today by the men he is presenting to. And justly so.
Look at the failure rates of "data projects". Still over 50%. And we don't name the culprits. We let them go and commit other failures on other projects. We have no professional standards. We have less professional standards than plumbers.
Ok?