Could Big Data have stopped a war?

It was a very strange experience - having actually lived through something and then, only understanding with crystal clarity, what had truly happened some forty odd years after the event! Somehow the horrors of the war seem even more real in hindsight than they did at the time.

Why is that I wondered?

At the time, Dwight D Eisenhower and then John F Kennedy were busy slipping US troops into Vietnam as ‘advisors’. They had not bothered consulting me first (not that surprising really, as I was just a snotty nosed school-boy at the time) but they had conveniently not bothered consulting Congress on the matter either!

By the time Lyndon B Johnson, Richard Nixon and later Gerald Ford were adding and then finally reducing American troops from a peak of nearly 500,000 men (well mostly men, but that’s a different story) – more than 58,000 would never come home! - I was too pre-occupied in trying to keep my young family fed to know, or care what they were up to.

At the time, I was concentrating on getting to grips with data analysis to try and figure out when we (The Premium Saving Bond office, where I then worked as a capacity manager) would need to increase the capacity of our mainframe computers and storage. We did not yet use the term ‘big data’ but we sure had a lot of it to play with!

As it turned out I was not the only person who had no idea what was going on regarding the Vietnam War. All of the American people, all of the Vietnamese people, all of the combatants on both sides, most of the commanding officers, practically all of the politicians and certainly the rest of the world were quite deliberately not told what was going on. When the ‘Vietnam Papers’ were released in 1971 it was clear that successive Presidents had deliberately deceived the American public, and everyone else as well!

Today we tend to think of fake news, misinformation and outright lies as very modern, but boy they were well perfected during that particular conflict.

You are probably beginning to wonder what all of this has got to do with Enterprise Big Data? I will try and explain.

Having been glued to our TV screen for almost 20 hours of riveting revelations it became abundantly clear that there were several distinct occasions during the 20 year conflict when a resolution could and should have been found and US troops could and should have been withheld or withdrawn, but successive US Presidents chose instead to escalate and extend the conflict further – more in pursuit of their own legacies or reputations – and in the name of ‘stopping communism’, rather than what was actually the more sensible or right course of action, especially as communism would suffer a pretty big collapse of its own as time transpired!

Time and again I found myself asking “how did the American public let them get away with that?’.

There is a saying that truth is always the first casualty of war, but surely someone knew what was actually going on? But did they?

How do you know whether things are going according to plan, assuming you have a plan in the first place?

In Vietnam this was all boiled down to one thing - ‘body count’ – how many enemy soldiers have we killed today, this week, this month etc.

So here we are, at last. It was all about data!

General William Westmoreland had calculated US victory would follow if they could reach the ‘tipping point’ – the point at which the US military were killing enemy soldiers at a higher rate than the North Vietnamese could ship replacements down the Ho Che Minh trail through Laos and Cambodia to reinforce their fighters in the South of Vietnam.

It turned out that this was never achieved, despite (largely anecdotal) data to the contrary!

A few days after watching the series I was discussing the new Enterprise Big Data analyst qualification with Jan Willem Middleburg, the author of the APMG EBD qualification scheme, and after our call I found myself asking the question “could that have happened today?”

If today’s Enterprise Big Data analysis techniques had been available then, could they have allowed a more accurate assessment of how well the war was actually going – and would this have made any difference to the way events transpired?

If enemy casualty data had been better and more accurately gathered, cleaned, wrangled and presented – and compared with other factors such as;

• American losses  
• The cost of resources,
• Equipment failure rates
• Weather conditions
• Geographical and topological regional breakdown
• Public opinion

We might have had results showing things such as;

• A much better understanding of actually enemy casualties, separating soldiers from civilians and removing duplications or distortions (many of which were deliberate)
• The value of better promotion and incentivisation of accurate data gathering
• The breakdown of casualties on both sides in particular engagement situations, allowing intelligence on how best to engage, and when not to.
• Graphical visualisation of the ongoing expenditure in comparison with casualty rates, showing the more cost-effective and successful engagement situations.
• Clearer and more supportive identification of equipment failure, allowing better selection and/or rectification and maintenance.
• Weather predictions and how conditions affected mortality, morale and efficiency / effect on equipment

My conclusion is that such techniques, if properly applied, could certainly have allowed the Generals and Presidents to have had a more accurate picture, and perhaps supported better decision making on their parts.

But would they have actually done things differently, or would their political machinations still have overridden common sense? Well not if we had also had today’s social media I wager!

How could they have kept the lid on what they were up to? Surely the American public would have discovered how badly the war was going, the fact that none of the Generals and Presidents actually thought it winnable, and the appalling things that were going on in their names sooner? And in this case surely public opinion would have demanded a resolution to the conflict years, if not decades earlier?

I suppose it is just an academic thought today, because that was then and now is now – but there are still plenty of very complex and complicated things going on in the world that the use of Enterprise Big Data can help resolve.

For example, in my next article I will, amongst other things, take a look at how these techniques can be used in today’s fields of medicine and health diagnosis and treatment – very topical because as I write the World Health Organization has just declared the Coronavirus outbreak a world pandemic! I can tell you one thing - it was big data analysis that led them to this conclusion, and big data analysis will also have a major role to play in helping find an eventual resolution!