Mobilizing The PCI Resistance, Part III: Quantifying The Huge SOX-404 Problem...

Previously, I wrote in Part I about "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer...", and in Part II, I wrote about the problems that management and auditors faced in 2005 and 2006 for the IT portions of SOX-404.

In Part III of this series, I will continue walking through the January 2006 GAIT summit slides, and show you the objective evidence that there was a real problem that needed to be solved, and our vision of what the solution was.

The Damage Of Bottom-Up Auditing

Actually, let me rewind a bit.  I didn't realize it at the time, but in 2005, I heard a great presentation by Patrick Gunderman that hinted at the magnitude and scale of the SOX-404 IT audit problem. Back then a Senior Manager in the KPMG audit practice.  He showed a slide that blew me away.

In the slide above, KPMG found that "The estimated percentage of deficiencies identified show IT controls accounting for the most (34 percent), followed distantly by revenue (13 percent), procure to pay (10 percent), and fixed assets (10 percent)."

What this means is that auditors were spending time digging around IT infrastructure, and finding lots of deficiencies.  Then for each one, management would either have to remediate, or argue with the auditors that it wasn't worth fixing, because an IT control failure would not result in an undetected material error.  Now, if the Enron and Worldcom failures were caused by rogue DBAs, then maybe this level of scrutiny was warranted.  But, something definitely doesn't seem right...

It’s estimated that as much as $3 billion was spent in the first year of SOX-404 to fix IT controls to remediate these findings. Ultimately, most of these findings were found not to be direct risks to accurate financial reports and did not result in a material weakness.  This is because they followed a bottom up versus a top-down, risk-based approach.

At the January 2006 GAIT Summit, we had publicly traded companies present how this problem was affecting them and their need for a better way.  Universally, they talked about the huge IT audit effort and fees associated with SOX-404 that was totally disproportionate to the risk.

These companies included (in no particular order), Goldman Sachs, Marathon Oil, Microsoft, Hewlett Packard, Chevron Phillips Chemical, Business Objects and so forth.

One of the most compelling data points was presented by Fawn Weaver at Intel.

This slide shows how 50% of the SOX-404 compliance effort was IT-related, which was generating almost 80% of the findings.  Yet, none of those findings represented a real risk to an undetected material error.  (So again, why was all that work performed?  It shouldn't have been.)

In my next post, I will write about how bottom-up auditing happens and our vision behind GAIT.  Next, I will write about the politics of GAIT, and how we assembled the constituencies, what was in it for them, and how I learned to use one of the most valuable tools in my career.

All of this helps (at least, in my mind) inform the PCI problem statement, as well as the strategy of how we can solve it.