Revitalize Your Audit Program Part 2Mar 14, 2020
This is the third in a series of blog posts covering the content of our October 2019 ASQ Audit Division conference presentation sessions: "Revitalizing Your Audit Program Using Engaging Audit Reporting Practices". This post goes behind the scenes to show the mechanics of creating charts from checklists in Word or other text document or spreadsheets.
Skip this post if you already:
- use software for reporting findings that creates charts that are meaningful and flexible,
- have your audit checklist in a spreadsheet set up to insert pivot tables and are confident in creating pivot tables.
Read this post if you:
- want to know how to get from a text checklist into a format that can be charted;
- want to confirm your spreadsheet checklist / audit protocol is properly formatted to be able to create a pivot table;
- have no idea what a pivot table is;
- need a quick refresher on how to make a pivot table and use it in different ways; or,
- want guidance to export findings from your software to be able to create charts presented here.
Watch the video if you:
- want to learn the basics of how to make a pivot table and several options for pivot charts.
Supplemental support: Get the How To Make a Pivot Table handout for explanation of key steps. The video above shows the steps and some pitfalls to avoid as a companion to the handout and this blog blog post.
Let's get started! In auditing hundreds of organizations, au dit reports that are limited to text in "ISO-speak" are a common factor that decreases engagement of leadership. One way to start bridging the gap and engaging leadership (and yourself) is to create visuals in the popular red-yellow-green format.
Problem 1. Our audit checklist is a MS Word or other text-based format. If you have ever tried to copy and paste text into a spreadsheet, you will know how annoying it is for a single question or paragraph to be translated into multiple rows in the spreadsheet. A hot tip is to browse the text document for any hard returns (line breaks) and remove them before you do the copy and paste into the spreadsheet.
You can put the line breaks back in the spreadsheet WITHIN a cell using Alt + Enter. Magic.
Once the spreadsheet rows are okay for you, then numbering the rows to be able to do the charts is next.
Problem 2. Numbering the rows while maintaining the connections to the audit protocol or standard can be a challenge. The ISO standards have Sub-Clauses which can be interpreted as text. Or after a sort, Clause 10 is listed after Clause 1 and before Clause 2.....frustrating! The snapshot below shows a hot tip to format a few extra columns that allow charts to be generated by Clause, Sub-Clause, or overall totals.
Problem 3. The categories of findings reported are often limited to bad news of only nonconformities and opportunities for improvement. And we wonder why audits are perceived to be negative and fearful! Including criteria-based positive findings are an important step to bridge this gap. Human beings thrive on acknowledgement. Within our company, Innovation Training & Consulting, Inc., we strive to distinguish different categories of findings tailored to the maturity of the organization audited. What will speak to their leadership and be understood? Too many categories of findings will confuse organizations at the basic level. More mature systems are worthy of discernment to clarify the feedback for leadership.
Your first decision on categories of findings is whether your auditors are able to review processes and systems deeply enough that they can recognize conformance and/or strengths. Does the audit program resourcing allow sufficient time for auditors to write fact-based conformance statements?
Are opportunities for improvement considered to be welcome feedback or distractions from all the other priorities? If auditors are pointing out issues with paper clips while the department is fighting dragons, the opportunities will be ignored and credibility of the auditors will slip. However, if the opportunities are meaningful within the context of current objectives and strategy, they may be welcome. Another issue is that often an opportunity for improvement is a soft-graded nonconformity. This is common when the auditors are not empowered through the audit program to give candid feedback.
The difference in the two bottom graphs in the slide below is that the opportunities for improvement are not included in the type of findings.
Problem 4. All nonconformities are not the same, but most reports do not distinguish the type! There are 3 types of nonconformities: INTENT is when there is a requirement in the standard that is not being addressed within our system (we missed it). These normally occur before registration, when a process is "leaned" and key requirements are deleted accidentally, or when the requirements change (e.g., new revision of ISO standard or customer contract).
IMPLEMENTATION is when our defined processes (written or unwritten) are not being done. These are commonly found yet auditors often do not investigate the impact on the customer or operations. Without understanding or communicating the resulting impact, these nonconformities are not respected. They can also look like paper clips in a dragon-fighting world.
EFFECTIVENESS is when we are faithfully following our defined processes but they do not work to deliver intended results. These nonconformities can identify valuable feedback and provide meaningful insights for operations and risk management. Unfortunately, many audit programs stop looking once implementation is verified. I invite you to look at any nonconformities from your organization and decide what type they are and whether or not impacts are evident. I would love to hear your results!
So.....why are we talking about this? Deciding whether you want to distinguish the types of nonconformities is a decision you should make. Does the distinction help separate out smaller issues from larger ones? Do they enlighten or confuse?
Problem 5. The bar or yardstick used by auditors can be highly variable and it is rare that the level is declared in the audit report. As a leader, if the report shows no nonconformities and the bar was low, the audit results are not meaningful feedback to guide improvement efforts. This approach also feeds a culture of disengagement that the management system and registration efforts are wasted efforts.
In contrast, if the mission, vision, and values inform the "bar," then this provides empowerment to the audit program to hold the organization to those standards. This leads to more meaningful and candid feedback from the audit program.
Another perspective is either replacing or supplementing the right/wrong thinking of conformity/nonconformity decisions with maturity based evaluation of the processes and systems. In this example, the ratings or scores are given which allows for seeing improvement (or slippage) visually.
The example chart below reflects a fully integrated audit program where the yellow reflects what a third party registrar might accept as conformance. This yellow leaves you with a bit of unease, right? That is intentional. To move to green in this scenario, the process must be shown to be effective and fully implemented. That includes feedback loops to maintain the controls to manage risk.
It is uncommon for leadership to receive an integrated and holistic chart from the variety of audits performed. Customizing reports for organizations is one of my favorite consulting projects! In contrast, it is common for leadership to get partial reports and perhaps conflicting reports from the variety of audits performed.
In summary, as you are trying to revitalize your audit program, we recommend going back to step one in the process shown below. How good do we want to be? What is the commitment level of the leadership? That answer will guide you to decide which of the above options will appeal to the leadership team now and perhaps identify next steps to improve their engagement in the management systems.
Lastly, review your current format of audit reporting and decide what next steps are appropriate for your organization. Another of our favorite consulting tasks is to audit your internal audit program. We identify strengths and key components to improve to achieve the results your organization seeks.
Contact us to explore how we can help you.
Sign up to get occasional emails from us that are relevant to your field and the standards of interest to you.
By providing your info, you will be signed up to receive helpful resources. We NEVER sell any information collected on our site. Our goal is to help people like you thrive. You may unsubscribe at any time.