How to audit an Authorised Engineering Organisation?

The Authorised Engineering Organisation (AEO) authorisation process consists of four stages: initial engagement, qualification for assessment, assessment, and surveillance. The first and second stages being preparatory stages, and the latter two consisting of the audit portion of the process. For those interested in learning about the four-stage process, we have covered it, in details, in a previous article about Authorised Engineering Organisation (AEO) Process.

In this article, we will focus on the audits involved in the process of obtaining and maintaining AEO authorisation. Contrary to popular belief, the audits do not stop after authorisation is granted. In fact, the only way to foster perpetual continual improvement, is if they never stop. So, what are the different types of audits prospective and existing AEOs must undergo, and what do they entail? Stay tuned with us in Edarasystems.

Assessment Audit of AEO standard

The authorisation assessment stage consists of two parts (Stage 3A – Assessment and Stage 3B – Finalisation) and is where the auditor examines evidence to ensure that it satisfies AEO requirements in accordance with industry best practice and in alignment with the proposed scope of authorisation for set scope of works.

If the auditor is satisfied with their findings, they will make a recommendation for the prospective AEO to be granted authorisation, and attest to its ability to manage the risk associated with their specified scope of engineering services when conducting work on Transport for NSW (TfNSW) assets. After initial authorisation, the AEO is placed under surveillance to monitor its performance, action management and to validate evidence of the implemented systems and processes for the scope of authorisation approved.

The concept of surveillance audit in AEO standard auditing

ISO 19011, the international standard that establishes the guidelines for auditing management systems defines audits as ‘a systematic, independent and documented verification process of objectively obtaining and evaluating audit evidence to determine whether specified criteria are met’.

To effectively measure the level of compliance with authorisation requirements, the Asset Management Branch (AMB) of TfNSW utilises a risk-based approach during surveillance audits and evaluates evidence that demonstrates an AEO’s active systems deployment in a project environment or organisational setting.

When does the first surveillance audit take place?

An AEO can expect its first surveillance audit to occur approximately 12 months from the time authorisation is granted, providing the AEO is involved in a Transport project or is in the process of delivering asset maintenance services. The audit’s aim is to establish a performance benchmark based on the proposed audit scope, to be able to report on the level of compliance with AEO requirements at deployment level.

How often do surveillance audits happen?

All AEOs are put on a surveillance audit program that is updated routinely by the Manager Audit & Compliance. The AMB adopts a risk-based approach with AEOs following the first surveillance audit to ensure:

• ongoing surveillance activities are planned around Transport projects presenting higher risk, and in consultation with interested parties
• The audit scope includes systems weaknesses identified in past audits

There are various risk considerations that condition the frequency of ongoing surveillance, including:

• initial authorisation assessment findings and maturity levels
• outstanding non-conformances
• the AEO’s scope of engineering services
• the Transport contracts the AEO is involved in
• the safety risks associated with the AEO services involved

Ongoing surveillance results are the only way for TfNSW to determine, with justified confidence, that AEOs are effectively employing approved systems. They also set the tone, in terms of frequency, complexity and emphasis areas for future surveillance audits.

How are AEOs notified of a surveillance audit?

AEOs are notified of a potential surveillance audit via a pre-audit questionnaire sent to them by the Manager Audit & Compliance who will use the information provided in the questionnaire to make a clear determination on whether the audit should take place, or be adjourned. In the event where the Manager Audit & Compliance decides to move forward with the audit, the AEO will be officially notified via email 3 months from the proposed audit date.

Because the AMB understands that this process may interfere with the AEO’s business activities, they always provide advance notice and consult heavily with the AEO throughout the planning phase, to ensure efficient and effective arrangements are made to facilitate the surveillance audit process, and lower the impact on business operations.

Special Audits of AEO standard

Special audits are audits that are carried out beyond the traditional surveillance audit program. They may be instigated suddenly if a risk is identified and require a prompt investigative response to be addressed. A special audit may be initiated due to a systemic issue or a serious incident, at which point the AMB will review the information provided and confer with AEO and Transport interested parties, to make the determination on whether or not to move forward with the special audit. If they do proceed with the audit, a systems auditor will be assigned the function of Audit Team Leader and necessary third parties will be contracted to join the audit team. The audit framework, timeframes and any other special provisions will be clearly defined in the terms of reference.