How Compliance Managers Can Reduce Manual File Review Work

Cristopher Wren is alleged to have described St Pauls Cathedral as an “Awful, pompous and artificial” sight. He meant, in the language of the 1600's, that it was awe-inspiring, magnificent and ingeniously made. The same applies to BAT AI-Filechecker. It is changing the back office and impacting daily lives in a way never before seen.

Why manual file reviews take so long

Why manual file reviews take so long is a source of vexation to all directors looking to improve efficiencies in their businesses.

The manual processes includes

Information may appear in:

In this article, we at BATSOFTWARE seek to show the reader how much of this can be eliminated. While some file checkers even recruit junior staff to carry out this activity, the new world will increasingly rely on AI tools to do this for them. BAT AI Filechecker strips out the information, pumps it into an AWS CLAUDE AGENTIC SDK, which is the foundational AI model used that provides the engine to check the documents.

The hidden cost of repetitive checking

BAT has a sister compliance operation that has been instrumental at helping develop its AI black box capability around file checking.

Put simply, the firm has the expertise to train the agents to check the files in a way that is compliant. This isn't just a question of meeting the FCA rules. As everyone in practice knows, the rules are just the starting point. The industry is actually governed by a set of principles and practices that create a standard expectation. The expectation is a long distance away from what the rule book may or may not say. As has often been said, the term "FACT FIND" does not exist in the FCA rulebook. COBS 9.2 requires firms to gather information about the client’s financial situation, investment objectives, knowledge and experience, risk profile, income, assets and commitments. But that's it. What exactly good looks like, is only found by experience, both in wider reviews, in S166 processes and in being subjected to FCA scrutiny. While the FCA will not define what good looks like, they certainly will be quick to spot what they don't like!

All this leads to experienced staff wasting valuable time finding simple documents and repeating that exercise again and again.

Generally, the hidden cost of repetitive checking includes

Because at the end of the day, file and the file check is not what actually happened, but only a record of what actually happened.

Separate evidence gathering from compliance judgement

Separate evidence gathering from compliance judgement

The biggest problem for compliance managers is with file checking when they are spending time trying to find documents. File checkers must be qualified, experienced and practicing. This means, for IFA work, that they are expensive, and working at level 4 for advice.

Using highly skilled staff spending time gathering information is not a cost-effective solution. But until the advent of AI, there was little alternative. Some checkers use semi-trained staff to categorize documents, perhaps write a summary and put yellow stickers into areas of the file that need to be looked at. But even this activity is rendered redundant by AI. AI can now strip out the data that may belong in 30 or 40 different submitted files, it can cross reference for accuracy or contradictions, and then deep dive and pass judgement on the quality.

Apply a risk-based review process

Apply a risk-based review process

It is such a simple theory - risk based compliance. Sounds so easy, and yet hides so much. The bigger the button, the more you press down on it. But that is no way to run a business, and just because it is a growth area - say ISAs in their early days some 25 years ago, when they were all the rage, does not mean that they present a risk in themselves. No the risks can lurk anywhere, and the key to reducing real risk, is in the identification of the risk. Once you have identified it and isolated it, then you can start to work on reducing it. Just thinking about it, reduces the risk. It is one area that both individuals and machines excel at, but on their own have unique limitations. Put them together and you have a dream team for reducing risk.

AI and structured automation together with human oversight will nearly always provide the greatest efficiency.

The AI can easily strip down the submitted files, that may amount to over fifty documents for a simple mortgage case.

The AI does not get tired, and simply strips out the data, and presents it to the human compliance manager, who then sees the data in a consistent format. Data discovery is repetitive, boring and can easily be automated. Risk ratings, references to vulnerability, charges, objectives all can be summarised at pace by the AI.

The missing data can then be displayed, or requested. It is then up to the manager to spot the discrepancies. Perhaps the individual has a particular business plan that allows for missing data? I mean missing FF may be fundamental, but not if there is a duplicate elsewhere that the machine has not picked up.

In more detail then. The BAT AI File Checker can be positioned as helping the firm to:

But every advice firm, and indeed every adviser, has its own:

How AI can support file checking

Give compliance professionals more time to exercise judgement

Compliance teams will continue to need experienced people who can:

However, those people should not spend most of their time manually locating information that technology can identify and organise.

The firms that benefit most from AI will not simply “automate compliance”. They will redesign their compliance processes so that:

The BAT AI File Checker helps compliance teams analyse advice files against a structured checking framework, identify relevant evidence and highlight areas requiring attention. When combined with BAT’s wider compliance monitoring, management information, procedures and Training and Competence tools, file checking becomes part of a connected compliance process rather than an isolated administrative task.

Why a generic AI checklist is not enough

One thing has struck me since building the BAT AI filechecker. There is a fairly well accepted list of filecheck points, used by our sister firm IFAC to check about a thousand files per annum, and itself adapted from various networks, independent firms, and from FCA own templates over years of experience. It is, in my opinion, the default file check form. But contact with the market taught us all a lesson. Firms use their own procedures, and from their own procedures comes their own checks against those same procedures. Forms proliferate, and guides, red lines and remedials differ. In 35 years in financial services, in several different roles, from adviser (fifteen years) to manager, compliance director, CEO and Chairman, I am repeatedly astonished to find so much that I do not understand. Nothing is generic in this game. The single biggest grouse continues to be the same today as when it all began for me in 1992 - the Fact Find. Never in history has a blank form created so much angst among advisers!

Generic simply will not do. When building our own BAT application, everything was built to be configurable. That too comes with drawbacks, not least that things take longer to develop, as the range of responses can be so much greater. But configurable it all is, and for those that want their own FF, then they either pass it to the team to input, or code it in themselves. It is all written in the latin alphabet, so the term code is a bit rich - any old fool can do it.

Connect file reviews with your wider compliance framework

A routine compliance audit for IFAs and mortgage advisers, consists normally of two parts. One is field based (remote nowadays) and involves a series of questions and answers, document discovery, process checking and finding out if the answers match the actual documented procedures. Does the new business book reveal a firm selling one product? or is it the proverbial monkeys throwing darts at a dart board for investment choices?

But almost as important is the final piece in the advice jigsaw - namely the advice file. Connecting these up with the compliance framework is essential to pass any due diligence audit, be it for business sale, to satisfy counter-party caution, or to respond to a complaint, or for the Financial Conduct Authority. The file check is key. Don't confuse file checks with simple document reviews made by admin staff. The file check should be independent, validated externally and carried out by qualified, practicing and experienced staff.

File reviews present a fascinating insight into the workings a the individual selling the product. It has often been noted that even a tiny sample, say of three files, shows a pattern emerging. The pattern tends to follow broad strokes of "fact finding sloppy" or "Hopeless reason why letters, but good advice" or some such, so a lot more work needs to be done to strip out the data on a case by case basis, and using some sort of system to examine files helps. More on the systems later.

But what action to take in the wider compliance framework for an adviser failing? It is obvious really. Training, remedial work, a set pattern of work to improve, perhaps by withholding commission earnings, or by incentivising on a presale versus post sale hook. Equally the file check ratios should always be increased if the adviser persistently fails. In between is the carrot and stick and keeping and documenting the results and the checks and the proscribed actions is key.

To summarise. The aim is to create a compliance map.

Compliance managers can also generate reports to feed into the compliance map.

A file check is only the beginning. Its findings should influence:

And managers can also turn checking results into management information

Turn checking results into management information

Manual checking has many drawbacks, but none is so consistent a thorn as MI reporting. Small firms of one or two advisers might consider here that they may cobble together results according to the demands at hand, this is a whole lot more complex for multi-RI firms.

Just look at what happened in FCA v Paul Reynolds. Formerly a director and adviser at a sole practitioner firm he was later banned and fined. After a notice of a routine supervisor visit from the regulator, he duly delved into his filing cabinet and pulled out his client files for a tidying up exercise. This then sprawled over into making amendments to the same files, and finally re-creating fact finds that were felt to be deficient.

But it came out later that he had added documents to the files only when he was aware that the FCA was coming to do an audit. What the lawyers call "retrospective re-papering” is a serious integrity issue.

And if you think that is bad, then the business of MI gets a whole lot more complex for multi-RI firms. If you are checking client files, then each fail leaves an imprint, and from the imprint you get a pattern. On our own system BAT, we have 95 reasons for a fail on each investment case alone. 95 on mortgages, 45 on pensions and 37 on protection! These can range from "provider research missing" to "wills and Power of attorney (missing)," and to "Vulnerable client", "Soft facts" and so on. Nearly 300 different reasons across all the product types.

What is extraordinary, is that if you pump in these reasons for fail onto a system, a pattern emerges, and from that pattern, adviser action can be implemented, and training prescribed. Spreadsheets cannot handle this depth of information.

How to introduce AI-assisted file checking safely

There is a craze for vibe coding. Ask the AI to create something, and it sits on your local server working away. You can even implement some sort of simple checker. It may be full of bugs, inconsistent and accessible only by you, but it can work in a fashion. But the world of IT changes dramatically with secure data, multiple log ins and regulation.

The key difficulties that BAT have encountered is around servers and databases, presentation of information, configurability, and tracking the results.

Firstly, to the infrastructure. The AI normally runs on a server, while the underlying database stores and controls the information the AI is permitted to use.

The server provides the computing grunt to runs the software. For the BAT AI File Checker, the server receives the documents submitted for checking and extracts the relevant text from all of the files (many times over 50 documents are submitted for a single file check).

The server powers the AI model and sends controlled instructions to the AI model, meaning the underlying client file-checking questions and rules. It compares the information found in different documents, summarises them and cross references them and generates findings. The server performs this work. In the BAT-AI file checker case it is a cloud-computing environment, sitting on an AWS server.

The database

The database stores the structured and extracted information in an organised form. For BAT, and the standard IFAs and mortgage brokers who use the system, this includes :

The database remembers and organises the information for the AI to work it's magic, and for it to be presented to the user in a structured way.

The AI application will ask the database for information when it needs it. The database returns only the data that is requested, and only to that particular user who is authorised to access. Don't forget, different users see different things! So the file checker view and the adviser or client view is likely to be three different things.

The original documents may not be stored directly in the database.

Long documents, PDFs, jpeg or png images and suitability reports are often held in secure document or object storage, such as an AWS S3 bucket. The database then just holds the access to those documents - a sort of set of keys with the following structure: which customer they relate to, which firm owns them, who uploaded them and when, their document type and where they are securely stored.

While many advisers call their fact find files "FACT FIND JOHN SMITH," many more will name their fact find "JOHN SMITH," and so the fun begins - what would the illustration look like?

Key controls

Confidential client information should not be exposed to an uncontrolled public AI service. A professional compliance application requires secure computing infrastructure and protected document storage. The controlled database must have strict user permissions. The AI should analyse only the information required for the authorised task, while the surrounding system will control access, record the results and maintain the audit trail.

A simplified BAT file-checking process will follow this path

The AI must never be allowed unrestricted access to every document in BAT. The surrounding software controls which information is supplied to the AI for each particular task and instructs the AI to delete that information.

The server and database address different security risks. Server security points. The server must be protecting the confidential information. There are no secrets to how this is done.

Database security

The database needs protection because it contains valuable and potentially sensitive records. This for BAT is a full-time role for one person.

Controls BAT use include:

in this way both the processing and storage is protected.

One of the most important controls for a software application is tenant separation. Each advice firm is treated as a tenant, and one of the most important requirements is ensuring that one firm cannot access another firm’s data.

This BAT achieves through a combination of:

So, to summarize a technical area around computing security. The AI operates within a secure application environment. The server performs the analysis, the database controls and stores structured information, and secure document storage holds the underlying client files. Access controls determine which information may be provided to the AI for each review.

The BAT AI File Checker

The BAT AI filechecker uses the AWS CLAUDE AGENTIC SDK, which is a foundational model that provides the engine to check the documents. In simple language, BAT point your files to the engine and await the results.

The detail is astonishing. The model will create a plan, execute that plan, provide feedback, check the content and provide references for cross checking.

BAT initially trained the agent against a relatively small sample of files – so every file was checked, according to a set framework of questions provided by our compliance partners, and manually each line was accurately amended. This helped to train the model. The key here is that this is what is called A Reasoning Model. It creates an answer, and validates it, presenting you the compliance checker with the results.

As the operators expected, the results get better and better at a relatively fast pace. In order to prove this we then built an accuracy checker on the back, of BAT, so that we can view the outputs, and view the changes, if any, made by compliance checker intervention, and the all-important thumbs up, thumbs down procedures used by compliance file checking staff.

Our system is grounded in compliance expertise. IT is compliance experts across multiple firms that have designed the questions the AI agents use to examine the files. The questions are configurable and editable by firm, and so the system is rooted in your own knowledge and built around your work, your processes and your desired outcomes. Put bluntly you can set the bar where you like. One large firm user discovered that advisers were inputting their cases onto their new business register using dumbed down terms. So Defined Benefit transfers were slipping through as pension switches, and simple Life insurance cases were actually very detailed inheritance tax planning work in progress, exposing the firm to high level of unforeseen radiation from the fall out of disappointments. What solution could possibly be easier than using the AI to extract the key document-data and put it onto a summary sheet for a compliance manager to sign off. A thirty word summary is used in their case, alongside the approximate categorisation.

What is the feedback like?

You get speed at scale - a defensible and auditable trail of work, and some firms are putting through the AI filter, literally several hundred cases each month. The system picks up completeness, scores the files and gives a traffic light to the adviser / paraplanner on whether to submit. If anyone had foretold that this would be what BAT is doing within three years, I would not have believed them.