100% File Checking vs Sample File Checking: Which Is Better for Advice Firms?

The regulator’s question has not changed

The IFA industry is built on compliance. 95% of the industry are doing the right thing and so compliance is an industry built on the back of the 5%. If we look back in time, we can see that scandals have abounded, from pension transfers in the 1990s, to Equitable Life, Endowments, PPI, and car finance.

Regulators have been asking the industry to clean up since 1988, and there are signs that they are getting fed up with waiting for improvements, and the industry is finally taking seriously their obligations. The consistent question the regulator asks every firm is this:

The stock response is through monitoring KPIs, sampling of file checks and trawling through the complaints, if any.

A changing FOS landscape

Arguably the biggest change in our colourful history is coming right now. It has barely been talked about but goes to the heart of many problems in the industry. The FOS is being neutered. This perhaps will stop the dual regulation merry-go-round and may even simplify Reason Why Letters, in the way that the FCA has long been calling for.

In March 2026, the Government announced reforms to the Financial Ombudsman Service , stating that the changes are intended to improve consistency and alignment with the FCA.

IFAs have rejected this drive for simplification, pointing to the list of damaging retrospective FOS rulings that have been churned out by FOS. Our sister company IFAC have tried challenging adjudicators who submit rulings on individual cases, even one containing blatant mathematical errors. Mostly this is to no avail. The maths on that one case, according to FOS, stands, whether incorrect or not.

We should now welcome the Financial Services and Markets Bill that reached the House of Lords on 19th May 2026, and as drafted will make the FOS answerable in part to the FCA. There are changes ahead.

The Bill is listed by Parliament as the Financial Services and Markets Bill [HL] , introduced in the House of Lords on 19 May 2026.

What changes for IFAs?

The panel of ombudsmen is abolished. One named “Financial Ombudsman” is appointed, approved by the Treasury and a new two-part complaints test is introduced. Statute now says “consumers must take responsibility for their own financial decisions.”

Why it matters

FOS can only uphold a complaint if the firm breached an FCA rule and the act was not fair and reasonable. Consumer responsibility is a mandatory consideration. FOS must also refer systemic issues to the FCA. More formal FCA and FOS coordination requirements will also follow, as reported by the FT on 16th March.

For IFAs, this means that FCA rules and principles will dominate, and while it may be a relief to see FOS reduced in stature from being the alternative regulator, it will also strengthen the hand of the FCA, and give them power to see the outputs your firm is making.

FCA will now take a closer read of your suitability reports given by FOS, and you would hope make further enquiries on cases that concern them. Is the problem systemic? We assume they will ask for more files of the same type. Indeed, the FOS is also now to be tasked with writing thematic reports on recent cases.

AI is the solution, and BAT has been developed to solve this problem.

AI-assisted advice creates a new oversight problem

Increasingly, AI assistants are becoming embedded into the front end of advice processes. Meeting discussions are being transcribed automatically. Fact finds are being built from conversational AI. Client summaries are generated instantly. Suitability letters are increasingly supported by AI reasoning and automation, and mostly these tools are improving efficiency significantly.

But are the results any better than before? If you do not check 90% of your work, how do you know?

AI-generated workflows can occasionally introduce subtle inconsistencies, assumptions, omissions, or contextual misunderstandings that may not be immediately visible at adviser level. This includes:

This is particularly important because the FCA’s guidance on the fair treatment of vulnerable customers expects firms to understand the needs of customers in vulnerable circumstances and make sure they are treated fairly.

Firms adopting AI at the front end of the advice journey must also bring in oversight at the back end.

When it comes to a relaxed conversation with a trusted financial adviser, and extracting key insights using automation, there are bound to be problems. AI-assisted advice creation may become more common, and may increase in size and speed and go international very fast.

Compliance directors, or SMF16s as we know them, at regulated firms know that the technology and its outputs require oversight and verification. Reviewing 100% of cases will become increasingly important.

Historically, many firms manually reviewed only a small percentage of cases because full oversight was commercially difficult to achieve, but if AI-assisted advice creation becomes the norm, sampling only 10% of cases becomes increasingly difficult to justify to external oversight, be it from regulators, prospective shareholders, suppliers or any other organisation conducting due diligence on your firm.

There is only one path, and that is to bring in AI-assisted oversight tools to support:

The easiest of these is data completeness checking. A very simple upload will enable firms to know if their work, on upload, is “complete”. What is not to like?

This is not so much to replace experienced file checkers, as to augment them and remove the lengthy and tiresome manual process of bringing the files onto a screen to use the eye to scan them. This will make the file checkers more efficient, and match what the advisers are doing at the front end.

The future compliance model is unlikely to be AI advisers replacing humans. It is AI-assisted advisers supported by AI-assisted compliance teams. This combination is where scalable oversight becomes achievable.

Ultimately, the firms that successfully combine experienced people with AI-enabled operational controls are likely to be the firms that scale most effectively over the next decade, because as AI adoption increases across the advice journey, different risks are emerging, and central to that is scale, speed and efficiency.

AI is both the answer and the problem, as I will show in this article. Or, rather, it is not the machine that is the problem, but the oversight and supervision of the machine.

Consolidation in the IFA world

Consolidation has been a key feature of the IFA world since the momentous changes wrought by RDR came into play. What was once a highly fragmented market of small independent firms has evolved into a mix of national consolidators, mid-sized regional players and local boutiques.

Smaller firms were and are facing bureaucratic compliance. Completing returns for the FCA is time consuming, and as the older ones retire the consolidators seem to win the succession parade, with deep pockets and smooth procedures.

Younger ones find it harder and harder to enter financial services. So they join a network and stay within the warm embrace of a national firm. Perhaps lacking the traditional sales skill taught by the life offices in the 1990s, these new entrants then rely on purchased clients, another reason to join larger firms and build a career inside their rails.

But M&A is not just about the small firms, because the bigger ones are also selling out to other big ones, often backed by private equity. The current craze for M&A is defined less by who is behind the buying but what is more important is what procedures and operations the senior managers run inside these larger operations.

That shift in focus that has triggered compliance concerns comes out of the FCA’s work on consolidation in the financial advice and wealth management sector. The FCA’s October 2024 portfolio letter for financial advisers and investment intermediaries set out its expectations for firms, and the FCA later published its multi-firm review of consolidation in financial advice and wealth management in October 2025.

The FCA said it had seen a significant increase in acquisitions of advice firms and adviser books over the previous two years and announced a multi-firm review into consolidation. The FCA stressed that firms involved in acquisitions must:

The letter also made clear that Consumer Duty had moved from implementation into active supervision and testing. No surprise really, but it is worth revisiting.

The FCA said firms must be able to evidence:

The regulator also stressed that it planned to be:

The 100% check cost problem

Remember the regulator’s question:

A network of IFAs deployed AI across its adviser base and customer service operations. When asked how it monitors outcomes, the firm pointed to its QA programme: a team of skilled compliance staff reviewing a selection of files, just ten per cent.

Can you sample just 10% of advice cases, and expect the FCA to accept this? This represents a typical quality assurance programme signed off by the directors. Sure, it is a good indicator, but is it sufficient, given the new focus outlined in FCA communications?

The remaining 90% are, of course, never reviewed.

The firm’s board report said no significant harm had been identified. But reporting no harm is not the same as proving none occurred. It means the reporting fell short. As one wise owl put it, “The absence of evidence is evidence of absence.” Put more bluntly, what the firm call compliance, the FCA call a gap.

Most advisers do at least two pieces of new business each week. So, you can now picture a large regional firm, with, say, just 20 advisers submitting 2,000 cases per annum.

If these are investment advisers, more than £50m of new client money will be coming through the firm’s agencies over the year, plus the existing FUM of perhaps £15m per adviser on average, a total of £300m.

Put bluntly, it is an operation with big responsibilities, and deserves strong structures of oversight.

When a conduct risk auditor puts the case like this, the firm will have little to show for it if signing off just 10% of cases.

In defence of the firm, because this is a typical structure today, checking four cases each week manually. A single file checked manually takes two hours, at least a day a week for a skilled and qualified member of staff spent doing nothing but checking the files. The firm would typically support a full-time compliance officer, a sizeable expense of perhaps £60k per annum salary.

From a compliance perspective, the impact is limited. But what else to do, pre-AI, to answer the question?

What if the issues are not evenly distributed?

QA Managers also have to field the same uncomfortable question once they sit with these numbers and percentages. What if the issues are not evenly distributed?

It would be little surprise if the interactions that were most likely to go wrong are also the ones least likely to appear in a random sample.

One firm BATSOFT came across had a notable difficulty with advisers learning to avoid the compliance radar, and gaming the system by putting cases through with slightly misleading titles.

If submitting a case for a pension drawdown, it can be covered off as a straightforward pension sale on the new business register, and the hidden dangers lurking in the sub-standard suitability report may remain undetected.

Equally, if a vulnerable flag means a case gets held up by QA file checkers, then the advisers are incentivised to conveniently mis-categorise the vulnerable customer.

The FCA has repeatedly highlighted the need for firms to monitor whether customers in vulnerable circumstances are receiving outcomes as good as those for other customers. Its review of firms’ treatment of customers in vulnerable circumstances reinforces why vulnerability assessment and outcome monitoring remain central compliance issues.

As AI agents begin providing guidance directly to customers, this risk increases. An AI agent can, in theory, dispense guidance around the clock, across thousands of conversations a day. Networks understand this, which is why they monitor thousands of files, but still they only capture a fraction of what is submitted.

Consumer Duty and IFAs

Consumer Duty sets a standard that means firms have to act to deliver good outcomes for retail customers, and they have to show the FCA how they are doing so.

Four outcomes cover products and services, price and value, consumer understanding, and consumer support. But the word that carries the most weight is the “evidence” feature.

The FCA cannot be expected to take your assertions. Regulators demand evidence and data. Specific data, tied to specific outcomes, showing what happened across all of the customer base and what the firm does when things fell short.

The FCA examined 180 firms in its review of the first annual Consumer Duty board reports . The FCA said firms must prepare a report for their governing body setting out the results of their monitoring of consumer outcomes and any actions required as a result of that monitoring.

Summary statistics were used. These built a picture of incomplete monitoring. A clear fail. The regulator makes the point repeatedly in its writings, statements, speeches and reports. Completing a process is one thing, but evidencing the outcome is another.

The FCA has continued to emphasise that Consumer Duty board reports should explain what outcome monitoring has found and what action the firm will take as a result.

When the FCA asks whether good outcomes are being delivered consistently across all customers, a sampling-based firm has only one realistic answer: a shrug of the shoulders, palms up. We know about the ones we checked.

Sampling has been the foundation of QA in financial services for decades, so this article has not been written lightly. What worked reasonably well in the past is no longer going to work, because the tools are now available to bridge the gap, and move firms from the basics of process compliance towards outcomes evidence.

The larger firms are also fed up with getting fines, and have called time on the world of yesterday. Consumer Duty is just the name given to this change.

Coverage, timing and consistency

The coverage problem is one big problem for the industry. Some 90% of all adviser files never get checked. Sampling does not cut the mustard.

It was designed for a world where failure is easy to control through process. Get the procedure right and you cannot get criticised by the regulator, they said.

But Consumer Duty does not let you off the hook by relying on process. Minor advice errors can blow up years later, and you owe it to your customers to have the best standards, and you get that by checking all files.

The timing problem

But there is also a timing problem. Sample-based reviews look backwards. Reading files after the event is in many cases too late to change the outcome.

While addendum letters can be written to the client, and contracts unwound, or compensation paid, this is incredibly rare, and arguably not an insurable event either.

All the incentive and energy is to make the case fit, or put the solution down to remedial training for the adviser, and leave the customer unaware of the slightly wrong turn.

The only way around this is pre-sale file checks, something that the FCA has been looking at and asking for since modern day regulation began in 1988.

But pre-sale checks are expensive to implement and the most experienced file checker is turned into a paraplanner, as individual items are omitted from the file, that the checker picks up. Total waste of time correcting shoddy work.

Most larger firms use a targeted pre-sale sign-off department, but the smaller firms rely on unqualified staff or paraplanners to spot missing documents, and comment verbally to the compliance director, who in turn scans the new business book for likely higher risk cases.

The consistency problem

The consistency problem is obvious with subjective post-sale file reviews.

One habit we have observed is for compliance staff to focus on particular advisers and product lines, as repeatedly checking these can make the work easier.

Advisers leave their own stamp on files. Weaknesses and strengths, good and bad repeat, and just following one adviser or firm makes the review task easier.

File checkers come to prefer some adviser files to others, and naturally lean that way. But of course they then miss other files, and other advisers when finding their magic percentage reviewed targets.

Random samples fail to pick up patterns by themselves, rather they can now be used to pick up bias in the file checker themselves.

File checkers reviewing the same interaction will often score it differently. That is a known issue across the industry.

If you send a sample of, say, 20 pension case files to the two best compliance specialists, you will nonetheless get different results and different scores, even when using the same file check forms.

Admittedly a pattern of sorts usually emerges, generally poor or generally good, but still, these discrepancies in the file checker procedures make it harder to draw reliable conclusions, and undermine the whole process. It is a subjective observation.

Solutions

Coverage, timing and consistency are all easily solved by AI.

By refining basic AI foundation models that are offered by the big tech giants, a set of pre-defined questions can be answered which together provide a series of answers, and generative AI can produce feedback on the same.

This can be done in real time, at adviser level, with results determining whether the file can go to the next stage.

The process can be technology neutral, so work across different systems and data sets, be it Word, PDF, Excel or JPEG scan document.

BAT AI Filechecker answers

BAT AI Filechecker examines all the documents and files submitted.

1. Document categorisation

Firstly, BAT will categorise the documents into Fact Find, Risk, Budget, Quotes, Research, Suitability Report and file notes. That alone is a time saver. Readers of the file, take note.

2. Completeness checking

Secondly, BAT will conduct a completeness check. It will extract the data, compare it to what inputs the compliance director has demanded, that is, the system is fully configurable, and show up what data is missing under the relevant section.

So under Fact Find there are a number of key requirements, all pretty obvious, such as date of birth, address, marriage status and so on. If any are missing, the file cannot proceed to the next step to be checked for suitability.

This file with missing data is already marked as unclear. The adviser has a steer to get the file fixed, and the file checker knows not to bother with that one.

The accuracy of this stage in BAT AI Filechecker is effectively 100%. It almost never makes a mistake. Astonishing.

3. Predicted grading

BAT will therefore give a predicted grade at this stage, based on Suitable, Unsuitable or Unclear.

The prediction is not a suitability judgement. After all, the most appalling file can contain good advice. But it is an indicator and about 90% accurate, in our experience.

4. Suitability assessment support

The actual suitability assessment is the next stage: stage three. This third stage is where BAT will provide a narrative of observations on the file, the so-called “suitability check”.

The narrative against all the pre-set, configurable and editable questions acts as a prompt for the file checker, who can amend or leave as they see fit.

As a general rule, the more the feedback, the more likely the file is to fail as unsuitable. The system will provide an estimated result.

At this stage the file checker clicks in and will amend or sign off the file check. All changes made are documented, and auditable, and the results show an extraordinary level of accuracy.

5. Trialling, reporting and accuracy evidence

BAT trials offer the users the opportunity to see the feedback and results on all files, and track how many times any manual overseer actually changes or disagrees with the end result.

BAT have proved to themselves that this is more accurate than individual file checkers for all but the most complex work, and external auditors are validating this as I write.

BAT has now built the reporting tool, to show you how accurate the AI tool may be for you, and will also give you a platform to report any issues to our lead developer.

You will see how many times a manual file checker has changed the text that BAT AI has input into the files, explaining the reason for the failure.

No changes equals 100% accurate. Thereafter, on a scale depending on how many corrections you are asked to make.

Move beyond sample-based file checking

BAT AI Filechecker helps advice firms review more files, identify missing evidence earlier and give compliance teams a clearer audit trail across the advice process.

Book a demo