Call back, known number
Any voice request that moves money gets a callback on a number you already had on file. Never the number the caller offers, never a reply to the thread.
Fraud no longer needs a forger
Deepfakes, voice clones and the new invoice fraud.
Payment fraud is one of the oldest entries on any risk register. What changed is the cost of running it convincingly, which has collapsed. This page covers the family: the cloned voice, the perfect invoice, and the safeguards that still hold.
TL;DR
- Fraud is not a new risk. AI collapsed the skill and cost of running it convincingly: voices can be cloned, video can be faked, and the sloppy tells people were trained to spot are gone.
- The targets are payments and credentials: a supplier's changed bank details, an urgent transfer, an executive's voice asking for an exception.
- Detection is the wrong bet. The safeguards that still hold are process: call back on a number you already had, verify bank changes out of band, and keep dual approval with no urgency exceptions.
- On your register this is an old entry with a new likelihood, not a new row. Re-score it; do not duplicate it.
- Deepfake fraud, voice cloning scams, invoice fraud and business email compromise are one family: impersonation aimed at your payments.
What actually changed?
Deepfake fraud is impersonation fraud that uses AI-generated media, a cloned voice, a synthetic video call or a fabricated document, to pose convincingly as a real person and move money or extract information. The crime is old. What changed is that the forgery is now free.
Impersonation fraud used to be limited by skill. A convincing forged invoice, a plausible voice on the phone, an email that read like your CEO wrote it: each took effort, and the effort showed. Most attempts carried tells, and staff were trained to spot exactly those tells: the odd phrasing, the wrong tone, the request that did not sound like the person.
Generative AI removed the effort without changing the crime. A voice can now be cloned from a small amount of recorded speech, the kind that exists publicly for almost anyone who has spoken on a webinar or a conference panel. Written impersonation now reads fluently in the style of the person it imitates. Video calls can be faked.
That is why this page sits in our series on the real risks of AI for a business: this is not a new risk, but an old one whose likelihood moved while its register entry stood still. That pattern has a name: AI disruption.
The call
Voice cloning scams: the call that sounds right
A voice cloning scam is simple: a call, a familiar voice, an urgent and slightly irregular request. The finance lead hears their CEO asking for a transfer before a deal closes. The bookkeeper hears a supplier chasing a payment to a new account. The voice is right, the context is plausible, and the urgency is the point, because urgency is what switches people from verifying to helping.
The uncomfortable part: recognising the voice was never a safeguard, it just felt like one. Familiarity was doing the work that verification should have been doing. AI did not break your process; it exposed that the process was resting on a human tell that can now be manufactured.
The same applies one step up: faked video calls exist, and executives are the easiest people to clone because they are the most recorded people in the business.
The email
Invoice fraud and BEC: the email that looks right
Invoice fraud and business email compromise are the volume end of the same family. The classic shapes: a real supplier's thread is hijacked and a payment is redirected to new bank details; a convincing invoice arrives for services close to ones you actually buy; someone senior appears to email accounts asking for a quick payment while they are travelling.
What AI changed is polish and scale. The giveaway grammar is gone, the tone matches the sender, and the follow-up nudge arrives at a believable interval. The Australian Cyber Security Centre's guidance on business email compromise is a good plain description of the shape, with steps for reporting and recovery.
One boundary worth naming: the phishing email itself, as a credential-stealing device, has its own page in this series. This page is about the money.
The safeguards that still hold
Bank changes, out of band
A change of supplier bank details is verified through a second channel you initiate, every time, no matter how routine the supplier.
Dual approval, no exceptions
Payments over a threshold need two people. The scam depends on urgency creating an exception, so the rule only works if urgency never does.
Slow is a feature
A genuine counterparty survives a one-hour delay. Treat manufactured urgency itself as the strongest signal you have.
On the register
An old entry with a new likelihood
The mistake is adding a new row called deepfake risk. You almost certainly already carry this risk under payment fraud or supplier impersonation, and what has changed is its likelihood and the reliability of the safeguards behind it, not its identity.
So the work is a re-score, not a new entry: the AI driver is named against the existing risk, the likelihood moves up, and the safeguard list is checked against the four above. If your current safeguard amounts to staff will recognise something odd, it no longer holds, and the entry should say so. The shape of a complete entry is on the register page, and the assessment page covers the re-scoring exercise itself.
Deepfake and invoice fraud FAQs
What is deepfake fraud?
Fraud that uses AI-generated media, a cloned voice, a synthetic video call or a fabricated image, to impersonate a real person convincingly enough to move money or extract information. The crime is ordinary impersonation fraud; the deepfake is the tool that removed the skill barrier.
How do voice cloning scams work?
An attacker builds a copy of a real person's voice from recorded speech, then calls a target with an urgent, plausible request, typically a payment or a change of bank details. The voice buys trust; the urgency discourages verification. The defence is a callback on a number you already had.
What is the difference between invoice fraud and business email compromise?
They overlap. Business email compromise is the method: control or convincing imitation of a business email account. Invoice fraud is a common goal: a payment redirected through a fake or altered invoice. A single scam often involves both.
Can software detect deepfakes reliably?
Not reliably enough to build your defence on. Detection is an arms race, and treating it as the safeguard recreates the original mistake of trusting your ability to spot fakes. Process safeguards, callbacks, out-of-band checks and dual approval, work whether or not the fake is detectable.
What should we do if we have been caught by one?
Move fast: contact your bank to attempt recall, report it, and preserve the messages. In Australia the ACSC's ReportCyber covers business email compromise with recovery steps; in the UK, the NCSC's guidance on phishing and scams covers reporting routes. Then re-score the register entry while the details are fresh.
Does this risk apply to small businesses?
Disproportionately. Smaller businesses often concentrate payment authority in one or two people and rely on personal familiarity in place of process, which is precisely the safeguard this class of fraud has defeated.
See which of the risks on your register AI is driving hardest, scored for your business, with a safeguard drafted for each. Free.
This is one risk. See the rest of yours.
AI risk series
Explore related topics
The real risks of AI for a business
The series hub: what counts as an AI risk, the four families, and where to start.
How to run an AI risk assessment
Three ways to do it, compared, and the five steps.
The AI risk register
What every entry carries, with worked examples.
The AI policy your business actually needs
The full template, free on the page, and how to make it yours.
What is AI disruption?
A plain definition: the change is in your risks and plans, not just your tools.
AI in risk management: what it can genuinely do
The four jobs AI does well, and the three things it must never own.
Guardrails, safeguards, controls: what AI actually needs
Three words untangled, and the four families of AI-era safeguards.
Prompt injection: the attack your register hasn't heard of
Instructions hidden in ordinary content, and the safeguards that limit the damage.
See your own AI risk picture
The risks AI is driving against your objectives, scored for your business.
