When I started testing a voice agent, the first things I watched were predictable: latency, interruption handling, whether the agent stopped when I spoke over it, and whether the voice sounded natural.
After making a few real outbound calls, my attention moved somewhere else surprisingly quickly.
Are we allowed to call this number right now?
That question came earlier than I expected.
Has this customer already opted out? Is this a marketing call or a service call? Is the recipient’s local time inside the permitted window? What consent or other basis supports the contact? Can the call be recorded or transcribed?
If those questions are unresolved, sending a number to a scheduler and telling the agent in its prompt to “follow the rules” is not much of a production architecture.
Creating a phone call is already an external side effect.
In Part 1, I argued that as voice models become better at the conversation itself, the harder enterprise questions move outward: CRM context, telephony, durable state, and business authority. Part 2 goes one layer deeper.
The ability to hold a conversation is not the same as the authority to contact a customer.
And once the call is connected, another authority question appears:
May the agent turn what the customer says into a real business action?
Without separating those questions, a voice agent can move very quickly from “a model that speaks well” to “an execution actor with too much authority.”
What changes when the enterprise initiates the conversation
A text chatbot is usually opened by the user. It still needs privacy controls and tool authorization, of course.
Outbound voice adds something else.
The enterprise initiates a real-world contact event.
A PSTN or SIP call reaches into a person’s day. Live audio can naturally contain names, addresses, account details, order numbers, and other personal information. During the same conversation, a customer may ask to reschedule an appointment, cancel a service, change a delivery address, or perform another action that changes business state.
I find it useful to separate three authorities:
- Contact authority — may we contact this person now?
- Data authority — what information may this conversation read or disclose?
- Action authority — which business mutations may be executed?
They are not interchangeable.
Permission to place the call does not imply permission to disclose account information. A fluent conversation does not imply permission to change an address.
“May we call?” and “May we act?” are different gates
In practice, three gates make the architecture easier to reason about.
The first is Pre-dial Compliance Admission.
Conceptually, the system resolves questions like these before a physical call is created:
Target → Purpose → Jurisdiction → Consent Provenance → External DNC → Internal Suppression → Local Calling Window → Recording Policy → ELIGIBLE
The second is In-call Business Authorization.
Being allowed to call someone does not grant the agent unlimited authority over business systems:
Speech → Semantic Intent → Authentication Context → Record / Operation Scope → Policy / Approval → Mutation
The third is Post-call Durable Evidence.
If the outcome of the conversation affects future behavior, a transcript is not enough:
Outcome → Durable State → Receipt / Audit Evidence → Retry / Suppression / Follow-up Eligibility
The distinction I keep coming back to is simple:
An agent knowing a policy and a runtime enforcing that policy are two different things.
A model can understand the sentence “respect Do Not Call rules.” That does not prove the DNC check ran before scheduling, the consent evidence is still valid, the recipient is inside a permitted local-time window, or an internal suppression flag is absent.
Discovering after the call that the number should never have been dialed is too late.
I care more about the rule shape than memorizing every jurisdiction
This is not legal advice. A real deployment needs jurisdiction- and use-case-specific legal and compliance review.
From an architecture perspective, what interested me was that the exact numbers and exceptions vary by country, but call admission almost never collapses into one universal boolean.
Australia’s ACMA guidance combines the Do Not Call Register with permitted calling times and identification requirements. Its guidance updated in September 2026 says telemarketing calls are generally limited to 9 a.m.–8 p.m. on weekdays and 9 a.m.–5 p.m. on Saturdays, with no telemarketing calls on Sundays or national public holidays. ACMA’s 2026 enforcement list also includes a telemarketing matter involving calls to registered numbers without consent, prohibited calling times, and missing required information at the start of calls.
ACMA — Dealing with telemarketing
ACMA — Investigations into spam and telemarketing
The United States has a different rule shape. In 2024, the FCC clarified that AI-generated voices fall within the TCPA’s concept of an “artificial” voice. That makes it unsafe to assume that a real-time generated voice sits outside artificial/prerecorded-voice rules simply because no static recording is being played. The FTC’s Telemarketing Sales Rule also distinguishes the National Do Not Call Registry from the seller’s own entity-specific Do Not Call list.
FCC — AI-Generated Voices in Robocalls
FTC — Complying with the Telemarketing Sales Rule
The UK’s ICO guidance distinguishes live marketing calls from automated calls. For live calls, TPS/CTPS screening and prior objections remain important. The ICO also currently notes that parts of the guidance are under review following changes from the Data (Use and Access) Act. That is another reason not to encode “AI voice call” as one fixed legal category without retaining purpose and modality as policy inputs.
Korea has an official telephone-solicitation opt-out system administered by the Fair Trade Commission and operated by the Korea Consumer Agency. Its FAQ also highlights the importance of keeping evidence of consent when consent is disputed. But that system should not be treated as the universal rule for every financial, B2B, service, or automated call.
What I want in the runtime is therefore not a copy of each country’s legal wording inside a prompt.
It is a policy model that retains facts such as country + purpose + modality + consent provenance + suppression + local time as explicit inputs.
“Don’t call me again” is not a transcript
This is one of my favorite operational test cases for a voice system.
Imagine that a customer says:
“Don’t call me again.”
An LLM can understand that sentence very well. It can classify it as a do-not-call intent and preserve the text perfectly in the transcript.
If the workflow stops there, however, the business task is incomplete.
The same customer can appear in tomorrow’s campaign, the scheduler can pick up the number again, and the system can place another call.
The flow needs to look more like this:
Customer Speech → DNC Intent → Deterministic Suppression Tool → Canonical CRM / Suppression State → Write Receipt → Future Admission Block
The semantic task belongs naturally to the model: does this utterance mean that the customer no longer wants to be contacted?
Turning that judgment into durable suppression state, confirming the write, and requiring future call admission to read that state are deterministic runtime responsibilities.
Understanding the sentence is not the same as completing the business operation.
One CallAllowed flag is usually not enough
Early in a prototype, it is tempting to reduce the whole problem to something like CallAllowed = true/false.
The requirements start separating as soon as the use case becomes real:
- May we place the call?
- May we record it?
- Is a recording notice required?
- How long may the transcript be retained?
- May this conversation collect certain sensitive data?
- Is cross-border processing allowed for this workload?
CallAllowed = true does not imply RecordingAllowed = true.
Compressing different lifecycle decisions into one flag is convenient in a PoC, but it becomes painful as soon as another country, purpose, or data policy is introduced.
I would rather treat compliance as a set of distinct policy outcomes than as one universal “phone call permitted” bit.
Voice is a conversation channel, not proof of identity
The more natural the voice interaction becomes, the easier it is to blur the authentication boundary.
A caller gives a name. They know part of an order number or date of birth. The conversation feels coherent. It becomes tempting to think, “This is probably the account holder.”
Conversation fluency and identity assurance are different problems.
Current NIST SP 800-63B guidance limits biometric use in authentication and explicitly says that biometric comparison based on voice shall not be used.
NIST SP 800-63B — Authenticators
The enterprise rule I take from that is straightforward:
Voice is a conversation channel, not proof of identity.
A low-risk conversation may continue. A sensitive account lookup, delivery-address change, contract modification, or payment may still require an authenticated context or step-up authentication.
Tool authority should be narrower than conversational ability
A customer can say anything to the agent.
That does not mean the agent should be able to execute anything.
For backend tools, I prefer a boundary like this:
Voice Agent → Proposed Intent / Action → Deterministic Tool Gateway → Business API
The agent proposes a semantic interpretation of what the customer wants.
The gateway checks the rest:
- Is this operation allowed at the current authentication level?
- Does this customer have authority over the record?
- Is this transition legal in the current business state?
- Does the amount or risk class require additional approval?
- Has an equivalent operation already succeeded?
The customer’s speech may sound perfectly natural, but from a security perspective it remains untrusted semantic input. Spoken prompt injection has to cross the same boundary.
This is not an argument that the model is unintelligent.
It is an argument that there is no reason to grant the model authority it does not need.
Sometimes the safest data is data the agent never receives
Payment information is an obvious example.
We can improve how accurately a voice model hears card numbers. We can also build better redaction around transcripts.
But if the workflow allows it, a stronger architecture is to remove that data from the voice agent’s working surface entirely.
Hand the customer to a secure DTMF or PCI-compliant payment flow, or to another authenticated channel, and return only a bounded result such as PAYMENT_SUCCESS to the agent.
The same principle extends beyond payments.
It is often simpler to keep high-risk data out of the model than to make the model increasingly sophisticated at handling it.
A security review is not a model benchmark
The same perspective changes how I review a voice AI vendor for enterprise use.
Conversation quality, latency, and tool-calling performance matter. But production review eventually asks different questions.
Where is the data processed? How long are audio and transcripts retained? How does deletion work? Which subprocessors are involved? How narrowly can credentials be scoped? Can recording and transcript behavior be controlled? What audit evidence exists when an incident occurs?
A vendor privacy or security page is useful evidence.
It is not the security acceptance itself.
This is not unique to voice AI. These are familiar enterprise SaaS questions. Voice simply puts real-time conversation + telephony + personal data + tool execution into the same session, which makes missing boundaries easier to notice.
What becomes important after the agent starts sounding human
Voice AI has become dramatically more natural in a short period.
The problems I touched in a voice PoC two years ago are not the same ones I care about now. Back then, the immediate challenge was how to listen, decide when a person had stopped speaking, respond quickly, and keep the pipeline from feeling broken.
A modern voice runtime can absorb much more of that conversation loop.
That progress makes the outer boundary more visible.
The important question becomes less “What can the agent say?” and more “What are we willing to let it do?”
A good voice agent is not necessarily the one that memorizes the most regulations. It may be the one that never receives authority it should not have in the first place.
Once those boundaries are in place, the next question is less about security and more about operations:
How do we run one permitted phone call as a durable business job without duplicates, with safe recovery when execution is ambiguous?
That is the next layer: Salesforce-owned business state, an execution scheduler, telephony, GET-based reconciliation, and explicit retry/replay boundaries.