Education Centre Guides & Checklists
Keeping AI software secure: security, privacy and compliance for Australian businesses
AI has made software dramatically faster to build. It hasn’t made it any easier to answer four questions: who can log in, what each person and each AI can reach, where your data lives, and which rules apply. The first half of this guide is for business owners. The second half, for CTOs and security leads, sets out our build standard control by control.
At a glance
Keeping AI software secure means protecting who can log in, what each person and AI agent can reach, where the data lives and which Australian rules apply. AI has made software faster to build, which means more apps touching customer data and more ways for it to leak. Australia recorded 1,205 notifiable data breaches in 2025, a record. This Promptwise AI guide explains why it matters, the standard practices every build should follow (two-factor login first), how AI changes the risks, what the Privacy Act asks of you, and how Promptwise AI builds security and compliance into every project.
- Breaches reported in 2025
- 1,205, a record (OAIC)
- Cost per cybercrime report
- A$56,600 average, small business (ASD)
- First thing to switch on
- Two-factor login on every account
- New rule, 10 Dec 2026
- Disclose significant automated decisions
Why this matters more than ever
The numbers in Australia are heading the wrong way
When software takes days to build instead of months, businesses build more of it, and every new app, automation and AI agent is another door into your customer data. Building faster is a huge advantage, as long as security moves just as fast.
Data breaches notified to the OAIC
A record 1,205 in 2025
716 of them came from malicious or criminal attacks. Up 8% on 2024, and the highest since the scheme began in 2018.
Source: OAIC, Notifiable Data Breaches statistics, 2025.
Average cost per cybercrime report, 2024–25
A$56,600 for a small business
Up 14% for small business. The ASD received over 84,700 cybercrime reports, about one every six minutes.
Source: Australian Signals Directorate, Annual Cyber Threat Report 2024–25.
What changed with Software 3.0
Five new risks when AI builds the software
Software 3.0 is the idea that a lot of software is now built by describing what you want to an AI model in plain English. None of this is a reason to avoid AI. It is a reason to build with clear rules from day one.
More software, built by more people
Tools that once needed a developer can now be built by anyone. Great for speed, bad for oversight if nobody keeps a list of what exists and what data it touches.
AI that acts, not just answers
An AI agent can send emails, update records or change settings. A mistake is no longer a bad paragraph; it can be a real action in your business.
Your data travels
Every time an app sends information to an AI model, it leaves your system. Where it goes, how long it is kept and whether it trains a model all matter.
Instructions can hide in content
AI follows plain-English instructions, so a cleverly worded email or web page can try to trick an agent. This is called prompt injection.
Code nobody checked
AI writes code fast, mistakes included, such as a password typed straight into the code. Someone experienced still has to review it.
How protection fits together
Five layers between an attacker and your data
No single control is enough. A well-built system stacks them, so a failure in one layer is caught by the next.
Monitoring and recoveryAudit trail, tested backups, a written incident plan
IdentitySingle sign-on and two-factor on every account
AccessLeast access for every person and every AI agent
AI guardrailsAllow-listed actions and a person approving anything that leaves the business
The standard practices
Nine basics every build should follow
Nothing new or exotic. These are the basics the ASD recommends for every Australian business, applied to every app, automation and AI agent you run.
Two-factor login (MFA)
A password plus a second proof, like an app prompt or a passkey.
Why: A stolen password alone can’t get in.
A password manager
Every account has a long, unique password stored securely.
Why: One leak doesn’t open everything.
Least access
Each person, and each AI, reaches only what its job needs.
Why: A mistake or breach stays small.
Encryption
Data is scrambled while it travels and while it is stored.
Why: Stolen data is unreadable.
Updates
Software and the tools it relies on are kept current.
Why: Most attacks use known, already-fixed holes.
Tested backups
Regular copies, kept separately, that you have actually restored.
Why: You recover from ransomware or a bad change.
An audit trail
A log of who, or which AI, changed what and when.
Why: You see what happened and fix it fast.
No secrets in code
Keys and passwords live in locked settings, not in the code.
Why: Code gets copied; secrets shouldn’t.
Test and live kept apart
Building and testing happen on sample data, away from real customers.
Why: A test can’t leak real records.
If you do one thing
Switch on two-factor login, everywhere
Email, CRM, accounting, hosting and every AI tool. The ASD lists multi-factor authentication among the Essential Eight, and it is one of the first three steps in its small business guidance.
Most account takeovers start with a stolen or guessed password. With two-factor on, the password alone isn’t enough. It matters even more with AI, because an agent often works through your accounts: take over the account and you take over the agent.
Not all two-factor is equal
Login methods, strongest to weakest
Passkey or security key
Strongest. Phishing-resistant, recommended by the ASD
Authenticator app prompt or code
Strong. A big step up from a password alone
Text message code
Weakest form of two-factor. Can be intercepted or talked out of people
Password only
Not enough on its own for any business account
Based on ASD guidance on implementing multi-factor authentication.
The extra rules for AI
Guardrails on every AI feature
On top of the basics, every AI feature gets its own guardrails. The most important: AI drafts, a person approves.
A person approves anything that leaves
Emails to customers, payments, posts and deletions wait for a human yes.
AI gets least access too
An agent that drafts replies doesn’t need your accounting. Each gets its own narrow permissions.
Outside content is information, not instructions
Emails, pages and files an AI reads can never tell it what to do.
Business-grade AI accounts
Business terms and settings where your data isn’t used to train models, with processing locations checked.
No personal information in public AI tools
The OAIC’s own best-practice advice for Australian businesses.
Every AI action logged
So it can be reviewed, and undone.
Storage and warehousing, explained
Where your data actually lives
Most business owners never see where their data sits. Here is the simple version of a well-built setup. Every account in it should be in your business’s name, not your builder’s.
One source of truth
Every screen and AI agent reads from one main database, rather than each keeping its own copy.
A reporting copy for bigger teams
A separate, read-only data warehouse feeds dashboards, so heavy reporting never slows down or risks the live system.
Kept only as long as needed
Old data you no longer need is risk with no upside. Each kind of record gets a retention period.
What Australian law asks of you
The rules, and what is changing
General information, not legal advice. Check your own situation with the OAIC’s tools or a lawyer.
1988
Privacy Act
The Australian Privacy Principles set the rules for handling personal information.
Feb 2018
Notifiable Data Breaches scheme
Covered businesses must notify serious breaches to affected people and the OAIC.
Dec 2024
APP 11.3
“Reasonable steps” to secure personal information now expressly include technical and organisational measures.
Oct 2025
Guidance for AI Adoption
The National AI Centre’s six essential practices replace the Voluntary AI Safety Standard.
10 Dec 2026
Automated decisions
Privacy policies must disclose computer programs that make or inform decisions significantly affecting people.
Who it covers
Businesses with annual turnover over A$3 million, plus some smaller ones, such as health service providers and businesses that trade in personal information. The OAIC’s small business checklist tells you which you are.
Securing what you hold
APP 11 requires reasonable steps to protect personal information, and to destroy or de-identify it once it is no longer needed.
When something goes wrong
A suspected breach must be assessed quickly, within 30 days at most, and serious ones notified.
Your industry
Health, finance and anything involving children can add rules on top, such as APRA’s CPS 234 for regulated entities.
Even if the Privacy Act doesn’t strictly cover you, following it is good practice. Your customers don’t check your turnover before they trust you with their details.
Best practice
A checklist for any build
Use it for any app, automation or AI agent, whoever builds it. If you can’t tick every line, that is where to start.
- Two-factor login is on for every account, including yours and your builder’s
- Every account is in your business’s name, with a list of who has access
- You know what data the build holds, where it is hosted and how long it is kept
- Each person and each AI can reach only what its job needs
- Anything that leaves the business waits for a person to approve it
- Keys and passwords are not written into the code
- Testing happened on sample data, not real customer records
- Backups exist, are kept separately, and have been restored at least once
- There is a log of who, or which AI, changed what
- Your privacy policy matches what the build does, including automated decisions
- There is a written plan for what happens if something goes wrong
- When someone stops working with you, their access goes the same day
The Promptwise AI way
How we handle security and compliance
Security isn’t a line item we add at the end. It is a step of its own in how we scope and build, and it wraps around every part of the software, as the six parts of a modern build show.
- 01
We map your data first
What the build holds, where it lives, who and which AI can see it, and how long it is kept, written down before anything is built.
- 02
Two-factor on every account
Yours, your team’s and ours. Passwords live in a password manager, never in emails or chats.
- 03
Least access, for people and AI
Each gets only what its job needs, and anything that leaves your business waits for a person to approve it.
- 04
No secrets in the code
Keys sit in locked settings on the hosting platform, never in the code or a shared document.
- 05
Sample data until it is ready
Prototypes, demos and testing run on made-up data. Real data goes in once the checks are done.
- 06
Every outside tool checked
A security and licence check before anything is installed, with the result logged.
- 07
Your accounts, your name
Hosting, database and AI accounts in your business’s name, so you can see and remove access at any time.
- 08
Privacy rules built in
We check what the Privacy Act asks of you and help update your privacy policy to match the build.
- 09
A handover you can follow
What was built, where the data lives, who has access and what to do if something goes wrong.
For CTOs and security leads
Our build standard, control by control
If you are the person who signs off a build, this is the detail: the standard every Promptwise AI build is scoped against, mapped to the ASD Essential Eight, the Australian Privacy Principles and the OWASP Top 10 for LLM Applications, and the evidence we hand over at the end.
Controls, and what they map to
| Area | Our standard | Maps to |
|---|---|---|
| Identity | Single sign-on through your identity provider where you have one. Phishing-resistant MFA for every admin and every account that can reach production or customer data. No shared logins. Access removed the same day someone leaves. | Essential Eight: MFA, restrict administrative privileges |
| Authorisation | Role-based access, denied by default. Permissions enforced in the database and server, not just hidden on screen. Narrow service accounts for every integration and AI agent. | APP 11.1, least privilege |
| Data protection | TLS in transit, encryption at rest. Personal information minimised before collection or before it reaches a model. Retention set per record type, then deletion or de-identification. Australian region wherever offered; any overseas processing listed. | APP 11.1, APP 11.2, APP 8 |
| Secrets | Held in the platform’s secret store, never in the repository, separate per environment, rotated at handover or on any suspected exposure. | APP 11.3 |
| Application security | Every change, AI-written code included, reviewed before it ships. Dependencies checked for security and licence before install, versions pinned, patches applied on a set rhythm. | Essential Eight: patch applications |
| Environments | Separate development, test and production. Synthetic data below production; production data is never copied down. | APP 11.1 |
| Logging | Audit trail of who, or which agent, read or changed what and when, with tamper-resistant retention agreed per build. | APP 1, APP 11 |
| Resilience | Automated backups held apart from production, a restore tested before go-live, recovery point and time targets agreed in writing. | Essential Eight: regular backups |
| Incident response | A written plan: who does what, who is told, and how a suspected breach is assessed within the 30 days the NDB scheme allows. | NDB scheme |
| Suppliers | Every platform and AI provider reviewed for data location, retention, training use, sub-processors and published assurance reports (such as ISO 27001 or SOC 2) before personal information reaches it. | APP 8, APP 11, OAIC AI guidance |
The AI-specific threat model: OWASP Top 10 for LLM Applications (2025)
LLM01
Prompt injection
Content the AI reads is data, never instructions. Agents call only allow-listed actions; anything with impact needs approval.
LLM02
Sensitive information disclosure
The AI receives only the fields a task needs; personal information masked where not needed; outputs checked before they leave.
LLM03
Supply chain
Models, packages and connectors from reviewed sources only, checked before install and logged.
LLM04
Data and model poisoning
Knowledge sources are owned and curated by you; nothing is learned from untrusted input.
LLM05
Improper output handling
Output validated against a fixed structure before anything acts on it; never run directly as code or a query.
LLM06
Excessive agency
Narrow permissions per agent and the autonomy level it has earned.
LLM07
System prompt leakage
No secrets or access rules in prompts; access is enforced by the system.
LLM08
Vector and embedding weaknesses
Search indexes respect the same permissions as the source records.
LLM09
Misinformation
Answers grounded in your own records, sourced where it matters, checked by a person when consequential.
LLM10
Unbounded consumption
Rate limits, spend caps and alerts on every AI integration.
Governance
A named owner for every AI system and an AI use register: what each does, what data it touches and what decisions it informs. AI features are tested against realistic and adversarial cases before launch and reviewed after. We follow the National AI Centre’s Guidance for AI Adoption (October 2025) and its six essential practices:
- Decide who is accountable
- Understand impacts and plan
- Measure and manage risk
- Be transparent about AI use
- Test and monitor
- Maintain human control
Shared responsibility
| Who | Responsible for |
|---|---|
| Hosting and AI platforms | Physical security, infrastructure, and the encryption and isolation their services provide |
| Promptwise AI | How the build is designed, configured, coded, connected and handed over against this standard |
| You | Who gets access, keeping MFA on, approving what the AI does, and your own policies and training |
At handover
The security pack
In plain English and in technical detail. It is the evidence your board, auditor or insurer will ask for.
- Data map and data-flow diagram
- Access list
- Supplier and sub-processor register, with data locations
- AI use register
- Backup and restore record
- Incident response plan
- Outside tool register with check results
Plainly
What we don’t claim
Promptwise AI is not ISO 27001 certified and does not hold a SOC 2 report. We are not a penetration testing firm and we are not lawyers.
Where a build needs an independent penetration test, a certification, an industry assessment such as APRA’s CPS 234, or legal advice, we say so at scoping, build it into the plan and bring in an accredited specialist.
What we do claim: a documented standard, applied to every build, with the evidence to show it.
Be honest
Why not just handle this yourself?
You could. Every practice here is public, and most are switched on in a few clicks. You likely won’t, though, and not because it’s hard. It is never urgent until the day it is, and by then the build is live, the data is in and the shortcuts are baked in.
What Promptwise AI adds is that security is planned before the first screen exists, checked at every step and handed over in writing, by someone who scopes the whole business rather than just the software. You get the speed of AI without trading away your customers’ trust.