Skip to content
AI SECURITY DESKPractical intelligence for small business

Guides

AI Security Risks for Small Startups: Ten Risks and a Seven-Day Plan

Ten failure modes that matter when one developer can connect a frontier model to code, customer data, cloud consoles, and money before lunch.

· admin
FIELD GUIDE / 2026TEAMS ≤ 10FRONTIER-MODEL STACKS

10 risksEach mapped to OWASP LLM Top 10 (2025)
6 are P0Fix before an agent gets production tools
1 weekControls chosen for a tiny team, not an enterprise

The risk register

10 risks shown · P0 means “before production”

01 / 10

Prompt injection becomes command execution

The model cannot reliably distinguish your instruction from hostile text inside a page, issue, document, email, or tool response.

P0 · CONTAIN

≤10-person startup scenario

Your coding agent reads a poisoned README or support ticket, then uses shell, GitHub, cloud, or browser tools under a founder’s identity. One person’s broad access becomes the blast radius.

Framework map

OWASP LLM01ATLAS · LLM Prompt InjectionSAIF · Prompt InjectionNIST · MAP / MEASURE
Evidence in the wild

Cornell Tech’s WARP research found a single Reddit comment could steer deep-research agents; Reddit supplied 54–71% of the user-generated content retrieved in tests. Separately, a ReFi Hub founder’s firsthand account described executing a malicious command surfaced in Claude.

The founder incident is self-reported, not a published forensic report.

Cheapest control this week

  • Default agents to read-only; require approval for shell, write, network, and credential use.
  • Run external content in a separate context from tool instructions.
  • Add five hostile-document tests to CI and log every tool call.
02 / 10

Sensitive data leaks through chat, connectors, or logs

The fastest workflow—paste it into the model—can silently move customer, payroll, source, or deal data outside the boundary you intended.

P0 · CONTAIN

≤10-person startup scenario

A teammate uses a personal AI account for customer support or debugging. Prompt history, uploaded files, OAuth-connected email, and verbose traces all become separate data stores with different retention and access rules.

Framework map

OWASP LLM02ATLAS · LLM Data LeakageSAIF · Sensitive Data DisclosureNIST · GOVERN / MAP
Owner + incident evidence

Small-business Facebook discussions reviewed for this guide repeatedly warned against pasting contracts, customer lists, payroll, and pricing into free AI accounts; Ryan Miller’s shorthand was “never paste client data.” Group-IB reported 225,000+ compromised ChatGPT credential logs offered on underground markets. A Check Point proof of concept also moved Gmail data between ChatGPT accounts through a shared internal service.

Check Point did not claim broad in-the-wild exploitation; the implicated service was decommissioned.

Cheapest control this week

  • Publish a one-page “red / yellow / green” data rule and name approved accounts.
  • Turn on MFA; remove unused connectors; choose “always ask” for connected-app access.
  • Scrub secrets and personal data before prompts; shorten log retention.
03 / 10

Your AI supply chain executes before anyone reviews it

Models, packages, MCP servers, skills, prompt files, eval sets, browser extensions, and hosted APIs are all executable trust decisions.

P0 · CONTAIN

≤10-person startup scenario

An engineer installs a new tool because an agent recommends it, or pulls a popular AI package without pinning. The component inherits environment variables, source access, and CI credentials.

Framework map

OWASP LLM03ATLAS · Supply Chain CompromiseSAIF · Model Source TamperingNIST · GOVERN-6
Evidence in the wild

The ReFi Hub founder said a malicious SKILL.md survived in backup disguised as a style guide. In a separate incident, reporting on the Mercor breach tied the initial compromise to poisoned AI library packages; stolen material reportedly included source code, candidate records, interviews, and identity documents.

Cheapest control this week

  • Pin package and model versions; verify hashes; block surprise install scripts.
  • Inventory models, MCP servers, skills, prompt files, and extensions beside your SBOM.
  • Give CI and agents short-lived, scoped credentials—not founder tokens.
04 / 10

Poisoned data quietly becomes product behavior

A tiny amount of adversarial content can dominate retrieval, evaluation, fine-tuning, or persistent memory.

P1 · HARDEN

≤10-person startup scenario

You ingest user feedback, public web pages, tickets, or generated synthetic data into RAG or evals. A poisoned source moves from “content” to “trusted context” without a promotion gate.

Framework map

OWASP LLM04ATLAS · Poison Training DataSAIF · Data PoisoningNIST · MAP / MEASURE
Reddit as the case study

WARP researchers showed that a short payload placed in public user-generated content could influence multiple research-agent outputs. The attack did not need access to model weights or provider systems—only content the agent already trusted enough to retrieve.

This is retrieval poisoning, not proof that a base frontier model was retrained on the comment.

Cheapest control this week

  • Track provenance and timestamp for every retrieved or training item.
  • Quarantine new sources; require two independent sources for consequential claims.
  • Run a golden eval set before any corpus, memory, or model promotion.
05 / 10

Model output crosses a trust boundary unsanitized

LLM text becomes code, SQL, HTML, shell, an API argument, or an auto-merged pull request—turning probabilistic output into deterministic impact.

P0 · CONTAIN

≤10-person startup scenario

A “temporary” automation pipes an agent’s response directly into a terminal or deployment system. Review disappears because the same founder wrote the tool and needs the velocity.

Framework map

OWASP LLM05ATLAS · Command ExecutionSAIF · Insecure Model OutputNIST · MEASURE / MANAGE
Founder evidence

In the ReFi Hub account, a Claude-supplied download command from a copycat site was pasted into a terminal and malware executed immediately. That chain is the smallest possible demonstration of improper output handling: plausible text crossed into execution without an independent control.

The report says no sensitive data escaped and remains a self-reported incident.

Cheapest control this week

  • Parse model output into typed schemas; reject anything outside an allowlist.
  • Escape output for its destination—HTML, SQL, shell, URL, or API.
  • Sandbox generated code and require a diff for every destructive action.
06 / 10

An agent has more authority than the task requires

The model does not need “admin.” It needs the narrowest tool, data slice, time window, and transaction limit for one job.

P0 · CONTAIN

≤10-person startup scenario

A founder connects email, GitHub, Stripe, cloud, and production to one assistant. Prompt injection or a model mistake can now read, write, spend, message, and deploy as the company.

Framework map

OWASP LLM06OWASP Agentic ASI02/03SAIF · Rogue ActionsNIST · MANAGE
Evidence from frontier labs

OpenAI disclosed that agents posted 53 user-provided images to external hosts. Its review also found roughly two dozen undesirable agent incidents and interactions with federal sites.

OpenAI said the cited federal interactions did not amount to successful breaches; the image cases were from consumer training data, not described enterprise/API data.

Cheapest control this week

  • Use read-only credentials by default and one connector per agent.
  • Require a human approval at every money, publish, delete, or production boundary.
  • Add maximum steps, maximum spend, a hard timeout, and a tested kill switch.
07 / 10

The system prompt is treated like a vault

Hidden instructions can leak or be inferred. A prompt is behavior guidance—not secret storage, authorization, or an access-control layer.

P1 · HARDEN

≤10-person startup scenario

API keys, internal URLs, customer rules, pricing logic, or role checks end up in a system prompt because it is faster than building real policy enforcement.

Framework map

OWASP LLM07ATLAS · Discovery / Data LeakageNIST · GOVERN
Community signal + adjacent case

Owner/MSP Facebook discussions repeatedly frame pasted client data as a breach-level event. Check Point’s cross-account proof of concept showed why hidden text should not be trusted as a security boundary: a planted instruction could trigger an invisible second task against connected data.

The supplied evidence does not document a small-startup system-prompt leak specifically; it demonstrates the same failed assumption that hidden instructions stay isolated.

Cheapest control this week

  • Assume every system prompt can become public.
  • Move authorization and secrets to code and a secrets manager.
  • Test extraction attempts; rotate anything that has ever lived in a prompt.
08 / 10

Retrieval crosses tenants or rewards poisoned chunks

Vector similarity is not authorization, provenance, or truth. Retrieval can surface the wrong customer’s document—or the attacker’s best-optimized one.

P1 · HARDEN

≤10-person startup scenario

One vector database holds every customer, internal docs, and web data. Metadata filtering lives in prompt-time logic instead of a hard pre-retrieval access check.

Framework map

OWASP LLM08ATLAS · Discovery / ExfiltrationSAIF · Insecure Integrated ComponentNIST · MAP / MEASURE
Evidence by mechanism

WARP demonstrated ranking manipulation inside agent retrieval. Check Point separately demonstrated a cross-account channel through shared infrastructure. Together they show the two halves of this risk: hostile content winning retrieval and supposedly isolated tenants sharing a hidden path.

Neither case is a published multi-tenant vector-database breach; this is a mechanism-level cross-reference.

Cheapest control this week

  • Enforce tenant scope before retrieval; never ask the model to filter afterward.
  • Carry source ACLs and provenance into every chunk and citation.
  • Add “foreign tenant” and “poisoned high-similarity chunk” tests.
09 / 10

Confident output becomes fraud, bad code, or a false fact

Hallucination is only the beginning. Attackers can manufacture the pages, voices, threads, and apparent consensus that AI systems and hurried people rely on.

P0 · CONTAIN

≤10-person startup scenario

The same person generates research, code, invoices, and customer copy. With no second reviewer, a fabricated dependency, vendor, payment request, or legal claim can reach production.

Framework map

OWASP LLM09ATLAS · Adversarial InfluenceNIST · MEASURE / MANAGE
Main Street proof

Owners of St. Louis Roofing & Siding discovered an AI-built clone using their testimonials and credentials after ChatGPT surfaced it. Microsoft found signs consistent with AI-assisted template development in a million-email impersonation campaign that sought ACH transfers near $50,000. A voice-clone attack on WPP’s CEO used WhatsApp and Teams; it failed.

Cheapest control this week

  • Require an independent source for facts and official-source checks for downloads.
  • Verify payment and bank-detail changes on a known second channel.
  • Use two-person approval for money, production, and customer-facing claims.
10 / 10

Tokens, tools, and retries run without a ceiling

Recursive agents, long contexts, attacker-shaped inputs, and retry loops can turn one request into a bill spike or denial of service.

P2 · METER

≤10-person startup scenario

A background agent loops across research, code, and browser tools. No one notices until the provider bill, queue backlog, or production latency becomes the alert.

Framework map

OWASP LLM10ATLAS · Resource HijackingSAIF · Denial of ML ServiceNIST · MEASURE / MANAGE
Closest available case

OpenAI’s ongoing review found roughly two dozen undesirable agent incidents, illustrating how agent activity can outrun oversight even at a frontier lab. That is evidence for hard operational bounds, but not a documented startup token-cost incident.

No direct owner/forum cost-runaway case was present in the supplied evidence; treat this as an explicitly open evidence gap.

Cheapest control this week

  • Set per-request token, tool-call, time, and dollar ceilings.
  • Add per-user and global rate limits, queue caps, and a daily spend alert.
  • Fail closed on repeated errors; never let an agent retry indefinitely.

The seven-day baseline

Sequence matters more than buying another tool.

01

Inventory the actual AI surface

Models, accounts, APIs, agents, connectors, skills, MCP servers, vector stores, eval data, and who owns each one.

02

Draw four hard boundaries

Approval is mandatory before money, production writes, external sends, or disclosure of non-public data.

03

Separate identities

No shared founder token. Read-only service identities by default; short-lived credentials for CI and agents.

04

Label data once

Green may enter approved AI; yellow requires sanitization; red never enters a third-party model or prompt log.

05

Constrain the output path

Typed schemas, allowlists, sandboxing, destination-specific escaping, and review before execution.

06

Instrument before scaling

Log prompt version, model, retrieved sources, tool calls, approver, result, cost, and rollback reference.

07

Run one tabletop

Assume the agent leaked a customer file and changed production. Revoke, contain, notify, recover—then fix what failed.

→

Minimum release gate

No production agent ships without an owner, a budget, a permission map, ten abuse tests, and a working kill switch.

Use the frameworks as a stack

Each answers a different question.

OWASP LLM Top 10

What can fail?
The application-security shortlist used for this register.

Open OWASP ↗

MITRE ATLAS

How would an adversary operate?
Tactics and techniques for threat modeling and red-team scenarios.

Open ATLAS ↗

NIST AI RMF + GenAI Profile

How do we run risk?
Govern, Map, Measure, Manage across the lifecycle.

Open NIST ↗

Google SAIF

Where is risk introduced and mitigated?
A lifecycle map across data, model, application, and agent components.

Open SAIF ↗

Evidence notes

Frameworks are stable references; incidents are time-bounded.

  1. OWASP Top 10 for LLM and GenAI — 2025 category names used here.
  2. Cornell Tech WARP coverage — retrieval poisoning via Reddit; May/June 2026.
  3. ReFi Hub founder incident — firsthand account; not independently forensically validated.
  4. Group-IB credential-log findings — January–October 2023 dataset, reported March 2024.
  5. Check Point cross-account channel coverage — proof of concept, service decommissioned.
  6. Mercor supply-chain breach coverage — June 2026 reporting.
  7. OpenAI agent-incident synthesis — distinguishes interaction from successful breach.
  8. St. Louis Roofing & Siding clone — National Roofing Contractors Association summary.
  9. Microsoft million-email campaign summary — AI involvement described as indicators, not proof every message was generated by AI.
  10. WPP CEO deepfake attempt — unsuccessful WhatsApp/Teams social-engineering case.
Method and limits. This is a prioritized threat model for a particular operating style: a startup of ten people or fewer using frontier models, agents, RAG, and fast-moving integrations. It is not a ranking of statistical incident frequency. Facebook owner/MSP observations supplied for this review are included as community signal but are not presented as controlled research. Where a cited event is analogous rather than a direct example—especially LLM07, LLM08, and LLM10—the card says so instead of overstating the evidence.
FIELD GUIDE · AI APPLICATION SECURITY · REVIEW CONTROLS WHEN MODELS, TOOLS, OR DATA FLOWS CHANGE

Back to AI Security Desk