Automating Test Data Fabrication with AI, Playwright and Faker (2026)

Every test suite starts with a users.json full of clean, sensible records, and every test suite eventually pays for it. The data goes stale, the same three rows never touch the edge cases that break production, and one day someone reuses [email protected] in two parallel tests and spends an afternoon debugging a unique-constraint collision that was never a real bug. This post is about replacing that brittle pile of fixtures with AI test data fabrication - a Playwright fixture that combines Faker for structure and a large language model for realism, so every test gets fresh, localized, schema-valid data on demand.

This is a companion to our pillar guide, Building a Self-Operating QA Agent with Playwright MCP. If you are also fighting brittle locators, pair it with Dynamic Self-Healing Locators in Playwright Fixtures.

Why do static test fixtures rot?

Static fixtures rot for the same reason any hardcoded snapshot of a moving system rots: the application keeps changing and the data stays frozen. A fixture written when the signup form had five fields silently becomes invalid the day someone adds a required phone field, and now your test fails for a reason that has nothing to do with what it was meant to check.

The deeper problem is coverage. A handful of curated records encodes exactly one worldview - usually a Western name, a Latin-script address, an adult birthday, and a happy-path email. It never exercises the 40-character surname, the hyphenated name, the Arabic name in native script, the leap-year date of birth, or the address with no postal code. Those are precisely the inputs that break real systems, and static fixtures are structurally incapable of covering them because someone would have to sit down and hand-write every variant.

Then there is state. Reused fixture values collide under parallel execution, tests start depending on rows other tests created, and your “isolated” suite quietly becomes an ordering-dependent mess. Fresh, per-test data fixes all three problems at once.

What does Faker do vs the LLM?

The key design decision is division of labor. Faker generates structure; the LLM generates realism. They are good at different things, and using each for what it is good at keeps your suite fast, cheap, and correct.

Faker is deterministic, offline, and instant. It is perfect for anything where you need a valid shape but do not care about meaning - a syntactically valid email, a UUID, an ISO date, a phone number matching a locale’s format, a numeric price. You should never pay for an LLM call to produce a random UUID.

The LLM earns its keep where meaning matters: a believable full name that actually goes with its email, a coherent shipping address where the city and emirate agree, a product review with realistic tone, a deliberately adversarial edge case (“a name that is valid but stresses Unicode normalization”). It is also the fastest way to get localized test data that respects real cultural structure rather than a mechanical locale swap.

The practical rule: reach for Faker first, and only escalate to the model for the fields where realism or edge-case intent actually changes what the test proves.

How do you wire it into a Playwright fixture?

Playwright’s fixtures are the natural home for this. A fixture runs before the test, hands the test exactly what it needs, and tears down cleanly - so each test gets its own isolated data with zero boilerplate in the test body.

Here is a testData fixture that generates a user by combining Faker structure with an LLM-enriched, schema-validated record.

// fixtures/test-data.ts
import { test as base } from '@playwright/test';
import { faker } from '@faker-js/faker';
import { z } from 'zod';

// 1. Define the contract once. Generated data must satisfy this.
const UserSchema = z.object({
  fullName: z.string().min(1).max(80),
  email: z.string().email(),
  locale: z.string(),
  address: z.object({
    line1: z.string().min(1),
    city: z.string().min(1),
    country: z.string().min(1),
  }),
  bio: z.string().max(280),
});

export type User = z.infer<typeof UserSchema>;

type Fixtures = {
  makeUser: (opts?: { locale?: string; edgeCase?: string }) => Promise<User>;
};

export const test = base.extend<Fixtures>({
  makeUser: async ({}, use, testInfo) => {
    // 2. Seed Faker per worker so parallel workers never collide,
    //    and the run is reproducible from the seed in the report.
    const seed = Number(process.env.DATA_SEED ?? testInfo.workerIndex + 1);
    faker.seed(seed);

    const factory = async (opts: { locale?: string; edgeCase?: string } = {}) => {
      const locale = opts.locale ?? 'en';

      // 3. Faker builds the structural skeleton - always valid, always free.
      const skeleton = {
        email: faker.internet.email().toLowerCase(),
        locale,
      };

      // 4. LLM enriches the fields that need realism / localization / edge cases.
      const enriched = await enrichWithLLM({
        email: skeleton.email,
        locale,
        edgeCase: opts.edgeCase,
      });

      // 5. Validate BEFORE the test sees it. Fall back to Faker on failure.
      const candidate = { ...skeleton, ...enriched };
      const parsed = UserSchema.safeParse(candidate);
      if (!parsed.success) {
        return fakerOnlyUser(locale); // deterministic safety net
      }
      return parsed.data;
    };

    await use(factory);
  },
});

export { expect } from '@playwright/test';

The enrichWithLLM call asks a model for structured output matching the schema (most current models support a JSON or tool-call response mode - request it so you get parseable data, not prose). Keep the prompt tight: pass the locale, the target schema, and any edge-case instruction, and ask for one record. [verify the exact structured-output flag for your provider and model.]

async function enrichWithLLM(input: {
  email: string; locale: string; edgeCase?: string;
}): Promise<Partial<User>> {
  const prompt = [
    `Generate one realistic user for locale "${input.locale}".`,
    `Return JSON with keys: fullName, address{line1,city,country}, bio.`,
    `fullName must be authentic for the locale, in the native script.`,
    input.edgeCase ? `Edge case to honor: ${input.edgeCase}.` : '',
    `Keep bio under 280 chars. Do not include the email; it is fixed.`,
  ].join('\n');

  const cached = cache.get(prompt);              // reproducibility + cost control
  if (cached) return cached;

  const record = await callModelJson(prompt);    // provider-specific JSON mode
  cache.set(prompt, record);
  return record;
}

Now a test just asks for what it needs and stays readable:

import { test, expect } from '../fixtures/test-data';

test('signs up an Arabic-locale user', async ({ page, makeUser }) => {
  const user = await makeUser({ locale: 'ar' });

  await page.goto('/signup');
  await page.getByRole('textbox', { name: 'Full name' }).fill(user.fullName);
  await page.getByRole('textbox', { name: 'Email' }).fill(user.email);
  await page.getByRole('button', { name: 'Create account' }).click();

  await expect(page.getByText(user.fullName)).toBeVisible();
});

How do you keep generated data schema-valid?

The single non-negotiable rule: validate before the test consumes the data. An LLM can hallucinate a field, drop a required key, or return a bio that blows past your length limit. If that reaches your test, you get a failure that looks like a product bug but is actually a data bug - the worst kind of noise.

Define the contract once (the UserSchema above) and run every candidate through it inside the fixture. On failure you have two sane options: retry the generation once, or fall back to a Faker-only record that is guaranteed valid. The fallback matters because it decouples your suite’s reliability from the model’s good behavior - a flaky API or a bad completion degrades to boring-but-valid data instead of a red build.

This is also where an API contract pays off twice: the same schema that validates your fabricated inputs can be the one your API enforces, so your test data and your production contract never drift apart.

How do you handle localization and RTL?

Localization is where fabricated data clearly beats static fixtures, because you can ask for authenticity you could never hand-write at scale.

For Arabic and other RTL locales, ask the model for names and addresses in native script, then assert two distinct things: that the value is stored correctly (round-trips through your API without mangling), and that it renders correctly (the UI applies dir="rtl" and does not mirror or truncate the text). These are different failure modes - a value can persist perfectly and still render as garbage.

For breadth, generate a batch - one record per target locale in a single call - and drive a data-driven test over the batch. You get real multi-locale coverage without maintaining a fixture file per language. For the visual side of RTL and responsive checks, see AI Responsive and Localization Testing with Vision Models.

How do you make runs reproducible?

Fresh data is a feature until a test fails and you cannot reproduce it. Determinism through seeding is how you keep the benefits of fabrication without losing reproducibility.

Two mechanisms work together. First, seed Faker with a fixed value so its pseudo-random stream is deterministic; derive that seed from testInfo.workerIndex so parallel workers stay isolated but each worker is repeatable. Log the seed into the test report. When CI goes red, you re-run with DATA_SEED=<that value> and get the identical structural data.

Second, cache LLM responses keyed by the prompt. Since the prompt is a pure function of locale and edge case, a cache hit returns the exact same record - which both stabilizes reruns and cuts cost and latency dramatically in CI. For fully offline or air-gapped runs, commit the cache so tests never hit the network at all; see Private and Offline LLMs for Secure QA for that setup. To keep the whole approach affordable at scale, Token-Efficient AI Testing in CI/CD covers batching and cache strategy in depth.

The bottom line

AI test data fabrication is not about replacing Faker with an LLM - it is about using each for what it does well. Faker gives you fast, free, valid structure; the model gives you realism, localization, and deliberate edge cases; a Playwright fixture delivers fresh isolated data to every test; a schema keeps it honest; and seeding keeps it reproducible. Do those five things and your suite stops testing the same three fake users and starts testing the messy, multilingual, edge-heavy reality your product actually ships into.

Start small: pick your flakiest data-dependent test, move it to a makeUser fixture with schema validation and a seed, and watch the class of “stale fixture” failures disappear. If you want help wiring this into an existing framework, that is exactly what our Test Automation Framework Engineering and AI-Augmented Test Generation teams do.