testing
Write or repair Elixir tests with ExUnit, sandbox isolation, async reliability, Mox, ExMachina, and LiveViewTest. Use for test files, test setup, or failing/flaky tests. NOT for investigating an application bug outside the test suite.
Install
npx skills add https://github.com/oliver-kriska/claude-elixir-phoenix/tree/main/plugins/elixir-phoenix/skills/testing
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install oliver-kriska-claude-elixir-phoenix@llmmart
git clone https://github.com/oliver-kriska/claude-elixir-phoenix.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole oliver-kriska/claude-elixir-phoenix collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Elixir Testing Reference
Ash projects: Use
DataCasewithAsh.Testhelpers; test actions via domain code interfaces, not directRepocalls. Seeash-frameworkskill.
Quick reference for Elixir testing patterns.
Iron Laws — Never Violate These
- ASYNC BY DEFAULT — Use
async: trueunless tests modify global state - SANDBOX ISOLATION — All database tests use Ecto.Adapters.SQL.Sandbox
- MOCK ONLY AT BOUNDARIES — Never mock database, internal modules, or stdlib
- BEHAVIOURS AS CONTRACTS — All mocks must implement a defined
@callbackbehaviour - BUILD BY DEFAULT — Use
build/2in factories;insert/2only when DB needed - NO PROCESS.SLEEP — Use
assert_receivewith timeout for async operations - VERIFY_ON_EXIT! — Always call in Mox tests setup
- FACTORIES MATCH SCHEMA REQUIRED FIELDS — Factory definitions must include all fields that have
validate_requiredin the schema changeset. Missing fields cause cascading test failures
Quick Decisions
Which Test Case?
| Testing | Use |
|---|---|
| Controller/API | use MyAppWeb.ConnCase |
| Context/Schema | use MyApp.DataCase |
| LiveView | use MyAppWeb.ConnCase + import Phoenix.LiveViewTest |
| Pure logic | use ExUnit.Case, async: true |
When to use async: true?
- ✅ Pure functions, no shared state
- ✅ Database tests with Sandbox (PostgreSQL)
- ❌ Tests modifying
Application.put_env - ❌ Tests using Mox global mode
Mock or not?
- ✅ Mock: External APIs, email services, file storage
- ❌ Don't mock: Database, internal modules, stdlib
build() or insert()?
- Use
build()by default for speed - Use
insert()only when you need DB ID, constraints, or persisted associations
Quick Patterns
# Setup chain
setup [:create_user, :authenticate]
# Pattern matching assertion
assert {:ok, %User{name: name}} = create_user(attrs)
# Async message assertion
assert_receive {:user_created, _}, 5000
# Mox setup
setup :verify_on_exit!
expect(MockAPI, :call, fn _ -> {:ok, "data"} end)
# LiveView async
html = render_async(view) # MUST call for assign_async
Common Anti-patterns
| Wrong | Right |
|---|---|
Process.sleep(100) |
assert_receive {:done, _}, 5000 |
insert(:user) in factory |
build(:user) in factory |
async: true with set_mox_global() |
async: false |
| Mock internal modules | Test through public API |
References
For detailed patterns, see:
${CLAUDE_SKILL_DIR}/references/exunit-patterns.md- Setup, assertions, tags${CLAUDE_SKILL_DIR}/references/mox-patterns.md- Behaviours, expect/stub, async${CLAUDE_SKILL_DIR}/references/liveview-testing.md- Forms, async, uploads${CLAUDE_SKILL_DIR}/references/factory-patterns.md- ExMachina, sequences, traits
Files (claude-elixir-phoenix)
-
references
-
exunit-patterns.md 4.8 KB
# ExUnit Patterns Reference ## Setup Chain ```elixir describe "admin actions" do setup [:create_user, :make_admin, :authenticate] test "admin can delete user", %{conn: conn, user: user, admin: admin} do # All context from setup chain available end end defp create_user(_context), do: %{user: insert(:user)} defp make_admin(%{user: user}), do: %{admin: insert(:user, role: :admin)} defp authenticate(%{conn: conn, admin: admin}), do: %{conn: log_in_user(conn, admin)} ``` ## Module Setup vs Test Setup ```elixir # setup_all - once per module, SEPARATE PROCESS setup_all do # For expensive operations that can be shared # CAUTION: Can't use Sandbox in async tests :ok end # setup - before each test, SAME PROCESS setup do %{user: insert(:user)} end ``` ## Tags ```elixir @moduletag :integration @tag :slow @tag timeout: 120_000 @tag :skip # Run only tagged tests # mix test --only integration # mix test --exclude slow ``` ## Assertions ```elixir # Pattern matching (preferred) assert {:ok, %User{name: name}} = create_user(attrs) assert name == "Jane" # Guards in pattern assert match?({:ok, %{id: id}} when is_integer(id), result) # Messages - waits up to timeout assert_receive {:user_created, user}, 5000 # Messages - must already be in mailbox assert_received {:notification, _} # Refute (prefer assert with negation for clarity) refute User.admin?(user) # Exceptions assert_raise ArithmeticError, fn -> 1 / 0 end assert_raise Ecto.NoResultsError, ~r/could not find/, fn -> Repo.get!(User, -1) end # Numeric with delta assert_in_delta 1.1, 1.15, 0.1 ``` ## DataCase Template ```elixir defmodule MyApp.DataCase do use ExUnit.CaseTemplate using do quote do alias MyApp.Repo import Ecto import Ecto.Changeset import Ecto.Query import MyApp.DataCase import MyApp.Factory end end setup tags do MyApp.DataCase.setup_sandbox(tags) :ok end def setup_sandbox(tags) do pid = Ecto.Adapters.SQL.Sandbox.start_owner!(MyApp.Repo, shared: not tags[:async]) on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end) end def errors_on(changeset) do Ecto.Changeset.traverse_errors(changeset, fn {msg, opts} -> Regex.replace(~r"%{(\w+)}", msg, fn _, key -> opts |> Keyword.get(String.to_existing_atom(key), key) |> to_string() end) end) end end ``` ## ConnCase Template ```elixir defmodule MyAppWeb.ConnCase do use ExUnit.CaseTemplate using do quote do import Plug.Conn import Phoenix.ConnTest import MyAppWeb.ConnCase import MyApp.Factory @endpoint MyAppWeb.Endpoint use MyAppWeb, :verified_routes end end setup tags do MyApp.DataCase.setup_sandbox(tags) {:ok, conn: Phoenix.ConnTest.build_conn()} end def log_in_user(conn, user) do token = MyApp.Accounts.generate_user_session_token(user) conn |> Phoenix.ConnTest.init_test_session(%{}) |> Plug.Conn.put_session(:user_token, token) end end ``` ## CI Partitioning Split tests across CI machines for faster runs: ```bash # CI config — run with 4 partitions MIX_TEST_PARTITION=1 mix test --partitions 4 MIX_TEST_PARTITION=2 mix test --partitions 4 # etc. ``` Database per partition in `config/test.exs`: ```elixir config :my_app, MyApp.Repo, database: "my_app_test#{System.get_env("MIX_TEST_PARTITION")}", pool: Ecto.Adapters.SQL.Sandbox ``` ## Seed-Based Flaky Test Debugging ExUnit randomizes test order by default. When tests fail intermittently, re-run with the specific seed: ```bash # Failed run shows seed mix test # "Randomized with seed 401472" # Reproduce exact order mix test --seed 401472 ``` If tests pass with `--seed` but fail randomly, you have state leakage between tests. Check: - Shared ETS tables or Application env - Global Mox mode without cleanup - Missing Sandbox ownership ## Running Test Subsets ```bash mix test test/file_test.exs # Single file mix test test/file_test.exs:42 # Single test at line mix test test/my_app_web/ # Directory mix test --only integration # Tagged tests mix test --exclude slow # Exclude tagged mix test --failed # Re-run failures only ``` ## Filtering Verbose Test Output When `--trace` or E2E test output (Playwright, Wallaby) is too noisy, filter for signal: ```bash # ExUnit --trace: show only test names and summary mix test test/file_test.exs --trace 2>&1 | \ grep -E '(^\s+\* test|^\s+\d+\) test|\d+ tests|failures)' # Playwright (via phoenix_test_playwright): filter results MIX_ENV=int_test mix test test/features/file_test.exs --trace 2>&1 | \ grep -E '(test |Finished|failure|✓|✗|success|Failed|assert|Error|PASS|FAIL|\d+ tests)' | \ tail -20 ``` **Rule**: When running E2E tests, always pipe through a filter to extract pass/fail signal. Raw output is too noisy to read in Claude Code. -
factory-patterns.md 2.9 KB
# Factory Patterns Reference ## ExMachina Style ```elixir defmodule MyApp.Factory do use ExMachina.Ecto, repo: MyApp.Repo def user_factory do %MyApp.Accounts.User{ name: sequence(:name, &"User #{&1}"), email: sequence(:email, &"user#{&1}@example.com") } end # Traits as functions def admin(user), do: %{user | role: :admin} def verified(user), do: %{user | verified_at: DateTime.utc_now()} end # Usage user = build(:user) |> admin() |> verified() |> insert() ``` ## Key Patterns ```elixir # BUILD by default (no DB hit) user = build(:user) # INSERT only when needed user = insert(:user) # Associations - use build in factory, insert when needed def post_factory do %Post{ title: "Test", author: build(:user) # NOT insert! } end # Sequences for uniqueness sequence(:email, &"user#{&1}@example.com") ``` ## Updating Factories for Required Fields When a schema adds fields to `@required_fields`, update ALL factories BEFORE running tests to prevent cascade failures: 1. Find all factories that build the affected struct 2. Add the new required fields with sensible defaults 3. Then run the test suite ```elixir # Schema added currency_code to @required_fields # -> Update factory FIRST: def deal_factory do %Deal{ title: sequence(:title, &"Deal #{&1}"), currency_code: :USD, # NEW required field area_unit: :square_feet # NEW required field } end ``` Skipping this step causes 20+ test failures that all have the same root cause (missing factory field) but look like unrelated failures. ## Anti-patterns ```elixir # ❌ insert() in factory definitions def post_factory do %Post{author: insert(:user)} # Creates record even on build()! end # ✅ Use build() in factories def post_factory do %Post{author: build(:user)} end # ❌ Hardcoded unique values insert(:user, email: "test@example.com") # Will fail on second run! # ✅ Use sequences insert(:user) # Uses sequence for email ``` ## Oban Testing ```elixir # config/test.exs config :my_app, Oban, testing: :manual # In test use Oban.Testing, repo: MyApp.Repo test "enqueues welcome email" do {:ok, user} = Accounts.create_user(%{email: "test@example.com"}) assert_enqueued worker: MyApp.WelcomeWorker, args: %{user_id: user.id}, queue: :mailers end test "processes job correctly" do assert :ok = perform_job(MyApp.WelcomeWorker, %{user_id: 1}) end test "drains queue" do assert %{success: 3, failure: 0} = Oban.drain_queue(queue: :default) end ``` ## Property Testing ```elixir use ExUnitProperties property "roundtrip encoding works" do check all data <- map_of(string(:alphanumeric), integer()) do assert data == data |> Jason.encode!() |> Jason.decode!() end end # Custom generator email_gen = gen all name <- string(:alphanumeric, min_length: 1), domain <- member_of(["gmail.com", "outlook.com"]) do "#{name}@#{domain}" end ``` -
liveview-testing.md 2.4 KB
# LiveView Testing Reference ## Mount and Interact ```elixir test "user can interact with counter", %{conn: conn} do {:ok, view, html} = live(conn, ~p"/counter") assert html =~ "Count: 0" html = view |> element("button", "Increment") |> render_click() assert html =~ "Count: 1" end ``` ## Form Testing ```elixir test "validates form on change", %{conn: conn} do {:ok, view, _html} = live(conn, ~p"/users/new") # Validation on change html = view |> form("#user-form", user: %{email: "invalid"}) |> render_change() assert html =~ "must be a valid email" # Submission view |> form("#user-form", user: %{email: "valid@example.com", name: "Jane"}) |> render_submit() assert_redirect(view, ~p"/users") end ``` ## Async Operations (CRITICAL) ```elixir test "loads data asynchronously", %{conn: conn} do {:ok, view, html} = live(conn, ~p"/dashboard") assert html =~ "Loading..." # MUST call render_async for assign_async html = render_async(view) assert html =~ "Dashboard Data" end ``` ## PubSub Testing ```elixir test "updates on broadcast", %{conn: conn} do {:ok, view, _html} = live(conn, ~p"/chat/room1") Phoenix.PubSub.broadcast(MyApp.PubSub, "chat:room1", {:new_message, "Hello!"}) # Re-render to see update assert render(view) =~ "Hello!" end ``` ## File Uploads ```elixir test "uploads file", %{conn: conn} do {:ok, view, _html} = live(conn, ~p"/upload") avatar = file_input(view, "#avatar-form", :avatar, [ %{ name: "photo.jpg", content: File.read!("test/fixtures/photo.jpg"), type: "image/jpeg" } ]) assert render_upload(avatar, "photo.jpg") =~ "100%" view |> form("#avatar-form") |> render_submit() assert render(view) =~ "Upload complete" end ``` ## Navigation Testing ```elixir # Patch (same LiveView, different params) assert_patch(view, ~p"/posts/#{post.id}") # Redirect (different LiveView or dead view) assert_redirect(view, ~p"/login") # Navigate within LiveView view |> element("a", "Next Page") |> render_click() ``` ## Common Mistakes ```elixir # ❌ Missing render_async for assign_async test "loads data" do {:ok, view, _html} = live(conn, ~p"/dashboard") assert render(view) =~ "Data" # Will fail - async not resolved! end # ✅ Call render_async test "loads data" do {:ok, view, _html} = live(conn, ~p"/dashboard") html = render_async(view) assert html =~ "Data" end ``` -
mox-patterns.md 2 KB
# Mox Patterns Reference ## Setup ```elixir # 1. Define behaviour defmodule MyApp.WeatherAPI do @callback get_temperature(String.t()) :: {:ok, float()} | {:error, term()} def get_temperature(city), do: impl().get_temperature(city) defp impl, do: Application.get_env(:my_app, :weather_api, MyApp.OpenWeatherAPI) end # 2. Define mock in test/support/mocks.ex Mox.defmock(MyApp.MockWeatherAPI, for: MyApp.WeatherAPI) # 3. Configure in test_helper.exs Application.put_env(:my_app, :weather_api, MyApp.MockWeatherAPI) ``` ## Usage ```elixir import Mox setup :verify_on_exit! test "fetches temperature" do expect(MockWeatherAPI, :get_temperature, fn "Chicago" -> {:ok, 72.0} end) assert {:ok, temp} = Weather.current_temp("Chicago") assert temp == 72.0 end # Stub for default behavior (not verified) stub(MockWeatherAPI, :get_temperature, fn _ -> {:ok, 70.0} end) # Multiple calls expect(MockWeatherAPI, :get_temperature, 3, fn _ -> {:ok, 70.0} end) ``` ## Async Tests with Mox ```elixir # For spawned processes - allow parent's expectations test "task uses parent's mock" do expect(MockAPI, :fetch, fn _ -> {:ok, "data"} end) parent = self() Task.async(fn -> Mox.allow(MockAPI, parent, self()) # Now can use mock end) |> Task.await() end # For GenServers - use global mode (requires async: false!) setup do set_mox_global() verify_on_exit!() :ok end ``` ## expect vs stub | Function | Verification | Use When | |----------|--------------|----------| | `expect/4` | Verified on exit | Testing specific call with specific args | | `stub/3` | NOT verified | Default behavior, not testing the call | ## Anti-patterns ```elixir # ❌ Missing verify_on_exit! setup do expect(MockAPI, :call, fn _ -> :ok end) :ok # Missing verify_on_exit!() end # ❌ async: true with Mox global mode use MyApp.DataCase, async: true setup do set_mox_global() # Race conditions! end # ❌ Mocking internal modules Mox.defmock(MockRepo, for: Ecto.Repo) # Never mock the database! ```
-
-
SKILL.md 3.2 KB
--- name: testing description: "Write or repair Elixir tests with ExUnit, sandbox isolation, async reliability, Mox, ExMachina, and LiveViewTest. Use for test files, test setup, or failing/flaky tests. NOT for investigating an application bug outside the test suite." effort: medium user-invocable: false paths: - "test/**/*_test.exs" - "test/support/**/*.ex" - "**/*factory*.ex" --- # Elixir Testing Reference > **Ash projects**: Use `DataCase` with `Ash.Test` helpers; test actions via domain code interfaces, not direct `Repo` calls. See `ash-framework` skill. Quick reference for Elixir testing patterns. ## Iron Laws — Never Violate These 1. **ASYNC BY DEFAULT** — Use `async: true` unless tests modify global state 2. **SANDBOX ISOLATION** — All database tests use Ecto.Adapters.SQL.Sandbox 3. **MOCK ONLY AT BOUNDARIES** — Never mock database, internal modules, or stdlib 4. **BEHAVIOURS AS CONTRACTS** — All mocks must implement a defined `@callback` behaviour 5. **BUILD BY DEFAULT** — Use `build/2` in factories; `insert/2` only when DB needed 6. **NO PROCESS.SLEEP** — Use `assert_receive` with timeout for async operations 7. **VERIFY_ON_EXIT!** — Always call in Mox tests setup 8. **FACTORIES MATCH SCHEMA REQUIRED FIELDS** — Factory definitions must include all fields that have `validate_required` in the schema changeset. Missing fields cause cascading test failures ## Quick Decisions ### Which Test Case? | Testing | Use | |---------|-----| | Controller/API | `use MyAppWeb.ConnCase` | | Context/Schema | `use MyApp.DataCase` | | LiveView | `use MyAppWeb.ConnCase` + `import Phoenix.LiveViewTest` | | Pure logic | `use ExUnit.Case, async: true` | ### When to use async: true? - ✅ Pure functions, no shared state - ✅ Database tests with Sandbox (PostgreSQL) - ❌ Tests modifying `Application.put_env` - ❌ Tests using Mox global mode ### Mock or not? - ✅ Mock: External APIs, email services, file storage - ❌ Don't mock: Database, internal modules, stdlib ### build() or insert()? - Use `build()` by default for speed - Use `insert()` only when you need DB ID, constraints, or persisted associations ## Quick Patterns ```elixir # Setup chain setup [:create_user, :authenticate] # Pattern matching assertion assert {:ok, %User{name: name}} = create_user(attrs) # Async message assertion assert_receive {:user_created, _}, 5000 # Mox setup setup :verify_on_exit! expect(MockAPI, :call, fn _ -> {:ok, "data"} end) # LiveView async html = render_async(view) # MUST call for assign_async ``` ## Common Anti-patterns | Wrong | Right | |-------|-------| | `Process.sleep(100)` | `assert_receive {:done, _}, 5000` | | `insert(:user)` in factory | `build(:user)` in factory | | `async: true` with `set_mox_global()` | `async: false` | | Mock internal modules | Test through public API | ## References For detailed patterns, see: - `${CLAUDE_SKILL_DIR}/references/exunit-patterns.md` - Setup, assertions, tags - `${CLAUDE_SKILL_DIR}/references/mox-patterns.md` - Behaviours, expect/stub, async - `${CLAUDE_SKILL_DIR}/references/liveview-testing.md` - Forms, async, uploads - `${CLAUDE_SKILL_DIR}/references/factory-patterns.md` - ExMachina, sequences, traits
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.