Claude Skill

testing

Write or repair Elixir tests with ExUnit, sandbox isolation, async reliability, Mox, ExMachina, and LiveViewTest. Use for test files, test setup, or failing/flaky tests. NOT for investigating an application bug outside the test suite.

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download oliver-kriska-claude-elixir-phoenix-plugins_elixir-phoenix_skills_testing-9767a82.zip · 7 KB
Part of oliver-kriska/claude-elixir-phoenix — 93 skills

Install

skills CLI npx skills add https://github.com/oliver-kriska/claude-elixir-phoenix/tree/main/plugins/elixir-phoenix/skills/testing
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install oliver-kriska-claude-elixir-phoenix@llmmart
Git git clone https://github.com/oliver-kriska/claude-elixir-phoenix.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole oliver-kriska/claude-elixir-phoenix collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Elixir Testing Reference

Ash projects: Use DataCase with Ash.Test helpers; test actions via domain code interfaces, not direct Repo calls. See ash-framework skill.

Quick reference for Elixir testing patterns.

Iron Laws — Never Violate These

  1. ASYNC BY DEFAULT — Use async: true unless tests modify global state
  2. SANDBOX ISOLATION — All database tests use Ecto.Adapters.SQL.Sandbox
  3. MOCK ONLY AT BOUNDARIES — Never mock database, internal modules, or stdlib
  4. BEHAVIOURS AS CONTRACTS — All mocks must implement a defined @callback behaviour
  5. BUILD BY DEFAULT — Use build/2 in factories; insert/2 only when DB needed
  6. NO PROCESS.SLEEP — Use assert_receive with timeout for async operations
  7. VERIFY_ON_EXIT! — Always call in Mox tests setup
  8. FACTORIES MATCH SCHEMA REQUIRED FIELDS — Factory definitions must include all fields that have validate_required in the schema changeset. Missing fields cause cascading test failures

Quick Decisions

Which Test Case?

Testing Use
Controller/API use MyAppWeb.ConnCase
Context/Schema use MyApp.DataCase
LiveView use MyAppWeb.ConnCase + import Phoenix.LiveViewTest
Pure logic use ExUnit.Case, async: true

When to use async: true?

  • ✅ Pure functions, no shared state
  • ✅ Database tests with Sandbox (PostgreSQL)
  • ❌ Tests modifying Application.put_env
  • ❌ Tests using Mox global mode

Mock or not?

  • ✅ Mock: External APIs, email services, file storage
  • ❌ Don't mock: Database, internal modules, stdlib

build() or insert()?

  • Use build() by default for speed
  • Use insert() only when you need DB ID, constraints, or persisted associations

Quick Patterns

# Setup chain
setup [:create_user, :authenticate]

# Pattern matching assertion
assert {:ok, %User{name: name}} = create_user(attrs)

# Async message assertion
assert_receive {:user_created, _}, 5000

# Mox setup
setup :verify_on_exit!
expect(MockAPI, :call, fn _ -> {:ok, "data"} end)

# LiveView async
html = render_async(view)  # MUST call for assign_async

Common Anti-patterns

Wrong Right
Process.sleep(100) assert_receive {:done, _}, 5000
insert(:user) in factory build(:user) in factory
async: true with set_mox_global() async: false
Mock internal modules Test through public API

References

For detailed patterns, see:

  • ${CLAUDE_SKILL_DIR}/references/exunit-patterns.md - Setup, assertions, tags
  • ${CLAUDE_SKILL_DIR}/references/mox-patterns.md - Behaviours, expect/stub, async
  • ${CLAUDE_SKILL_DIR}/references/liveview-testing.md - Forms, async, uploads
  • ${CLAUDE_SKILL_DIR}/references/factory-patterns.md - ExMachina, sequences, traits
Files (claude-elixir-phoenix)
  • references
    • exunit-patterns.md 4.8 KB
      # ExUnit Patterns Reference
      
      ## Setup Chain
      
      ```elixir
      describe "admin actions" do
        setup [:create_user, :make_admin, :authenticate]
      
        test "admin can delete user", %{conn: conn, user: user, admin: admin} do
          # All context from setup chain available
        end
      end
      
      defp create_user(_context), do: %{user: insert(:user)}
      defp make_admin(%{user: user}), do: %{admin: insert(:user, role: :admin)}
      defp authenticate(%{conn: conn, admin: admin}), do: %{conn: log_in_user(conn, admin)}
      ```
      
      ## Module Setup vs Test Setup
      
      ```elixir
      # setup_all - once per module, SEPARATE PROCESS
      setup_all do
        # For expensive operations that can be shared
        # CAUTION: Can't use Sandbox in async tests
        :ok
      end
      
      # setup - before each test, SAME PROCESS
      setup do
        %{user: insert(:user)}
      end
      ```
      
      ## Tags
      
      ```elixir
      @moduletag :integration
      @tag :slow
      @tag timeout: 120_000
      @tag :skip
      
      # Run only tagged tests
      # mix test --only integration
      # mix test --exclude slow
      ```
      
      ## Assertions
      
      ```elixir
      # Pattern matching (preferred)
      assert {:ok, %User{name: name}} = create_user(attrs)
      assert name == "Jane"
      
      # Guards in pattern
      assert match?({:ok, %{id: id}} when is_integer(id), result)
      
      # Messages - waits up to timeout
      assert_receive {:user_created, user}, 5000
      
      # Messages - must already be in mailbox
      assert_received {:notification, _}
      
      # Refute (prefer assert with negation for clarity)
      refute User.admin?(user)
      
      # Exceptions
      assert_raise ArithmeticError, fn -> 1 / 0 end
      assert_raise Ecto.NoResultsError, ~r/could not find/, fn -> Repo.get!(User, -1) end
      
      # Numeric with delta
      assert_in_delta 1.1, 1.15, 0.1
      ```
      
      ## DataCase Template
      
      ```elixir
      defmodule MyApp.DataCase do
        use ExUnit.CaseTemplate
      
        using do
          quote do
            alias MyApp.Repo
            import Ecto
            import Ecto.Changeset
            import Ecto.Query
            import MyApp.DataCase
            import MyApp.Factory
          end
        end
      
        setup tags do
          MyApp.DataCase.setup_sandbox(tags)
          :ok
        end
      
        def setup_sandbox(tags) do
          pid = Ecto.Adapters.SQL.Sandbox.start_owner!(MyApp.Repo, shared: not tags[:async])
          on_exit(fn -> Ecto.Adapters.SQL.Sandbox.stop_owner(pid) end)
        end
      
        def errors_on(changeset) do
          Ecto.Changeset.traverse_errors(changeset, fn {msg, opts} ->
            Regex.replace(~r"%{(\w+)}", msg, fn _, key ->
              opts |> Keyword.get(String.to_existing_atom(key), key) |> to_string()
            end)
          end)
        end
      end
      ```
      
      ## ConnCase Template
      
      ```elixir
      defmodule MyAppWeb.ConnCase do
        use ExUnit.CaseTemplate
      
        using do
          quote do
            import Plug.Conn
            import Phoenix.ConnTest
            import MyAppWeb.ConnCase
            import MyApp.Factory
            @endpoint MyAppWeb.Endpoint
            use MyAppWeb, :verified_routes
          end
        end
      
        setup tags do
          MyApp.DataCase.setup_sandbox(tags)
          {:ok, conn: Phoenix.ConnTest.build_conn()}
        end
      
        def log_in_user(conn, user) do
          token = MyApp.Accounts.generate_user_session_token(user)
          conn
          |> Phoenix.ConnTest.init_test_session(%{})
          |> Plug.Conn.put_session(:user_token, token)
        end
      end
      ```
      
      ## CI Partitioning
      
      Split tests across CI machines for faster runs:
      
      ```bash
      # CI config — run with 4 partitions
      MIX_TEST_PARTITION=1 mix test --partitions 4
      MIX_TEST_PARTITION=2 mix test --partitions 4
      # etc.
      ```
      
      Database per partition in `config/test.exs`:
      
      ```elixir
      config :my_app, MyApp.Repo,
        database: "my_app_test#{System.get_env("MIX_TEST_PARTITION")}",
        pool: Ecto.Adapters.SQL.Sandbox
      ```
      
      ## Seed-Based Flaky Test Debugging
      
      ExUnit randomizes test order by default. When tests fail
      intermittently, re-run with the specific seed:
      
      ```bash
      # Failed run shows seed
      mix test  # "Randomized with seed 401472"
      
      # Reproduce exact order
      mix test --seed 401472
      ```
      
      If tests pass with `--seed` but fail randomly, you have
      state leakage between tests. Check:
      
      - Shared ETS tables or Application env
      - Global Mox mode without cleanup
      - Missing Sandbox ownership
      
      ## Running Test Subsets
      
      ```bash
      mix test test/file_test.exs         # Single file
      mix test test/file_test.exs:42      # Single test at line
      mix test test/my_app_web/           # Directory
      mix test --only integration         # Tagged tests
      mix test --exclude slow             # Exclude tagged
      mix test --failed                   # Re-run failures only
      ```
      
      ## Filtering Verbose Test Output
      
      When `--trace` or E2E test output (Playwright, Wallaby) is too
      noisy, filter for signal:
      
      ```bash
      # ExUnit --trace: show only test names and summary
      mix test test/file_test.exs --trace 2>&1 | \
        grep -E '(^\s+\* test|^\s+\d+\) test|\d+ tests|failures)'
      
      # Playwright (via phoenix_test_playwright): filter results
      MIX_ENV=int_test mix test test/features/file_test.exs --trace 2>&1 | \
        grep -E '(test |Finished|failure|✓|✗|success|Failed|assert|Error|PASS|FAIL|\d+ tests)' | \
        tail -20
      ```
      
      **Rule**: When running E2E tests, always pipe through a filter
      to extract pass/fail signal. Raw output is too noisy to read
      in Claude Code.
      
    • factory-patterns.md 2.9 KB
      # Factory Patterns Reference
      
      ## ExMachina Style
      
      ```elixir
      defmodule MyApp.Factory do
        use ExMachina.Ecto, repo: MyApp.Repo
      
        def user_factory do
          %MyApp.Accounts.User{
            name: sequence(:name, &"User #{&1}"),
            email: sequence(:email, &"user#{&1}@example.com")
          }
        end
      
        # Traits as functions
        def admin(user), do: %{user | role: :admin}
        def verified(user), do: %{user | verified_at: DateTime.utc_now()}
      end
      
      # Usage
      user = build(:user) |> admin() |> verified() |> insert()
      ```
      
      ## Key Patterns
      
      ```elixir
      # BUILD by default (no DB hit)
      user = build(:user)
      
      # INSERT only when needed
      user = insert(:user)
      
      # Associations - use build in factory, insert when needed
      def post_factory do
        %Post{
          title: "Test",
          author: build(:user)  # NOT insert!
        }
      end
      
      # Sequences for uniqueness
      sequence(:email, &"user#{&1}@example.com")
      ```
      
      ## Updating Factories for Required Fields
      
      When a schema adds fields to `@required_fields`, update ALL factories
      BEFORE running tests to prevent cascade failures:
      
      1. Find all factories that build the affected struct
      2. Add the new required fields with sensible defaults
      3. Then run the test suite
      
      ```elixir
      # Schema added currency_code to @required_fields
      # -> Update factory FIRST:
      def deal_factory do
        %Deal{
          title: sequence(:title, &"Deal #{&1}"),
          currency_code: :USD,          # NEW required field
          area_unit: :square_feet        # NEW required field
        }
      end
      ```
      
      Skipping this step causes 20+ test failures that all have the same
      root cause (missing factory field) but look like unrelated failures.
      
      ## Anti-patterns
      
      ```elixir
      # ❌ insert() in factory definitions
      def post_factory do
        %Post{author: insert(:user)}  # Creates record even on build()!
      end
      
      # ✅ Use build() in factories
      def post_factory do
        %Post{author: build(:user)}
      end
      
      # ❌ Hardcoded unique values
      insert(:user, email: "test@example.com")  # Will fail on second run!
      
      # ✅ Use sequences
      insert(:user)  # Uses sequence for email
      ```
      
      ## Oban Testing
      
      ```elixir
      # config/test.exs
      config :my_app, Oban, testing: :manual
      
      # In test
      use Oban.Testing, repo: MyApp.Repo
      
      test "enqueues welcome email" do
        {:ok, user} = Accounts.create_user(%{email: "test@example.com"})
      
        assert_enqueued worker: MyApp.WelcomeWorker,
                        args: %{user_id: user.id},
                        queue: :mailers
      end
      
      test "processes job correctly" do
        assert :ok = perform_job(MyApp.WelcomeWorker, %{user_id: 1})
      end
      
      test "drains queue" do
        assert %{success: 3, failure: 0} = Oban.drain_queue(queue: :default)
      end
      ```
      
      ## Property Testing
      
      ```elixir
      use ExUnitProperties
      
      property "roundtrip encoding works" do
        check all data <- map_of(string(:alphanumeric), integer()) do
          assert data == data |> Jason.encode!() |> Jason.decode!()
        end
      end
      
      # Custom generator
      email_gen = gen all
          name <- string(:alphanumeric, min_length: 1),
          domain <- member_of(["gmail.com", "outlook.com"]) do
        "#{name}@#{domain}"
      end
      ```
      
    • liveview-testing.md 2.4 KB
      # LiveView Testing Reference
      
      ## Mount and Interact
      
      ```elixir
      test "user can interact with counter", %{conn: conn} do
        {:ok, view, html} = live(conn, ~p"/counter")
        assert html =~ "Count: 0"
      
        html = view
          |> element("button", "Increment")
          |> render_click()
      
        assert html =~ "Count: 1"
      end
      ```
      
      ## Form Testing
      
      ```elixir
      test "validates form on change", %{conn: conn} do
        {:ok, view, _html} = live(conn, ~p"/users/new")
      
        # Validation on change
        html = view
          |> form("#user-form", user: %{email: "invalid"})
          |> render_change()
      
        assert html =~ "must be a valid email"
      
        # Submission
        view
        |> form("#user-form", user: %{email: "valid@example.com", name: "Jane"})
        |> render_submit()
      
        assert_redirect(view, ~p"/users")
      end
      ```
      
      ## Async Operations (CRITICAL)
      
      ```elixir
      test "loads data asynchronously", %{conn: conn} do
        {:ok, view, html} = live(conn, ~p"/dashboard")
        assert html =~ "Loading..."
      
        # MUST call render_async for assign_async
        html = render_async(view)
        assert html =~ "Dashboard Data"
      end
      ```
      
      ## PubSub Testing
      
      ```elixir
      test "updates on broadcast", %{conn: conn} do
        {:ok, view, _html} = live(conn, ~p"/chat/room1")
      
        Phoenix.PubSub.broadcast(MyApp.PubSub, "chat:room1", {:new_message, "Hello!"})
      
        # Re-render to see update
        assert render(view) =~ "Hello!"
      end
      ```
      
      ## File Uploads
      
      ```elixir
      test "uploads file", %{conn: conn} do
        {:ok, view, _html} = live(conn, ~p"/upload")
      
        avatar = file_input(view, "#avatar-form", :avatar, [
          %{
            name: "photo.jpg",
            content: File.read!("test/fixtures/photo.jpg"),
            type: "image/jpeg"
          }
        ])
      
        assert render_upload(avatar, "photo.jpg") =~ "100%"
      
        view |> form("#avatar-form") |> render_submit()
        assert render(view) =~ "Upload complete"
      end
      ```
      
      ## Navigation Testing
      
      ```elixir
      # Patch (same LiveView, different params)
      assert_patch(view, ~p"/posts/#{post.id}")
      
      # Redirect (different LiveView or dead view)
      assert_redirect(view, ~p"/login")
      
      # Navigate within LiveView
      view |> element("a", "Next Page") |> render_click()
      ```
      
      ## Common Mistakes
      
      ```elixir
      # ❌ Missing render_async for assign_async
      test "loads data" do
        {:ok, view, _html} = live(conn, ~p"/dashboard")
        assert render(view) =~ "Data"  # Will fail - async not resolved!
      end
      
      # ✅ Call render_async
      test "loads data" do
        {:ok, view, _html} = live(conn, ~p"/dashboard")
        html = render_async(view)
        assert html =~ "Data"
      end
      ```
      
    • mox-patterns.md 2 KB
      # Mox Patterns Reference
      
      ## Setup
      
      ```elixir
      # 1. Define behaviour
      defmodule MyApp.WeatherAPI do
        @callback get_temperature(String.t()) :: {:ok, float()} | {:error, term()}
      
        def get_temperature(city), do: impl().get_temperature(city)
        defp impl, do: Application.get_env(:my_app, :weather_api, MyApp.OpenWeatherAPI)
      end
      
      # 2. Define mock in test/support/mocks.ex
      Mox.defmock(MyApp.MockWeatherAPI, for: MyApp.WeatherAPI)
      
      # 3. Configure in test_helper.exs
      Application.put_env(:my_app, :weather_api, MyApp.MockWeatherAPI)
      ```
      
      ## Usage
      
      ```elixir
      import Mox
      
      setup :verify_on_exit!
      
      test "fetches temperature" do
        expect(MockWeatherAPI, :get_temperature, fn "Chicago" ->
          {:ok, 72.0}
        end)
      
        assert {:ok, temp} = Weather.current_temp("Chicago")
        assert temp == 72.0
      end
      
      # Stub for default behavior (not verified)
      stub(MockWeatherAPI, :get_temperature, fn _ -> {:ok, 70.0} end)
      
      # Multiple calls
      expect(MockWeatherAPI, :get_temperature, 3, fn _ -> {:ok, 70.0} end)
      ```
      
      ## Async Tests with Mox
      
      ```elixir
      # For spawned processes - allow parent's expectations
      test "task uses parent's mock" do
        expect(MockAPI, :fetch, fn _ -> {:ok, "data"} end)
        parent = self()
      
        Task.async(fn ->
          Mox.allow(MockAPI, parent, self())
          # Now can use mock
        end)
        |> Task.await()
      end
      
      # For GenServers - use global mode (requires async: false!)
      setup do
        set_mox_global()
        verify_on_exit!()
        :ok
      end
      ```
      
      ## expect vs stub
      
      | Function | Verification | Use When |
      |----------|--------------|----------|
      | `expect/4` | Verified on exit | Testing specific call with specific args |
      | `stub/3` | NOT verified | Default behavior, not testing the call |
      
      ## Anti-patterns
      
      ```elixir
      # ❌ Missing verify_on_exit!
      setup do
        expect(MockAPI, :call, fn _ -> :ok end)
        :ok  # Missing verify_on_exit!()
      end
      
      # ❌ async: true with Mox global mode
      use MyApp.DataCase, async: true
      setup do
        set_mox_global()  # Race conditions!
      end
      
      # ❌ Mocking internal modules
      Mox.defmock(MockRepo, for: Ecto.Repo)  # Never mock the database!
      ```
      
  • SKILL.md 3.2 KB
    ---
    name: testing
    description: "Write or repair Elixir tests with ExUnit, sandbox isolation, async reliability, Mox, ExMachina, and LiveViewTest. Use for test files, test setup, or failing/flaky tests. NOT for investigating an application bug outside the test suite."
    effort: medium
    user-invocable: false
    paths:
      - "test/**/*_test.exs"
      - "test/support/**/*.ex"
      - "**/*factory*.ex"
    ---
    
    # Elixir Testing Reference
    
    > **Ash projects**: Use `DataCase` with `Ash.Test` helpers; test actions via domain code interfaces, not direct `Repo` calls. See `ash-framework` skill.
    
    Quick reference for Elixir testing patterns.
    
    ## Iron Laws — Never Violate These
    
    1. **ASYNC BY DEFAULT** — Use `async: true` unless tests modify global state
    2. **SANDBOX ISOLATION** — All database tests use Ecto.Adapters.SQL.Sandbox
    3. **MOCK ONLY AT BOUNDARIES** — Never mock database, internal modules, or stdlib
    4. **BEHAVIOURS AS CONTRACTS** — All mocks must implement a defined `@callback` behaviour
    5. **BUILD BY DEFAULT** — Use `build/2` in factories; `insert/2` only when DB needed
    6. **NO PROCESS.SLEEP** — Use `assert_receive` with timeout for async operations
    7. **VERIFY_ON_EXIT!** — Always call in Mox tests setup
    8. **FACTORIES MATCH SCHEMA REQUIRED FIELDS** — Factory definitions must include all fields that have `validate_required` in the schema changeset. Missing fields cause cascading test failures
    
    ## Quick Decisions
    
    ### Which Test Case?
    
    | Testing | Use |
    |---------|-----|
    | Controller/API | `use MyAppWeb.ConnCase` |
    | Context/Schema | `use MyApp.DataCase` |
    | LiveView | `use MyAppWeb.ConnCase` + `import Phoenix.LiveViewTest` |
    | Pure logic | `use ExUnit.Case, async: true` |
    
    ### When to use async: true?
    
    - ✅ Pure functions, no shared state
    - ✅ Database tests with Sandbox (PostgreSQL)
    - ❌ Tests modifying `Application.put_env`
    - ❌ Tests using Mox global mode
    
    ### Mock or not?
    
    - ✅ Mock: External APIs, email services, file storage
    - ❌ Don't mock: Database, internal modules, stdlib
    
    ### build() or insert()?
    
    - Use `build()` by default for speed
    - Use `insert()` only when you need DB ID, constraints, or persisted associations
    
    ## Quick Patterns
    
    ```elixir
    # Setup chain
    setup [:create_user, :authenticate]
    
    # Pattern matching assertion
    assert {:ok, %User{name: name}} = create_user(attrs)
    
    # Async message assertion
    assert_receive {:user_created, _}, 5000
    
    # Mox setup
    setup :verify_on_exit!
    expect(MockAPI, :call, fn _ -> {:ok, "data"} end)
    
    # LiveView async
    html = render_async(view)  # MUST call for assign_async
    ```
    
    ## Common Anti-patterns
    
    | Wrong | Right |
    |-------|-------|
    | `Process.sleep(100)` | `assert_receive {:done, _}, 5000` |
    | `insert(:user)` in factory | `build(:user)` in factory |
    | `async: true` with `set_mox_global()` | `async: false` |
    | Mock internal modules | Test through public API |
    
    ## References
    
    For detailed patterns, see:
    
    - `${CLAUDE_SKILL_DIR}/references/exunit-patterns.md` - Setup, assertions, tags
    - `${CLAUDE_SKILL_DIR}/references/mox-patterns.md` - Behaviours, expect/stub, async
    - `${CLAUDE_SKILL_DIR}/references/liveview-testing.md` - Forms, async, uploads
    - `${CLAUDE_SKILL_DIR}/references/factory-patterns.md` - ExMachina, sequences, traits
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related