Make Elixir Tests Faster
Everybody wants faster tests, and nobody wants to spend more time than necessary writing them. In this article, I will show a few ways to make Elixir tests faster.
Run Tests In Parallel
The most obvious way to make tests faster is to run them in parallel. ExUnit of course makes this easy
with async: true:
defmodule MyTest do
use ExUnit.Case, async: true
# ...
But you have to remember
to add async: true
to every test, and remembering is hard,
especially when there are multiple people working on the codebase. Luckily,
Credo
has a check called
PassAsyncInTestCases
. Just add the following to your
.credo.exs
in the Refactoring Opportunities
section:
{Credo.Check.Refactor.PassAsyncInTestCases, []},
Credo will now fail if you don’t explicity add an async
value (either true
or false).
If you aren’t already using Credo, I’d recommend adding it. You can disable the checks you don’t care about or that would be too time-consuming to fix right now. I’d recommend running it in CI at the very least, but you might also want to run it before checking code in (via a commit hook or perhaps in your shipit script), or even directly in your editor.
Run Fewer Tests
mix test has a --stale option that only runs tests that reference files which have
changed since the last time you ran tests with this option. If you get in the habit of running mix test --stale,
you will end up saving time by running fewer tests. It’s probably a good idea
to run the full suite before pushing code.
When running mix test --stale, you might be surprised at the number of tests that run. If your
project has a big web of module dependencies, then changing one module might cause a lot of other modules to change,
causing mix test --stale to run a lot of tests.
This is where the Boundary
library comes in handy. It will help you enforce
boundaries between different sections of your codebase, which will reduce the number of modules that get
compiled when you make a change, which will cause fewer tests to run via mix test --stale,
which will result in a faster test suite.
Installing Boundary is not hard. Fixing boundary violations can take time though. I’d recommend installing it as soon as possible, and then allowing existing boundary violations so that you can at least get alerted to new boundary violations. Over time, you can clean up the existing violations to further decrease compilation times and increase stale-test speed.
Shameless plug: my Reflective Software business partner and I can help with cleaning up your boundary violations so that you can continue building features.
Sleep Less
When testing some asynchronous code, your test sometimes has to wait for the async action to complete before
it can successfully assert on the state. The easy way to do this is just drop in :timer.sleep(1_000)
and hope that the async action has finished. Unfortunately, this has the dual downsides of making the test suite
slower and more flaky.
A better way is to retry your assertion repeatedly until it passes. The Moar
library (of which
I am an author) has a Retry
module with retry_for
and rescue_for!
functions:
# repeatedly retry up to 5 seconds for the element with ID "status" to be "finished"
Moar.Retry.rescue_for!(5000, fn ->
assert view |> element("#status") |> render() == "finished"
end)
# repeatedly retry up to 5 seconds for the element with ID "status" to be "finished",
# and then make a different assertion
Moar.Retry.retry_for(5000, fn ->
view |> element("#status") |> render() == "finished"
end)
assert view |> element("#total") |> render() == "8,675,309"
There is a downside to all that retrying and waiting: you are doing unnecessary work which may slow down the test suite (but probably not as much as sleeping for a long time does). Another approach is to send a message to your test when the thing you are waiting for has finished. Look for a future article about how to do that.