uTest for Scala

Previous
Quiz
Next

uTest is a minimalist testing framework for the Scala programming language. It is simple and has important features needed for writing and running tests. So, uTest is a good choice for developers who want to focus on writing effective tests without focusing on configurations and extraneous features. It streamlined nature for the testing process.

You can dedicate more time to developing your application code. Whether you're working with Scala on the JVM, Scala.js, and Scala Native, uTest supports consistent experience across different platforms.

Why uTest?

uTest has various advantages over other testing libraries like, ScalaTest, MUnit, ScalaCheck, and Specs2. Some of these advantages are given below.

  • It gives coloured, clearly formatted, and readable test results
  • It has uniform syntax for defining simple and nested tests
  • It has uniform syntax for executing tests
  • It supports for Scala, Scala.js, and Scala Native

Setting Up uTest

You can add the following dependency to your build.sbt file to start using uTest -

libraryDependencies += "com.lihaoyi" %% "utest" % "0.8.4" % "test"
testFrameworks += new TestFramework("utest.runner.Framework")

You can use for Scala.js and Scala-Native projects -

libraryDependencies += "com.lihaoyi" %%% "utest" % "0.8.4" % "test"

You need to add following for Scala-Native -

nativeLinkStubs := true

Writing Tests

Define your test suite by extending TestSuite and overriding the tests method. Here is an example of a simple test -

package test.utest.examples

import utest._

object HelloTests extends TestSuite {
  val tests = Tests {
    test("test1") {
      // Resolved: Remove the exception
      assert(1 + 1 == 2)
    }
    test("test2") {
      1
    }
    test("test3") {
      val a = List[Byte](1, 2)
      // Resolved: Access a valid index
      assert(a(1) == 2)
    }
  }
}

Note that you should have this dependency in your build.sbt file -

name := "MyProject"

version := "0.1"

scalaVersion := "2.13.14"

libraryDependencies += "com.lihaoyi" %% "utest" % "0.8.4" % "test"

testFrameworks += new TestFramework("utest.runner.Framework")

Running Tests

You can run all tests in your project with -

sbt clean compile
sbt test

The output will be -

-------------------------------- Running Tests --------------------------------
+ test.utest.examples.HelloTests.test1 13ms
+ test.utest.examples.HelloTests.test2 0ms  1                         
+ test.utest.examples.HelloTests.test3 1ms                            
[info] Tests: 3, Passed: 3, Failed: 0
[success] Total time: 4 s, completed 08-Aug-2024, 5:36:55 pm

You need to use their full path to run specific tests -

sbt "testOnly test.utest.examples.HelloTests.test1"

The output will be -

[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for Test / testOnly
[success] Total time: 1 s, completed 08-Aug-2024, 5:41:48 pm

For running multiple tests (or groups of tests) -

sbt "testOnly test.utest.examples.{HelloTests,NestedTests}"

The output will be -

[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for Test / testOnly
[success] Total time: 0 s, completed 08-Aug-2024, 5:43:17 pm

Nesting Tests

uTest supports nested tests. So you can test related groups together. For example -

package test.utest.examples

import utest._

object NestedTests extends TestSuite {
  val tests = Tests {
    val x = 1
    test("outer1") {
      val y = x + 1

      test("inner1") {
        assert(x == 1, y == 2)
        (x, y)
      }
      test("inner2") {
        val z = y + 1
        assert(z == 3)
      }
    }
    test("outer2") {
      test("inner3") {
        assert(x > 1)
      }
    }
  }
}

Asynchronous Tests

uTest can handle asynchronous tests that return a Future[T]. For example -

import utest._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

object AsyncTest extends TestSuite {
  def getFromDB(): Future[Int] = Future { 42 }

  val tests = Tests {
    test("get something from database") {
      getFromDB().map(v => assert(v == 42))
    }
  }
}

Additional Features

uTest provides several additional features to make testing easier. Some of these given as follows -

Arrow Assert (==>)

It is syntactic sugar for assertions.

test("arrow assert") {
  val name = "Baeldung"
  name.length ==> 8
}

Before and After Execution

You can run code before and after tests.

object BeforeAfterTest extends TestSuite {
  println("This is executed before the tests")
  override def utestAfterAll() = {
    println("This method will be executed after all the tests")
  }
  val tests = Tests {
    test("simple test") {
      assert(1 == 1)
    }
    test("simple test 2") {
      assert(2 == 2)
    }
  }
}

Intercept

You can verify that a block of code throws an exception.

override val tests = Tests {
  def funnyMethod: String = throw new RuntimeException("Uh oh...")
  test("Handle an exception") {
    val ex = intercept[RuntimeException] {
      funnyMethod
    }
    assert(ex.getMessage == "Uh oh...")
  }
}

Retry

Retry flaky tests multiple times before considering them failed.

def flakyMethod: Int = Random.nextInt(4)
test("retry flaky test") - retry(3) {
  val value = flakyMethod
  assert(value > 2)
}

Nested Tests

You can write nested test blocks to share code between tests.

override def tests: Tests = Tests {
  test("outer test") - {
    val list = List(1,2)
    println("This is an outer level of the test.")
    test("inner test 1") - {
      val list2 = List(10,20)
      list.zip(list2) ==> List((1,10), (2,20))
    }
    test("inner test 2") - {
      val str = List("a", "b")
      list.zip(str) ==> List((1,"a"), (2,"b"))
    }
  }
  test("outer test 2") - {
    println("there is no nesting level here") 
    assert(true)
  }
}

Smart Asserts

uTest includes macro-powered smart asserts that provide useful debugging information in the error message. These asserts print out the names, types, and values of any local variables used in the expression that failed.

val x = 1
val y = "2"
assert(
  x > 0,
  x == y
)

// utest.AssertionError: x == y
// x: Int = 1
// y: String = 2

Shared Setup Code and Fixtures

You can define shared setup code and fixtures. Each nested test block gets its own copy of any mutable variables defined within it, avoiding inter-test interference. For example -

package test.utest.examples

import utest._

object SeparateSetupTests extends TestSuite {
  val tests = Tests {
    var x = 0
    test("outer1") {
      x += 1
      test("inner1") {
        x += 2
        assert(x == 3) // += 1, += 2
        x
      }
      test("inner2") {
        x += 3
        assert(x == 4) // += 1, += 3
        x
      }
    }
    test("outer2") {
      x += 4
      test("inner3") {
        x += 5
        assert(x == 9) // += 4, += 5
        x
      }
    }
  }
}

If you want the mutable fixtures to be truly shared between tests, define them outside the Tests block:

package test.utest.examples

import utest._

object SharedFixturesTests extends TestSuite {
  var x = 0
  val tests = Tests {
    test("outer1") {
      x += 1
      test("inner1") {
        x += 2
        assert(x == 3) // += 1, += 2
        x
      }
      test("inner2") {
        x += 3
        assert(x == 7) // += 1, += 2, += 1, += 3
        x
      }
    }
    test("outer2") {
      x += 4
      test("inner3") {
        x += 5
        assert(x == 16) // += 1, += 2, += 1, += 3, += 4, += 5
        x
      }
    }
  }
}

Smart Asserts and Advanced Assertions

uTest includes advanced assertions such as assertMatch and compileError.

Assert Match

You can check if a value matches a particular shape using Scala's pattern matching syntax.

assertMatch(Seq(1, 2, 3)) { case Seq(1, 2) => }
// AssertionError: Matching failed Seq(1, 2, 3)

Compile Error

Assert that a fragment of code fails to compile

compileError("true * false").check(
  """
  compileError("true * false").check(
                 ^
  """,
  "value * is not a member of Boolean"
)

Configuring uTest

uTest provides several ways to configure the testing framework, like, per-run setup/teardown, custom output formatting, and suite retries. For example, of customizing the framework -

class CustomFramework extends utest.runner.Framework {
  override def setup() = {
    println("Setting up CustomFramework")
  }
  override def teardown() = {
    println("Tearing down CustomFramework")
  }
}

testFrameworks += new TestFramework("test.utest.CustomFramework")

Troubleshooting

If you encounter issues where uTest does not recognize your tests. You are using the correct dependency for Scala.js -

libraryDependencies += "com.lihaoyi" %%% "utest" % "0.3.0" % "test"

This ensures that the correct Scala.js-enabled dependency is used instead of the JVM dependency.

Print Page
Previous
Next
Advertisements