When you develop a program, dealing with errors and side effects is unavoidable. However carefully you write the code, there will be problems you didn’t think of, and a program that keeps evolving accumulates technical debt, and new problems keep appearing.

Problems are a terrible thing, both for the individual developer and for the company selling the program. Even a problem that is simple to solve means financial loss if it reaches users. That is why programming has kept evolving toward forms that let us write programs safely, and developers have come up with all sorts of approaches. One of them is a methodology called Railway-Oriented Programming (ROP from here on).

Let's take a trip
Let's take a trip

Side Effects

First, let’s look at side effects in more detail. A side effect is something that happens inside a function (or procedure) and affects the world outside it. Concretely, that means cases like these.

  • A function manipulates a variable that lives outside it
  • Bad data arrives over the network and affects the program
  • An error occurs inside a function and causes a problem in the program

There are plenty of other cases besides these. These days it’s widely known that referencing or changing outside values from inside a function is a bad idea, so side effects mostly come up as problems caused by I/O. So many developers work hard on exception handling to deal with it. But the problem many developers overlook is the last one, an error occurs inside a function and causes a problem in the program. It can happen even in the simplest code. If anything, mistakes happen because the code is simple. Take the following code, for example.

// Kotlin
fun getFirstElement(list: List<Int>): Int {
  return list[0]
}

This is a tiny function that fetches the first value in a list. At a glance there’s nothing wrong with it, but it breaks when the list is empty. A problem like this is easy to fix, of course, but people get it wrong all the time when actually writing the code.

Side effects make the flow of a program hard to predict, and when you modify code someone else wrote without accounting for side effects, unexpected problems can follow. There are various ways to deal with this.

Ways of Dealing With It

Beyond plain branching, there are many ways to handle side effects. Before getting to ROP, let’s look at the other approaches first. Broadly, they fall into two camps.

  • LBYL (Look Before You Leap)
  • EAFP (Easier to Ask for Forgiveness than Permission)

LBYL means look before you leap, and EAFP means it’s easier to ask for forgiveness than permission. If you’ve studied Python you’ve probably heard of both. LBYL means checking conditions explicitly inside the logic. Like this:

// Kotlin
fun getFirstElement(list: List<Int>): Int? {
  if (list.isEmpty()) {
    return null
  }
  return list[0]
}

This code anticipates an empty list coming in as the parameter and handles the exception up front with a branch. EAFP, on the other hand, handles side effects through exception handling. It looks like this.

// Kotlin
fun getFirstElement(list: List<Int>): Int? {
  return try {
    list[0]
  } catch (e: Exception) {
    null
  }
}

Unlike LBYL, this code handles the side effect by writing the correct logic first and catching the exception if one occurs. Just as the name says, you act first and ask forgiveness for the exception afterward.

Python prefers EAFP over LBYL, but I don’t think either style is better than the other. Each simply fits certain situations. Let’s look at the use cases for these approaches in a bit more detail.

LBYL

Pure Functions

If you aren’t dealing with I/O that has to interact with the outside world, you can solve the side effect problem by writing pure functions. A pure function is one that always returns the same value for the same arguments. Which is the same as saying its result is predictable. The following function is pure.

fun sum(a: Int, b: Int): Int {
  return a + b
}

Exceptions that could raise errors still need to be handled.

Since a program runs on top of a computer system, it can never be completely pure the way mathematics is. That can make the boundary of a pure function feel fuzzy. Floating point is one example.

var num1: Double = 0.0
for (i in 0 until 10) {
  num1 += 1.0 / 3
}
val num2: Double = 1.0 / 3 * 10
println(num1 == num2) // false

Mathematically, num1 and num2 are the same value, so this should print true. But because of the limits of floating point, it prints false. If a function handles floating point numbers like this, can you really call it pure?

To resolve this, you need to decide on an implementation spec that fits the program’s purpose. Taking floating point as the example again, if you don’t need much precision you can round at a suitable place, and if you need exact calculation you can build an object that computes decimals exactly using strings instead of the Double type.

The Guard Clause Pattern

The guard clause pattern puts the defensive conditions at the very start of the logic. Calling it a pattern makes it sound complicated, but in practice it’s just a few if statements.

// JavaScript
function authorize(user) {
  if (user.role !== 'admin') return false
  if (user.isBlocked) return false

  // logic shown only to authorized users
}

As you can see, it’s very simple. The point of the guard clause pattern is to keep the defensive conditions at the top of the logic and to avoid nested ifs, which makes the function more readable. Swift even has a guard statement built into the language.

// Swift
func authorize(user: User) throws -> Bool {
  // unlike if, the body runs when the condition is NOT met
  guard user.role == .admin else { return false }
  guard !user.isBlocked else { return false }

  // logic shown only to authorized users
}

EAFP

try-catch

Pure functions make results predictable, but they don’t solve external I/O or the problems a developer failed to notice. Most software built these days almost always deals with external I/O, so we need another solution. try-catch is one.

Many languages have supported try-catch for a long time, so it’s an exception handling method most developers know well.

// Kotlin
try {
  // code that may throw
} catch (e: Exception) {
  // code to run when an exception is thrown
}
// JavaScript
try {
  // code that may throw
} catch (e) {
  // code to run when an exception is thrown
}
# Python
try:
  # code that may throw
except Exception as e:
  # code to run when an exception is thrown

The syntax varies a little from language to language, but the shape is nearly the same. With try-catch, the code that may throw goes in the try block and the code to run when an exception occurs goes in the catch block. When an exception is thrown, the code in the catch block runs. So where should try-catch be used? Usually in the higher-level logic that calls a function, while the function being called just throws. It doesn’t matter whether the error is one the developer anticipated or not.

// Kotlin
fun authorize(user: User) {
  if (user.role != Role.ADMIN) {
    throw RuntimeException("Permission denied.")
  }
}

fun login() {
  try {
    authorize(User(name = "kciter", role = Role.USER))
  } catch (e: Exception) {
    println(e.message)
  }
}

There’s nothing much wrong with try-catch, but it does have a readability problem. try-catch doesn’t flow sequentially. When an error occurs, control jumps to the catch clause, and if you use something like finally, you have to check whether you arrived there from try or from catch. So unless the program is going to just exit, the developer has to make sure the logic continues without trouble no matter which clause it finishes in.

There’s also the problem that the developer has to know in advance which errors a function throws. When there are many custom errors, that can hurt productivity.

That doesn’t mean try-catch is bad. In a program that must never panic, like a server, try-catch is very useful.

fun main() {
  val server = ServerSocket(8080)
  println("Server is running on port ${server.localPort}")

  while (true) {
    val socket = server.accept()
    val reader = Scanner(socket.getInputStream())
    val writer = socket.getOutputStream()
    println("Client connected: ${socket.inetAddress.hostAddress}")

    thread {
      while (true) {
        try {
          val text = reader.nextLine()
          writer.write(text.toByteArray(Charset.defaultCharset()))
        } catch (e: Exception) {
          println(e.message)
          socket.close()
          break
        }
      }
    }
  }
}
For reliability, a server has to stay alive as long as it can

Functors and Monads

Functors and monads are concepts you hear about often once you get into functional programming. They can feel difficult, since they’re foreign to what most of us learned when we first started programming, and explanations sometimes bring in mathematics. But looked at one at a time, they aren’t hard concepts. Explaining functional programming is outside the scope of this article, so I’ll skip the hard theory and look at functors and monads in simple terms.

Before functors and monads, we need to look at types. In functional programming, types are an important concept for function composition. Just like the mathematical definition, a function in the programming world has a domain and a range.

Domain, range, and codomain
Domain, range, and codomain

A function’s domain and range are sets, and a programming language expresses them as types.

Boolean = {true, false}
Short = {-32768, ..., 0, ..., 32767}
Int = {-2147483648, ..., 0, ..., 2147483647}
...

So the following function can be seen as transforming the Int set into the Int set. The domain is the parameter type and the range is the return type.

fun divide(a: Int, b: Int): Int = a / b

But this function isn’t pure, because when b is 0 it raises a DivideByZero error. In that case you can’t say its range is the whole of Int. You could handle it with a branch, but if you want the error itself to be part of the range, you need another approach. That may sound difficult, but it’s actually quite easy. The Int set can’t hold an error, so we need a new set. That is, we make a new type.

Bundled together into one type
Bundled together into one type

This idea is what a functor is. Next, how a functor is implemented in code.

Functor

Using the idea of a functor, we can extend a type into a new type, and that lets us hold an error. Before looking at code, let’s start with the conceptual picture.

The image shows that a functor is like a box. There’s a value inside the box, and we take it out (unwrap value), apply a function (apply function) to it, and put it back in the box (rewrap value). Why go to all this trouble? To deal with the problems that come up when applying a function to a value. Look at the next image.

This time we applied a divide-by-zero function to the functor, and naturally an error occurs. Here the developer can handle the exception with appropriate logic. The error object obtained from that exception handling goes into the functor.

In code, it looks like this. I’ll use Kotlin for the examples.

class Functor<T>(private val value: T) {
  fun <R> map(f: (T) -> R): Functor<R> =
    Functor(f(this.value))
}

In a functor, the function that takes a function and transforms the value is usually called map. Look familiar? Yes. We’ve already been using functors all along! Now let’s look at code that uses the Functor class above.

class Functor<T>(private val value: T) {
  fun <R> map(f: (T) -> R): Functor<R> =
    Functor(f(this.value))

  override fun toString(): String =
    "Functor($value)"
}

fun main() {
  val functor = Functor(1)
  val result = functor.map { it + 1 }
  println(result) // Functor(2)
}

Very simple code. The functor holds the value 1, and map returns that value plus 1. This shows that a functor lets you apply a function that transforms the value. Now let’s build something a little more involved with a functor. This time we’ll implement a functor that knows whether its value is null.

sealed class Option<out T> {
  data class Some<T>(val value: T): Option<T>()
  object None: Option<Nothing>()

  companion object {
    fun <T> of(value: T?): Option<T> = when (value) {
      null -> None
      else -> Some(value)
    }
  }

  override fun toString(): String =
    when (this) {
      is Some -> "Some($value)"
      is None -> "None"
    }
}

fun <T, R> Option<T>.map(f: (T) -> R): Option<R> =
  when (this) {
    is Option.Some -> Option.of(f(this.value))
    is Option.None -> Option.None
  }

fun main() {
  val option = Option.of("Hello, World!")
  val result1 = option.map { it.toIntOrNull() }
  val result2 = option.map { it.length }

  println(result1) // None
  println(result2) // Some
}

We’ve implemented a functor called Option that, when a value is applied, checks whether it is null and classifies it as the None type if so and Some if there is a value. With this you can prevent problems like NullPointerException. And in a language with pattern matching, it can be used even more safely, like this.

fun main() {
  val option = Option.of("Hello, World!")
  val result = when (option) {
    is Some -> option.value
    is None -> "None"
  }

  // result is guaranteed not to be null
  println(result) // Hello, World!
}

What if, instead of using a functor to check for null, we used one to check for errors? Let’s implement a functor that does that.

sealed class Result<out V, out E> {
  data class Success<out V>(val value: V): Result<V, Nothing>()
  data class Failure<out E>(val error: E): Result<Nothing, E>()

  companion object {
    fun <V> of(f: () -> V): Result<V, Throwable> = try {
      Success(f())
    } catch (e: Throwable) {
      Failure(e)
    }
  }

  override fun toString(): String =
    when (this) {
      is Success -> "Success($value)"
      is Failure -> "Failure($error)"
    }
}

fun <V, R, E> Result<V, E>.map(f: (V) -> R): Result<R, E> =
  when (this) {
    is Result.Success -> Result.of { f(value) }
    is Result.Failure -> this
  }

It’s almost the same as Option. The difference is that instead of checking for null, it uses try-catch to check for a Throwable. It can be used like this.

fun main() {
  val result = Result.of { 1 + 2 }
    .map { it / 0 }
  println(result) // Failure(error=java.lang.ArithmeticException: / by zero)
}

If an error occurs in the of method, it returns the Failure type, and if not, Success. Likewise, if an error occurs while transforming the value with map, it returns Failure, and otherwise Success. This lets us handle errors safely. And just as with Option, pattern matching works here too.

fun main() {
  val result = Result.of { 1 + 2 }
    .map { it / 0 }
    .map { it * 2 }

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  }
}

So far there’s no problem, but a situation like the following can come up.

fun sum(a: Int, b: Int): Result<Int, Throwable> = Result.of { a + b }
fun divide(a: Int, b: Int): Result<Int, Throwable> = Result.of { a / b }

fun main() {
  val result = Result.of { 5 }
    .map { sum(it, 10) } // Result<Result<Int, Throwable>, Throwable>
    .map { divide(it, 0) } // the types don't match, so this is a compile error

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  } // java.lang.ArithmeticException: / by zero
}

When every function uses the Result functor type to report errors, as in this code, map runs into the problem of wrapping a box in another box. To solve it, we need to transform the value without wrapping it in a box again. The concept for that is the monad.

Monad

Monads are rumored to be extremely difficult, and that reputation even produced material like The Monad Fear. But set the theory aside and take it one piece at a time, and it isn’t so hard after all.

Let's put the math terms away for now
Let's put the math terms away for now

Earlier I said a monad can resolve the nesting of functors. Monads in programming were in fact created for this. And a lot of developers are already using monads. Look at the following code.

val list = listOf(1, 2, 3, 4, 5)
val result = list
  .flatMap {
    listOf(it, it + 1) // the listOf function returns a List<T>
  }

If you’ve ever dealt with a function called flatMap, this code will look familiar. When you need to return a list again while transforming a list, you use flatMap. Where map would have turned List<Int> into List<List<Int>>, flatMap can produce List<Int>. Whatever flatMap’s function returns is used as the value, as it is. That is a monad.

So a monad resolves nesting. Let’s implement one using the Result functor.

sealed class Result<out V, out E> {
  data class Success<out V>(val value: V): Result<V, Nothing>()
  data class Failure<out E>(val error: E): Result<Nothing, E>()

  companion object {
    fun <V> of(f: () -> V): Result<V, Throwable> = try {
      Success(f())
    } catch (e: Throwable) {
      Failure(e)
    }
  }

  override fun toString(): String =
    when (this) {
      is Success -> "Success($value)"
      is Failure -> "Failure($error)"
    }
}

fun <V, R, E> Result<V, E>.map(f: (V) -> R): Result<R, E> =
  when (this) {
    is Result.Success -> Result.of { f(value) }
    is Result.Failure -> this
  }

// flatMap uses the returned value as it is
fun <V, R, E> Result<V, E>.flatMap(f: (V) -> Result<R, E>): Result<R, E> =
  when (this) {
    is Result.Success -> f(this.value)
    is Result.Failure -> this
  }

This is the code for the Result functor. The map function follows the character of a functor, and the flatMap function follows the character of a monad. flatMap uses the returned value as it is. Now let’s use the monad to solve the problem that couldn’t be solved in the functor example.

fun sum(a: Int, b: Int): Result<Int, Throwable> = Result.of { a + b }
fun divide(a: Int, b: Int): Result<Int, Throwable> = Result.of { a / b }

fun main() {
  val result = Result.of { 5 }
    .flatMap { sum(it, 10) }
    .flatMap { divide(it, 0) } // the types match!

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  } // java.lang.ArithmeticException: / by zero
}

Now the problem is solved. There is a lot of theory about monads, but if you only care about the practical side, an implementation can be this simple. By now you can see that functors and monads make it possible to handle exceptions in a different way from before. Let’s get to ROP properly.

Railway-Oriented Programming

ROP is a methodology, based on the functional paradigm, for controlling side effects. It isn’t widely known as a methodology, but Rust, which doesn’t support try-catch, follows part of the ROP philosophy instead. So I think it’s good to know.

// Rust example
use std::fs::File;

fn main() {
  let f = File::open("hello.txt"); // returns a Result

  let f = match f {
    Ok(file) => file, // if the file opened fine, return the file handle
    Err(error) => {
      panic!("There was a problem opening the file: {:?}", error) // handle the error
    },
  };
}

ROP itself is extremely simple. Briefly, it says that logic splits into success or failure, and you lay a new track for each, and that this is how you build reliable software.

Success or failure
Success or failure

For this, it basically uses the Result monad object implemented above. It doesn’t matter how the errors are checked. Understanding the philosophy behind ROP matters more. ROP follows these principles.

  • Every function runs sequentially.
  • Every function is divided into success or failure.
  • The program must never panic.

Even with principles this simple, ROP is a powerful methodology as a way of thinking. When we program, we are always abstracting functionality. ROP abstracts functionality as a railway track, and every function that makes up the track is divided into success or failure. Abstracting this way means you divide functionality into units of a size that can be split into success and failure, which makes implementation and refactoring easier.

Also, because every function runs sequentially, the flow of the program is easier to understand and readability improves. Through these advantages, ROP helps you build reliable software. Now let’s look at a few more things about Result.

The Recovery Track

Once Result has been implemented, the explanation of ROP is nearly complete. But there is one concept I haven’t covered yet: recovery. ROP has three kinds of track.

  • The success track
  • The failure track
  • The recovery track

The success track is simple. The logic runs the way we imagined in the best case. On the failure track, a problem occurs partway along (while a function is running) and the step fails. The recovery track is for a problem on the failure track that can be recovered from, after which we move back to the success track. We already saw how the success and failure tracks are built while implementing Result in the functor and monad sections, so let’s look at how to recover.

The function that builds the recovery track is implemented under the name rescue or recover. It doesn’t matter which name you use. Let’s look at the code.

sealed class Result<out V, out E> {
  data class Success<out V>(val value: V): Result<V, Nothing>()
  data class Failure<out E>(val error: E): Result<Nothing, E>()
}

// Other functions...

fun <V, E> Result<V, E>.recover(f: (E) -> V): Result.Success<V> {
  return when (this) {
    is Result.Success -> this
    is Result.Failure -> Result.Success(f(error))
  }
}

fun main() {
  val result = sum(5, 10)
    .flatMap { divide(it, 0) }
    .recover { 0 } // after the recovery track it is always a Success

  println(result.value) // 0
}

I implemented a function called recover so that failures can be handled. As the code shows, it’s very simple.

Constraining Error Types

With try-catch, it’s hard to know which errors might occur. So people often look inside the function they’re calling and branch on the throw, or pattern match on the type, at the call site. With Result, errors can be distinguished and handled. For instance:

sealed class NumberException: RuntimeException() {
  data class DivideByZero(override val message: String): NumberException()
  data class TooBig(override val message: String): NumberException()
  data class TooSmall(override val message: String): NumberException()
}

fun sum(a: Int, b: Int): Result<Int, NumberException> {
  val result = a + b
  if (result > 100) return Result.Failure(NumberException.TooBig("Too Big"))
  if (result < 0) return Result.Failure(NumberException.TooSmall("Too Small"))

  return Result.Success(result)
}

fun divide(a: Int, b: Int): Result<Int, NumberException> {
  if (b == 0) return Result.Failure(NumberException.DivideByZero("Divide By Zero"))
  return Result.Success(a / b)
}

fun main() {
  val result = sum(5, 10)
    .flatMap { divide(it, 0) }
    .recover {
      when (it) {
        is NumberException.DivideByZero -> -1
        is NumberException.TooBig -> 100
        is NumberException.TooSmall -> 0
      }
    }

  println(result.value) // -1
}

Look at the recover and when parts. Types constrained with sealed class are handled through pattern matching, with the compiler checking that every case is covered. This lets you manage exceptions even more safely.

Monad Comprehension

We learned earlier that with flatMap, boxes don’t have to be nested. Most of the time you can write clean code with flatMap alone, but a case like the following can come up.

fun main() {
  val result = getUserById(1)
    .flatMap { user ->
      getAllPosts()
        .map { posts ->
          posts.filter { it.userId == user.id } // user is needed here
        }
    }

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  }
}

Even with flatMap, this code gets more and more nested and complicated. Cases like this, where a later step needs a value from an earlier one, often force you to write nested code. To solve this, you can use something called Monad Comprehension. Kotlin, which this article uses for most of its examples, doesn’t support it, though. The representative languages that do are Scala and Haskell. Let’s look at Monad Comprehension through a Scala example.

def getUserById(id: Int): Either[Exception, User] = {
  // ...
}

def getAllPosts(): Either[Exception, List[Post]] = {
  // ...
}

def main(args: Array[String]) = {
  val result = for {
    user <- getUserById(1)
    posts <- getAllPosts().map(_.filter(_.userId == user.id))
  } yield posts.map(_.title)

  result match {
    case Right(posts) => println(posts)
    case Left(e) => println(e)
  }
}

The for ~ yield part of this code is the syntax called For Comprehension, syntactic sugar that makes Monad Comprehension easy to use. This is how the nesting gets removed. In Kotlin, you can use something called Context Receivers to imitate this syntax.

fun main() {
  val result: Result<List<String>, Throwable> = binding {
    val user = getUserById(1).bind()
    val posts = getAllPosts().bind()
    posts.filter { it.userId == user.id }.map { it.title }
  }

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  }
}

The implementation using Context Receivers is outside the scope of this article, so I’ll skip it. If you’re curious, see the Arrow documentation.

The Nested Container Problem

What if you want to use Result together with another monad? For example, you might want to use it with the Option monad built above. Look at the following code.

fun getUserById(id: Int): Result<Option<User>, Throwable> {
  // ...
}

fun getPostsByUserId(userId: Int): Result<List<Post>, Throwable> {
  // ...
}

fun main() {
  val result = getUserById(1)
    .flatMap { user ->
      when (user) { // user is of type Option<User>
        is Option.Some -> {
          getPostsByUserId(user.value.id)
            .map { posts -> 
              posts.map { it.title } 
            }
        }
        is Option.None -> Result.Failure(Throwable("User not found"))
      }
    }

  when (result) {
    is Result.Success -> println(result.value)
    is Result.Failure -> println(result.error)
  }
}

Because the getUserById function has the type Result<Option<User>, Throwable>, the code has to peel off the box with pattern matching in the middle. Here you could solve it by replacing Option with a nullable, but when several monads are actually in use, the code can get more and more complicated. This can be a real problem when you already rely on another monad as your main one. Using Mono and Flux for reactive programming in a Spring environment, or using an Rx-family library, are examples.

To solve this, you need a concept called Higher-Kinded Types (HKT from here on). Unfortunately, apart from a few languages, HKT support is rare, so this problem isn’t easy to solve. Kotlin, used throughout this article, doesn’t provide it.

Scala supports HKT. With it you can implement something called a Monad Transformer, which solves the problem.

def getUserById(id: Int): Either[Exception, Option[User]] = {
  Right(Option(User(1, 30)))
}

def getPostsByUserId(userId: Int): Either[Exception, List[Post]] = {
  Right(List(Post("A"), Post("B")))
}

def main(args: Array[String]): Unit = {
  val result = for {
    // the OptionT type comes from the cats library
    user <- OptionT(getUserById(1))
    posts <- OptionT.liftF(getPostsByUserId(user.id))
  } yield posts.map(_.title)

  result.value match {
    case Right(posts) => println(posts)
    case Left(e) => println(e)
  }
}

The nested code is gone and the result is a bit cleaner. Unfortunately, this isn’t possible in languages that don’t support it. So a developer who wants to adopt ROP needs to consider their environment.

Closing

With ROP you can code a little more safely and intuitively. It can be hard to use in some environments, though, so take that into account. And even with ROP, I don’t recommend using Result for every function, since that can hurt readability. Use it only for the functions that need it.