One Type, Many Formats: Export a Catalog as JSON, YAML, CSV, XML, MessagePack, TOON and Avro

Welcome! This tutorial is for Scala developers who have finished Your First Schema and can derive a Schema and round-trip a value through JSON. You don't need any prior experience with YAML, XML, CSV, TOON, MessagePack or Avro to follow along.

1. Introduction

By the end of this tutorial, you will understand:

How one Schema value drives a codec for every format ZIO Blocks supports
How to get a codec with Schema#derive and a format object, and how to encode and decode with it
What the text formats (JSON, YAML, XML, TOON) and the binary formats (MessagePack, Avro) look like for the same data
Why CSV accepts only flat records, and how a small flat row type solves that

We learn these concepts by exporting a small catalog of Scala books, one format at a time:

Model the catalog
Encode it to JSON the long way
Swap in YAML
Swap in XML
Swap in TOON
Swap in MessagePack
Swap in Avro
Flatten the catalog for CSV

We recommend reading from top to bottom — each section builds on the previous one.

Each format lives in its own module. To follow along, add the schema module and the six format modules to your build.sbt:

// Scala 2.13 and Scala 3.3 to 3.8
libraryDependencies ++= Seq(
  "dev.zio" %% "zio-blocks-schema"             % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-yaml"        % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-xml"         % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-toon"        % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-messagepack" % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-avro"        % "0.0.56",
  "dev.zio" %% "zio-blocks-schema-csv"         % "0.0.56"
)

// Scala 3.9 and newer (also needs scalacOptions += "-experimental"): the same modules with the zio-blocks-next- prefix
libraryDependencies ++= Seq(
  "dev.zio" %% "zio-blocks-next-schema"             % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-yaml"        % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-xml"         % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-toon"        % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-messagepack" % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-avro"        % "0.0.56",
  "dev.zio" %% "zio-blocks-next-schema-csv"         % "0.0.56"
)

The JSON codec ships inside zio-blocks-schema itself, so it needs no extra module. The code in this tutorial is the same for every Scala version.

2. Background: One Schema, Many Codecs

In Your First Schema, we derived a Schema[Book] and got a JSON codec for free. JSON is only one of many formats your data may need: YAML for configuration files, XML for older services, CSV for spreadsheets, MessagePack or Avro for compact messages between services, and TOON for compact prompts to large language models. In most libraries, each of those formats means another set of encoders and decoders to write and keep in sync with the case class.

ZIO Blocks treats a format as a plug-in. Each format module provides a format object — YamlFormat, XmlFormat, AvroFormat and so on — that knows how to turn any Schema into a codec for that format. The schema describes the shape of the data once; the format object decides how that shape is written. So adding a format to a program means adding its module, importing its format object, and writing one line that derives the codec. In this tutorial, we watch that one line produce seven very different outputs from the same catalog.

3. Model the Catalog

Let's start with the data. A Book has a title, an author, a publication year and a list of tags, and a Catalog holds a list of books. As in the first tutorial, each type gets a derived schema in its companion object:

import zio.blocks.schema._

case class Book(title: String, author: String, year: Int, tags: List[String])

object Book {
  implicit val schema: Schema[Book] = Schema.derived
}

case class Catalog(books: List[Book])

object Catalog {
  implicit val schema: Schema[Catalog] = Schema.derived
}

The code above:

Book stores its authors as one String and its tags as a List[String].
Catalog wraps a List[Book], so the whole export is one value with one top-level name.
Schema.derived derives each schema at compile time; Catalog.schema finds Book.schema for the list's elements.

Next, we fill the catalog with three books about Scala:

val catalog = Catalog(
  List(
    Book("Zionomicon", "John A. De Goes, Adam Fraser", 2024, List("zio", "effects")),
    Book("Programming in Scala", "Martin Odersky, Lex Spoon, Bill Venners", 2008, List("scala", "fundamentals")),
    Book("Functional Programming in Scala", "Paul Chiusano, Rúnar Bjarnason", 2014, List("scala", "fp"))
  )
)

The code above builds the one value we export in every section that follows. Notice that every author list contains a comma — that detail matters when we reach CSV.

4. Encode to JSON the Long Way

In the first tutorial, toJsonString and fromJson did all the work. Behind those shortcuts sits a codec — a value that can both encode and decode one type in one format. Let's get the JSON codec explicitly, because the same three steps work for every other format in this tutorial:

import java.nio.charset.StandardCharsets.UTF_8
import zio.blocks.schema.json.JsonFormat

val jsonCodec = Catalog.schema.derive(JsonFormat)

val jsonBytes = jsonCodec.encode(catalog)

The code above:

import zio.blocks.schema.json.JsonFormat brings the JSON format object into scope.
Catalog.schema.derive(JsonFormat) walks the catalog's schema and builds a JsonCodec[Catalog].
jsonCodec.encode(catalog) writes the catalog and returns the result as an Array[Byte].

The bytes are UTF-8 text, so we turn them into a String to read them, and then decode the same bytes back:

println(new String(jsonBytes, UTF_8))
// {"books":[{"title":"Zionomicon","author":"John A. De Goes, Adam Fraser","year":2024,"tags":["zio","effects"]},{"title":"Programming in Scala","author":"Martin Odersky, Lex Spoon, Bill Venners","year":2008,"tags":["scala","fundamentals"]},{"title":"Functional Programming in Scala","author":"Paul Chiusano, Rúnar Bjarnason","year":2014,"tags":["scala","fp"]}]}

jsonCodec.decode(jsonBytes) == Right(catalog)
// res1: Boolean = true

The code above prints the JSON text and then checks the round trip. JsonCodec#decode returns an Either[SchemaError, Catalog], just like fromJson, and true tells us that decoding gave back exactly the catalog we started with.

These are the three steps we repeat for every format: derive a codec from the schema, encode a value to bytes, decode the bytes back. The toJsonString shortcut from the first tutorial asks the schema for this same JSON codec and calls JsonCodec#encodeToString on it.

5. Swap In YAML

Now for the moment this tutorial is about. To export the catalog as YAML, we change only the format object. This step uses the zio-blocks-schema-yaml module:

import zio.blocks.schema.yaml.YamlFormat

val yamlCodec = Catalog.schema.derive(YamlFormat)

val yamlBytes = yamlCodec.encode(catalog)

The code above:

import zio.blocks.schema.yaml.YamlFormat brings the YAML format object into scope.
Catalog.schema.derive(YamlFormat) is the JSON line with JsonFormat replaced by YamlFormat; Catalog.schema already says everything the YAML codec needs.
yamlCodec.encode(catalog) writes the catalog as YAML bytes.

Let's print the result and check the round trip:

println(new String(yamlBytes, UTF_8))
// books:
//   - title: Zionomicon
//     author: John A. De Goes, Adam Fraser
//     year: 2024
//     tags:
//       - zio
//       - effects
//   - title: Programming in Scala
//     author: Martin Odersky, Lex Spoon, Bill Venners
//     year: 2008
//     tags:
//       - scala
//       - fundamentals
//   - title: Functional Programming in Scala
//     author: Paul Chiusano, Rúnar Bjarnason
//     year: 2014
//     tags:
//       - scala
//       - fp

yamlCodec.decode(yamlBytes) == Right(catalog)
// res3: Boolean = true

The code above prints the YAML and decodes it back. Notice how the same structure looks in YAML: the books field becomes a block sequence with one - entry per book, each book's fields become key: value lines, and the nested tags list becomes a sequence of its own.

6. Swap In XML

XML follows the same pattern. This step uses the zio-blocks-schema-xml module:

import zio.blocks.schema.xml.XmlFormat

val xmlCodec = Catalog.schema.derive(XmlFormat)

val xmlBytes = xmlCodec.encode(catalog)

The code above:

import zio.blocks.schema.xml.XmlFormat brings the XML format object into scope.
Catalog.schema.derive(XmlFormat) builds the XML codec from the same schema.
xmlCodec.encode(catalog) writes the catalog as XML bytes.

Let's print the XML and decode it back:

println(new String(xmlBytes, UTF_8))
// <Catalog><books><item><title>Zionomicon</title><author>John A. De Goes, Adam Fraser</author><year>2024</year><tags><item>zio</item><item>effects</item></tags></item><item><title>Programming in Scala</title><author>Martin Odersky, Lex Spoon, Bill Venners</author><year>2008</year><tags><item>scala</item><item>fundamentals</item></tags></item><item><title>Functional Programming in Scala</title><author>Paul Chiusano, Rúnar Bjarnason</author><year>2014</year><tags><item>scala</item><item>fp</item></tags></item></books></Catalog>

xmlCodec.decode(xmlBytes) == Right(catalog)
// res5: Boolean = true

The code above prints one compact line of XML. Every XML document needs a single root element, and the codec names it after the type, so the catalog becomes <Catalog>. Each field becomes a child element with the field's name, and each list element is wrapped in an <item> element — both for the books and for each book's tags.

tip

To print indented XML for people to read, pass a writer configuration: xmlCodec.encodeToString(catalog, WriterConfig.pretty) with import zio.blocks.schema.xml.WriterConfig.

7. Swap In TOON

TOON (Token-Oriented Object Notation) is a text format designed to carry structured data to large language models in fewer tokens than JSON. This step uses the zio-blocks-schema-toon module:

import zio.blocks.schema.toon.ToonFormat

val toonCodec = Catalog.schema.derive(ToonFormat)

val toonBytes = toonCodec.encode(catalog)

The code above:

import zio.blocks.schema.toon.ToonFormat brings the TOON format object into scope.
Catalog.schema.derive(ToonFormat) builds the TOON codec from the same schema.
toonCodec.encode(catalog) writes the catalog as TOON bytes.

Let's print the TOON text and decode it back:

println(new String(toonBytes, UTF_8))
// books[3]:
//   - title: Zionomicon
//     author: John A. De Goes, Adam Fraser
//     year: 2024
//     tags[2]: zio,effects
//   - title: Programming in Scala
//     author: Martin Odersky, Lex Spoon, Bill Venners
//     year: 2008
//     tags[2]: scala,fundamentals
//   - title: Functional Programming in Scala
//     author: Paul Chiusano, Rúnar Bjarnason
//     year: 2014
//     tags[2]: scala,fp

toonCodec.decode(toonBytes) == Right(catalog)
// res7: Boolean = true

The code above prints the catalog as TOON. Notice what TOON leaves out compared with JSON: there are no braces and no quotes around keys or plain strings. Each list states its length up front — books[3] and tags[2] — and a list of plain values such as tags fits on one line as zio,effects.

8. Swap In MessagePack

The formats so far produce text. MessagePack is a binary format: it stores the same structure as JSON, but with compact type tags instead of punctuation. This step uses the zio-blocks-schema-messagepack module:

import zio.blocks.schema.msgpack.MessagePackFormat

val msgpackCodec = Catalog.schema.derive(MessagePackFormat)

val msgpackBytes = msgpackCodec.encode(catalog)

The code above:

import zio.blocks.schema.msgpack.MessagePackFormat brings the MessagePack format object into scope.
Catalog.schema.derive(MessagePackFormat) builds the MessagePack codec from the same schema.
msgpackCodec.encode(catalog) writes the catalog as binary bytes.

Binary bytes aren't readable as text, so instead of printing them as a String, we write a small helper that shows the first 16 bytes in hexadecimal:

def hexPrefix(bytes: Array[Byte]): String =
  bytes.take(16).map(b => f"$b%02x").mkString(" ")

The code above:

bytes.take(16) keeps only the first 16 bytes.
f"$b%02x" formats each byte as two hexadecimal digits.
mkString(" ") joins the digits with spaces.

Now let's compare the size with JSON, look at the first bytes and check the round trip:

println(s"MessagePack: ${msgpackBytes.length} bytes, JSON: ${jsonBytes.length} bytes")
// MessagePack: 298 bytes, JSON: 360 bytes

println(hexPrefix(msgpackBytes))
// 81 a5 62 6f 6f 6b 73 93 84 a5 74 69 74 6c 65 aa

msgpackCodec.decode(msgpackBytes) == Right(catalog)
// res10: Boolean = true

The code above shows that the MessagePack export is smaller than the JSON one. The hex prefix still follows the catalog's structure: 81 starts a map with one entry, a5 starts a 5-byte string, 62 6f 6f 6b 73 spells books, 93 starts an array of three books, and 84 starts the first book as a map with four entries, whose first key is title. Field names travel inside the bytes, just as they do in JSON.

9. Swap In Avro

Avro is a binary format that is popular with Apache Kafka and big-data tools. This step uses the zio-blocks-schema-avro module, which builds on the Apache Avro library and runs on the JVM only — it is not available for Scala.js:

import zio.blocks.schema.avro.AvroFormat

val avroCodec = Catalog.schema.derive(AvroFormat)

val avroBytes = avroCodec.encode(catalog)

The code above:

import zio.blocks.schema.avro.AvroFormat brings the Avro format object into scope.
Catalog.schema.derive(AvroFormat) builds the Avro codec from the same schema.
avroCodec.encode(catalog) writes the catalog as binary bytes.

Let's compare the size with the previous two formats, look at the first bytes and check the round trip:

println(s"Avro: ${avroBytes.length} bytes, MessagePack: ${msgpackBytes.length} bytes, JSON: ${jsonBytes.length} bytes")
// Avro: 219 bytes, MessagePack: 298 bytes, JSON: 360 bytes

println(hexPrefix(avroBytes))
// 06 14 5a 69 6f 6e 6f 6d 69 63 6f 6e 38 4a 6f 68

avroCodec.decode(avroBytes) == Right(catalog)
// res13: Boolean = true

The code above shows that Avro is the smallest export so far. Its bytes contain no field names at all: 06 is the number of books (Avro writes 3 as 06), 14 is the length of the first title (10), and the next ten bytes spell Zionomicon. A reader needs the Avro schema to know which value is which field, and the codec derived that from Catalog.schema as well — AvroCodec#avroSchema returns it.

ZIO Blocks has two more binary formats, both JVM only: Thrift, derived with ThriftFormat from zio-blocks-schema-thrift, and BSON, derived with BsonCodecDeriver from zio-blocks-schema-bson.

10. Flatten the Catalog for CSV

CSV is different from every format so far. A CSV file is a table: one row per record, one column per field, and every cell is plain text. There is no place in a cell for a list, so a Book with its tags doesn't fit. The CSV deriver checks this when we derive the codec and throws an exception. Let's see the message. This step uses the zio-blocks-schema-csv module:

import zio.blocks.schema.csv.CsvFormat

println(scala.util.Try(Book.schema.derive(CsvFormat)))
// Failure(java.lang.UnsupportedOperationException: CSV does not support sequence fields.)

The code above imports CsvFormat from zio.blocks.schema.csv and wraps the derivation in scala.util.Try(...), so we can print the failure instead of stopping the program.

The exception is thrown at run time, when we call Schema#derive, not at compile time. The CSV deriver rejects lists, maps, Option fields and sealed traits in the same way.

The fix is a flat row type that holds only the columns we want in the table. Let's define BookRow and turn each book into a row:

case class BookRow(title: String, author: String, year: Int)

object BookRow {
  implicit val schema: Schema[BookRow] = Schema.derived
}

val rows = catalog.books.map(book => BookRow(book.title, book.author, book.year))

val csvCodec = BookRow.schema.derive(CsvFormat)

The code above derives the CSV codec for BookRow, which works because every field is a plain value. Notice that we derive the codec for one row, not for the catalog — a CSV codec encodes and decodes one record at a time.

CSV is also the one text format in this tutorial: CsvFormat is a TextFormat, so its codec writes characters into a java.nio.CharBuffer instead of returning bytes. Let's write a small helper that encodes one row into a buffer and returns the line as a String:

import java.nio.CharBuffer

def toCsvLine(row: BookRow): String = {
  val buffer = CharBuffer.allocate(256)
  csvCodec.encode(row, buffer)
  buffer.flip()
  buffer.toString
}

val csvLines = rows.map(toCsvLine)

The code above:

CharBuffer.allocate(256) creates a buffer large enough for one line.
csvCodec.encode(row, buffer) writes the row's fields, separated by commas and ending with a line break.
buffer.flip() switches the buffer from writing to reading, so buffer.toString returns what was written.

The codec writes data rows only; the header line comes from CsvCodec#headerNames, the list of column names. Let's print the header and the three lines:

println(csvCodec.headerNames.mkString(","))
// title,author,year

csvLines.foreach(line => println(line.stripLineEnd))
// Zionomicon,"John A. De Goes, Adam Fraser",2024
// Programming in Scala,"Martin Odersky, Lex Spoon, Bill Venners",2008
// Functional Programming in Scala,"Paul Chiusano, Rúnar Bjarnason",2014

The code above prints a small CSV table. Notice the author column: a value that contains a comma is wrapped in double quotes, so the commas inside "John A. De Goes, Adam Fraser" don't split the cell. The derived codec always uses the default CSV settings — a comma between fields, double quotes and a \r\n line ending.

Finally, let's decode each line back into a BookRow and check the round trip:

csvLines.map(line => csvCodec.decode(CharBuffer.wrap(line))) == rows.map(Right(_))
// res17: Boolean = true

The code above decodes each line with CsvCodec#decode and compares the results with the original rows. The tags are not in the CSV, so the round trip holds for BookRow, not for Book — choosing which fields go into the table is part of designing a CSV export.

11. Putting It Together

This example combines all the concepts we've learned — one Catalog schema, a codec derived for each of the six structured formats with the same Schema#derive call, a round trip through each, and a flat BookRow for CSV:

schema-examples/src/main/scala/onetypemanyformats/CompleteExample.scala
package onetypemanyformats

import java.nio.CharBuffer
import java.nio.charset.StandardCharsets.UTF_8
import zio.blocks.schema._
import zio.blocks.schema.avro.AvroFormat
import zio.blocks.schema.csv.CsvFormat
import zio.blocks.schema.json.JsonFormat
import zio.blocks.schema.msgpack.MessagePackFormat
import zio.blocks.schema.toon.ToonFormat
import zio.blocks.schema.xml.XmlFormat
import zio.blocks.schema.yaml.YamlFormat

/**
 * One Type, Many Formats — Complete Example
 *
 * Derives JSON, YAML, XML, TOON, MessagePack and Avro codecs from one Catalog
 * schema, round-trips each, then exports a flat BookRow as CSV.
 *
 * Run with: cd schema-examples && sbt "runMain
 * onetypemanyformats.CompleteExample"
 */
object CompleteExample extends App {

  case class Book(title: String, author: String, year: Int, tags: List[String])

  object Book {
    implicit val schema: Schema[Book] = Schema.derived
  }

  case class Catalog(books: List[Book])

  object Catalog {
    implicit val schema: Schema[Catalog] = Schema.derived
  }

  val catalog = Catalog(
    List(
      Book("Zionomicon", "John A. De Goes, Adam Fraser", 2024, List("zio", "effects")),
      Book("Programming in Scala", "Martin Odersky, Lex Spoon, Bill Venners", 2008, List("scala", "fundamentals")),
      Book("Functional Programming in Scala", "Paul Chiusano, Rúnar Bjarnason", 2014, List("scala", "fp"))
    )
  )

  // One schema, six structured formats: only the format object changes
  val jsonCodec = Catalog.schema.derive(JsonFormat)
  val yamlCodec = Catalog.schema.derive(YamlFormat)
  val xmlCodec  = Catalog.schema.derive(XmlFormat)
  val toonCodec = Catalog.schema.derive(ToonFormat)
  val msgCodec  = Catalog.schema.derive(MessagePackFormat)
  val avroCodec = Catalog.schema.derive(AvroFormat)

  val jsonBytes = jsonCodec.encode(catalog)
  println(new String(jsonBytes, UTF_8))
  println(jsonCodec.decode(jsonBytes) == Right(catalog))

  val yamlBytes = yamlCodec.encode(catalog)
  println(new String(yamlBytes, UTF_8))
  println(yamlCodec.decode(yamlBytes) == Right(catalog))

  val xmlBytes = xmlCodec.encode(catalog)
  println(new String(xmlBytes, UTF_8))
  println(xmlCodec.decode(xmlBytes) == Right(catalog))

  val toonBytes = toonCodec.encode(catalog)
  println(new String(toonBytes, UTF_8))
  println(toonCodec.decode(toonBytes) == Right(catalog))

  val msgBytes = msgCodec.encode(catalog)
  println(s"MessagePack: ${msgBytes.length} bytes, round trip: ${msgCodec.decode(msgBytes) == Right(catalog)}")

  val avroBytes = avroCodec.encode(catalog)
  println(s"Avro: ${avroBytes.length} bytes, round trip: ${avroCodec.decode(avroBytes) == Right(catalog)}")

  // CSV accepts only flat records, so export a BookRow per book
  case class BookRow(title: String, author: String, year: Int)

  object BookRow {
    implicit val schema: Schema[BookRow] = Schema.derived
  }

  val rows     = catalog.books.map(book => BookRow(book.title, book.author, book.year))
  val csvCodec = BookRow.schema.derive(CsvFormat)

  def toCsvLine(row: BookRow): String = {
    val buffer = CharBuffer.allocate(256)
    csvCodec.encode(row, buffer)
    buffer.flip()
    buffer.toString
  }

  val csvLines = rows.map(toCsvLine)
  println(csvCodec.headerNames.mkString(","))
  csvLines.foreach(line => println(line.stripLineEnd))
  println(csvLines.map(line => csvCodec.decode(CharBuffer.wrap(line))) == rows.map(Right(_)))
}

12. Running the Examples

All examples in this tutorial have corresponding runnable Scala files in the schema-examples project. To get them, clone the repository and change into the examples project:

git clone https://github.com/zio/zio-blocks.git
cd zio-blocks/schema-examples

Run them in order to follow the tutorial step by step.

Model the Catalog and Encode to JSON

This example models the catalog, derives its JSON codec with Schema#derive, prints the JSON and checks the round trip.

schema-examples/src/main/scala/onetypemanyformats/Concept1JsonExample.scala

schema-examples/src/main/scala/onetypemanyformats/Concept1JsonExample.scala
package onetypemanyformats

import java.nio.charset.StandardCharsets.UTF_8
import zio.blocks.schema._
import zio.blocks.schema.json.JsonFormat

/**
 * One Type, Many Formats — Concept 1: Model the Catalog and Encode to JSON
 *
 * Models the catalog, derives its JSON codec with Schema#derive, prints the
 * JSON and checks the round trip.
 *
 * Run with: cd schema-examples && sbt "runMain
 * onetypemanyformats.Concept1JsonExample"
 */
object Concept1JsonExample extends App {

  case class Book(title: String, author: String, year: Int, tags: List[String])

  object Book {
    implicit val schema: Schema[Book] = Schema.derived
  }

  case class Catalog(books: List[Book])

  object Catalog {
    implicit val schema: Schema[Catalog] = Schema.derived
  }

  val catalog = Catalog(
    List(
      Book("Zionomicon", "John A. De Goes, Adam Fraser", 2024, List("zio", "effects")),
      Book("Programming in Scala", "Martin Odersky, Lex Spoon, Bill Venners", 2008, List("scala", "fundamentals")),
      Book("Functional Programming in Scala", "Paul Chiusano, Rúnar Bjarnason", 2014, List("scala", "fp"))
    )
  )

  val jsonCodec = Catalog.schema.derive(JsonFormat)

  val jsonBytes = jsonCodec.encode(catalog)

  println(new String(jsonBytes, UTF_8))

  println(jsonCodec.decode(jsonBytes) == Right(catalog))
}