The blog Standardizing IO Interfaces for Scala Libraries describes the design of two standard interfaces: Writable and Readable. It is implemented in the Geny Library.

The Problem

Different libs don’t know each other but they need to consume data from each other or product data for others. Many libs have both read and write operations. Data can be exchanged in non-stream or stream way:

  • Materialize String or Array[Byte] in memory. Not good for big data. Also it creates many short-lived objects that need to be garbarge collected.
  • Use java.io.InputStream or java.io.OutputStream. It has limited data sources and it is not easy to to convert the your data to one of its source format.

Usually different libs define different intefaces that use differnt low-level APIs such as java.io.Writer, java.io.OutputStream, java.nio.channel.Chaneel and data types such as Array[Byte], Iterator[String].

One solution is to starndardize the IO inerfaces.

Standard Interface

The interface is defined as:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
// a source of bytes that can be written to an OutputStream
trait Writable {
  def writeByteTo(out: OutputStream): Unit
}

// a source of bytes that be read from an InputStream
// it is a subtype of Writable that every Readable can
// be transformed into a Writable by replacing an OutputStream
// by an InputStream.
trait Readable extends Writable {
  def readByteThrough[T](f: InputStream => T): T
  def writeBytesTo(out: OutputStream): Unit = readBytesThrough(Internal.transfer(_, out))
}

The Readable doesn’t provide an InputStream directly. It uses a callback to allow data source to perform any necessary cleanup actions after the reading is complete.

For streaming data, the concepts of push and pull are important. It is flexible to use push-based Writable for data destinations and pull-based Readable for data sources. Any pull-based Readable can be trivially used as a push-based Writable, and any method receiving a push-based Writable can also receive a pull-based Readable. This is reflected in type inheritence hierarchy trait Readable extends Writable.

why???

When you use Writable, you want “push” data. When you use Readable, you want “pull” data.