This is a note for learning Akk stream.

Introduction

Stream processing manipulates data sources that are intermittent or unbounded. It features compositional building-blocks and flow control control.

Flow control is a mechanism that control the rate of source events. Reactive stream uses non-blocking back-pressure. Reactive streams standard defines a set of interaction rules (the specificaiton), a set of types (the SPI), and a technology compliance test kit (TCK). Reactive streams is a low level API.

Back pressure means that the flow is constrained by consumer demand via subscription.request(N) where N is the allowed messages.

Akka stream provides a high-level API for streams processing. It implements the Reactive Streams protocol internally.

In Akka, the “steps” for the processing pipeline (the Graph), are referred to as stages. They are often called operators if fluent DSL API methods, such as map, filter, are used.

The term “stage” is more general and also means various fan-in and fan-out shapes. Ther are three main shapes: Source, Flow, Sink. The typical workflow is Source.via(Flow).to(Sink).

Running a stream is performed by materializer that has a type of ActorMaterializer. Use .async operator or async operators to run stream in parallel. The last parameter of all stages is the materialized value – that is a yield value after the stage runs. The runWith method keeps the materialized value. The soruce.to(sink).run() would return the materialized value of the source.

Failure and Rate

A failure is an unexpected event that prevents a stream from functioning. An error is an expected coded-for condition. However, onError event is used for stream failure.

An onError signal is terminal. It can be sent out-of-bounds and is not part of the stream’s data model.

You can use recover[T](pf: PartialFunction[Throwable, T]) to recover a failure, use RestartFlow.withBackoff to restart.

Some operators are rate-aware. For example, conflate conbmines elements from upstream while control rate. extrapolate extrapolates elements from last seen upstream element for faster downstream.