This is a note of the Rich Hickey’s talk The Language of the System. Transcript.

The talk is about system thinking in programming language.

Introduction

a programming language sort of defines the world. actual programs need to do interactions with the world.

Language + Runtime

  • Memory model
  • Calling conventions
  • Resource management
  • Coordination
  • Abstraction
  • Types

A system is bigger than a program

  • Stand together
  • Ensemble of programs (services)
  • No global managers as with runtimes/OS
  • Connected

Language

  • Communication: programer -> programer
  • Programming language: programer -> computer
  • System language: program -> program
Program System
Application libss Application as services
Runtime and core libs ?
Language primitives Protocols and formats

The missing part on the right middle could be RMI, DCOM, CORBA etc.

JSON is not extensible. Protocol buffers support new verison but not new types. Other desired features are self-describing, schemas (in or out of band) and generic processors and intermediaries.

System

On wires inside a system, most values are ephermeral, often require names and need to distinguish value from reference.

Most system names are global thus namespace is critical. In programs, most names are local and verbs of function names.

The view of a sysstems as a hierarchy of services that looks like connected objects – that’s the wrong picture. A system is like a flow of production line: input-process-output.

Flow view

  • transform values: values on I/O or from/to storage
  • move values: queues rule.
  • route values: pub/sub.
  • record/remeber values: epochal time model via atoms, refs and agents.
  • keep parts separate
  • flow vs places

It is good to use UUID to name values because value names are not identities.

Failures are common in a system:

  • Partial
  • Independent
  • Timeout
  • Retry, idempotency

Systems are dynamic and have membership, capactiy, discovery, scalability, elasticity etc.

Systematic Approaches

Hoslistic Approach

  • Erlang: good for telecom, not generic
  • fundamental language units are services
  • bespoke communication
  • specified model (actors)

Heterogeneous Approach

  • Cross-language/runtime/platform
  • The language of the system
    • Protocols
    • Formats
    • Simple services

The missing part in system is simple services.

Program System
Application libss Application as services
Runtime and core libs Simple services
Language primitives Protocols and formats

Simple System Services

  • Communication: message queues
  • Coordination: Zoo Keeper
  • Control flow: workflow services
  • Memory: memcache, redis
  • Storage: S3, K/V stores et al
  • more and smaller

The interface for system services is missing.

Systems should use more values. Language should use plain data that is the format sent over wires. The system failure model is the only failure model. Systems are dynamic and data-driven, so should your language.

Call to Action

  • Building simple services rather than language libs
  • Avoid bespoke protocols and formats
  • Consider the abstraction of your service
  • Design to be composed