StreamLD Core

Draft Community Group Report,

This version:
https://openfaster.org/streamld/core
Issue Tracking:
GitHub
Editor:
OpenFASTER Editors

Abstract

StreamLD is a protocol for real-time, resumable Linked Data event streaming. This document defines the core event envelope and cursor model that every StreamLD transport binding builds on. Part of the StreamLD module.

← Back to the StreamLD module index

1. Introduction

StreamLD defines an append-only, per-stream event log. Each event is assigned a *sequence number* by the server at write time: a strictly increasing, per-stream integer starting at 1. Sequence numbers are the sole ordering primitive — StreamLD defines no timestamp-based ordering.

2. The Event Envelope

Every StreamLD event is a `streamld:EventEnvelope`. Its normative field list is defined by the SHACL shape at `streamld/model/envelope.ttl` and summarized below.

The table below is generated from streamld/model/envelope.ttl by generator/generate_streamld_docs.py. Do not edit it by hand — edit the SHACL shape and regenerate.

2.1. EventEnvelope

Field Type Required
isSnapshot boolean Yes
payload (node) Yes
sequence integer Yes
streamId string Yes

2.2. ReplicationFrame

Field Type Required
cursor integer Yes
event (node) Yes

2.3. SubscriptionRequest

Field Type Required
after integer No
stream (node) Yes

2.4. GapSignal

Field Type Required
oldestAvailable integer No

3. The Cursor

A StreamLD *cursor* is a sequence number: the position of the last event a subscriber has already received. A subscription request of the form "everything after cursor C" MUST be interpreted as "all events with sequence number greater than C, in ascending sequence order."

3.1. Gap handling

If a server can no longer satisfy a subscription request because the requested cursor has aged out of the stream’s retention window, it MUST respond with a `streamld:GapSignal` rather than silently resuming from an incorrect position. A client that receives a `streamld:GapSignal` MUST perform a full historical read of the stream before re-subscribing live.

4. Conformance

A StreamLD-conformant server MUST:

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

References

Normative References

[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://datatracker.ietf.org/doc/html/rfc2119