Rewrite ADR29 to be about EventSource/EventSink

Author: ch1bo

docs/adr/2023-11-07_029-streaming-persistence.md

@@ -1,8 +1,8 @@
 ---
 slug: 29
 title: |
-  29. EventServer abstraction for event persistence
-authors: [@cardenaso11, @quantumplation]
+  29. EventSource & EventSink abstractions
+authors: [@cardenaso11, @quantumplation, @ch1bo]
 tags: []
 ---
 
@@ -11,30 +11,62 @@ Draft
 
 ## Context
 
-The Hydra node represents a significant engineering asset, providing layer 1 monitoring, peer to peer consensus, durable persistence, and an isomorphic Cardano ledger. Because of this, it is being eyed as a key building block not just in Hydra based applications, but other protocols as well.
+* The Hydra node represents a significant engineering asset, providing layer 1 monitoring, peer to peer consensus, durable persistence, and an isomorphic Cardano ledger. Because of this, it is being eyed as a key building block not just in Hydra based applications, but other protocols as well.
 
-One remaining difficulty in integrating Hydra into a fully productionalized software stack is the persistence model.  Currently, the Hydra node achieves durability by writing a sequence of "StateChanged" events to disk. If a node is interrupted, upon restarting, it can replay just these events, ignoring their corresponding effects, to recover the same internal state it had when it was interrupted. However, downstream consumers don't have the same luxury.
+* Currently the `hydra-node` uses a very basic persistence mechanism for it's internal `HeadState`, that is saving `StateChanged` events to file on disk and reading them back to load and re-aggregate the `HeadState` upon startup.
+  - Some production setups would benefit from storing these events to a service like Amazon Kinesis data stream instead of local files.
 
-We propose generalizing the persistence mechanism to open the door to a plugin based approach to extending the Hydra node.
+* The `hydra-node` websocket-based API is the only available event stream right now and might not fit all purposes.
+  - See also ADR [3](/adr/3) and [25](/adr/25)
+  - Internally, this is realized as a single `Server` handle which can `sendOutput :: ServerOutput tx -> m ()`
+  - These `ServerOutput`s closely relate to `StateChanged` events and `ClientEffect`s are yielded by the logic layer often together with the `StateChanged`. For example:
+  ```hs
+  onInitialChainAbortTx newChainState committed headId =
+    StateChanged HeadAborted{chainState = newChainState}
+      <> Effects [ClientEffect $ ServerOutput.HeadIsAborted{headId, utxo = fold committed}]
+  ```
+
+* Users of `hydra-node` are interested to add alternative implementations for storing, loading and consuming events of the Hydra protocol.
 
 # Decision
 
-We propose adding a "persistence combinator", which can combine one or more "persistenceIncremental" instances.
+* We create two new interfaces in the `hydra-node` architecture:
+
+  - ```data EventSource e m = EventSource { getEvents :: m [e] }```
+  - ```data EventSink e m = EventSink { putEvent :: e -> m () }```
+
+* We realize our current `PersistenceIncremental` used for persisting `StateChanged` events is both an `EventSource` and an `EventSink`
 
-When appending to the combinator, it will forward the append to each persistence mechanism.
+* We drop the `persistence` from the main handle `HydraNode tx m`, add **one** `EventSource` and allow **many** `EventSinks`
 
-As the node starts up, as part of recovering the node state, it will also ensure "at least once" semantics for each persistence mechanism. It will maintain a notion of a "primary" instance, from which events are loaded, and "secondary instances", which are given the opportunity to re-observe each event.
+```hs
+data HydraNode tx m = HydraNode
+  { -- ...
+  , eventSource :: EventSource (StateChanged tx) m
+  , eventSinks :: [EventSink (StateChanged tx) m]
+  }
+```
 
-Each persistence mechanism will be responsible for it's own durability; for example, it may maintain its own checkpoint, and only re-broadcast the event if its after the checkpoint.
+* The `stepHydraNode` main loop does call `putEvent` on all `eventSinks` in sequence.
+ 
+  - TBD: transaction semantics? what happens if one fails?
 
-The exact implementation details of the above are left unspecified, to allow some flexibility and experimentation on the exact mechanism to realize these goals.
+* The default `hydra-node` main loop does use the file-based `EventSource` and a single file-based `EventSink` (using the same file).
+
+* TBD: As the node starts up, when loading events from the `EventSource`, it will also ensure "at least once" semantics for each `EventSink`.
+
+* We realize that the `EventSource` and `EventSink` handles, as well as their aggregation in `HydraNode` are used as an API by forks of the `hydra-node` and try to minimize changes to it.
 
 ## Consequences
 
-Here are the consequences we forsee from this change:
-- The default operation of the node remains unchanged
-- Projects forking the hydra node have a natively supported mechanism to extend node persistence
-- These extensions can preserve robust "at least once" semantics for each hydra event
-- Sundae Labs will build a "Save transaction batches to S3" proof of concept extension
-- Sundae Labs will build a "Scrolls source" proof of concept integration
-- This may also enable a future ADRs for dynamically loaded plugins without having to fork the Hydra node at all
+* TODO: Naming conflicts / overloaded terms -> should resolve by calling something not "Event"?
+
+* The default operation of the `hyda-node` remains unchanged
+* The API `Server` can be modelled and refactored as an `EventSink`
+  - TBD: Do `Network` and `Chain` parts qualify as `EventSink`s as well or shall those be triggered by `Effect`s still?
+* Projects forking the hydra node have a natively supported mechanism to extend node persistence
+* These extensions can preserve robust "at least once" semantics for each hydra event
+* Sundae Labs can build a "Save transaction batches to S3" proof of concept `EventSink`
+* Sundae Labs can build a "Scrolls source" `EventSink`
+* Sundae Labs can build a "Amazon Kinesis" `EventSource` and `EventSink`
+* Extension points like `EventSource` and `EventSink` could be dynamically loaded as plugins without having to fork `hydra-node` (maybe in a future ADR)