
ElixirConf 2018 - Docker and OTP Friends or Foes - Daniel Azuma
About this video
Daniel Azuma's ElixirConf 2018 talk on whether Docker and OTP conflict. He shows they can coexist and uses Horde and DeltaCrdt, with a live tank-game demo, to keep state alive across container kills.
Why it appears in the archive
Recommended for its practical strategy for preserving state on ephemeral containers, and for the live demo.
AI-generated summary and labels from the source dataset; verify important details against the video. Source & credits.
From the Hacker News discussion
Horde is indeed a really interesting Elixir library. It's based on the CRDT library, DeltaCrdt [0], also by Derek Kraan. He discusses DeltaCrdt here [1] and Horde here [2]. BTW, Horde is a distributed supervisor, i.e. analogous to what regular Elixir OTP supervisors do, except that it can restart processes on different cluster nodes. It also provides a distributed process registry. Daniel Azuma of Google gave a great 39 minute talk at ElixirConf 2018, "Docker and OTP: Friends or Foes?" [3], where he shows Horde (and DeltaCrdt) in action to keep a multi-player tank video game going while the Docker container it's running in gets killed and and the Elixir processes get respawned on another container. The state is persisted across Elixir nodes in other containers in an instance of DeltaCrdt. (Again, the talk is really good -- if the above sounds interesting, you'll want to check it out [3]). [0] [LINK] [1] [LINK] [2] [LINK] [3] [VIDEO]
Read original comment ↗Honestly, in most scenarios, you kill the vm and let it restart with new code (just like you would in a kubernetes pod, and in fact using kubernetes pods with BEAM languages is also a valid strategy). If you must keep some state preserved, there's even ways to do it without erlang's hot code reloading strategies, I would recommend watching Dan Azuma's fantastic video (stick around for the live demo): [VIDEO]
Read original comment ↗You can use Docker + containers with OTP, they are not mutually exclusive. Daniel Azuma gave a talk at the most recent Elixirconf about this subject.[0] [0]: [VIDEO]
Read original comment ↗In the elixir community CRDTs are well known for various HA; AP distributed state strategies, Phoenix Presence uses CRDTs to track processes and broadcast them over pubsub channels. This makes it trivial to represent "user state" in chat rooms, Sophie DiBenedetto describes using it for writing a trivial chat app: [LINK] For something a bit wilder, you can track your state machines across a distributed cluster and be able to pass and preserve state across the cluster using CRDTs, as is done in Dan Azuma's (live) demo of a tank game: [VIDEO] (worth it for the demo at 29m)
Read original comment ↗> you should store the state in the external system It's an old talk, but this one has a strategy for robustly persisting state across ephemeral containers. The example here is running a multiplayer game server while doing code updates etc. using Horde and CRDTs [VIDEO]
Read original comment ↗Can a LiveView process migrate to a different machine/VM? Liveview makes no assumptions about the client connection or even stickiness of client to nodes in a cluster. If you store critical data in the liveview you could be in for some trouble. Always store important stuff into a recoverable, network aware datastore. The nice thing is that liveview is designed to be tolerant to client or client connection failures, you should design to be able to repopulate state sanely in the on_mount call. If you want some sort of ephermal network state that lives on one node and not the database only (e.g. a game), the liveview should connect into that. If you want to store that in a genserver, a strategy like this is doable: [VIDEO]
Read original comment ↗