Communication method for new topologies

[frm]

Is there any preferred method of communication between processes? In particular using the underlying Erlang Node connections vs implementing something using gen_tcp or equivalent.

[michalmuskala]

In general, from what I've heard in "battle stories" it might be advantageous to use the erlang distribution as a control plane and have separate tcp connections for actually sending data.

Right now big messages may clog the pipe and stall the heartbeat messages leading to a disconnection. At Lambda Days Peer (from GRISP) said, though, that they are looking into fixing this one.

Another consideration is that things like partisan don't use the default message sending and implement a completely separate layer for sending messages across processes (there might be even multiple "channels" between nodes). If we want to leverage this, I think we should do some sort of interface for delivering a message instead of using the built-in send.

[chrismccord]

The topology handles the remote pid communication. So our default disterl topology will use send, but others can implement their own mechanisms.

[frm]

In Firenest.Topology we have the following definition: @callback send(t, node, name, message :: term) :: :ok | {:error, term}

But in the case of using something akin to gen_tcp, wouldn't the node have to be an IP? At most, we can assume the topology can internally map nodes to IP addresses or sockets.

[bitwalker]

The topology by definition would be responsible for translating the node name to something meaningful for its transport. The most trivial example could be using DNS A records to get an IP in the case of using :gen_tcp - in fact I think that's more or less how disterl does things today when you get down to the meat of it.