Module membership
This module is a membership
library for Tarantool based on a gossip protocol.
This library builds a mesh from multiple Tarantool instances. The mesh monitors itself, helps members discover everyone else in the group and get notified about their status changes with low latency. It is built upon the ideas from Consul or, more precisely, the SWIM algorithm.
The membership
module works over UDP protocol and can operate even before
the box.cfg
initialization.
A member is represented by the table with the following fields:
uri
(string) is a Uniform Resource Identifier.status
(string) is a string that takes one of the values below.alive
: a member that replies to ping-messages isalive
and well.suspect
: if any member in the group cannot get a reply from any other member, the first member asks three otheralive
members to send a ping-message to the member in question. If there is no response, the latter becomes asuspect
.dead
: asuspect
becomesdead
after a timeout.left
: a member gets theleft
status after executing the leave() function.Note
The gossip protocol guarantees that every member in the group becomes aware of any status change in two communication cycles.
incarnation
(number) is a value incremented every time the instance becomes asuspect
,dead
, or updates its payload.payload
(table) is auxiliary data that can be used by various modules.timestamp
(number) is a value offiber.time64()
which:- corresponds to the last update of
status
orincarnation
; - is always local;
- does not depend on other members’ clock setting.
- corresponds to the last update of
Below is an example of the table:
tarantool> membership.myself()
---
uri: localhost:33001
status: alive
incarnation: 1
payload:
uuid: 2d00c500-2570-4019-bfcc-ab25e5096b73
timestamp: 1522427330993752
...
Below is a list of membership
’s common, encryption, subscription
functions, and options.
Name | Use |
---|---|
Common functions | |
init(advertise_host, port) | Initialize the membership
module. |
myself() | Get the member data structure of the current instance. |
get_member(uri) | Get the member data structure for a given URI. |
members() | Obtain a table with all members known to the current instance. |
pairs() | Shorthand for
pairs(membership.members()) . |
add_member(uri) | Add a member to the group. |
probe_uri(uri) | Check if the member is in the group. |
broadcast() | Discover members in LAN by sending a UDP broadcast message. |
set_payload(key, value) | Update myself().payload and
disseminate it. |
leave() | Gracefully leave the group. |
is_encrypted() | Check if encryption is enabled. |
Encryption functions | |
set_encryption_key(key) | Set the key for low-level message encryption. |
get_encryption_key() | Retrieve the encryption key in use. |
Subscription functions | |
subscribe() | Subscribe for the members table updates. |
unsubscribe() | Remove the subscription. |
Options | |
PROTOCOL_PERIOD_SECONDS | Direct ping period. |
ACK_TIMEOUT_SECONDS | ACK message wait time. |
ANTI_ENTROPY_PERIOD_SECONDS | Anti-entropy synchronization period. |
SUSPECT_TIMEOUT_SECONDS | Timeout to mark a suspect
dead . |
NUM_FAILURE_DETECTION_SUBGROUPS | Number of members to ping a
suspect indirectly. |
Common functions:
-
membership.
init
(advertise_host, port)¶ Initialize the
membership
module. This binds a UDP socket to0.0.0.0:<port>
, sets theadvertise_uri
parameter to<advertise_host>:<port>
, andincarnation
to1
.The
init()
function can be called several times, the old socket will be closed and a new one opened.If the
advertise_uri
changes during the nextinit()
, the old URI is consideredDEAD
. In order to leave the group gracefully, use the leave() function.Parameters: - advertise_host (
string
) – a hostname or IP address to advertise to other members - port (
number
) – a UDP port to bind
Return: true
Rtype: boolean
Raises: socket bind error
- advertise_host (
-
membership.
myself
()¶ Return: the member data structure of the current instance. Rtype: table
-
membership.
get_member
(uri)¶ Parameters: - uri (
string
) – the given member’sadvertise_uri
Return: the member data structure of the instance with the given URI.
Rtype: - uri (
-
membership.
members
()¶ Obtain all members known to the current instance.
Editing this table has no effect.
Return: a table with URIs as keys and corresponding member data structures as values. Rtype: table
-
membership.
pairs
()¶ A shorthand for
pairs(membership.members())
.Return: Lua iterator It can be used in the following way:
for uri, member in membership.pairs() -- do something end
-
membership.
add_member
(uri)¶ Add a member with the given URI to the group and propagate this event to other members. Adding a member to a single instance is enough as everybody else in the group will receive the update with time. It does not matter who adds whom.
Parameters: - uri (
string
) – theadvertise_uri
of the member to add
Return: true
ornil
in case of an errorRtype: boolean
Raises: parse error if the URI cannot be parsed
- uri (
-
membership.
probe_uri
(uri)¶ Send a message to a member to make sure it is in the group. If the member is
alive
but not in the group, it is added. If it already is in the group, nothing happens.Parameters: - uri (
string
) – theadvertise_uri
of the member to ping
Return: true
if the member responds within 0.2 seconds, otherwiseno response
Rtype: boolean
Raises: ping was not sent
if the hostname could not be resolved- uri (
-
membership.
broadcast
()¶ Discover members in local network by sending a UDP broadcast message to all networks discovered by a
getifaddrs()
C call.Return: true
if broadcast was sent,false
ifgetaddrinfo()
fails.Rtype: boolean
-
membership.
set_payload
(key, value)¶ Update
myself().payload
and disseminate it along with the member status.Increments
incarnation
.Parameters: - key (
string
) – a key to set in payload table - value – auxiliary data
Return: true
Rtype: boolean
- key (
-
membership.
leave
()¶ Gracefully leave the
membership
group. The node will be marked with theleft
status and no other members will ever try to reconnect it.Return: true
Rtype: boolean
-
membership.
is_encrypted
()¶ Return: true
if encryption is enabled,false
otherwise.Rtype: boolean
Encryption functions:
-
membership.
set_encryption_key
(key)¶ Set the key used for low-level message encryption. The key is either trimmed or padded automatically to be exactly 32 bytes. If the
key
value isnil
, the encryption is disabled.The encryption is handled by the
crypto.cipher.aes256.cbc
Tarantool module.For proper communication, all members must be configured to use the same encryption key. Otherwise, members report either
dead
ornon-decryptable
in their status.Parameters: - key (
string
) – encryption key
Return: nil
.- key (
-
membership.
get_encryption_key
()¶ Retrieve the encryption key that is currently in use.
Return: encryption key or nil
if the encryption is disabled.Rtype: string
Subscription functions:
-
membership.
subscribe
()¶ Subscribe for updates in the members table.
Return: a fiber.cond
object broadcasted whenever the members table changes.Rtype: object
-
membership.
unsubscribe
(cond)¶ Remove subscription on
cond
obtained by the subscribe() function.The
cond
’s validity is not checked.Parameters: - cond – the
fiber.cond
object obtained from subscribe()
Return: nil
.- cond – the
Below is a list of membership
options. They can be set as follows:
options = require('membership.options')
options.<option> = <value>
-
options.PROTOCOL_PERIOD_SECONDS
¶
Period of sending direct pings. Denoted as
T'
in the SWIM protocol.
-
options.ACK_TIMEOUT_SECONDS
¶
Time to wait for ACK message after a ping. If a member is late to reply, the indirect ping algorithm is invoked.
-
options.ANTI_ENTROPY_PERIOD_SECONDS
¶
Period to perform the anti-entropy synchronization algorithm of the SWIM protocol.
-
options.SUSPECT_TIMEOUT_SECONDS
¶
Timeout to mark
suspect
members asdead
.
-
options.NUM_FAILURE_DETECTION_SUBGROUPS
¶
Number of members to try pinging a
suspect
indirectly. Denoted ask
in the SWIM protocol.