Module net.box
The net.box
module contains connectors to remote database systems. One
variant, to be discussed later, is for connecting to MySQL or MariaDB or PostgreSQL
(see SQL DBMS modules reference). The other variant, which
is discussed in this section, is for connecting to Tarantool server instances via a
network.
For a quick start with net.box
, refer to the tutorial.
You can call the following methods:
require('net.box')
– to get anet.box
object (namednet_box
for examples in this section)net_box.connect()
– to connect and get a connection object (namedconn
for examples in this section)- other
net.box()
routines, passingconn:
, to execute requests on the remote database system conn:close
– to disconnect
All net.box
methods are fiber-safe, that is, it is safe to share and use the
same connection object across multiple concurrent fibers. In fact that is perhaps
the best programming practice with Tarantool. When multiple fibers use the same
connection, all requests are pipelined through the same network socket, but each
fiber gets back a correct response. Reducing the number of active sockets lowers
the overhead of system calls and increases the overall server performance. However
for some cases a single connection is not enough – for example, when
it is necessary to prioritize requests or to use different authentication IDs.
Most net.box
methods accept the last {options}
argument, which can be:
{timeout=...}
. For example, a method whose last argument is{timeout=1.5}
will stop after 1.5 seconds on the local node, although this does not guarantee that execution will stop on the remote server node.{buffer=...}
. For an example, see the buffer module.{is_async=...}
. For example, a method whose last argument is{is_async=true}
will not wait for the result of a request. See the is_async description.{on_push=... on_push_ctx=...}
. For receiving out-of-band messages. See the box.session.push() description.{return_raw=...}
(since version 2.10.0). If set totrue
, net.box returns response data wrapped in a MsgPack object instead of decoding it to Lua. The default value isfalse
. For an example, see option description below.
The diagram below shows possible connection states and transitions:
![net_states.png](https://www.tarantool.io/en/doc/latest/_images/net_states.png)
On this diagram:
net_box.connect()
method spawns a worker fiber, which will establish the connection and start the state machine.- The state machine goes to the
initial
state. - Authentication and schema upload.
It is possible later on to re-enter the
fetch_schema
state fromactive
to trigger schema reload. - The state changes to the
graceful_shutdown
state when the state machine receives a box.shutdown event from the remote host (see conn:on_shutdown()). Once all pending requests are completed, the state machine switches to theerror
(error_reconnect
) state. - The transport goes to the
error
state in case of an error. It can happen, for example, if the server closed the connection. If thereconnect_after
option is set, instead of the ‘error’ state, the transport goes to theerror_reconnect
state. conn.close()
method sets the state toclosed
and kills the worker. If the transport is already in theerror
state,close()
does nothing.
Below is a list of all net.box
functions.
Name | Use |
---|---|
net_box.connect() net_box.new() net_box.self |
Create a connection |
conn:ping() | Execute a PING command |
conn:wait_connected() | Wait for a connection to be active or closed |
conn:is_connected() | Check if a connection is active or closed |
conn:wait_state() | Wait for a target state |
conn:close() | Close a connection |
conn.space.space-name:select{field-value} | Select one or more tuples |
conn.space.space-name:get{field-value} | Select a tuple |
conn.space.space-name:insert{field-value} | Insert a tuple |
conn.space.space-name:replace{field-value} | Insert or replace a tuple |
conn.space.space-name:update{field-value} | Update a tuple |
conn.space.space-name:upsert{field-value} | Update a tuple |
conn.space.space-name:delete{field-value} | Delete a tuple |
conn:eval() | Evaluate the expression in a string and execute it |
conn:call() | Call a stored procedure |
conn:watch() | Subscribe to events broadcast by a remote host |
conn:on_connect() | Define a connect trigger |
conn:on_disconnect() | Define a disconnect trigger |
conn:on_shutdown() | Define a shutdown trigger |
conn:on_schema_reload() | Define a trigger when schema is modified |
conn:new_stream() | Create a stream |
stream:begin() | Begin a stream transaction |
stream:commit() | Commit a stream transaction |
stream:rollback() | Rollback a stream transaction |
-
net_box.
connect
(URI[, {option[s]}])¶ Create a new connection. The connection is established on demand, at the time of the first request. It can be re-established automatically after a disconnect (see
reconnect_after
option below). The returnedconn
object supports methods for making remote requests, such as select, update or delete.Parameters: - URI (
string
) – the URI of the target for the connection - options –
the supported options are shown below:
user/password
: two options to connect to a remote host other than through URI. For example, instead ofconnect('username:userpassword@localhost:3301')
you can writeconnect('localhost:3301', {user = 'username', password='userpassword'})
.wait_connected
: a connection timeout. By default, the connection is blocked until the connection is established, but if you specifywait_connected=false
, the connection returns immediately. If you specify this timeout, it will wait before returning (wait_connected=1.5
makes it wait at most 1.5 seconds).Note
If
reconnect_after
is greater than zero, thenwait_connected
ignores transient failures. The wait completes once the connection is established or is closed explicitly.reconnect_after
: a number of seconds to wait before reconnecting. The default value, as with the otherconnect
options, isnil
. Ifreconnect_after
is greater than zero, then anet.box
instance will attempt to reconnect if a connection is lost or a connection attempt fails. This makes transient network failures transparent to the application. Reconnection happens automatically in the background, so requests that initially fail due to connection drops fail, are transparently retried. The number of retries is unlimited, connection retries are made after any specified interval (for example,reconnect_after=5
means that reconnect attempts are made every 5 seconds). When a connection is explicitly closed or when the Lua garbage collector removes it, then reconnect attempts stop.call_16
: [since 1.7.2] a new binary protocol command for CALL innet.box
connections by default. The new CALL is not backward compatible with previous versions. It no longer restricts a function to returning an array of tuples and allows returning an arbitrary MsgPack/JSON result, including scalars, nil and void (nothing). The old CALL is left intact for backward compatibility. It will not be present in the next major release. All programming language drivers will gradually be switched to the new CALL. To connect to a Tarantool instance that uses the old CALL, specifycall_16=true
.connect_timeout
: a number of seconds to wait before returning “error: Connection timed out”.fetch_schema
: a boolean option that controls fetching schema changes from the server. Default:true
. If you don’t operate with remote spaces, for example, run onlycall
oreval
, setfetch_schema
tofalse
to avoid fetching schema changes which is not needed in this case.Important
In connections with
fetch_schema == false
, remote spaces are unavailable and the on_schema_reload triggers don’t work.required_protocol_version
: a minimum version of the IPROTO protocol supported by the server. If the version of the IPROTO protocol supported by the server is lower than specified, the connection will fail with an error message. Withrequired_protocol_version = 1
, all connections fail where the IPROTO protocol version is lower than1
.required_protocol_features
: specified IPROTO protocol features supported by the server. You can specify one or morenet.box
features from the table below. If the server does not support the specified features, the connection will fail with an error message. Withrequired_protocol_features = {'transactions'}
, all connections fail where the server hastransactions: false
.
net.box feature Use IPROTO feature ID IPROTO versions supporting the feature streams
Requires streams support on the server IPROTO_FEATURE_STREAMS 1 and newer transactions
Requires transactions support on the server IPROTO_FEATURE_TRANSACTIONS 1 and newer error_extension
Requires support for MP_ERROR MsgPack extension on the server IPROTO_FEATURE_ERROR_EXTENSION 2 and newer watchers
Requires remote watchers support on the server IPROTO_FEATURE_WATCHERS 3 and newer To learn more about IPROTO features, see IPROTO_ID and the IPROTO_FEATURES key.
Return: conn object Rtype: userdata Examples:
net_box = require('net.box') conn = net_box.connect('localhost:3301') conn = net_box.connect('127.0.0.1:3302', {wait_connected = false}) conn = net_box.connect('127.0.0.1:3303', {reconnect_after = 5, call_16 = true}) conn = net_box.connect('127.0.0.1:3304', {required_protocol_version = 4, required_protocol_features = {'transactions', 'streams'}, })
- URI (
-
net_box.
new
(URI[, {option[s]}])¶ new()
is a synonym forconnect()
. It is retained for backward compatibility. For more information, see the description of net_box.connect().
-
object
self
¶ For a local Tarantool server, there is a pre-created always-established connection object named
net_box.self
. Its purpose is to make polymorphic use of thenet_box
API easier. Thereforeconn = net_box.connect('localhost:3301')
can be replaced byconn = net_box.self
.However, there is an important difference between the embedded connection and a remote one:
- With the embedded connection, requests which do not modify data do not yield. When using a remote connection, due to the implicit rules any request can yield, and the database state may have changed by the time it regains control.
- All the options passed to a request (as
is_async
,on_push
,timeout
) will be ignored.
-
object
conn
¶ -
conn:
ping
([options])¶ Execute a PING command.
Parameters: - options (
table
) – the supported option istimeout=seconds
Return: true on success, false on error
Rtype: boolean
Example:
net_box.self:ping({timeout = 0.5})
- options (
-
conn:
wait_connected
([timeout])¶ Wait for connection to be active or closed.
Parameters: - timeout (
number
) – in seconds
Return: true when connected, false on failure.
Rtype: boolean
Example:
net_box.self:wait_connected()
- timeout (
-
conn:
is_connected
()¶ Show whether connection is active or closed.
Return: true if connected, false on failure. Rtype: boolean Example:
net_box.self:is_connected()
-
conn:
wait_state
(state[s][, timeout])¶ [since 1.7.2] Wait for a target state.
Parameters: - states (
string
) – target states - timeout (
number
) – in seconds
Return: true when a target state is reached, false on timeout or connection closure
Rtype: boolean
Examples:
-- wait infinitely for 'active' state: conn:wait_state('active') -- wait for 1.5 secs at most: conn:wait_state('active', 1.5) -- wait infinitely for either `active` or `fetch_schema` state: conn:wait_state({active=true, fetch_schema=true})
- states (
-
conn:
close
()¶ Close a connection.
Connection objects are destroyed by the Lua garbage collector, just like any other objects in Lua, so an explicit destruction is not mandatory. However, since close() is a system call, it is good programming practice to close a connection explicitly when it is no longer needed, to avoid lengthy stalls of the garbage collector.
Example:
conn:close()
-
conn.space.<space-name>:select({field-value, ...} [, {options}])
conn.space.space-name:select
({...})
is the remote-call equivalent of the local callbox.space.space-name:select
{...}
(see details). For an additional option see Module buffer and skip-header.Example:
conn.space.testspace:select({1,'B'}, {timeout=1})
Note
Due to the implicit yield rules a local
box.space.space-name:select
{...}
does not yield, but a remoteconn.space.space-name:select
{...}
call does yield, so global variables or database tuples data may change when a remoteconn.space.space-name:select
{...}
occurs.
-
conn.space.<space-name>:get({field-value, ...} [, {options}])
conn.space.space-name:get(...)
is the remote-call equivalent of the local callbox.space.space-name:get(...)
(see details).Example:
conn.space.testspace:get({1})
-
conn.space.<space-name>:insert({field-value, ...} [, {options}])
conn.space.space-name:insert(...)
is the remote-call equivalent of the local callbox.space.space-name:insert(...)
(see details). For an additional option see Module buffer and skip-header.Example:
conn.space.testspace:insert({2,3,4,5}, {timeout=1.1})
-
conn.space.<space-name>:replace({field-value, ...} [, {options}])
conn.space.space-name:replace(...)
is the remote-call equivalent of the local callbox.space.space-name:replace(...)
(see details). For an additional option see Module buffer and skip-header.Example:
conn.space.testspace:replace({5,6,7,8})
-
conn.space.<space-name>:update({field-value, ...} [, {options}])
conn.space.space-name:update(...)
is the remote-call equivalent of the local callbox.space.space-name:update(...)
(see details). For an additional option see Module buffer and skip-header.Example:
conn.space.Q:update({1},{{'=',2,5}}, {timeout=0})
-
conn.space.<space-name>:upsert({field-value, ...} [, {options}])
conn.space.space-name:upsert(...)
is the remote-call equivalent of the local callbox.space.space-name:upsert(...)
. (see details). For an additional option see Module buffer and skip-header.
-
conn.space.<space-name>:delete({field-value, ...} [, {options}])
conn.space.space-name:delete(...)
is the remote-call equivalent of the local callbox.space.space-name:delete(...)
(see details). For an additional option see Module buffer and skip-header.
-
conn:
eval
(Lua-string[, {arguments}[, {options}]])¶ conn:eval(Lua-string)
evaluates and executes the expression in Lua-string, which may be any statement or series of statements. An execute privilege is required; if the user does not have it, an administrator may grant it withbox.schema.user.grant(username, 'execute', 'universe')
.To ensure that the return from
conn:eval
is whatever the Lua expression returns, begin the Lua-string with the word “return”.Examples:
tarantool> --Lua-string tarantool> conn:eval('function f5() return 5+5 end; return f5();') --- - 10 ... tarantool> --Lua-string, {arguments} tarantool> conn:eval('return ...', {1,2,{3,'x'}}) --- - 1 - 2 - [3, 'x'] ... tarantool> --Lua-string, {arguments}, {options} tarantool> conn:eval('return {nil,5}', {}, {timeout=0.1}) --- - [null, 5] ...
-
conn:
call
(function-name[, {arguments}[, {options}]])¶ conn:call('func', {'1', '2', '3'})
is the remote-call equivalent offunc('1', '2', '3')
. That is,conn:call
is a remote stored-procedure call. The return fromconn:call
is whatever the function returns.Limitation: the called function cannot return a function, for example if
func2
is defined asfunction func2 () return func end
thenconn:call(func2)
will return “error: unsupported Lua type ‘function’”.Examples:
tarantool> -- create 2 functions with conn:eval() tarantool> conn:eval('function f1() return 5+5 end;') tarantool> conn:eval('function f2(x,y) return x,y end;') tarantool> -- call first function with no parameters and no options tarantool> conn:call('f1') --- - 10 ... tarantool> -- call second function with two parameters and one option tarantool> conn:call('f2',{1,'B'},{timeout=99}) --- - 1 - B ...
-
conn:
watch
(key, func)¶ Subscribe to events broadcast by a remote host.
Parameters: - key (
string
) – a key name of an event to subscribe to - func (
function
) – a callback to invoke when the key value is updated
Return: a watcher handle. The handle consists of one method –
unregister()
, which unregisters the watcher.To read more about watchers, see the Functions for watchers section.
The method has the same syntax as the box.watch() function, which is used for subscribing to events locally.
Watchers survive reconnection (see the
reconnect_after
connection option). All registered watchers are automatically resubscribed when the connection is reestablished.If a remote host supports watchers, the
watchers
key will be set in the connectionpeer_protocol_features
. For details, check the net.box features table.Note
Keep in mind that garbage collection of a watcher handle doesn’t lead to the watcher’s destruction. In this case, the watcher remains registered. It is okay to discard the result of
watch
function if the watcher will never be unregistered.Example 1:
Server:
-- Broadcast value 42 for the 'foo' key. box.broadcast('foo', 42)
Client:
conn = net.box.connect(URI) local log = require('log') -- Subscribe to updates of the 'foo' key. w = conn:watch('foo', function(key, value) assert(key == 'foo') log.info("The box.id value is '%d'", value) end)
If you don’t need the watcher anymore, you can unregister it using the command below:
w:unregister()
Example 2:
The
net.box module
provides the ability to monitor updates of a configuration stored in a Tarantool-based configuration storage by watching path or prefix changes. In the example below, conn:watch() is used to monitor updates of a configuration stored by the/myapp/config/all
path:net_box = require('net.box') local conn = net_box.connect('127.0.0.1:4401') local log = require('log') conn:watch('config.storage:/myapp/config/all', function(key, value) log.info("Configuration stored by the '/myapp/config/all' key is changed") end)
You can find the full example here: config_storage.
- key (
-
conn:
request
(... {is_async=...})¶ {is_async=true|false}
is an option which is applicable for allnet_box
requests includingconn:call
,conn:eval
, and theconn.space.space-name
requests.The default is
is_async=false
, meaning requests are synchronous for the fiber. The fiber is blocked, waiting until there is a reply to the request or until timeout expires. Before Tarantool version 1.10, the only way to make asynchronous requests was to put them in separate fibers.The non-default is
is_async=true
, meaning requests are asynchronous for the fiber. The request causes a yield but there is no waiting. The immediate return is not the result of the request, instead it is an object that the calling program can use later to get the result of the request.This immediately-returned object, which we’ll call “future”, has its own methods:
future:is_ready()
which will return true when the result of the request is available,future:result()
to get the result of the request (returns the response or nil in case it’s not ready yet or there has been an error),future:wait_result(timeout)
to wait until the result of the request is available and then get it, or throw an error if there is no result after the timeout exceeded,future:discard()
to abandon the object.
Typically a user would say
future=request-name(...{is_async=true})
, then either loop checkingfuture:is_ready()
until it is true and then sayrequest_result=future:result()
, or sayrequest_result=future:wait_result(...)
. Alternatively the client could check for “out-of-band” messages from the server by callingpairs()
in a loop – see box.session.push().A user would say
future:discard()
to make a connection forget about the response – if a response for a discarded object is received then it will be ignored, so that the size of the requests table will be reduced and other requests will be faster.Examples:
-- Insert a tuple asynchronously -- tarantool> future = conn.space.bands:insert({10, 'Queen', 1970}, {is_async=true}) --- ... tarantool> future:is_ready() --- - true ... tarantool> future:result() --- - [10, 'Queen', 1970] ... -- Iterate through a space with 10 records to get data in chunks of 3 records -- tarantool> while true do future = conn.space.bands:select({}, {limit=3, after=position, fetch_pos=true, is_async=true}) result = future:wait_result() tuples = result[1] position = result[2] if position == nil then break end print('Chunk size: '..#tuples) end Chunk size: 3 Chunk size: 3 Chunk size: 3 Chunk size: 1 --- ...
Typically
{is_async=true}
is used only if the load is large (more than 100,000 requests per second) and latency is large (more than 1 second), or when it is necessary to send multiple requests in parallel then collect responses (sometimes called a “map-reduce” scenario).Note
Although the final result of an async request is the same as the result of a sync request, it is structured differently: as a table, instead of as the unpacked values.
-
conn:
request
(... {return_raw=...}) {return_raw=true}
is ignored for:- Methods that return
nil
:begin
,commit
,rollback
,upsert
,prepare
. index.count
(returns number).
For
execute
, the option is applied only to data (rows
). Metadata is decoded even if{return_raw=true}
.Example:
local c = require('net.box').connect(uri) local mp = c.eval('eval ...', {1, 2, 3}, {return_raw = true}) mp:decode() -- {1, 2, 3}
The option can be useful if you want to pass a response through without decoding or with partial decoding. The usage of MsgPack object can reduce pressure on the Lua garbage collector.
- Methods that return
-
conn:
new_stream
([options])¶ Create a stream.
Example:
-- Start a server to create a new stream local conn = net_box.connect('localhost:3301') local conn_space = conn.space.test local stream = conn:new_stream() local stream_space = stream.space.test
-
-
object
stream
¶ -
stream:
begin
([txn_isolation])¶ Begin a stream transaction. Instead of the direct method, you can also use the
call
,eval
or execute methods with SQL transaction.Parameters: - txn_isolation – transaction isolation level
-
stream:
commit
()¶ Commit a stream transaction. Instead of the direct method, you can also use the
call
,eval
or execute methods with SQL transaction.Examples:
-- Begin stream transaction stream:begin() -- In the previously created ``accounts`` space with the primary key ``test``, modify the fields 2 and 3 stream.space.accounts:update(test_1, {{'-', 2, 370}, {'+', 3, 100}}) -- Commit stream transaction stream:commit()
-
stream:
rollback
()¶ Rollback a stream transaction. Instead of the direct method, you can also use the
call
,eval
or execute methods with SQL transaction.Example:
-- Test rollback for memtx space space:replace({1}) -- Select return tuple that was previously inserted, because this select belongs to stream transaction space:select({}) stream:rollback() -- Select is empty, stream transaction rollback space:select({})
-
With the net.box
module, you can use the following
triggers:
-
conn:
on_connect
([trigger-function[, old-trigger-function]])¶ Define a trigger for execution when a new connection is established, and authentication and schema fetch are completed due to an event such as
net_box.connect
.If a trigger function issues
net_box
requests, they must be asynchronous ({is_async = true}
). An attempt to wait for request completion withfuture:pairs()
orfuture:wait_result()
in the trigger function will result in an error.If the trigger execution fails and an exception happens, the connection’s state changes to ‘error’. In this case, the connection is terminated, regardless of the
reconnect_after
option’s value. Can be called as many times as reconnection happens, ifreconnect_after
is greater than zero.Parameters: - trigger-function (
function
) – the trigger function. Takes theconn
object as the first argument. - old-trigger-function (
function
) – an existing trigger function to replace withtrigger-function
Return: nil or function pointer
- trigger-function (
-
conn:
on_disconnect
([trigger-function[, old-trigger-function]])¶ Define a trigger for execution after a connection is closed. If the trigger function causes an error, the error is logged but otherwise is ignored. Execution stops after a connection is explicitly closed, or once the Lua garbage collector removes it.
Parameters: - trigger-function (
function
) – the trigger function. Takes theconn
object as the first argument - old-trigger-function (
function
) – an existing trigger function to replace withtrigger-function
Return: nil or function pointer
- trigger-function (
-
conn:
on_shutdown
([trigger-function[, old-trigger-function]])¶ Define a trigger for shutdown when a box.shutdown event is received.
The trigger starts in a new fiber. While the
on_shutdown()
trigger is running, the connection stays active. It means that the trigger callback is allowed to send new requests.After the trigger return, the
net.box
connection goes to thegraceful_shutdown
state (check the state diagram for details). In this state, no new requests are allowed. The connection waits for all pending requests to be completed.Once all in-progress requests have been processed, the connection is closed. The state changes to
error
orerror_reconnect
(if thereconnect_after
option is defined).Servers that do not support the
box.shutdown
event or IPROTO_WATCH just close the connection abruptly. In this case, theon_shutdown()
trigger is not executed.Parameters: - trigger-function (
function
) – the trigger function. Takes theconn
object as the first argument - old-trigger-function (
function
) – an existing trigger function to replace withtrigger-function
Return: nil or function pointer
- trigger-function (
-
conn:
on_schema_reload
([trigger-function[, old-trigger-function]])¶ Define a trigger executed when some operation has been performed on the remote server after schema has been updated. So, if a server request fails due to a schema version mismatch error, schema reload is triggered.
If a trigger function issues
net_box
requests, they must be asynchronous ({is_async = true}
). An attempt to wait for request completion withfuture:pairs()
orfuture:wait_result()
in the trigger function will result in an error.Parameters: - trigger-function (
function
) – the trigger function. Takes theconn
object as the first argument - old-trigger-function (
function
) – an existing trigger function to replace withtrigger-function
Return: nil or function pointer
Note
If the parameters are
(nil, old-trigger-function)
, then the old trigger is deleted.If both parameters are omitted, then the response is a list of existing trigger functions.
Find the detailed information about triggers in the triggers section.
- trigger-function (