Tarantool 1.10

Release 1.10.4

Release type: stable (lts). Release date: 2019-09-26. Tag: 1-10-4.

Overview

1.10.4 is the next stable (lts) release in the 1.10 series. The label ‘stable’ means we have had systems running in production without known crashes, bad results or other showstopper bugs for quite a while now.

This release resolves about 50 issues since 1.10.3.

Compatibility

Tarantool 1.10.x is backward compatible with Tarantool 1.9.x in binary data layout, client-server protocol and replication protocol. Please upgrade using the box.schema.upgrade() procedure to unlock all the new features of the 1.10.x series when migrating from 1.9 version.

Functionality added or changed

(Engines) Improve dump start/stop logging. When initiating memory dump, print
how much memory is going to be dumped, expected dump rate, ETA, and the recent write rate. Upon dump completion, print observed dump rate in addition to dump size and duration.
(Engines) Look up key in reader thread. If a key isn’t found in the tuple cache,
we fetch it from a run file. In this case disk read and page decompression is done by a reader thread, however key lookup in the fetched page is still performed by the TX thread. Since pages are immutable, this could as well be done by the reader thread, which would allow us to save some precious CPU cycles for TX (gh-4257).
(Core) Improve box.stat.net
(gh-4150).
(Core) Add idle to downstream status in box.info.
When a relay sends a row it updates last_row_time value with the current time. When box.info() is called, idle is set to current_time - last_row_time.

(Replication) Print corrupted rows on decoding error.

Improve row printing to log. Print the header row by row, 16 bytes in a row, and format output to match xxd output:

[001] 2019-04-05 18:22:46.679 [11859] iproto V> Got a corrupted row:
[001] 2019-04-05 18:22:46.679 [11859] iproto V> 00000000: A3 02 D6 5A E4 D9 E7 68 A1 53 8D 53 60 5F 20 3F
[001] 2019-04-05 18:22:46.679 [11859] iproto V> 00000010: D8 E2 D6 E2 A3 02 D6 5A E4 D9 E7 68 A1 53 8D 53

(Lua) Add type of operation to space trigger parameters.

For example, a trigger function may now look like this:

function before_replace_trig(old, new, space_name, op_type)
    if op_type == 'INSERT' then
        return old
    else
        return new
    end
end

(gh-4099).

(Lua) Add debug.sourcefile() and debug.sourcedir() helpers
(and debug.__file__ and debug.__dir__ shortcuts) to determine the location of a current Lua source file. Part of (gh-4193).
(HTTP client) Add max_total_connections option in addition to
max_connections to allow more fine-grained tuning of libcurl connection cache. Don’t restrict the total connections` with a constant value by default, but use libcurl’s default, which scales the threshold according to easy handles count (gh-3945).

Bugs fixed

(Vinyl) Fix assertion failure in vy_tx_handle_deferred_delete
(gh-4294).
(Vinyl) Don’t purge deleted runs from vylog on compaction. Cherry-picked from (gh-4218).
(Vinyl) Don’t throttle DDL
(gh-4238).
(Vinyl) Fix deferred DELETE statement lost on commit
(gh-4248).
(Vinyl) Fix assertion while recovering dumped statement
(gh-4222).
(Vinyl) Reset dump watermark after updating memory limit
(gh-3864).
(Vinyl) Be pessimistic about write rate when setting dump watermark
(gh-4166).
(Vinyl) Fix crash if space is dropped while space.get is reading from it
(gh-4109).
(Vinyl) Fix crash during index build
(gh-4152).
(Vinyl) Don’t compress L1 runs
(gh-2389).
(Vinyl) Account statements skipped on read.
(Vinyl) Take into account primary key lookup in latency accounting.
(Vinyl) Fix vy_range_update_compaction_priority hang.
(Vinyl) Free region on vylog commit instead of resetting it and clean up region after allocating surrogate statement.
(Vinyl) Increase even more the open file limit in systemd unit file.
(Vinyl) Increment min range size to 128MB
(Memtx) Cancel checkpoint thread at exit
(gh-4170).
(Core) Fix crash for update with empty tuple
(gh-4041).
(Core) Fix use-after-free in space_truncate
(gh-4093).
(Core) Fix error while altering index with sequence
(gh-4214).
(Core) Detect a new invalid json path case
(gh-4419).
(Core) Fix empty password authentication
(gh-4327).
(Core) Fix txn::sub_stmt_begin array size.
(Core) Account index.pairs in box.stat.SELECT().
(Replication) Disallow bootstrap of read-only masters
(gh-4321).
(Replication) Enter orphan mode on manual replication configuration change
(gh-4424).
(Replication) Set last_row_time to now in relay_new and relay_start. PR 4431.
(Replication) Stop relay on subscribe error
(gh-4399).
(Replication) Init coio watcher before join/subscribe
(gh-4110).
(Replication) Allow to change instance id during join
(gh-4107).
(Replication) Fix garbage collection logic.
(Replication) Revert packet boundary checking for iproto.
(Replication) Do not abort replication on ER_UNKNOWN_REPLICA.
(Replication) Reduce effects of input buffer fragmentation on large cfg.readahead.
(Replication) Fix upgrade from 1.7 (it doesn’t recognize IPROTO_VOTE request type).
(Replication) Fix memory leak in call / eval in the case when a transaction
is not committed (gh-4388).
(Lua) Fix fio.mktree() error reporting
(gh-4044).
(Lua) Fix segfault on ffi.C_say() without filename
(gh-4336).
(Lua) Fix segfault on json.encode() on a recursive table
(gh-4366).
(Lua) Fix pwd.getpwall() and pwd.getgrall() hang on CentOS 6
and FreeBSD 12 (gh-4447, gh-4428).
(Lua) Fix a segfault during initialization of a cipher from crypto module
(gh-4223).
(HTTP client) Reduce stack consumption during waiting for a DNS resolving result
(gh-4179).
(HTTP client) Increase max outgoing header size to 8 KiB
(gh-3959).
(HTTP client) Verify “headers” option stronger
(gh-4281, gh-3679).
(HTTP client) Use bundled libcurl rather than system-wide by default
(gh-4318, gh-4180, gh-4288, gh-4389, gh-4397).
(HTTP client) This closes several known problems that were fixed in recent
libcurl versions, including segfaults, hangs, memory leaks and performance problems.
(LuaJIT) Fix overflow of snapshot map offset
Part of (gh-4171).
(LuaJIT) Fix rechaining of pseudo-resurrected string keys
Part of (gh-4171).
(LuaJIT) Fix fold machinery misbehaves
(gh-4376).
(LuaJIT) Fix for debug.getinfo(1,'>S')
(gh-3833).
(LuaJIT) Fix string.find recording
(gh-4476).
(LuaJIT) Fixed a segfault when unsinking 64-bit pointers.
(Misc) Increase even more the open file limit in systemd unit file.
(Misc) Raise error in tarantoolctl when box.cfg() isn’t called
(gh-3953).
(Misc) Support systemd’s NOTIFY_SOCKET on OS X
(gh-4436).
(Misc) Fix coio_getaddrinfo() when 0 timeout is passed
(affects netbox’s connect_timeout) (gh-4209).
(Misc) Fix coio_do_copyfile() to perform truncate of destination
(affects fio.copyfile()) (gh-4181).
(Misc) Make hints in coio_getaddrinfo() optional.
(Misc) Validate msgpack.decode() cdata size argument
(gh-4224).
(Misc) Fix linking with static openssl library
(gh-4437).

Deprecations

(Core) Deprecate rows_per_wal in favor of wal_max_size.
Part of (gh-3762).

Release 1.10.3

Release type: stable (lts). Release date: 2019-04-01. Tag: 1-10-3.

Release: v. 1.10.3.

Overview

1.10.3 is the next stable (lts) release in the 1.10 series. The label ‘stable’ means we have had systems running in production without known crashes, bad results or other showstopper bugs for quite a while now.

This release resolves 69 issues since 1.10.2.

Compatibility

Functionality added or changed

(Engines) Randomize vinyl index compaction
(gh-3944).
(Engines) Throttle tx thread if compaction doesn’t keep up with dumps
(gh-3721).
(Engines) Do not apply run_count_per_level to the last level
(gh-3657).
(Server) Report the number of active iproto connections
(gh-3905).
(Replication) Never keep a dead replica around if running out of disk space
(gh-3397).
(Replication) Report join progress to the replica log
(gh-3165).
(Lua) Expose snapshot status in box.info.gc()
(gh-3935).
(Lua) Show names of Lua functions in backtraces in fiber.info()
(gh-3538).
(Lua) Check if transaction opened
(gh-3518).

Bugs fixed

(Engines) Tarantool crashes if DML races with DDL
(gh-3420).
(Engines) Recovery error if DDL is aborted
(gh-4066).
(Engines) Tarantool could commit in the read-only mode
(gh-4016).
(Engines) Vinyl iterator crashes if used throughout DDL
(gh-4000).
(Engines) Vinyl doesn’t exit until dump/compaction is complete
(gh-3949).
(Engines) After re-creating secondary index no data is visible
(gh-3903).
(Engines) box.info.memory().tx underflow
(gh-3897).
(Engines) Vinyl stalls on intensive random insertion
(gh-3603).
(Server) Newer version of libcurl explodes fiber stack
(gh-3569).
(Server) SIGHUP crashes tarantool
(gh-4063).
(Server) checkpoint_daemon.lua:49: bad argument #2 to ‘format’
(gh-4030).
(Server) fiber:name() show only part of name
(gh-4011).
(Server) Second hot standby switch may fail
(gh-3967).
(Server) Updating box.cfg.readahead doesn’t affect existing connections
(gh-3958).
(Server) fiber.join() blocks in ‘suspended’ if fiber has cancelled
(gh-3948).
(Server) Tarantool can be crashed by sending gibberish to a binary socket
(gh-3900).
(Server) Stored procedure to produce push-messages never breaks on client disconnect
(gh-3559).
(Server) Tarantool crashed in lj_vm_return
(gh-3840).
(Server) Fiber executing box.cfg() may process messages from iproto
(gh-3779).
(Server) Possible regression on nosqlbench
(gh-3747).
(Server) Assertion after improper index creation
(gh-3744).
(Server) Tarantool crashes on vshard startup (lj_gc_step)
(gh-3725).
(Server) Do not restart replication on box.cfg if the configuration didn’t change
(gh-3711).
(Replication) Applier times out too fast when reading large tuples
(gh-4042).
(Replication) Vinyl replica join fails
(gh-3968).
(Replication) Error during replication
(gh-3910).
(Replication) Downstream status doesn’t show up in replication.info unless the channel is broken
(gh-3904).
(Replication) replication fails: tx checksum mismatch
(gh-3883).
(Replication) Rebootstrap crashes if master has replica’s rows
(gh-3740).
(Replication) After restart tuples revert back to their old state which was before replica sync
(gh-3722).
(Replication) Add vclock for safer hot standby switch
(gh-3002).
(Replication) Master row is skipped forever in case of wal write failure
(gh-2283).
(Lua) space:frommap():tomap() conversion fail
(gh-4045).
(Lua) Non-informative message when trying to read a negative count of bytes from socket
(gh-3979).
(Lua) space:frommap raise “tuple field does not match…” even for nullable field
(gh-3883).
(Lua) Tarantool crashes on net.box.call after some uptime with vshard internal fiber
(gh-3751).
(Lua) Heap use after free in lbox_error
(gh-1955).
(Misc) http.client doesn’t honour ‘connection: keep-alive’
(gh-3955).
(Misc) net.box wait_connected is broken
(gh-3856).
(Misc) Mac build fails on Mojave
(gh-3797).
(Misc) FreeBSD build error: no SSL support
(gh-3750).
(Misc) ‘http.client’ sets invalid (?) reason
(gh-3681).
(Misc) Http client silently modifies headers when value is not a “string” or a “number”
(gh-3679).
(Misc) yaml.encode uses multiline format for ‘false’ and ‘true’
(gh-3662).
(Misc) yaml.encode encodes ‘null’ incorrectly
(gh-3583).
(Misc) Error object message is empty
(gh-3604).
(Misc) Log can be flooded by warning messages
(gh-2218).

Deprecations

Deprecate console=true option for net.box.new().

Release 1.10.2

Release type: stable (lts). Release date: 2018-10-13. Tag: 1-10-2.

Release: v. 1.10.2.

This is the first stable (lts) release in the 1.10 series. Also, Tarantool 1.10.2 is a major release that deprecates Tarantool 1.9.2. It resolves 95 issues since 1.9.2.

Tarantool 1.10.x is backward compatible with Tarantool 1.9.x in binary data layout, client-server protocol and replication protocol. You can upgrade using the box.schema.upgrade() procedure.

The goal of this release is to significantly increase vinyl stability and introduce automatic rebootstrap of a Tarantool replica set.

Functionality added or changed:

(Engines) support ALTER for non-empty vinyl spaces (gh-1653).

(Engines) tuples stored in the vinyl cache are not shared among the indexes of the same space (gh-3478).

(Engines) keep a stack of UPSERTS in vy_read_iterator (gh-1833).

(Engines) box.ctl.reset_stat(), a function to reset vinyl statistics (gh-3198).

(Server) configurable syslog destination (gh-3487).

(Server) allow different nullability in indexes and format (gh-3430).

(Server) allow to back up any checkpoint, not just the last one (gh-3410).

(Server) a way to detect that a Tarantool process was started / restarted by tarantoolctl (TARANTOOLCTL and TARANTOOL_RESTARTED env vars) (gh-3384, gh-3215).

(Server) net_msg_max configuration parameter to restrict the number of allocated fibers (gh-3320).

(Replication) display the connection status if the downstream gets disconnected from the upstream (box.info.replication.downstream.status = disconnected) (gh-3365).

(Replication) replica-local spaces (gh-3443)

(Replication) replication_skip_conflict, a new option in box.cfg{} to skip conflicting rows in replication (gh-3270)

(Replication) remove old snapshots which are not needed by replicas (gh-3444)

(Replication) log records which tried to commit twice (gh-3105)

(Lua) new function fiber.join() (gh-1397).

(Lua) new option names_only to tuple:tomap() (gh-3280).

(Lua) support custom rock servers (server and only-server options for tarantoolctl rocks command) (gh-2640).

(Lua) expose on_commit/on_rollback triggers to Lua (gh-857).

(Lua) new function box.is_in_txn() to check if a transaction is open (gh-3518).

(Lua) tuple field access via a json path (by number, name, and path); (gh-1285).

(Lua) new function space:frommap() (gh-3282).

(Lua) new module utf8 that implements libicu’s bindings for use in Lua (gh-3290, gh-3385).

Version:

Tarantool 1.10