Module fio

Overview

Tarantool supports file input/output with an API that is similar to POSIX syscalls. All operations are performed asynchronously. Multiple fibers can access the same file simultaneously.

The fio module contains:

• functions for common pathname manipulations,
• functions for directory or file existence and type checks,
• functions for common file manipulations, and
• constants which are the same as POSIX flag values (for example fio.c.flag.O_RDONLY = POSIX O_RDONLY).

Index

Below is a list of all fio functions and members.

Common pathname manipulations

fio.pathjoin(partial-string[, partial-string ...])¶

Concatenate partial string, separated by ‘/’ to form a path name.

Parameters:	partial-string (string) – one or more strings to be concatenated.
Return:	path name
Rtype:	string

Example:

tarantool> fio.pathjoin('/etc', 'default', 'myfile')
---
- /etc/default/myfile
...

fio.basename(path-name[, suffix])¶

Given a full path name, remove all but the final part (the file name). Also remove the suffix, if it is passed.

Note that the basename of a path with a trailing slash is an empty string. It is different from how the Unix basename program interprets such a path.

Parameters:	path-name (string) – path name suffix (string) – suffix
Return:	file name
Rtype:	string

Example:

tarantool> fio.basename('/path/to/my.lua', '.lua')
---
- my
...

Example with a trailing slash:

tarantool> fio.basename('/path/to/')
---
-
...

fio.dirname(path-name)¶

Given a full path name, remove the final part (the file name).

Parameters:	path-name (string) – path name
Return:	directory name, that is, path name except for file name.
Rtype:	string

Example:

tarantool> fio.dirname('/path/to/my.lua')
---
- '/path/to/'

fio.abspath(file-name)¶

Given a final part (the file name), return the full path name.

Parameters:	file-name (string) – file name
Return:	directory name, that is, path name including file name.
Rtype:	string

Example:

tarantool> fio.abspath('my.lua')
---
- '/path/to/my.lua'
...

Directory or file existence and type checks

Functions in this section are similar to some Python os.path functions.

fio.path.exists(path-name)¶

Parameters:	path-name (string) – path to directory or file.
Return:	true if path-name refers to a directory or file that exists and is not a broken symbolic link; otherwise false
Rtype:	boolean

fio.path.is_dir(path-name)¶

Parameters:	path-name (string) – path to directory or file.
Return:	true if path-name refers to a directory; otherwise false
Rtype:	boolean

fio.path.is_file(path-name)¶

Parameters:	path-name (string) – path to directory or file.
Return:	true if path-name refers to a file; otherwise false
Rtype:	boolean

fio.path.is_link(path-name)¶

Parameters:	path-name (string) – path to directory or file.
Return:	true if path-name refers to a symbolic link; otherwise false
Rtype:	boolean

fio.path.lexists(path-name)¶

Parameters:	path-name (string) – path to directory or file.
Return:	true if path-name refers to a directory or file that exists or is a broken symbolic link; otherwise false
Rtype:	boolean

Common file manipulations

fio.umask(mask-bits)¶

Set the mask bits used when creating files or directories. For a detailed description type man 2 umask.

Parameters:	mask-bits (number) – mask bits.
Return:	previous mask bits.
Rtype:	number

Example:

tarantool> fio.umask(tonumber('755', 8))
---
- 493
...

fio.lstat(path-name)¶

fio.stat(path-name)¶

Returns information about a file object. For details type man 2 lstat or man 2 stat.

Parameters:	path-name (string) – path name of file.
Return:	(If no error) table of fields which describe the file’s block size, creation time, size, and other attributes. (If error) two return values: null, error message.
Rtype:	table.

Additionally, the result of fio.stat('file-name') will include methods equivalent to POSIX macros:

• is_blk() = POSIX macro S_ISBLK,
• is_chr() = POSIX macro S_ISCHR,
• is_dir() = POSIX macro S_ISDIR,
• is_fifo() = POSIX macro S_ISFIFO,
• is_link() = POSIX macro S_ISLINK,
• is_reg() = POSIX macro S_ISREG,
• is_sock() = POSIX macro S_ISSOCK.

For example, fio.stat('/'):is_dir() will return true.

Example:

tarantool> fio.lstat('/etc')
---
- inode: 1048577
  rdev: 0
  size: 12288
  atime: 1421340698
  mode: 16877
  mtime: 1424615337
  nlink: 160
  uid: 0
  blksize: 4096
  gid: 0
  ctime: 1424615337
  dev: 2049
  blocks: 24
...

fio.mkdir(path-name[, mode])¶

fio.rmdir(path-name)¶

Create or delete a directory. For details type man 2 mkdir or man 2 rmdir.

Parameters:	path-name (string) – path of directory. mode (number) – Mode bits can be passed as a number or as string constants, for example S_IWUSR. Mode bits can be combined by enclosing them in braces.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.mkdir('/etc')
---
- false
...

fio.chdir(path-name)¶

Change working directory. For details type man 2 chdir.

Parameters:	path-name (string) – path of directory.
Return:	(If success) true. (If failure) false.
Rtype:	boolean

Example:

tarantool> fio.chdir('/etc')
---
- true
...

fio.listdir(path-name)¶

List files in directory. The result is similar to the ls shell command.

Parameters:	path-name (string) – path of directory.
Return:	(If no error) a list of files. (If error) two return values: null, error message.
Rtype:	table

Example:

tarantool> fio.listdir('/usr/lib/tarantool')
---
- - mysql
...

fio.glob(path-name)¶

Return a list of files that match an input string. The list is constructed with a single flag that controls the behavior of the function: GLOB_NOESCAPE. For details type man 3 glob.

Parameters:	path-name (string) – path-name, which may contain wildcard characters.
Return:	list of files whose names match the input string
Rtype:	table

Possible errors: nil.

Example:

tarantool> fio.glob('/etc/x*')
---
- - /etc/xdg
  - /etc/xml
  - /etc/xul-ext
...

fio.tempdir()¶

Return the name of a directory that can be used to store temporary files.

Example:

tarantool> fio.tempdir()
---
- /tmp/lG31e7
...

fio.tempdir() stores the created temporary directory into /tmp by default. Since version 2.4.1, this can be changed by setting the TMPDIR environment variable. Before starting Tarantool, or at runtime by os.setenv().

Example:

tarantool> fio.tempdir()
---
- /tmp/lG31e7
...
tarantool> fio.mkdir('./mytmp')
---
- true
...

tarantool> os.setenv('TMPDIR', './mytmp')
---
...

tarantool> fio.tempdir()
---
- ./mytmp/506Z0b
...

fio.cwd()¶

Return the name of the current working directory.

Example:

tarantool> fio.cwd()
---
- /home/username/tarantool_sandbox
...

fio.copytree(from-path, to-path)¶

Copy everything in the from-path, including subdirectory contents, to the to-path. The result is similar to the cp -r shell command. The to-path should not be empty.

Parameters:	from-path (string) – path-name. to-path (string) – path-name.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.copytree('/home/original','/home/archives')
---
- true
...

fio.mktree(path-name)¶

Create the path, including parent directories, but without file contents. The result is similar to the mkdir -p shell command.

Parameters:	path-name (string) – path-name.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.mktree('/home/archives')
---
- true
...

fio.rmtree(path-name)¶

Remove the directory indicated by path-name, including subdirectories. The result is similar to the rm -rf shell command.

Parameters:	path-name (string) – path-name.
Return:	(If no error) true. (If error) two return values: null, error message.
Rtype:	boolean

Example:

tarantool> fio.rmtree('/home/archives')
---
- true
...

fio.link(src, dst)¶

fio.symlink(src, dst)¶

fio.readlink(src)¶

fio.unlink(src)¶

Functions to create and delete links. For details type man readlink, man 2 link, man 2 symlink, man 2 unlink.

Parameters:	src (string) – existing file name. dst (string) – linked name.
Return:	(If no error) fio.link and fio.symlink and fio.unlink return true, fio.readlink returns the link value. (If error) two return values: false|null, error message.

Example:

tarantool> fio.link('/home/username/tmp.txt', '/home/username/tmp.txt2')
---
- true
...
tarantool> fio.unlink('/home/username/tmp.txt2')
---
- true
...

fio.rename(path-name, new-path-name)¶

Rename a file or directory. For details type man 2 rename.

Parameters:	path-name (string) – original name. new-path-name (string) – new name.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.rename('/home/username/tmp.txt', '/home/username/tmp.txt2')
---
- true
...

fio.utime(file-name[, accesstime[, updatetime]])¶

Change the access time and possibly also change the update time of a file. For details type man 2 utime. Times should be expressed as number of seconds since the epoch.

Parameters:	file-name (string) – name. accesstime (number) – time of last access. default current time. updatetime (number) – time of last update. default = access time.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.utime('/home/username/tmp.txt')
---
- true
...

fio.copyfile(path-name, new-path-name)¶

Copy a file. The result is similar to the cp shell command.

Parameters:	path-name (string) – path to original file. new-path-name (string) – path to new file.
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.copyfile('/home/user/tmp.txt', '/home/usern/tmp.txt2')
---
- true
...

fio.chown(path-name, owner-user, owner-group)¶

fio.chmod(path-name, new-rights)¶

Manage the rights to file objects, or ownership of file objects. For details type man 2 chown or man 2 chmod.

Parameters:	owner-user (string) – new user uid. owner-group (string) – new group uid. new-rights (number) – new permissions
Return:	null

Example:

tarantool> fio.chmod('/home/username/tmp.txt', tonumber('0755', 8))
---
- true
...
tarantool> fio.chown('/home/username/tmp.txt', 'username', 'username')
---
- true
...

fio.truncate(path-name, new-size)¶

Reduce file size to a specified value. For details type man 2 truncate.

Parameters:	path-name (string) – new-size (number) –
Return:	(If no error) true. (If error) two return values: false, error message.
Rtype:	boolean

Example:

tarantool> fio.truncate('/home/username/tmp.txt', 99999)
---
- true
...

fio.sync()¶

Ensure that changes are written to disk. For details type man 2 sync.

Return:	true if success, false if failure.
Rtype:	boolean

Example:

tarantool> fio.sync()
---
- true
...

fio.open(path-name[, flags[, mode]])¶

Open a file in preparation for reading or writing or seeking.

Parameters:	path-name (string) – Full path to the file to open. flags (number) – Flags can be passed as a number or as string constants, for example ‘O_RDONLY’, ‘O_WRONLY’, ‘O_RDWR’. Flags can be combined by enclosing them in braces. On Linux the full set of flags as described on the Linux man page is: O_APPEND (start at end of file), O_ASYNC (signal when IO is possible), O_CLOEXEC (enable a flag related to closing), O_CREAT (create file if it doesn’t exist), O_DIRECT (do less caching or no caching), O_DIRECTORY (fail if it’s not a directory), O_EXCL (fail if file cannot be created), O_LARGEFILE (allow 64-bit file offsets), O_NOATIME (no access-time updating), O_NOCTTY (no console tty), O_NOFOLLOW (no following symbolic links), O_NONBLOCK (no blocking), O_PATH (get a path for low-level use), O_SYNC (force writing if it’s possible), O_TMPFILE (the file will be temporary and nameless), O_TRUNC (truncate) … and, always, one of: O_RDONLY (read only), O_WRONLY (write only), or O_RDWR (either read or write). mode (number) – Mode bits can be passed as a number or as string constants, for example S_IWUSR. Mode bits are significant if flags include O_CREAT or O_TMPFILE. Mode bits can be combined by enclosing them in braces.
Return:	(If no error) file handle (abbreviated as ‘fh’ in later description). (If error) two return values: null, error message.
Rtype:	userdata

Possible errors: nil.

Note that since version 2.4.1 fio.open() returns a descriptor which can be closed manually by calling the :close() method, or it will be closed automatically when it has no references, and the garbage collector deletes it.

Keep in mind that the number of file descriptors is limited, and they can become exhausted earlier than the garbage collector will be triggered to collect not used descriptors. It is always good practice to close them manually as soon as possible.

Example 1:

tarantool> fh = fio.open('/home/username/tmp.txt', {'O_RDWR', 'O_APPEND'})
---
...
tarantool> fh -- display file handle returned by fio.open
---
- fh: 11
...

Example 2:

Using fio.open() with tonumber('N', 8) to set permissions as an octal number:

tarantool> fio.open('x.txt', {'O_WRONLY', 'O_CREAT'}, tonumber('644',8))
---
- fh: 12
...

object file-handle¶

file-handle:close()¶

Close a file that was opened with fio.open. For details type man 2 close.

Parameters:	fh (userdata) – file-handle as returned by fio.open().
Return:	true if success, false if failure.
Rtype:	boolean

Example:

tarantool> fh:close() -- where fh = file-handle
---
- true
...

file-handle:pread(count, offset)¶

file-handle:pread(buffer, count, offset)

Perform random-access read operation on a file, without affecting the current seek position of the file. For details type man 2 pread.

Parameters:	fh (userdata) – file-handle as returned by fio.open() buffer – where to read into (if the format is pread(buffer, count, offset)) count (number) – number of bytes to read offset (number) – offset within file where reading begins

If the format is pread(count, offset) then return a string containing the data that was read from the file, or empty string if failure.

If the format is pread(buffer, count, offset) then return the data to the buffer. Buffers can be acquired with buffer.ibuf.

Example:

tarantool> fh:pread(25, 25)
---
- |
  elete from t8//
  insert in
...

file-handle:pwrite(new-string, offset)¶

file-handle:pwrite(buffer, count, offset)

Perform random-access write operation on a file, without affecting the current seek position of the file. For details type man 2 pwrite.

Parameters:	fh (userdata) – file-handle as returned by fio.open() new-string (string) – value to write (if the format is pwrite(new-string, offset)) buffer (cdata) – value to write (if the format is pwrite(buffer, count, offset)) count (number) – number of bytes to write offset (number) – offset within file where writing begins
Return:	true if success, false if failure.
Rtype:	boolean

If the format is pwrite(new-string, offset) then the returned string is written to the file, as far as the end of the string.

If the format is pwrite(buffer, count, offset) then the buffer contents are written to the file, for count bytes. Buffers can be acquired with buffer.ibuf.

tarantool> ibuf = require('buffer').ibuf()
---
...

tarantool> fh:pwrite(ibuf, 1, 0)
---
- true
...

file-handle:read([count])¶

file-handle:read(buffer, count)

Perform non-random-access read on a file. For details type man 2 read or man 2 write.

Note

fh:read and fh:write affect the seek position within the file, and this must be taken into account when working on the same file from multiple fibers. It is possible to limit or prevent file access from other fibers with fiber.cond() or fiber.channel().

Parameters:

fh (userdata) – file-handle as returned by fio.open(). buffer – where to read into (if the format is read(buffer, count)) count (number) – number of bytes to read

Return:

If the format is read() – omitting count – then read all bytes in the file. If the format is read() or read([count]) then return a string containing the data that was read from the file, or empty string if failure. If the format is read(buffer, count) then return the data to the buffer. Buffers can be acquired with buffer.ibuf. In case of an error the method returns nil, err and sets the error to errno.

tarantool> ibuf = require('buffer').ibuf()
---
...

tarantool> fh:read(ibuf:reserve(5), 5)
---
- 5
...

tarantool> require('ffi').string(ibuf:alloc(5),5)
---
- abcde

file-handle:write(new-string)¶

file-handle:write(buffer, count)

Perform non-random-access write on a file. For details type man 2 write.

Note

fh:read and fh:write affect the seek position within the file, and this must be taken into account when working on the same file from multiple fibers. It is possible to limit or prevent file access from other fibers with fiber.cond() or fiber.channel().

Parameters:	fh (userdata) – file-handle as returned by fio.open() new-string (string) – value to write (if the format is write(new-string)) buffer (cdata) – value to write (if the format is write(buffer, count)) count (number) – number of bytes to write
Return:	true if success, false if failure.
Rtype:	boolean

If the format is write(new-string) then the returned string is written to the file, as far as the end of the string.

If the format is write(buffer, count) then the buffer contents are written to the file, for count bytes. Buffers can be acquired with buffer.ibuf.

Example:

tarantool> fh:write("new data")
---
- true
...
tarantool> ibuf = require('buffer').ibuf()
---
...
tarantool> fh:write(ibuf, 1)
---
- true
...

file-handle:truncate(new-size)¶

Change the size of an open file. Differs from fio.truncate, which changes the size of a closed file.

Parameters:	fh (userdata) – file-handle as returned by fio.open().
Return:	true if success, false if failure.
Rtype:	boolean

Example:

tarantool> fh:truncate(0)
---
- true
...

file-handle:seek(position[, offset-from])¶

Shift position in the file to the specified position. For details type man 2 seek.

Parameters:	fh (userdata) – file-handle as returned by fio.open(). position (number) – position to seek to offset-from (string) – ‘SEEK_END’ = end of file, ‘SEEK_CUR’ = current position, ‘SEEK_SET’ = start of file.
Return:	the new position if success
Rtype:	number

Possible errors: nil.

Example:

tarantool> fh:seek(20, 'SEEK_SET')
---
- 20
...

file-handle:stat()¶

Return statistics about an open file. This differs from fio.stat which return statistics about a closed file. For details type man 2 stat.

Parameters:	fh (userdata) – file-handle as returned by fio.open().
Return:	details about the file.
Rtype:	table

Example:

tarantool> fh:stat()
---
- inode: 729866
  rdev: 0
  size: 100
  atime: 140942855
  mode: 33261
  mtime: 1409430660
  nlink: 1
  uid: 1000
  blksize: 4096
  gid: 1000
  ctime: 1409430660
  dev: 2049
  blocks: 8
...

file-handle:fsync()¶

file-handle:fdatasync()¶

Ensure that file changes are written to disk, for an open file. Compare fio.sync, which is for all files. For details type man 2 fsync or man 2 fdatasync.

Parameters:	fh (userdata) – file-handle as returned by fio.open().
Return:	true if success, false if failure.

Example:

tarantool> fh:fsync()
---
- true
...

FIO constants

fio.c¶

Table with constants which are the same as POSIX flag values on the target platform (see man 2 stat).

Example:

tarantool> fio.c
---
- seek:
    SEEK_SET: 0
    SEEK_END: 2
    SEEK_CUR: 1
  mode:
    S_IWGRP: 16
    S_IXGRP: 8
    S_IROTH: 4
    S_IXOTH: 1
    S_IRUSR: 256
    S_IXUSR: 64
    S_IRWXU: 448
    S_IRWXG: 56
    S_IWOTH: 2
    S_IRWXO: 7
    S_IWUSR: 128
    S_IRGRP: 32
  flag:
    O_EXCL: 2048
    O_NONBLOCK: 4
    O_RDONLY: 0
    <...>
...