-
Notifications
You must be signed in to change notification settings - Fork 122
Open Files Limit
TokuDB stored user defined databases in files. In the current design, there is a one to one correspondence between user defined databases and files. One could store a user defined databases in multiple files, or multiple databases in a single file, or some other mapping of databases to files. For this topic, the mapping is irrelevant. What does matter is how TokuDB handles the case when a file can not be opened. We currently abort the server. We want to fail the database open instead.
TokuDB maintains some meta data file that describe the operational environment. There are also lock files that are used to gain exclusive access to various directories. These meta data files are opened when the environment is opened. If these meta data files can not be created, then the environment open should fail.
TokuDB uses a recovery log to contain transactional work that has occurred since the last checkpoint. The recovery log is implemented as a sequence of files. TokuDB usually has the most recent log file open. When a log file gets too big, it gets closed and a new log file is created. Since another thread can consume the last file descriptor before the new TokuDB log file is created, TokuDB's log file creation can fail. The probability of this event is very small, so nothing will be done to close the window.
TokuDB fsync's the directory in which new files are created. A file descriptor for the directory is obtained by the opendir (3) function. We should fail the create if the directory can not be fsync'ed.
sysconf on linux queries status files to get various information like the page size or the number of processors in the system. When these files can not be opened due to the open file limit, these operations will fail. We will call these functions during library initialization and cache the answers so that failures after library initialization do not happen.