Significantly reduce file descriptors consumption #3085
+86
−27
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Significantly reduce file descriptors consumption (by 30-50%). From 8,000 to 5,800 using the example below.
Details
Reduce number of sockets created by closing off one half of the socketpair in child and parent after forking. This results in 30% decrease of the total number of sockets allocated, from 8k to some 5.8k on my test config with 10 workers and 10 sockets.
Usual scenario involving shared pipe / socketpair in parent / child in POSIX / Linux is that two descriptors returned by the system call are then split with first descriptor used by parent and then the second is closed off by parent and the second used by a child and the first is closed by the child (or vice versa). OpenSIPS breaks this rule by keeping both descriptors open in both parent and child, effectively wasting up descriptor space. This could lead to hitting a system-wide limit or some excessive system load due to enormous size of the descriptor table.
Solution
pt[N].ipc_pipe[2] is replaced with just a single int. Initially set to be the parent's end of the pipe, in the child after fork() we do dup2 of the second fd using parent's end as a destination. This closes master's end (in child) and ensures FD numbers for IPC are the same in both parent and child, so that pt[N].ipc_pipe does not need to differ (helping with debugging for example). Then the original child's pipe is closed off in child. In master we simply close the child's end once fork is done.
Compatibility
As it only has access to read-only end of the pipe.
This seems to be copied and pasted into other places too:
The solution might be to create special API to request sending a wake up API at module creation time. Or perhaps have a ping-pong RPC API? The module would just call main proc and get it to return the call ASAP?
OpenSIPS Configs Tested
opensips1.cfg
opensips2.cfg