Too many close() syscall when collecting data

[jimmyxian]

In my environment, about 1000000 close() syscalls produced by my own collector every 10 seconds. Almost takes 300ms when doing close FDs.
After analysis, I find the following reasons.

• It will close FDs(from 3 to MAXFD ) before launching a subprocess when using close_fds=True parameter in subprocess.Popen.
  https://github.com/OpenTSDB/tcollector/blob/master/tcollector.py#L1372
  /usr/lib/python2.7/subprocess.py

        def _close_fds(self, but):
            if hasattr(os, 'closerange'):
                os.closerange(3, but)
                os.closerange(but + 1, MAXFD)
            else:
                for i in xrange(3, MAXFD):
                    if i == but:
                        continue
                    try:
                        os.close(i)
                    except:
                        pass

• In my env, MAXFD=1000000

Also, after review the history patch, I find the patch which set close_fds from False to True. 7b5659d#diff-50bd42e8d38bbdf7c24f69ae5e25165f @tsuna

Any good idea to solve to this?

[vasiliyk]

Xian, could you reconfirm that you can still reproduce the bug?