Worker_reactor_try_to_exit() (ERRNO 9101): worker exit timeout, forced termination

[warmbook]

Please answer these questions before submitting your issue.

1. What did you do? If possible, provide a simple script for reproducing the error.
   启动服务，用浏览器快速刷新访问该服务，直到临近max_request，缓慢刷新直到触发 WorkerExit

use Swoole\Thread\Map;
use Swoole\WebSocket\Server;
use Swoole\WebSocket\Frame;

list($connections)=Swoole\Thread::getArguments();

$setting=[
// 'group'=>'nobody',
// 'user'=>'nobody',
// 'reload_async'=>TRUE, // 异步事件完成后再退出 Worker 进程
// 'max_wait_time'=>60, // 等待异步事件完成的最大时长
'max_request'=>32, // 每个worker进程处理请求数
// 'max_connection'=>1024, // 并发连接数
// 'buffer_input_size'=>102410244,
// 'package_max_length'=>102410244,
// 'upload_max_filesize'=>102410244,
// 'enable_deadlock_check'=>FALSE, // 检测协程死锁，默认开启
'hook_flags'=>SWOOLE_HOOK_NATIVE_CURL|SWOOLE_HOOK_FILE|SWOOLE_HOOK_TCP|SWOOLE_HOOK_SLEEP,
// 'open_http2_protocol'=>TRUE,
// 'http_compression'=>FALSE, // 压缩默认是开启的，但是一方面nginx层有gzip压缩，另一方面安卓版微信浏览器br压缩存在bug，故需关闭
// 'log_level'=>SWOOLE_LOG_WARNING,
'log_file'=>'/path/to/logs'.DIRECTORY_SEPARATOR.'swoole.txt',
// 'document_root'=>FCPATH,
// 'enable_coroutine'=>TRUE,
// 'enable_static_handler'=>TRUE, //允许访问静态文件
// 'heartbeat_check_interval'=>10, // 心跳间隔
// 'heartbeat_idle_time'=>30, // 超时断开
];
if(!$connections){
// if($memory=env('SWOOLE_SERVER_MEMORY')) ini_set('memory_limit', $memory);
if(!file_exists($setting['log_file'])) fclose(fopen($setting['log_file'],'w+'));
// chown($setting['log_file'],'nobody'); // 多线程模式下读取不到nobody的uid
chmod($setting['log_file'],0777);

$connections=new Map();
$setting+=[
    'init_arguments'=>fn()=>[$connections],
    'worker_num'=>1, // 默认为CPU核数
];

}

$server=new Server('0.0.0.0',80,SWOOLE_THREAD);
$server->set($setting);

$server->on('WorkerStart',function(Server $server,$workerId){
echo '[ ',microtime(TRUE),' ] WorkerStart:',$workerId,PHP_EOL;
});

$server->on('WorkerExit',function(Server $server,$workerId){
echo '[ ',microtime(TRUE),' ] WorkerExit:',$workerId,PHP_EOL;
});

$server->on('request',function($request,$response){
$response->end(microtime());
});

$server->on('open',function(Server $ws,$request){
echo '[ ',microtime(TRUE),' ] open:',$ws->worker_id,' > ',$request->fd,PHP_EOL;
});
$server->on('message',function(Server $ws,Frame $frame){
var_dump($frame->data);
if(!$frame->data){
$ws->push($frame->fd,new Frame());
return;
}
});
$server->on('connect',function(Server $ws,$fd){
echo '[ ',microtime(TRUE),' ] connect:',$ws->worker_id,' > ',$fd,PHP_EOL;
});
$server->on('close',function(Server $ws,$fd){
echo '[ ',microtime(TRUE),' ] close:',$ws->worker_id,' > ',$fd,PHP_EOL;
});

$server->start();

2. What did you expect to see?
   WorkerExit应该平滑重启，而不是卡死。并且不应该出现这两个wrning

3. What did you see instead?
   触发WorkerExit后的请求会无响应直到报502，同时日志中会出现 WARNING Worker_reactor_try_to_exit() (ERRNO 9101): worker exit timeout, forced termination ，如果有websocket连着并且设置了心跳，后面就会以心跳周期告警 WARNING ReactorImpl::after_removal_failure(): failed to delete events[fd=24#0, type=0, events=512], Error: No such file or directory[2]

4. What version of Swoole are you using (show your php --ri swoole)?
   swoole

Swoole => enabled
Author => Swoole Team [email protected]
Version => 6.0.0
Built => Dec 18 2024 14:06:46
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
thread => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 3.1.4+quic 24 Oct 2023
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
c-ares => 1.24.0
zlib => 1.3.1
brotli => E16777225/D16777225
zstd => 1.5.2
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
coroutine_pgsql => enabled
coroutine_odbc => enabled
coroutine_sqlite => enabled

Directive => Local Value => Master Value
swoole.enable_library => On => On
swoole.enable_fiber_mock => Off => Off
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

5. What is your machine environment used (show your uname -a & php -v & gcc -v) ?
   Linux iZuf68ujmphbn8maxiqqp6Z 5.10.134-12.2.al8.x86_64 编译出错 #1 SMP Thu Oct 27 10:07:15 CST 2022 x86_64 x86_64 x86_64 GNU/Linux

Swoole 6.0.0 (cli) (built: Dec 18 2024 14:08:31) (ZTS)

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/10/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20200825 (Alibaba 10.2.1-3 2.32) (GCC)