在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务
1、安装 Yii2 队列扩展，执行以下命令
composer require --prefer-dist yiisoft/yii2-queue
2、查看 composer.json，已经安装成功
"require": {
"yiisoft/yii2-queue": "^2.1"
},
3、把多个队列组件注册到控制台，编辑 \console\config\main.php
"require": {
"yiisoft/yii2-queue": "^2.1"
},
4、多个队列组件的配置，编辑 \environments\dev\common\config\main-local.php、\environments\prod\common\config\main-local.php
'copyAssetQueue' => [ // 复制资源文件队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:copy:asset', // 队列键前缀
'ttr' => 10 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'qqCwVideoMultipartUploadQueue' => [ // 企鹅号的内容网站应用的视频文件分片上传队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:qq:cw:video:multipart:upload', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'pubArticleQueue' => [ // 发布文章队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:pub:article', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
'sourceCallbackQueue' => [ // 来源回调队列
'class' => 'yii\queue\redis\Queue',
'redis' => 'redis', // Redis 连接组件或它的配置
'channel' => 'cpa:queue:source:callback', // 队列键前缀
'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
'as log' => 'yii\queue\LogBehavior',
],
5、定义每个被发送到队列的任务的基础类，新建 \common\jobs\Job.php
<?php
/**
* Created by PhpStorm.
* User: terryhong
* Date: 2018/9/7
* Time: 下午4:54
*/
namespace common\jobs;
use yii\base\BaseObject;
use yii\queue\JobInterface;
/**
* 队列的任务的基础类
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class Job extends BaseObject implements JobInterface
{
public function execute($queue){
}
}
6、每个被发送到队列的任务应该被定义为一个单独的类。基于官方示例，如果您需要下载并保存一个文件，该类可能看起来如下
<?php
/**
* Created by PhpStorm.
* User: Qiang Wang
* Date: 2018/10/18
* Time: 10:39
*/
namespace common\jobs;
/**
* 下载并保存一个文件
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class DownloadJob extends Job
{
public $url;
public $file;
public function execute($queue)
{
file_put_contents($this->file, file_get_contents($this->url));
}
}
7、在 qq 应用中定义对应的下载类，新建 \qq\jobs\DownloadJob.php
<?php
/**
* Created by PhpStorm.
* User: Qiang Wang
* Date: 2018/10/19
* Time: 10:28
*/
namespace qq\jobs;
/**
* 下载并保存一个文件
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class DownloadJob extends \common\jobs\DownloadJob
{
}
8、作业状态的测试，编辑 \qq\rests\qq_cw_app\IndexAction.php
<?php
/**
* @link http://www.yiiframework.com/
* @copyright Copyright (c) 2008 Yii Software LLC
* @license http://www.yiiframework.com/license/
*/
namespace qq\rests\qq_cw_app;
use Yii;
use qq\jobs\DownloadJob;
use yii\data\ActiveDataProvider;
use yii\base\InvalidConfigException;
/**
*
* For more details and usage information on IndexAction, see the [guide article on rest controllers](guide:rest-controllers).
*
* @author Qiang Wang <shuijingwanwq@163.com>
* @since 1.0
*/
class IndexAction extends \yii\rest\IndexAction
{
/**
* Prepares the data provider that should return the requested collection of the models.
* @return ActiveDataProvider
* @throws InvalidConfigException if the configuration is invalid.
*/
protected function prepareDataProvider()
{
// 将作业推送到队列并获得其ID
$id = Yii::$app->copyAssetQueue->push(new DownloadJob([
'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
'file' => './qq/runtime/robots.txt',
]));
// 这个作业等待执行
var_dump(Yii::$app->copyAssetQueue->isWaiting($id));
// Worker 从队列获取作业，并执行它
var_dump(Yii::$app->copyAssetQueue->isReserved($id));
// Worker 作业执行完成
var_dump(Yii::$app->copyAssetQueue->isDone($id));
echo $id;
exit;
}
}
9、打开网址：http://api.channel-pub-api.localhost/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行
bool(true) bool(false) bool(false) 1
10、查看 Redis ，队列已经存入，如图1
600;O:19:"qq\jobs\DownloadJob":2:{s:3:"url";s:45:"http://wjdev2.chinamcloud.com:8661/robots.txt";s:4:"file";s:23:"./qq/runtime/robots.txt";}
11、info 命令打印关于队列状态的信息，一个任务状态为等待
.\yii copy-asset-queue/info
Jobs - waiting: 1 - delayed: 0 - reserved: 0 - done: 0
12、run 命令获取并执行循环中的任务，直到队列为空，如图2
.\yii copy-asset-queue/run --verbose=1 --isolate=1 --color=0 .\yii copy-asset-queue/info
2018-10-19 11:07:30 [pid: 36016] - Worker is started 2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Started 2018-10-19 11:07:30 [2] qq\jobs\DownloadJob (attempt: 1, pid: 36016) - Done (0.255 s) 2018-10-19 11:07:30 [pid: 36016] - Worker is stopped (0:00:00) Jobs - waiting: 0 - delayed: 0 - reserved: 0 - done: 1
13、查看 Redis ，键：cpa:queue:copy:asset.messages、cpa:queue:copy:asset.waiting 均已经被删除，如图3
14、文件 \qq\runtime\robots.txt 已经被保存，如图4
15、为了使开发过程更加友好，您可以向 Yii2 调试模块添加一个面板。面板显示 计数器和队列任务列表。编辑 \environments\dev\api\config\main-local.php、\environments\dev\backend\config\main-local.php、\environments\dev\frontend\config\main-local.php、\environments\dev\qq\config\main-local.php、\environments\dev\rpc\config\main-local.php、\environments\dev\wx\config\main-local.php
if (!YII_ENV_TEST) {
// configuration adjustments for 'dev' environment
$config['bootstrap'][] = 'debug';
$config['modules']['debug'] = [
'class' => 'yii\debug\Module',
'panels' => [
'queue' => 'yii\queue\debug\Panel',
],
];
$config['bootstrap'][] = 'gii';
$config['modules']['gii'] = [
'class' => 'yii\gii\Module',
];
}
16、查看 Yii2 调试模块，已经增加 queue 面板，如图5
17、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。
\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini
[program:yii-copy-asset-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 1 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini
[program:yii-qq-cw-video-multipart-upload-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 1 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini
[program:yii-pub-article-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 1 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini
[program:yii-source-callback-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 1 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log
18、容器升级完毕后，执行命令，如图6
php /sobey/www/channel-pub-api/init --env=Development --overwrite=All php /sobey/www/channel-pub-api/yii migrate --interactive=0
19、打开网址：http://wjdev2.chinamcloud.com:8661/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行
bool(true) bool(false) bool(false) 1
20、info 命令打印关于队列状态的信息，一个任务状态为等待
php /sobey/www/channel-pub-api/yii copy-asset-queue/info
Jobs - waiting: 1 - delayed: 0 - reserved: 0 - done: 0
21、手动运行 queue/listen 命令监听和处理队列任务
php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
2018-10-19 15:56:04 [pid: 1151] - Worker is started 2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Started 2018-10-19 15:56:05 [5] qq\jobs\DownloadJob (attempt: 1, pid: 1151) - Error (0.031 s) > yii\base\ErrorException: file_put_contents(./qq/runtime/robots.txt): failed to open stream: No such file or directory ^C2018-10-19 15:56:13 [pid: 1151] - Worker is stopped (0:00:09)
22、作业状态的测试，编辑 \qq\rests\qq_cw_app\IndexAction.php，设置 file 为绝对路径
// 将作业推送到队列并获得其ID
$id = Yii::$app->copyAssetQueue->push(new DownloadJob([
'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
'file' => '/sobey/www/channel-pub-api/qq/runtime/robots.txt',
]));
23、升级至开发环境后，手动运行 queue/listen 命令监听和处理队列任务，队列任务执行成功，但是 Supervisor 配置未生效，未自动处理队列中的任务，一直处于等待中
php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
2018-10-19 17:04:57 [pid: 1717] - Worker is started 2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Started 2018-10-19 17:04:57 [10] qq\jobs\DownloadJob (attempt: 1, pid: 1717) - Done (0.021 s) ^C2018-10-19 17:05:01 [pid: 1717] - Worker is stopped (0:00:04)
24、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。startsecs 的值调整为 0
\build\c_files\etc\supervisord.d\yii-copy-asset-queue-worker.ini
[program:yii-copy-asset-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 0 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-qq-cw-video-multipart-upload-queue-worker.ini
[program:yii-qq-cw-video-multipart-upload-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 0 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-pub-article-queue-worker.ini
[program:yii-pub-article-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 0 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log
\build\c_files\etc\supervisord.d\yii-source-callback-queue-worker.ini
[program:yii-source-callback-queue-worker] process_name = %(program_name)s_%(process_num)02d command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0 autostart = true autorestart = true startsecs = 0 stopwaitsecs = 10 numprocs = 4 redirect_stderr = true stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log
25、Supervisor 配置已生效，会自动处理队列中的任务，/sobey/www/channel-pub-api/qq/runtime/robots.txt 文件的生成时间时刻在更新中，如图7
近期评论