Queues
Introduction
Laravel now offers Horizon, a beautiful dashboard and configuration system for your Redis powered queues. Check out the full Horizon documentation for more information.
Laravel queues provide a unified API across a variety of different queue backends, such as Beanstalk, Amazon SQS, Redis, or even a relational database. Queues allow you to defer the processing of a time consuming task, such as sending an email, until a later time. Deferring these time consuming tasks drastically speeds up web requests to your application.
The queue configuration file is stored in config/queue.php
. In this file you will find connection configurations for each of the queue drivers that are included with the framework, which includes a database, Beanstalkd, Amazon SQS, Redis, and a synchronous driver that will execute jobs immediately (for local use). A null
queue driver is also included which discards queued jobs.
Connections Vs. Queues
Before getting started with Laravel queues, it is important to understand the distinction between "connections" and "queues". In your config/queue.php
configuration file, there is a connections
configuration option. This option defines a particular connection to a backend service such as Amazon SQS, Beanstalk, or Redis. However, any given queue connection may have multiple "queues" which may be thought of as different stacks or piles of queued jobs.
Note that each connection configuration example in the queue
configuration file contains a queue
attribute. This is the default queue that jobs will be dispatched to when they are sent to a given connection. In other words, if you dispatch a job without explicitly defining which queue it should be dispatched to, the job will be placed on the queue that is defined in the queue
attribute of the connection configuration:
// This job is sent to the default queue...
Job::dispatch();
// This job is sent to the "emails" queue...
Job::dispatch()->onQueue('emails');
Some applications may not need to ever push jobs onto multiple queues, instead preferring to have one simple queue. However, pushing jobs to multiple queues can be especially useful for applications that wish to prioritize or segment how jobs are processed, since the Laravel queue worker allows you to specify which queues it should process by priority. For example, if you push jobs to a high
queue, you may run a worker that gives them higher processing priority:
php artisan queue:work --queue=high,default
Driver Notes & Prerequisites
Database
In order to use the database
queue driver, you will need a database table to hold the jobs. To generate a migration that creates this table, run the queue:table
Artisan command. Once the migration has been created, you may migrate your database using the migrate
command:
php artisan queue:table
php artisan migrate
Redis
In order to use the redis
queue driver, you should configure a Redis database connection in your config/database.php
configuration file.
Redis Cluster
If your Redis queue connection uses a Redis Cluster, your queue names must contain a key hash tag. This is required in order to ensure all of the Redis keys for a given queue are placed into the same hash slot:
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => '{default}',
'retry_after' => 90,
],
Blocking
When using the Redis queue, you may use the block_for
configuration option to specify how long the driver should wait for a job to become available before iterating through the worker loop and re-polling the Redis database.
Adjusting this value based on your queue load can be more efficient than continually polling the Redis database for new jobs. For instance, you may set the value to 5
to indicate that the driver should block for five seconds while waiting for a job to become available:
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => 90,
'block_for' => 5,
],
Other Driver Prerequisites
The following dependencies are needed for the listed queue drivers:
- Amazon SQS:
aws/aws-sdk-php ~3.0
- Beanstalkd:
pda/pheanstalk ~4.0
- Redis:
predis/predis ~1.0
Creating Jobs
Generating Job Classes
By default, all of the queueable jobs for your application are stored in the app/Jobs
directory. If the app/Jobs
directory doesn't exist, it will be created when you run the make:job
Artisan command. You may generate a new queued job using the Artisan CLI:
php artisan make:job ProcessPodcast
The generated class will implement the Illuminate\Contracts\Queue\ShouldQueue
interface, indicating to Laravel that the job should be pushed onto the queue to run asynchronously.
Class Structure
Job classes are very simple, normally containing only a handle
method which is called when the job is processed by the queue. To get started, let's take a look at an example job class. In this example, we'll pretend we manage a podcast publishing service and need to process the uploaded podcast files before they are published:
<?php
namespace App\Jobs;
use App\Podcast;
use App\AudioProcessor;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
class ProcessPodcast implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $podcast;
/**
* Create a new job instance.
*
* @param Podcast $podcast
* @return void
*/
public function __construct(Podcast $podcast)
{
$this->podcast = $podcast;
}
/**
* Execute the job.
*
* @param AudioProcessor $processor
* @return void
*/
public function handle(AudioProcessor $processor)
{
// Process uploaded podcast...
}
}