How to implement Queue Worker/API in Drupal 8?

Before I start my writing about Queue Worker, it is important to understand about cron job in Drupal 8.

In generic terms, cron is a scheduled job which runs periodically at fixed intervals. cron manages typical tasks such as database maintenance, sending bulk emails, fetching data from a third-party on regular basis.

It manages short running tasks with fewer resources and it will automatically time-out while performing time-consuming tasks such as fetching data from a third- party.

In Drupal you can write cron like:-

[php]

function hook_cron() {
  //code.
}

[/php]

in .module file of your module.

So let’s take an example:-
There is a module namely – my_module.
so in my_module.module file write

[php]

function my_module_cron() {
$mailManager = \Drupal::service(‘plugin.manager.mail’);
$mailManager->mail($module, $key, $to, $langcode, $params = array(), $reply = NULL, $send = TRUE);
}

/**
* Implements hook_mail().
*/
function my_module_mail($key, &$message, $params) {
switch ($key) {
case ‘key’:
$message[‘subject’] = t(‘Test’);
$message[‘body’][] = ‘Hello World!’;
break;
}
}

[/php]

Whenever cron is executed it will send an email.

drush commands to run a cron manually.

drush core-cron

alias

drush cron

Disadvantages:-

1. You can not run an individual cron.
2. All hook_cron runs at same time for all modules according to module weight
   Queue Worker A Powerful manual cron system.
3. When cron fails for a module, the other cron of different modules will be not executed.
4. There is no logging information that can tell us which module was the culprit for cron failure.
5. Cron can’t handle large and complex tasks in huge quantity.
6. You can’t run cron with time limits.

QueueWorker:-

The following disadvantages compel us to use queue API of Drupal 8 which is known as Queue Worker.

Queue Worker offers a lot of advantages such as :

1. Offers flexibility to run queue with time limits
2. You can individual queue at one time as compared to cron which runs all the cron
3. Efficient as compared to cron, can handle resource intensive tasks
4. If any failure occurs, you can revert the item back to queue to process that again
5. Multiple queues can run without interrupting other work
6. You can log for each item of queue easily

Queue Worker rules out the disadvantage of hook_cron and hence solve the issues with hook cron very efficiently.

Here are some of the basics concepts of Queue API along with an example:

The Queue API in Drupal allows you to complete tasks at a later stage. We can insert items in the queue and process them when we require. We can run Queue either by running cron or by manually using Drush commands.

Some very basic concepts of Queue working is FIFO.

Components of Queue API.

1. QueueInterface:-
   • It plays a very important role here. Its implementation represents queue, which means the object of a class that implements QueueInterface.
2. DatabaseQueue:-
   • It is a reliable type of queue which makes ensure that it does not timeout. All the items are processed without any error and in (FIFO) order.
3. Roles of Queue Object:-
   • The typical role of the queue object is to create items, claim them from the queue and delete them when they have been processed.
   • You can also release the items if processing is not finished or another Queue needs to be processed before removing them from Queue.

Here’s an example of Queue Worker.

You have a user query form on your website. See the attached screenshot of Form. So, whenever a user submits a query instead of sending mail at run-time we will store these details in Queue Worker and later run Queue Worker manually whenever we want to receive emails.

Folder Structure. See attached screenshot.

my_module.info.yml

[php]

name: My Module
description: ‘Example how to use Queue Worker’
package: Custom
type: module
core: 8.x

[/php]

my_module.routing.yml

[php]

my_module.query_form:
path: ‘/user-query-form’
defaults:
_form: ‘\Drupal\my_module\Form\UserQueryForm’
_title: ‘Query Form’
requirements:
_permission: ‘access content’

[/php]

UserQueryForm.php

Screenshot:

Code:-

[php]

<?php

namespace Drupal\my_module\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;

/**
* Class UserQueryForm.
*
* @package Drupal\my_module\Form
*/
class UserQueryForm extends FormBase {

/**
* {@inheritdoc}
*/
public function getFormId() {
return ‘user_query_form’;
}

/**
* {@inheritdoc}
*/
public function buildForm(array $form, FormStateInterface $form_state) {
$form[‘name’] = [
‘#type’ => ‘textfield’,
‘#attributes’ => [
‘placeholder’ => ‘Username’,
],
‘#required’ => true,
];
$form[’email’] = [
‘#type’ => ’email’,
‘#attributes’ => [
‘placeholder’ => ‘Email’,
],
‘#required’ => true,
];
$form[‘query’] = [
‘#type’ => ‘textarea’,
‘#attributes’ => [
‘placeholder’ => ‘Query’,
],
‘#description’ => ‘Upto 200 characters allowed’,
‘#required’ => true,
];
$form[‘submit’] = [
‘#type’ => ‘submit’,
‘#value’ => ‘Send’,

];
return $form;
}

/**
* {@inheritdoc}
*/
public function validateForm(array &$form, FormStateInterface $form_state) {

}

/**
* {@inheritdoc}
*/
public function submitForm(array &$form, FormStateInterface $form_state) {
/** @var QueueFactory $queue_factory */
$queue_factory = \Drupal::service(‘queue’);
/** @var QueueInterface $queue */
$queue = $queue_factory->get(’email_processor’);
$item = new \stdClass();
$item->username = $form_state->getValue(‘name’);
$item->email = $form_state->getValue(’email’);
$item->query = $form_state->getValue(‘query’);
$queue->createItem($item);
}
}

[/php]

First create QueueFactory object from the service container statically and use this object to instantiate a queue object of email_proceesor.

Here we are calling createItem() function to insert items in Queue.

So, whenever a user submits query form an item will be inserted in the Queue.

To see the items in Queue use this drush command:

drush queue-list

EmailEventBase.php

The Queue Worker

The Queue Worker is Plugin so we create EmailEventBase.php inside Plugin Folder as attached screenshot of folder structure.

[php]

<?php

/**
* Handles sending of email.
*
* PHP Version 5
*/

namespace Drupal\my_module\Plugin\QueueWorker;

use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Queue\QueueWorkerBase;
use Drupal\Core\Mail\MailManager;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
*
* @inheritdoc
*/
class EmailEventBase extends QueueWorkerBase implements ContainerFactoryPluginInterface {

/**
*
* @var Drupal\Core\Mail\MailManager
*/
protected $mail;
/**
* constructor
*/
public function __construct(MailManager $mail) {
$this->mail = $mail;
}

/**
* {@inheritdoc}
*/
public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
return new static(
$container->get(‘plugin.manager.mail’)
);
}

/**
* Processes a single item of Queue.
*
*/
public function processItem($data) {
$params[‘subject’] = t(‘query’);
$params[‘message’] = $data->query;
$params[‘from’] = $data->email;
$params[‘username’] = $data->username;
$to = \Drupal::config(‘system.site’)->get(‘mail’);
$this->mail->mail(‘my_module’,’query_mail’,$to,’en’,$params,NULL,true);
}
}

[/php]

In processItem() function we are processing individual items of Queue.

Now, let’s create a CronEventProcessor plugin that will use this logic on Cron runs or when we will run manually using drush command:

CronEventProcessor.php

[php]

<?php
/**
*
* PHP Version 5
*/

namespace Drupal\my_module\Plugin\QueueWorker;

/**
*
* @QueueWorker(
* id = "email_processor",
* title = "My custom Queue Worker",
* cron = {"time" = 10}
* )
*/
class CronEventProcessor extends EmailEventBase {

}

[/php]

In this file, we have written annotation. So by writing annotation, we are telling Drupal that whenever cron runs process this queue for 10 seconds.

Whenever cron runs Queue Worker will load all the plugins that cron as their key in annotations. The id in annotation is user for creating object of queue.

in .module file of your module you need to write hook_mail to send mail.

[php]

<?php

/**
* Implements hook_mail()
* @param type $key
* @param type $message
* @param type $params
*/
function my_module_mail($key, &$message, $params) {
switch ($key) {
case ‘query_mail’:
$body = "Hi ".$params[‘username’]
. " Thanks for posting your Query"
. $params[‘message’]
. "Your username: " . $params[‘username’]

. "Your email: " . $params[’email’];

$message[‘from’] = $params[‘from’];
$message[‘subject’] = $params[‘subject’];
$message[‘body’][] = Drupal\Core\Mail\MailFormatHelper::htmlToText($body);
break;
}
}

[/php]

To run this queue manually we can use drush command like:-

drush queue-run email_processor –time-limit=10

Conclusion:-

You should use Queue API instead of native hook_cron(). It has many benefits as compared to hook_cron.

Manage your resource intensive tasks on your website using Queue API.