Writing Reliable Drush Commands for Drupal Batch Operations

Large Drupal updates are rarely difficult because of the individual record being changed. The difficulty comes from changing thousands of records without exhausting PHP memory, holding database locks for too long, or leaving the site in an uncertain state when a command stops halfway through.

A Drush command is a practical way to run maintenance tasks from the terminal. It can process entities in predictable chunks, report progress, return useful exit codes, and run under cron or a deployment pipeline without depending on a browser session.

This approach suits common Drupal work such as updating legacy fields, rebuilding derived data, assigning taxonomy terms, migrating content, or refreshing media metadata. It also gives developers more control than an administrator clicking through a web-based batch screen.

For an Australian site, operational details matter. A news publisher in Sydney may need to run maintenance outside morning traffic, while an organisation serving Perth users may schedule tasks in a different local time zone. If content contains personal information, the command should also support the site’s obligations under the Privacy Act 1988 and the Australian Privacy Principles.

Choose The Right Batch Pattern

There are three related patterns in Drupal: the Batch API, the Queue API, and a chunked Drush command. The Batch API is designed mainly for web requests and provides browser progress pages. The Queue API stores work items for later processing and is a strong choice when tasks need retries or long-term persistence.

A Drush command can perform chunked processing directly. It fetches a limited set of entity IDs, processes them, clears references, and continues until no records remain. This is often the simplest solution for a one-off migration or a scheduled maintenance operation.

The command should not load every entity at once. A site with 100,000 nodes may work during testing with loadMultiple() across the complete result set, then fail in production because the PHP process consumes several hundred megabytes. Chunking keeps the working set bounded.

For work that may be interrupted for days, use a queue instead. A command can populate queue items, while a worker handles each item independently. For a controlled task that should finish in one terminal run, cursor-based chunking is usually easier to understand and monitor.

Design The Command Around Services

Custom Drush commands belong in a custom module. A class such as ContentMaintenanceCommands can extend DrushCommands, receive Drupal services through dependency injection, and expose a method with the #[DrushCommand] attribute in modern Drush versions.

The command should receive the entity type manager and a logger rather than calling \Drupal::service() throughout the implementation. Dependency injection makes the class easier to test and avoids hiding its dependencies. It also makes future changes, such as replacing node storage with media storage, less invasive.

A basic class structure looks like this:

<?php

namespace Drupal\site_tools\Commands;

use Drush\Attributes as CLI;
use Drush\Commands\DrushCommands;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Psr\Log\LoggerInterface;

final class ContentMaintenanceCommands extends DrushCommands {

  public function __construct(
    private readonly EntityTypeManagerInterface $entityTypeManager,
    private readonly LoggerInterface $logger,
  ) {
    parent::__construct();
  }

  #[CLI\Command(name: 'site-tools:refresh-summaries')]
  #[CLI\Argument(name: 'bundle', description: 'The node bundle to process.')]
  #[CLI\Option(name: 'chunk', description: 'Number of entities per pass.')]
  #[CLI\Usage(name: 'drush site-tools:refresh-summaries article --chunk=50')]
  public function refreshSummaries(
    string $bundle,
    array $options = ['chunk' => 50],
  ): void {
    // Processing code goes here.
  }

}

The exact attribute syntax depends on the Drush version used by the project. Check the installed Drush documentation before copying an example between major versions. A command can also use older annotations on projects that have not moved to PHP attributes.

services:
  site_tools.commands:
    class: Drupal\site_tools\Commands\ContentMaintenanceCommands
    arguments: ['@entity_type.manager', '@logger.channel.site_tools']
    tags:
      - { name: drush.command }

A dedicated logger channel is preferable to sending every detail to standard output. Terminal output should show useful progress, while logs should preserve diagnostic information for later review.

Process Entities In Predictable Chunks

A cursor is safer than a page number for many maintenance jobs. The command stores the last processed entity ID and requests records with a higher ID on the next pass. If records are updated during processing, a cursor is less likely to skip or duplicate items than a changing offset.

The following example targets article nodes. The refreshSummary() method represents the business rule and should be replaced with the actual site-specific operation:

public function refreshSummaries(
  string $bundle,
  array $options = ['chunk' => 50],
): void {
  $chunk_size = max(1, (int) $options['chunk']);
  $storage = $this->entityTypeManager->getStorage('node');
  $last_id = 0;
  $processed = 0;

  do {
    $query = $storage->getQuery()
      ->accessCheck(FALSE)
      ->condition('type', $bundle)
      ->condition('nid', $last_id, '>')
      ->sort('nid', 'ASC')
      ->range(0, $chunk_size);

    $ids = $query->execute();

    if (!$ids) {
      break;
    }

    $entities = $storage->loadMultiple($ids);

    foreach ($entities as $entity) {
      try {
        $this->refreshSummary($entity);
        $entity->save();
        $processed++;
        $last_id = (int) $entity->id();
      }
      catch (\Throwable $exception) {
        $this->logger->error(
          'Could not process node @nid: @message',
          [
            '@nid' => $entity->id(),
            '@message' => $exception->getMessage(),
          ],
        );
      }
    }

    $this->output()->writeln(
      sprintf('Processed %d entities; cursor is %d.', $processed, $last_id)
    );

    $storage->resetCache(array_keys($entities));
  } while (count($ids) === $chunk_size);

  $this->logger->notice('Finished processing @count entities.', [
    '@count' => $processed,
  ]);
}

The method should validate the bundle and option values before processing. It should also define what happens when one entity fails. Continuing may be correct for independent records, but a migration that must remain internally consistent may need to stop immediately and return a non-zero exit status.

Avoid using range(0, $chunk_size) without a stable sort. Database row order is not guaranteed, and an unstable query can process records unpredictably. Also remember that accessCheck(FALSE) is intentional for administrative maintenance, so the command must be restricted to trusted operators.

Compare Processing Strategies

The best implementation depends on how long the task runs, whether it can be resumed, and how much feedback operators need. A short command that updates a few hundred records does not need the same machinery as a national membership website processing millions of rows.

Pattern	Best Use	Strength	Main Risk
Chunked Drush loop	Controlled maintenance and migrations	Simple, fast, easy to run manually	A failed process needs a cursor or rerun strategy
Drupal Batch API	Browser-driven administrative operations	Familiar progress interface	Web request timeouts and session dependence
Queue API with workers	Large or retryable workloads	Durable items and independent retries	More code and operational monitoring
Direct database update	Simple field transformations	Very fast for suitable operations	Bypasses entity hooks, validation, and revisions
Drush command calling a service	Reusable business logic	Keeps command thin and testable	Requires deliberate service design

A direct SQL update should be treated carefully. Drupal entity saves invoke hooks, field handling, revisions, search indexing, and other integrations. Bypassing that lifecycle can leave the site inconsistent even when the database query succeeds.

A useful compromise is to put the transformation in a service and let both a Drush command and a queue worker call it. This keeps the business rule in one place while allowing the project to change its execution method later.

Handle Errors, Transactions, And Resumability

A database transaction can protect a small group of related writes. For example, if updating a node also creates a related record, those changes should either both succeed or both roll back. A transaction around an entire multi-hour command is usually a poor choice because it holds locks and creates a large rollback scope.

Use one transaction per entity or per small chunk where appropriate. Do not assume that catching an exception automatically reverses a failed save; explicitly use the database connection’s transaction API when atomic behaviour is required.

Commands should produce a meaningful summary: processed records, skipped records, failures, and elapsed time. For unattended runs, write a machine-readable log or store a state marker. A shell script or CI job can then detect failure from the command’s exit code rather than parsing prose.

Resumability needs a clear definition. A cursor stored only in a local variable disappears when the process stops. For a task that may be restarted, store progress in State API, a dedicated tracking entity, or queue items. Include the operation version in the stored state so a later code change does not resume an old task with incompatible assumptions.

Make Production Runs Safe

Add a dry-run option before allowing a command to write data. In dry-run mode, load the same records and calculate the same changes, but report what would happen without calling save(). This exposes incorrect entity queries and unexpected bundles before production data is changed.

Use a small default chunk, then increase it after measuring memory, database load, and execution time. On a modest Australian VPS, a chunk of 25 or 50 may be safer than 500. A larger Sydney agency platform with a dedicated database may tolerate more, but capacity should be measured rather than assumed.

Take a database backup and test the command against a staging copy. Confirm that configuration is imported, required modules are enabled, and the command can be interrupted safely. Never test an unverified bulk update during a high-traffic period such as a major ticket release or a public campaign.

Content-related jobs should account for media and editorial workflows. For example, a command that changes image references should be reviewed alongside this Drupal media library guide, since media entities, file usage, alt text, and editorial permissions may all be involved.

The command should also protect private data in logs. Avoid printing full entity fields, email addresses, or uploaded filenames when they are not required. This is particularly important for Australian organisations handling health, education, membership, or customer information under the Privacy Act.

Test And Schedule The Command

Start with unit tests for the transformation service and kernel tests for entity queries. A kernel test can create nodes, run the command or service, and verify field values without depending on a complete production-like website. Include records that are missing optional fields, already processed, unpublished, translated, or subject to revisions.

Test interruption explicitly. Stop a command after one chunk, run it again, and confirm that records are not silently skipped. Test a failed entity and verify whether the intended behaviour is to continue, retry, or stop. Check that cache resets prevent stale entities from affecting later iterations.

For scheduling, use the project’s existing cron or deployment tooling. A server running Australian Eastern Time may shift between AEST and AEDT, so a task scheduled around business hours should be checked across daylight-saving changes. A Perth-based operation and a Melbourne-based editorial team may also use different maintenance windows.

A typical cron entry might call Drush from the project root:

15 2 * * * cd /var/www/site && vendor/bin/drush site-tools:refresh-summaries article --chunk=50 >> /var/log/site-tools.log 2>&1

Use an explicit PHP and Drush path where the hosting environment has several PHP versions. Ensure the cron user can read the Drupal settings, write required files, and access the database, but does not have broader server privileges than necessary.

Recommendations For Reliable Operations

A well-designed command is small, observable, and safe to rerun. Keep the Drush class focused on input, progress, and exit behaviour, while a service performs the actual content transformation. This structure prevents terminal-specific code from spreading through the application.

Before enabling an automated run, apply these practices:

Add --dry-run, --chunk, and, where useful, --limit options.
Use stable cursor-based queries with explicit sorting.
Reset entity caches after each processed chunk.
Log failures without exposing personal or confidential content.
Record progress when the operation must resume after interruption.

Treat each bulk command as a maintenance tool with a lifecycle: test it, document its permissions, monitor its output, and remove or disable it when the migration is complete. That discipline keeps Drupal administration predictable as the site, editorial team, and data volume grow.