Creating a Custom Drupal Service to Aggregate API Data
Drupal sites often need information from several external systems: product feeds, event platforms, weather services, stock APIs, review tools, or internal business applications. Placing those HTTP requests directly in controllers makes the code difficult to test, reuse and maintain. A custom Drupal service provides a single, dependable layer for retrieving, combining and normalising that data.
This approach is especially useful for Australian websites that bring together local stock levels, delivery estimates, venue listings or content from different providers. A site serving visitors in Sydney, Melbourne and Brisbane may need consistent handling of Australian Eastern time, Australian dollars and provider outages, even when the upstream APIs have different formats and response times.
Plan The Service Around A Clear Responsibility
A service should own the work of communicating with external APIs and turning their responses into a predictable internal structure. Controllers, block plugins and preprocess functions should ask for aggregated data rather than knowing endpoint URLs, authentication details, JSON field names or retry rules.
For example, an ApiAggregator service could request products from a catalogue API and promotions from a second endpoint, then return an array such as:
[
[
'id' => 'product-123',
'title' => 'Australian Cotton Shirt',
'price' => 89.95,
'currency' => 'AUD',
'available' => TRUE,
'source' => 'catalogue',
],
]
This normalised result gives Twig templates and Drupal render arrays a stable contract. If one supplier changes product_name to name, the transformation can be updated in the service without changing every template that displays the result.
Keep the service independent from presentation. It should not build markup, inspect the current route or return a complete render array unless that is its explicit purpose. A focused service can later be used by a JSON:API enhancement, a scheduled import, a Views integration or a command-line task.
Register A Drupal Service With Dependency Injection
Create a module, such as api_aggregator, with an api_aggregator.info.yml file:
name: API Aggregator
type: module
description: Aggregates data from external API providers.
core_version_requirement: ^10.3 || ^11
package: Custom
Register the service in api_aggregator.services.yml. Drupal will construct the object and inject the HTTP client, configuration factory and cache backend:
services:
api_aggregator.data:
class: Drupal\api_aggregator\Service\ApiAggregator
arguments:
- '@http_client'
- '@config.factory'
- '@cache.data'
- '@logger.factory'
The PHP class should implement ContainerInjectionInterface only when it is being created through a container-aware plugin or another special mechanism. Ordinary services do not need to implement that interface. Their dependencies are supplied through the constructor, which makes the class easier to test with mocked objects.
<?php
namespace Drupal\api_aggregator\Service;
use Drupal\Core\Cache\CacheBackendInterface;
use Drupal\Core\Config\ConfigFactoryInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use GuzzleHttp\ClientInterface;
final class ApiAggregator {
public function __construct(
private ClientInterface $httpClient,
private ConfigFactoryInterface $configFactory,
private CacheBackendInterface $cache,
LoggerChannelFactoryInterface $loggerFactory,
) {
$this->logger = $loggerFactory->get('api_aggregator');
}
private object $logger;
}
Drupal’s http_client service is built on Guzzle and handles the common transport layer. Injecting it is preferable to calling new Client() inside the class because Drupal can configure the client and automated tests can replace it with a mock. The configuration factory gives the service access to endpoint URLs, API keys and timeout settings without hard-coding deployment-specific values.
Fetch And Normalise Data From Several Providers
A public method can coordinate multiple private fetch methods. The requestJson() helper centralises HTTP options, status validation and JSON decoding, while each provider-specific method understands that provider’s response shape.
public function getProducts(): array {
$settings = $this->configFactory
->get('api_aggregator.settings');
$cache_id = 'api_aggregator:products';
if ($cached = $this->cache->get($cache_id)) {
return $cached->data;
}
$catalogue = $this->requestJson(
$settings->get('catalogue_endpoint'),
[
'headers' => [
'Accept' => 'application/json',
'Authorization' => 'Bearer ' . $settings->get('catalogue_token'),
],
'query' => ['market' => 'au'],
]
);
$promotions = $this->requestJson(
$settings->get('promotion_endpoint'),
[
'headers' => ['Accept' => 'application/json'],
'query' => ['currency' => 'AUD'],
]
);
$products = $this->mergeProducts($catalogue, $promotions);
$this->cache->set($cache_id, $products, time() + 900);
return $products;
}
The mergeProducts() method should use a reliable key, such as a supplier SKU or external ID, rather than matching titles. It can convert decimal strings to floats, map availability flags to booleans and apply a consistent date format. If an upstream source reports prices in cents, the conversion belongs here, before the data reaches Drupal’s display layer.
When API records have different identifiers, define a small mapping strategy. A catalogue item might use sku, while a promotion uses product_code; both can become id in the normalised result. For local retailers, this prevents errors when inventory is sourced from a Sydney warehouse system and promotional data comes from a national marketing platform.
Avoid silently accepting malformed responses. Check that the decoded value is an array, confirm required keys exist and discard incomplete records with a logged warning. A single invalid promotion should not necessarily prevent valid catalogue products from appearing, but authentication failures or a completely unavailable primary source may require a controlled exception.
Add Caching, Configuration And Failure Handling
External API calls should be cached because repeated page requests can consume rate limits and slow down the site. Drupal’s cache backend is suitable for short-lived responses, while longer-lived imports can use a queue or cron task. A fifteen-minute cache may work for product promotions; live transport information may need a shorter period, and an editorial feed may be acceptable for an hour.
Use cache tags when cached data should be invalidated by a configuration change, and cache contexts when the result differs by language, market or user state. If every visitor receives the same Australian catalogue, a single cache item is appropriate. If prices vary by customer group, the cache key must include that variation so one visitor cannot receive another visitor’s result.
Configuration belongs in Drupal configuration rather than source code:
api_aggregator.settings:
catalogue_endpoint: 'https://api.example.test/products'
promotion_endpoint: 'https://api.example.test/promotions'
request_timeout: 5
Sensitive tokens should be supplied through environment variables, settings.php overrides or a secrets manager instead of being committed to exported configuration. A configuration form can let administrators choose endpoints and cache periods, but it should never display a full secret after it has been saved. For Australian organisations, API handling should also be reviewed against contractual requirements and the Australian Privacy Principles when personal information is involved; operational notes can point administrators to the site’s privacy policy.
Guzzle throws exceptions for transport failures when configured with the default behaviour, so the service should catch RequestException and related runtime exceptions. Log useful diagnostic details without recording access tokens or personal data:
private function requestJson(string $url, array $options): array {
try {
$response = $this->httpClient->request('GET', $url, $options + [
'timeout' => 5,
'connect_timeout' => 2,
'http_errors' => TRUE,
]);
$data = json_decode((string) $response->getBody(), TRUE, 512, JSON_THROW_ON_ERROR);
if (!is_array($data)) {
throw new \UnexpectedValueException('API response was not an array.');
}
return $data;
}
catch (\Throwable $exception) {
$this->logger->error('API request failed: @message', [
'@message' => $exception->getMessage(),
]);
return [];
}
}
Returning an empty array is suitable when the consuming page can safely show no external results. Other applications should throw a domain-specific exception so a controller can display a status message or use a fallback cache. The important part is to choose the behaviour deliberately rather than allowing an unhandled provider error to produce a white screen.
Consume The Service In Drupal Features
A controller can request the service through its constructor and pass the returned data into a render array:
namespace Drupal\api_aggregator\Controller;
use Drupal\api_aggregator\Service\ApiAggregator;
use Drupal\Core\Controller\ControllerBase;
use Symfony\Component\DependencyInjection\ContainerInterface;
final class ProductController extends ControllerBase {
public function __construct(
private ApiAggregator $aggregator,
) {}
public static function create(ContainerInterface $container): static {
return new static(
$container->get('api_aggregator.data'),
);
}
public function listing(): array {
return [
'#theme' => 'api_aggregator_products',
'#products' => $this->aggregator->getProducts(),
'#cache' => [
'max-age' => 0,
],
];
}
}
The render cache metadata deserves attention. If the controller output contains externally cached data, set a suitable max-age, or attach cacheability metadata that reflects the service’s refresh period. A service cache and Drupal render cache operate at different layers; changing one does not automatically invalidate the other.
A block plugin is often a better choice for a reusable dashboard panel or homepage component. The plugin should inject the same service and render a small, focused result. Avoid making several independent API calls during one page request when the data can be collected once and shared through the service cache.
For scheduled aggregation, use a queue worker or cron task instead of waiting for a visitor’s request. The worker can fetch pages of records, store them as content or custom entities, and retry temporary failures. This pattern is useful for Australian event directories, where listings from Perth, Adelaide and regional areas may arrive at different times and should remain searchable even when one provider is briefly offline.
Test the service separately from Drupal’s theme layer. Unit tests can mock the HTTP client and verify that successful responses are normalised, malformed JSON is rejected and provider errors are logged. Kernel tests can confirm service registration, configuration access and Drupal cache behaviour. A small, injected service therefore becomes a maintainable integration boundary rather than a collection of API calls scattered across the site.
For editorial integrations, the same pattern can aggregate structured items from a publication or partner site, then expose only approved fields to Drupal editors. A visually oriented source such as style publication might contribute article metadata while Drupal controls moderation, local navigation and display rules. This separation keeps external content useful without allowing an upstream format to dictate the structure of the Australian website.