Using Drupal's Migrate API to import CSV data into nodes
Drupal’s Migrate API turns a structured source file into Drupal entities through a repeatable pipeline. Each row passes through a source plugin, optional processing plugins and a destination plugin that creates or updates the final object. For a controlled import of CSV records into nodes, that separation makes field mapping easier to review than a one-off script and leaves a migration record that can be rerun and monitored.
This guide uses a common Australian scenario: a council, venue or community-market organiser importing stallholders, events, prices and contact details for listings in Sydney or Melbourne. The same principles apply to supplier catalogues, member directories and training records. A technical owner can prepare the source and migration definition, while site editors can review the resulting content types, fields and access rules without becoming full-time developers.
The important first step is to treat the CSV as data, not as a miniature database. Headers, identifiers, date formats and missing values need explicit decisions before anything is imported. Those decisions matter when a file contains Australian mobile numbers, AUD prices or DD/MM/YYYY dates, and when downstream systems expose the nodes to staff, members or the public.
Planning a repeatable node import
Start with the destination, not the spreadsheet. List every node type involved and the fields each row must populate. For a market event, that might include a title, summary, start date, venue, organiser reference, price and accessibility information. Check the target field types before designing transformations: a Unix timestamp stored in a plain-text field will fail validation, while a multi-value field requires an array rather than one comma-separated string.
Choose a stable source identifier for every row. A numeric ID supplied by the original system is preferable to a title, organiser name or venue because those values may change or contain duplicates. Migrate can use that identifier to maintain a map between source records and destination nodes, which is essential when the file will be imported again after corrections.
Define what happens when data disappears from the CSV. A repeat import should not automatically delete a node merely because its row is missing. Missing records may indicate a bad export rather than a deliberate removal. Retaining historical nodes, recording source status or running an explicit reconciliation process gives the site owner control over destructive changes.
Preparing Australian CSV data
Save the source as UTF-8 CSV with one header row and consistent column names. Remove any byte-order mark that an application such as Excel has added, check that quoted fields containing commas contain one pair of quotes, and avoid mixing line endings within the same file. Duplicate headings, blank rows and inconsistent capitalisation can otherwise create ambiguous source fields that are difficult to diagnose after the import.
Australian formats deserve particular attention. Dates are commonly written as DD/MM/YYYY, prices normally include AUD, and mobile numbers often begin with 04. Decide whether to retain the leading zero or convert numbers to international E.164 format, such as +614…. Event times also need an explicit timezone: Sydney and Melbourne observe daylight-saving changes, so a fixed UTC conversion can move a local event to the wrong day or hour.
Personal and commercial information should be reviewed before the file reaches Drupal. Have the project owner assess whether the Privacy Act 1988 (Cth) applies to member, attendee or staff records, and import only fields required for the stated purpose. Addresses used later for commercial email may also bring the Spam Act 2003 into consideration. Imported prices and product claims remain subject to the Australian Consumer Law, regardless of how efficiently the data was loaded.
Defining and testing the migration
Drupal’s core Migrate module provides the framework, while the contributed migrate_source_csv module supplies the CSV source plugin. Install and enable both modules, then place the migration definition in a small custom module so it can be deployed consistently. Using a separate module also avoids storing migration configuration as an unexplained collection of files in an active site’s configuration database.
A simplified migration definition for market events might look like this:
id: import_market_events
label: Import Australian market events
source:
plugin: csv
path: /var/data/market-events.csv
header_row_count: 1
keys:
- source_id
process:
field_event_date:
plugin: format_date
source: start_date
format: 'd/m/Y'
timezone: Australia/Sydney
destination:
plugin: node
default_bundle: event
The CSV should contain headers such as source_id, title, body and start_date, matching the names referenced by the migration. The event value is the machine name of the destination content type, while field_event_date must match the machine name of the corresponding field. Additional processing plugins can convert values, split multi-value fields, look up taxonomy terms or apply default values without changing the original CSV.
Test against a representative copy rather than the full production file. Check migrated node counts, field values, formatting, relationships and unpublished or moderation states. A successful command can still produce incorrect content if the source header is mapped to the wrong destination property, so inspecting sample nodes is more reliable than relying on the absence of an error message.
Running, updating and integrating imports
Use the migration status screen or Drush to review dependencies and completed items before importing. A typical Drush workflow starts with drush migrate:status, followed by drush migrate:import import_market_events. Run the first import in a development or staging environment, confirm the results and document the source file’s checksum. That checksum makes it possible to identify exactly which version of the CSV produced a given set of nodes.
Re-running a migration with a stable source key allows Migrate’s mapping to locate previously created destination entities. Confirm the update behaviour in a test site: a changed row should update the mapped node, and an unchanged row should not create a second copy. A flat CSV has no query to poll for changes automatically, so do not assume high-water processing will behave as it does with a database source. Incremental imports may require filtering changed rows or maintaining a separate change log.
When imported listings also need to feed a booking or display application, exposing Drupal’s normal entity data through JSON:API is often preferable to maintaining a separate endpoint. This REST API endpoint guide offers useful Drupal-side context for deciding how external applications should read the imported nodes.
Keep the CSV outside the public web root and deliver it through a controlled channel such as SFTP. Restrict read permissions because the source may contain information that was deliberately excluded from the published node. Record each run’s operator, timestamp, source checksum, row count and outcome. Where the source and destination plugins support it, rehearse rollback in staging before relying on it for a production correction.
Comparing common import approaches
Several approaches can perform the import, but they suit different levels of control and operational complexity. Migrate is particularly useful when field transformations, source-to-destination mappings and a documented execution history matter. It also allows the import definition to live alongside other custom Drupal code.
Feeds provides a more familiar interface for routine editorial imports and can suit teams that primarily need to map columns and schedule recurring jobs. A custom module is useful when the import requires unusual validation, external API calls or high-volume processing. A separate script can move very large files quickly, but it must handle Drupal entity creation, access controls and updates correctly without the protections and visibility provided by the migration framework.
The practical differences are:
| Approach | Strongest fit | Main advantages | Main limitations |
|---|---|---|---|
| Migrate API with a CSV source | Controlled or repeatable structured imports | Explicit transformations, mappings, status tracking and dependency handling | Requires configuration knowledge and careful testing |
| Feeds module | Regular imports managed by site editors | Accessible interface and useful scheduling options | Complex migration dependencies and rollback workflows can be harder to model |
| Custom Drupal module | Bespoke logic or integration with other systems | Full control over validation, queues and processing | Requires ongoing development, testing and security review |
| External script or database loader | Very large files with simple requirements | Fast bulk processing and broad language support | Bypasses normal Drupal workflows unless carefully integrated |
For an existing Drupal site with established content types, the Migrate API and CSV source plugin usually offer the strongest balance of control and maintainability. The choice should reflect the file size, transformation complexity, frequency of updates and the number of people responsible for operating the import.
A dependable import checklist
A small pilot should use rows containing the most awkward real-world records: missing optional values, long descriptions, accents, embedded quotes and unusual dates. Include rows that will change between imports, because update behaviour is usually harder to verify than initial creation. Compare a sample of source records with their destination nodes and investigate every mismatch before increasing the batch size.
Once the migration behaves as expected, apply the following controls:
- Test with a representative subset before processing the complete file.
- Give every record an immutable source identifier that will not change.
- Validate encoding, headers, dates, currency values and quoted delimiters.
- Store the CSV outside the web root and restrict access to sensitive fields.
- Compare imported node counts and representative field values with the source.
- Record the migration version, source checksum, operator, timestamp and outcome of each run.
Keep the migration definition, sample CSV and operating instructions together in version control. A documented and repeatable import is safer than a sequence of manual uploads, especially when a council, small business or community organisation needs to explain where its node data came from and how it can be corrected.