A normal JSON array must be parsed as one document, so file_get_contents() followed by json_decode() can exhaust memory. For export and import pipelines you control, newline-delimited JSON (NDJSON) allows one complete JSON object per line and true bounded-memory processing.
Last updated: September 26, 2026.
$file = new SplFileObject('events.ndjson', 'r');
$lineNumber = 0;
while (!$file->eof()) {
$line = trim($file->fgets());
$lineNumber++;
if ($line === '') continue;
try {
$record = json_decode($line, true, 512, JSON_THROW_ON_ERROR);
processRecord($record);
} catch (JsonException $e) {
throw new RuntimeException("Invalid JSON on line {$lineNumber}", 0, $e);
}
}Only the current line and decoded record are held in memory. The line number makes malformed input actionable without logging sensitive record contents.
Choose a streamable format
NDJSON works when each record can stand alone and does not contain literal unescaped line breaks. A large conventional array such as [{...},{...}] needs a streaming parser that understands nesting and strings; splitting on commas is incorrect. If you control the producer, NDJSON is usually simpler to resume and inspect.
Process in restartable batches
Validate required fields, write a bounded batch in one database transaction, record progress, then release the batch. Choose an idempotency key so retrying a partially processed file does not duplicate records. Apply limits to line length, nesting depth, record count, and total bytes.
- Open the file from a controlled path.
- Validate UTF-8 and JSON syntax for each record.
- Transform and write a bounded batch.
- Commit progress or store the last completed record ID.
- Move completed and failed files to separate managed locations.
Handle encoding and cleanup
PHP’s JSON functions require valid UTF-8. Repair a documented source encoding before parsing or reject invalid input using the invalid UTF-8 JSON workflow. Close or release handles when finished and apply the existing temporary-file cleanup strategy. Avoid calling json_validate() immediately before json_decode(); the PHP manual notes that this parses the same data twice.
