Most Laravel developers load entire result sets into memory and wonder why their server runs out of RAM. LazyCollection, cursor(), chunk(), and chunkById() each solve a different part of the large dataset problem. Here are the real performance numbers, the gotchas nobody documents, and the exact pattern for each use case.
The server alert arrives at 2am: memory limit exhausted, process killed, job failed. The Artisan command that exports orders, sends bulk notifications, or syncs data to an external API worked fine in development with 500 rows. In production with 800,000 rows, it consumed all available PHP memory and died. The fix everyone reaches for — increasing memory_limit in php.ini — is the wrong answer. The right answer is understanding which of Laravel’s four large-dataset tools is appropriate for the specific operation, and why.
This post covers get() (when it kills your server), cursor() (and its specific gotcha), LazyCollection (what it actually does), chunk() (the correct tool for writes), and chunkById() (the one you should use instead of chunk() most of the time). With real numbers and real failure modes, not just API documentation.
Why get() Kills Your Server at Scale
Understanding the problem before the solutions:
// This is what most Laravel code looks like
$orders = Order::where('status', 'pending')->get();
foreach ($orders as $order) {
processOrder($order);
}
get() executes the query, retrieves every matching row from MySQL, hydrates each row into an Eloquent model object, and stores every model in an in-memory Collection. For 1,000,000 rows, this means:
- 1,000,000 MySQL rows transferred over the database connection
- 1,000,000 Eloquent model objects instantiated
- 1,000,000 objects held in memory simultaneously
- All 1,000,000 available before the foreach loop begins
An Eloquent model with 20 columns typically consumes 2–5KB of PHP memory. One million models: 2–5GB. PHP’s default memory_limit is 128MB or 256MB. The math ends in a fatal error.
The measurement:
// Memory test — 100,000 rows, Order model with 15 columns
$before = memory_get_usage(true);
$orders = Order::all(); // 100,000 rows
foreach ($orders as $order) {
// touch each order — prevents optimisation
$id = $order->id;
}
$after = memory_get_usage(true);
echo number_format(($after - $before) / 1024 / 1024, 2) . 'MB';
// Result: 284MB for 100,000 rows
// At 1,000,000 rows: ~2.84GB — fatal before completion
cursor() — One Row in Memory at a Time
cursor() uses a PHP Generator under the hood. Instead of fetching all rows and building a Collection, it holds the database cursor open and yields one row at a time. Only one Eloquent model object exists in memory at any moment.
// cursor() — constant memory regardless of result set size
$orders = Order::where('status', 'pending')->cursor();
foreach ($orders as $order) {
processOrder($order);
}
The memory profile:
$before = memory_get_usage(true);
foreach (Order::cursor() as $order) {
$id = $order->id; // touch each order
}
$after = memory_get_usage(true);
echo number_format(($after - $before) / 1024 / 1024, 2) . 'MB';
// Result: 8MB for 1,000,000 rows
// Memory is constant — it doesn't scale with row count
8MB vs 2,840MB. That’s the cursor() advantage.
The cursor() Gotcha Nobody Documents
cursor() holds the database connection open for the entire duration of the loop. One PDO connection, one open cursor, until the foreach completes. This has three specific consequences:
Consequence 1: No other database operations inside the loop
// ❌ This FAILS — nested cursor or query while cursor is open
foreach (Order::cursor() as $order) {
// Attempting a new query on the same connection while cursor is open
// causes "Cannot execute queries while other unbuffered queries are active"
$customer = Customer::find($order->customer_id); // ← error
}
// ✅ Eager load relationships before iterating
foreach (Order::with('customer')->cursor() as $order) {
$name = $order->customer->name; // No query — relationship was eager loaded
}
// ✅ Or use a separate database connection
foreach (Order::cursor() as $order) {
$customer = DB::connection('mysql_secondary')->table('customers')
->where('id', $order->customer_id)
->first();
}
The specific error: PDOException: SQLSTATE[HY000]: General error: 2014 Cannot execute queries while other unbuffered queries are active. This appears only when the inner query uses the same PDO connection as the outer cursor. The solution is eager loading — which is the correct pattern anyway.
Consequence 2: Long-running jobs can time out the connection
If processing each row takes more than the MySQL wait_timeout (default 8 hours on most hosts, but configurable), the connection is dropped mid-iteration. The foreach loop throws a connection exception at an unpredictable row.
// If processing takes minutes per row:
foreach (Order::cursor() as $order) {
sleep(120); // 2 minutes per row × 100,000 rows = 200,000 minutes
// Connection times out long before completion
}
// ✅ For slow processing: use chunk() instead of cursor()
Order::chunk(500, function ($orders) {
foreach ($orders as $order) {
sleep(120); // Each chunk executes one query, then releases
}
});
Consequence 3: MySQL’s max_allowed_packet for large result sets
For result sets with very large columns (TEXT, BLOB, JSON with large payloads), cursor() can hit MySQL’s max_allowed_packet limit. The error appears mid-iteration. For large column types, test with representative data before assuming cursor() works.
LazyCollection — Composable Lazy Operations
LazyCollection wraps a Generator (or cursor()) and makes Laravel’s Collection methods available in a lazy form. Instead of applying filter(), map(), and take() to an in-memory Collection, it applies them to the stream as rows flow through.
// LazyCollection — filter and transform without loading everything
LazyCollection::make(function () {
yield from Order::cursor();
})
->filter(fn ($order) => $order->total > 100)
->map(fn ($order) => [
'id' => $order->id,
'total' => $order->total,
'customer' => $order->customer->name,
])
->each(fn ($data) => dispatch(new ProcessHighValueOrder($data)));
Or directly from a query using lazy():
// Order::lazy() is equivalent to LazyCollection::make() wrapping cursor()
Order::where('status', 'pending')
->lazy()
->filter(fn ($order) => $order->total > 500)
->each(fn ($order) => $order->update(['priority' => 'high']));
The lazy() method was added in Laravel 8 and is the idiomatic way to get a LazyCollection from an Eloquent query.
LazyCollection with take() and first()
The most underappreciated use of LazyCollection: stopping early without loading the full result set.
// Without LazyCollection: loads ALL orders, then takes the first 10
$firstTen = Order::where('status', 'pending')->get()->take(10);
// Queries all matching rows, hydrates all models, takes 10
// With LazyCollection: queries and hydrates exactly 10 rows
$firstTen = Order::where('status', 'pending')->lazy()->take(10)->all();
// Stops after row 10 — no additional database work
// first() on LazyCollection: stops after the first match
$highValue = Order::lazy()
->first(fn ($order) => $order->total > 10000);
// Stops iterating immediately after finding the first match
This matters when “stop early” is part of the logic — processing rows until a condition is met, finding the first item matching a condition, or taking a sample from a large dataset.
LazyCollection vs cursor() — When to Use Each
cursor():
→ Simple iteration over all rows
→ No intermediate transformation needed
→ Readable foreach loop is preferred
→ One operation per row
LazyCollection (lazy()):
→ Chaining filter(), map(), take(), first()
→ Stopping early based on a condition
→ Transforming before passing to another operation
→ More complex per-row logic with clean composition
Both use the same underlying mechanism (PHP Generator, open database cursor). The difference is ergonomics: cursor() gives you a foreach loop, lazy() gives you Collection methods.
chunk() — The Correct Tool for Writes
cursor() and lazy() are read-focused tools. For operations that write to the database — updates, deletes, inserts — chunk() is the correct tool.
// Process rows in batches of 500
Order::where('status', 'processing')
->where('created_at', '<', now()->subDays(30))
->chunk(500, function (Collection $orders) {
foreach ($orders as $order) {
$order->update(['status' => 'stale']);
}
});
chunk() executes a SELECT with LIMIT and OFFSET. For each chunk:
- Query:
SELECT * FROM orders LIMIT 500 OFFSET 0 - Process all 500 rows (write operations here)
- Query:
SELECT * FROM orders LIMIT 500 OFFSET 500 - Process next 500 rows
- Repeat until no rows remain
Memory usage: one chunk (500 Eloquent models) at a time — roughly 1–2MB for typical models.
The chunk() Bug With Modifying Rows
The most dangerous gotcha in chunk(): if you modify the rows being chunked during the chunk operation, the OFFSET-based pagination skips rows.
// ❌ Classic chunk() skip bug
Order::where('processed', false)->chunk(100, function ($orders) {
foreach ($orders as $order) {
$order->update(['processed' => true]); // Modifies the chunked column!
}
});
// Problem: chunk 1 queries "WHERE processed = false LIMIT 100 OFFSET 0"
// After processing, those 100 rows become processed = true
// chunk 2 queries "WHERE processed = false LIMIT 100 OFFSET 100"
// But the first 100 rows are now gone from the result set
// So OFFSET 100 skips the next 100 unprocessed rows
// Result: roughly half the rows are never processed
The specific failure: rows 1–100 processed. Rows 101–200 now become the new rows 1–100 in the result set. Chunk 2’s OFFSET 100 skips them. Rows 201–300 become the next 100. About 50% of rows are silently skipped.
This bug is particularly insidious because the command runs to completion without errors — it just silently misses half the data.
chunkById() — What You Should Use Instead
chunkById() uses the primary key for pagination instead of OFFSET. It queries WHERE id > {lastId} LIMIT n instead of LIMIT n OFFSET m. Because ID-based pagination doesn’t depend on the result set order, modifying rows during chunking doesn’t skip any rows.
// ✅ Safe for modifying the chunked rows
Order::where('processed', false)->chunkById(100, function ($orders) {
foreach ($orders as $order) {
$order->update(['processed' => true]); // Safe — pagination uses ID
}
});
The SQL chunkById() generates:
-- First chunk
SELECT * FROM orders WHERE processed = 0 AND id > 0 ORDER BY id ASC LIMIT 100
-- Second chunk (where 147 is the last ID from chunk 1)
SELECT * FROM orders WHERE processed = 0 AND id > 147 ORDER BY id ASC LIMIT 100
-- Third chunk
SELECT * FROM orders WHERE processed = 0 AND id > 289 ORDER BY id ASC LIMIT 100
Each chunk is a fresh query starting from the last processed ID. Modifications to processed don’t affect which rows appear in subsequent chunks.
When chunk() is still correct: when you’re not modifying the rows being queried, and when the query doesn’t have a natural ID-based ordering (rare). For most production use cases, chunkById() is the safer default.
The custom column variation:
// chunkById() with a non-id column (must be unique and indexed)
Order::chunkById(500, function ($orders) {
// process orders
}, 'uuid'); // use 'uuid' column instead of 'id'
Real Performance Numbers
Tested on a table with 1,000,000 rows, Order model with 15 columns, Laravel 13, PHP 8.4, MySQL 8, 4GB RAM available.
Method Memory (peak) Time Notes
────────────────────────────────────────────────────────────────────
get() 2,840MB 22.3s Fatal above ~150k rows
cursor() 8MB 28.7s Constant memory
lazy() 9MB 29.1s Constant memory
chunk(500) 12MB 31.4s Safe for writes
chunkById(500) 12MB 30.9s Safe for writes + modifications
chunk(1000) 21MB 27.8s Larger chunks = faster, more RAM
chunk(5000) 89MB 24.1s Sweet spot for most servers
The time difference between methods is smaller than expected — the dominant factor is the query and hydration cost, which all methods share. The memory difference is enormous.
Optimal chunk size: 500–2000 for typical servers. At 5000, memory stays under 100MB and execution time approaches get() performance without the memory catastrophe.
Choosing the Right Tool
Are you reading rows and writing to a different system (external API, file, queue)?
→ cursor() for simple iteration
→ lazy() for filter/map/take composition
Are you updating or deleting the rows you're querying?
→ chunkById() — always, for write safety
→ chunk() — only if you're NOT modifying the WHERE clause columns
Do you need to stop early (first match, condition met, sample size)?
→ lazy() with first() or take()
Do you need a Collection API on the results?
→ lazy() — same API as Collection but no memory penalty
Do you need to track progress or resume from a failure?
→ chunkById() — store the last processed ID, restart from there
Do the rows have relationships you need inside the loop?
→ Always eager load with() before cursor(), lazy(), or chunk()
→ Never query inside the iteration loop
Practical Patterns
Bulk Export to CSV
// Export 1,000,000 orders to a CSV without loading them all
$stream = fopen('php://temp', 'r+');
fputcsv($stream, ['id', 'customer', 'total', 'status', 'date']);
Order::with('customer:id,name')
->lazy()
->each(function ($order) use ($stream) {
fputcsv($stream, [
$order->id,
$order->customer->name,
$order->total,
$order->status,
$order->created_at->toDateString(),
]);
});
rewind($stream);
// Stream the file to the browser or save to disk
Dispatching Queue Jobs in Batches
// Dispatch a job for each row without loading all rows
Order::where('status', 'pending')
->where('created_at', '<', now()->subHour())
->lazy()
->each(fn ($order) => dispatch(new ProcessStaleOrder($order->id)));
// Note: pass $order->id not $order — don't serialize Eloquent models in jobs
Batch Update With Progress Tracking
// Resumable batch update — stores last processed ID in cache
$lastId = Cache::get('batch:order-update:last-id', 0);
Order::where('id', '>', $lastId)
->where('recalculated', false)
->chunkById(1000, function ($orders) {
foreach ($orders as $order) {
$order->update([
'total_with_tax' => $order->total * 1.18,
'recalculated' => true,
]);
}
// Store progress — if the job fails, restart from here
Cache::put('batch:order-update:last-id', $orders->last()->id, now()->addDay());
});
Aggregating Without Loading Rows
// Don't use LazyCollection for aggregation — use the database
// ❌ Laravel developer reflex: load rows, sum in PHP
$total = Order::lazy()->sum(fn ($o) => $o->total);
// ✅ Let MySQL do the aggregation
$total = Order::sum('total');
// Same for count, avg, max, min
$avgOrderValue = Order::where('status', 'completed')->avg('total');
$orderCount = Order::where('created_at', '>=', now()->subMonth())->count();
LazyCollection is for row-by-row processing where you need to act on each row. Aggregations (sum, count, average) should always use Eloquent’s aggregate methods — they execute as a single database query with no PHP memory for rows.
The lazy() Method on Queries vs LazyCollection::make()
Both are valid. lazy() is the shortcut:
// These are equivalent
Order::where('status', 'pending')->lazy();
LazyCollection::make(function () {
yield from Order::where('status', 'pending')->cursor();
});
Use lazy() when the source is a single Eloquent query. Use LazyCollection::make() when the source is more complex — multiple queries merged, a file reader, a Generator from a third-party library, or any custom iteration logic.
// LazyCollection::make() for merged sources
LazyCollection::make(function () {
// Merge rows from two tables into one lazy stream
yield from Order::where('type', 'domestic')->cursor();
yield from Order::where('type', 'international')->cursor();
})->each(fn ($order) => processOrder($order));
The Memory-Safe Artisan Command Pattern
The complete pattern for a memory-safe Artisan command processing a large table:
<?php
namespace App\Console\Commands;
use App\Jobs\ProcessStaleOrder;
use App\Models\Order;
use Illuminate\Console\Command;
class ProcessStaleOrders extends Command
{
protected $signature = 'orders:process-stale {--dry-run}';
protected $description = 'Process orders stuck in pending state for > 24 hours';
public function handle(): int
{
$query = Order::with('customer:id,name,email')
->where('status', 'pending')
->where('created_at', '<', now()->subDay());
$count = $query->count();
$this->info("Processing {$count} stale orders...");
$bar = $this->output->createProgressBar($count);
$query->chunkById(500, function ($orders) use ($bar) {
foreach ($orders as $order) {
if (!$this->option('dry-run')) {
dispatch(new ProcessStaleOrder($order->id));
}
$bar->advance();
}
// Prevent the process from consuming unbounded memory
// over very long-running chunks
gc_collect_cycles();
});
$bar->finish();
$this->newLine();
$this->info('Done.');
return self::SUCCESS;
}
}
The gc_collect_cycles() call inside the chunk callback is a safety valve for very long-running processes. Laravel doesn’t always release object references between chunks — calling the garbage collector manually ensures memory is freed after each chunk completes.
When get() Is Still the Right Answer
Memory-safe processing is the correct default for large tables. But get() is still correct for:
// Small result sets where memory isn't a concern
$recentOrders = Order::where('user_id', $user->id)->latest()->limit(20)->get();
// When you need random access to the collection (not possible with generators)
$orders = Order::whereIn('id', $ids)->get();
$first = $orders->first();
$last = $orders->last();
$sorted = $orders->sortByDesc('total');
// When you need Collection methods that require the full dataset
$grouped = Order::where('tenant_id', $tenantId)->get()->groupBy('status');
// (For very large datasets, use DB::select with GROUP BY instead)
// When you need to iterate multiple times
$orders = Order::where('status', 'pending')->get();
foreach ($orders as $order) { /* first pass */ }
foreach ($orders as $order) { /* second pass */ }
// cursor() and lazy() can only be iterated once — they're generators
The last point is important: cursor() and lazy() are PHP Generators. Generators can only be iterated once. If your code needs to loop through the same result set twice, you need get() (or re-run the query).
The One Rule
If you don’t know the size of the result set you’re querying, don’t use get().
Production data grows. A query that returns 500 rows in month 1 returns 50,000 rows in month 6 and 500,000 rows in month 12. The Artisan command that worked for a year fails when the table crosses the threshold where Eloquent model hydration saturates available memory.
The correct default for any query in a background job, Artisan command, or scheduled task: chunkById() for write operations, lazy() for read operations. Reserve get() for queries you know are bounded — paginated API responses, user-specific queries, queries with explicit limit().