Laravel Chores: Resumable Bulk Data Operations | Mohamed Said [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://msaied.com) [ Home ](https://msaied.com) [ Projects ](https://msaied.com/projects) [ Articles ](https://msaied.com/articles) [ Certificates ](https://msaied.com/certificates) [ Contact ](https://msaied.com#contact-section) [ ](https://github.com/EG-Mohamed) [ Home ](https://msaied.com) [ Projects ](https://msaied.com/projects) [ Articles ](https://msaied.com/articles) [ Certificates ](https://msaied.com/certificates) [ Contact ](https://msaied.com#contact-section) [ home ](https://msaied.com) [ articles ](https://msaied.com/articles) Laravel Chores: Resumable, Checkpointed Data Operations for Large Datasets On this page 1. [ The Problem: Interrupted Bulk Operations ](#the-problem-interrupted-bulk-operations) 2. [ Key Features ](#key-features) 3. [ Writing a Chore ](#writing-a-chore) 4. [ Running, Pausing, and Resuming ](#running-pausing-and-resuming) 5. [ Handling Failures ](#handling-failures) 6. [ Installation ](#installation) 7. [ Limitations to Know ](#limitations-to-know) 8. [ Takeaways ](#takeaways) ![Laravel Chores: Resumable, Checkpointed Data Operations for Large Datasets](https://cdn.msaied.com/570/2aac658a2b082a4442b384819c0b54fe.png) [ Laravel ](https://msaied.com/articles?category=laravel) [ Composer Pacakge ](https://msaied.com/articles?category=composer-pacakge) #Laravel #PHP #Composer Package #Data Migration #Artisan #Bulk Operations Laravel Chores: Resumable, Checkpointed Data Operations for Large Datasets ============================================================================ 18 Aug 2026 4 min read ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said Table of contents 1. [ 01 The Problem: Interrupted Bulk Operations ](#the-problem-interrupted-bulk-operations) 2. [ 02 Key Features ](#key-features) 3. [ 03 Writing a Chore ](#writing-a-chore) 4. [ 04 Running, Pausing, and Resuming ](#running-pausing-and-resuming) 5. [ 05 Handling Failures ](#handling-failures) 6. [ 06 Installation ](#installation) 7. [ 07 Limitations to Know ](#limitations-to-know) 8. [ 08 Takeaways ](#takeaways) The Problem: Interrupted Bulk Operations ---------------------------------------- Backfilling ten million rows is straightforward until a deploy restarts the server four hours in. Without checkpointing, you're left guessing which records were already processed and restarting from scratch. **Laravel Chores**, by [Amr Lotfy Saleh](https://github.com/AmrLotfy), solves this by persisting progress to the database after every batch, inspired by Shopify's `maintenance_tasks` gem for Rails. Key Features ------------ - **Checkpointed progress** — the last processed ID is written to a `chore_runs` table after each batch; a crash loses at most one batch - **Keyset pagination** — pages by primary key rather than offset, avoiding the classic bug where updating rows shifts them out of the current chunk - **Failure isolation** — exceptions are logged to a failures table and skipped; the run continues - **No extra infrastructure** — state lives in your database; no Redis or queue workers required - **Six Artisan commands** — scaffold, run, list, pause, inspect failures, and retry - **CI-friendly output** — JSON mode and distinct exit codes: `0` clean, `1` completed with failures, `2` fatal Writing a Chore --------------- A chore class needs two methods: `collection()` returns the query, and `process()` handles one record. Scaffold one with: ```bash php artisan make:chore NormalizePhoneNumbers ``` ```php namespace App\Chores; use AmrLotfy\Chores\Chore; use App\Models\User; use Illuminate\Contracts\Database\Eloquent\Builder; class NormalizePhoneNumbers extends Chore { public int $batchSize = 500; public function collection(): Builder { return User::whereNotNull('phone') ->where('phone', 'not like', '+%'); } public function process($record): void { $record->update([ 'phone' => PhoneNumber::parse($record->phone, 'EG')->toE164(), ]); } } ``` Batching, progress tracking, and failure logging are handled by the package around your two methods. The default batch size is 500 and can be overridden per class or via the config file. Running, Pausing, and Resuming ------------------------------ ```bash php artisan chore:run NormalizePhoneNumbers ``` Progress checkpoints after each batch. If the run is interrupted, the same command resumes from the last saved position. Use `chore:pause` to stop at the next batch boundary, and `chore:list` to view available chores and their run history. > **Idempotency note:** The checkpoint is per batch, not per record. Records inside an in-flight batch may be re-examined after a resume. Write `process()` to be idempotent where possible — the phone normalization example above is safe because already-normalized numbers no longer match the `collection()` query. For recurring work, compose with Laravel's task scheduler: ```php $schedule->command('chore:run PurgeExpiredRecords')->monthly(); ``` Handling Failures ----------------- A record that throws an exception is logged and skipped; the run continues. Afterwards: ```bash php artisan chore:failures NormalizePhoneNumbers php artisan chore:retry NormalizePhoneNumbers ``` This split is valuable on long runs: a hundred malformed records out of ten million shouldn't kill a four-hour job, and retrying only the failures after a fix is far cheaper than rerunning everything. Installation ------------ Requires PHP 8.2+, Laravel 12 or 13, and supports MySQL, PostgreSQL, and SQLite. ```bash composer require amrlotfy/laravel-chores php artisan vendor:publish --tag=chores-migrations php artisan migrate ``` The config file lets you adjust the chore class directory, default batch size, table names, and a sleep interval between batches for throttling load on busy databases. ### Limitations to Know Chores run in the foreground with a single worker per chore, and the collection requires an orderable primary key (auto-increment or ULID). Queued and parallel execution are on the roadmap. Takeaways --------- - Checkpointed batches mean interrupted runs resume rather than restart - Keyset pagination prevents the offset-drift bug on mutable datasets - Failure isolation keeps a long job alive despite bad records - Zero infrastructure overhead — just two database tables - Idempotent `process()` methods are the key to safe resumption --- Source: [Laravel Chores: Resumable Data Operations and Cleanups — Laravel News](https://laravel-news.com/laravel-chores) Found this useful? [ ](https://twitter.com/intent/tweet?url=https%3A%2F%2Fmsaied.com%2Farticles%2Flaravel-chores-resumable-checkpointed-data-operations-for-large-datasets&text=Laravel+Chores%3A+Resumable%2C+Checkpointed+Data+Operations+for+Large+Datasets) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fmsaied.com%2Farticles%2Flaravel-chores-resumable-checkpointed-data-operations-for-large-datasets) Frequently Asked Questions ---------------------------- 3 questions Q01 What happens if a Laravel Chores run is interrupted mid-way? Laravel Chores writes the last processed ID to a `chore_runs` database table after each batch. When you re-run the same chore command, it picks up from that checkpoint rather than starting over. At most one batch of work is repeated. Q02 Does Laravel Chores require Redis or a queue worker? No. All state is stored in your existing database using two tables. There is no dependency on Redis, queue workers, or any external service. Q03 How does Laravel Chores handle records that throw exceptions during processing? Exceptions are caught per record, logged to a failures table, and the run continues. After the job finishes you can inspect failures with `chore:failures` and reprocess them with `chore:retry`, avoiding a full re-run. Continue reading More Articles --------------- [ View all ](https://msaied.com/articles) [ ![Read-Through Disks and Debounced Listeners in Laravel 13.26](https://cdn.msaied.com/568/580ae69765054f5f750614e4d977ff56.png) Laravel 13.26 Filesystem Queue ### Read-Through Disks and Debounced Listeners in Laravel 13.26 Laravel 13.26 ships a read-through filesystem driver for lazy storage migration, extends #\[DebounceFor\] to que... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 18 Aug 2026 4 min read Read ](https://msaied.com/articles/read-through-disks-and-debounced-listeners-in-laravel-1326) [ ![Laravel Read-Through Filesystem: Lazy Storage Migration Between Buckets](https://cdn.msaied.com/569/7759742f96ca5582353a70049ec950e1.png) Laravel Filesystem Storage Migration ### Laravel Read-Through Filesystem: Lazy Storage Migration Between Buckets Laravel 13.26 ships a read-through filesystem driver that transparently checks a primary disk, falls back to a... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 18 Aug 2026 4 min read Read ](https://msaied.com/articles/laravel-read-through-filesystem-lazy-storage-migration-between-buckets) [ ![Object Storage Migrations with Laravel's Read-Through Filesystem](https://cdn.msaied.com/565/f830d15d4a1287d381fa05e631ea2aba.png) Laravel 13 Object Storage S3 ### Object Storage Migrations with Laravel's Read-Through Filesystem Laravel 13 introduces a read-through filesystem driver that lets you migrate from S3 to R2 without downtime. N... ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MJ.jpg) Mohamed Said 18 Aug 2026 4 min read Read ](https://msaied.com/articles/object-storage-migrations-with-laravels-read-through-filesystem) [ ![Mohamed Said](https://cdn.msaied.com/01KT78WE565VEMM3PSNQAAB0MH.png) Mohamed Said Laravel Backend Engineer ](https://msaied.com)Senior Backend Engineer specializing in Laravel, scalable SaaS platforms, APIs, and cloud infrastructure. I build secure, high-performance web applications that help businesses grow. Explore - [Home](https://msaied.com) - [Projects](https://msaied.com/projects) - [Articles](https://msaied.com/articles) - [Certificates](https://msaied.com/certificates) - [Contact](https://msaied.com#contact-section) Connect - [ hello@msaied.com ](mailto:hello@msaied.com) - [ +20 109 461 9204 ](tel:+201094619204) © 2026 Mohamed Said. All rights reserved. [ ](https://github.com/EG-Mohamed) [ ](https://www.linkedin.com/in/msaiedm/) [ ](https://wa.me/201094619204) [ ](mailto:hello@msaied.com) [ ](https://drive.google.com/file/u/0/d/1MF20IPRJyzfy32mhEutjL5EpSls0w2Q8/view)