Daemonizing PHP

Using PHP to run your backend processes

By Andy Blyler (@ablyler) and Jon Kuperman (@insitedesignlab)

WHOIS Andy Blyler

Lead Backup Team Software Engineer @ Barracuda
Working on the Backup product since 2006

WHOIS Jon Kuperman

Software Engineer @ Barracuda on the Backup Team
Organizer of the Ann Arbor PHP Meetup

Presentation Agenda

What We Do

How We Do It

What Is A Daemon

Using Our Forking Library

Real World Results

What we do

How we do it

What is a Daemon

dae·mon (/ˈdēmən/) Noun

  1. A background process that handles requests for services and is normally parallelized via forking or threading.

fork·ing (/fɔrkɪn/) Verb

  1. Dividing a process into a parent and a child.

thread·ing (/THredɪn/) Verb

  1. Executing part of a program independently of other parts.

This talk is just going to focus on process forking

Our Forking Library


Actively maintained for 5+ years.

Open source just days ago!

The Copy Thumbnailer

Without Forking

Processes files one at a time

Some file types take multiple seconds

Introduce Forking

/* setup forking daemon */
$server = new fork_daemon();

/* registered call back function */
function generate($files)
  foreach ($files as $file)
/* add work units */
$files = db_work_get();

/* process work blocking mode */

Documents blocking Images


define("IMAGES", 1);
define("DOCUMENTS", 2);

$server->max_children_set(2, IMAGES);
$server->max_children_set(5, DOCUMENTS);
$server->register_child_run("generate_images", IMAGES);
$server->register_child_run("generate_documents", DOCUMENTS);

$images = db_work_get('images');
$documents = db_work_get('documents');

$server->addwork($images, '', IMAGES);
$server->addwork($documents, '', DOCUMENTS);

Open Database Connections

Resource Management

/* setup forking daemon */
$server = new fork_daemon();

function prefork()


/* setup forking daemon */
$server = new fork_daemon();

function helper()
   pcntl_exec('/usr/bin/image_processor', array('-l'));

Storing the work queue

Flat Files

Local = Fast!


Does Not Scale

Cannot Share





Sharing DB Queues

         id: 29323023
      ctime: 2013-03-22 13:34:33
     module: generate_thumbnails
       slot: image_raw
server_lock: 32342
 start_time: 2013-03-22 13:34:35
update_time: NULL
error_count: 0
       data: {"user_id":999,"image_id":3234123}


Cloud processes a petabyte every 1.5 days

Copy is generating 125 thumbnails per second

In the last 4 months Copy has added 1.3 million users.

Daemonizing PHP

Using PHP to run your backend processes

GitHub Repo


By Andy Blyler (@ablyler) and Jon Kuperman (@insitedesignlab)