+1 571-297-6383 | info@sonjara.com

Managing long-running processes

Although we are used to most things we do through a website being pretty much instantaneous, every so often we run across a task that may take a little time, such as transcoding a video from one format to another, or indexing a large number of documents in a SOLR database. This causes us a few problems as web developers:
  • We don't want our page request to hang while the process is running
  • We would prefer to pass information back to the user regarding the progress that is being made
  • We don't want the request to time out
  • Ideally we don't want to be tying up a request handler in the web server for extended periods of time.

Fortunately Fakoli provides a mechanism for handling this type of problem, via the process component. The process component allows you to respawn web requests as background processes that execute outside of the web server process. For the examples in this article I will use Fakoli's solr component, which provides full text indexing of documents in Fakoli document libraries. Indexing these documents can take time, depending on the size of the document libraries and the server resources available.

Step 1. Modifying your Action Handler to Spawn a Background Process

The first step in converting to a background process model is to take your action handler and modify it to perform the process spawn. Here is our original action handler, where all the SOLR indexing is performed in the web request:


$document_library_id = checkNumeric($_GET["document_library_id"]);

SolrManager::indexLibrary($document_library_id, true, $process);

To convert it to spawn a background process, we modify it as follows:

Fakoli::using("solr", "process");

$document_library_id = checkNumeric($_GET["document_library_id"]);

if (!$process)
	// We are in a user request - spawn the background process and return the ID
	$process = new ProcessManager();
	echo $process->run("/action/solr/index_library?document_library_id=$document_library_id");
	// We are in the spawned background process - begin indexing the library
	SolrManager::indexLibrary($document_library_id, true, $process);

Step 2. Reporting on your Progress

Once you have your background process running, you need to report back to the user periodically on how things are going. The ProcessManager object provides this functionality. The initial status of the process is "Starting". We set it to "Running" at the start of our computation, then send periodic updates. Finally when we are finished we set the status to "Completed".

$count = count($documents);
if ($process)
  $process->setProgress("Running", "Indexing $count files", 0);

$c = 0;

foreach($documents as $document)
   // ... Process the documents, then update the counter
  if ($process)
    $process->setProgress("Running", "Indexing $c of $count", ($c * 100 / $count));

$process->setProcess("Completed", "All done!", 100);

Step 3. Presenting Progress to the User

Now we have all our background processing wired up, we just need to provide a progress UI to keep the user informed of what is happening. Fakoli makes this pretty simple, using a Javascript class called BackgroundProcess(). Here is how we would set up a background process call from a button in our HTML:

<a class='button' href='#' 
   onclick='new BackgroundProcess("Indexing Library",
       {onComplete: function() { window.location.reload(); } }); 
       return false;'>Index Files</a>

The Javascript object manages the whole process of displaying a modal popup with a progress bar, and then calling your onComplete action once the background process has completed. The UI looks like this:

Progress Bar Snapshot

Step 4. Profit ??

Every once in a while you'll run across something like a long-running report, or a computationally intensive activity that you will need to handle in this fashion. And now you can, thanks to Fakoli.

Fakoli events may be coming to a calendar near you » « Hello, TinyMCE 4!


* indicates required field