Tyler Sommer dot com

There are many ways to skin a cat, they say. This rings true for nearly anything in life, especially with software development. Not only are there best practices and common design patterns, but a language can offer different ways to accomplish even the simplest of tasks.

Many times, given multiple solutions, I’ve wondered what the fastest or most efficient way is. For example, in PHP, determining if a string begins with a sub-string has at least three obvious solutions: strpos, substr or preg_match. We can obviously apply basic wisdom to these ideas: preg_match is definitely not the fastest; the entire regular expression library must be called. However, there is no substitute for hard evidence.

Below is a very simple (abridged) class I’ve written that allows registering test functions and then executing those tests a number of times while keeping track of the results. It disregards the 10% highest and lowest results for a decently accurate mean.

<?php

class Benchmark {

	// ...

	public function execute() {
		$adjustment = round($this->_length * .1, 0);

		echo "Running " . count($this->_tests) . " tests, {$this->_length} times each...\nThe {$adjustment} highest and lowest results will be disregarded.\n\n";

		foreach ($this->_tests as $name => $test) {
			$results = array();

			for ($x = 0; $x < $this->_length; $x++) {
				$start = time() + microtime();

				call_user_func($test);

				$results[] = round((time() + microtime()) - $start, 10);

			}

			sort($results);

			// remove the lowest and highest 10% (leaving 80% of results)
			for ($x = 0; $x < $adjustment; $x++) {
				array_shift($results);
				array_pop($results);
			}

			$avg = array_sum($results) / count($results);

			echo "For {$name}, out of " . count($results) . " runs, average time was: " . sprintf("%.10f", $avg) . " secs.\n";

			$this->_results[$name] = $avg;
		}

		asort($this->_results);
		reset($this->_results);

		$fastestResult = each($this->_results);

		reset($this->_results);

		echo "\n\nResults:\n";
		printf("%-25s	%-20s	%s\n", "Test Name", "Time", "+ Interval");

		foreach ($this->_results as $name => $result) {
			$interval = $result - $fastestResult["value"];

			printf("%-25s	%-20s	%s\n", $name, sprintf("%.10f", $result), "+" . sprintf("%.10f", $interval));

		}
	}
}

To use the Benchmark type, create a new instance, register tests and call the execute method.

Consider the following snippet to test our original question: (Note that I am using anonymous functions, so you’ll need PHP >= 5.3 for this to work as-is)

$bm = new Benchmark();
$bm->register("substr()", function() {
	$str = "document_test";

	$result = (substr($str, 0, 9) == "document_");
});

$bm->register("strpos()", function() {
	$str = "document_test";

	$result = (strpos("document_", $str) === 0);
});

$bm->register("preg_match()", function() {
	$str = "document_test";

	$result = (preg_match("/^(document_)/", $str) == 1);
});

$bm->execute();

/*
 OUTPUT:

 Running 3 tests, 1000 times each...
 The 100 highest and lowest results will be disregarded.

 For substr(), out of 800 runs, average time was: 0.0000043887 secs.
 For strpos(), out of 800 runs, average time was: 0.0000040096 secs.
 For preg_match(), out of 800 runs, average time was: 0.0000057400 secs.

 Results:
 Test Name                    Time                    + Interval
 strpos()                     0.0000040096            +0.0000000000
 substr()                     0.0000043887            +0.0000003791
 preg_match()                 0.0000057400            +0.0000017304
*/

If you execute this code repeatedly, you should get pretty consistent results (although likely not identical to mine). Using strpos is always marginally faster than the other two, with preg_match being quite a bit slower.

Obviously, this is pretty limited in its uses. Connecting to non-local (or even any external) resources is going to invalidate basically every result because you can’t necessarily account for time spent “communicating” but there are lots of other things you can test.

Download the complete source here.