sklar.com/blog

Fast Multiple String Replacement in PHP

At work, we added a language filter to Ning Pro last month. It lets Network Creators have naughty words (for the Network Creator's definition of "naughty") replaced with * characters.

A straightforward way to do this in PHP is to pass an array of words to look for and their replacements to a function like str_replace() or str_ireplace(). Or, similarly, use a regular expression that gloms the search terms together (and potentially checks word boundaries.) There are assorted WordPress plugins that work like this.

The problem with this approach is that it's really slow. Especially if you have a lot of words you're looking for. The amount of time it takes to do the search and replace grows in proportion to the number of words you're looking for. This is particularly unfortunate because usually, none of the words are ever found!

For our language filter, we took a different approach. We've packaged it up into a PHP extension called Boxwood and releasing it today as open source. (Find it on github: http://github.com/ning/boxwood.)

With Boxwood, you can have your list of search terms be as long as you like -- the search and replace algorithm doesn't get slower with more words on the list of words to look for. It works by building a trie of all the search terms and then scans your subject text just once, walking down elements of the trie and comparing them to characters in your text. It supports US-ASCII and UTF-8, case-sensitive or insensitive matching, and has some English-centric word boundary checking logic.

Take it for a drive and let us know what you think!

PHP Microbenchmarking

I just posted on the Ning code blog about the PHP microbenchmarking framework we released:

I'm pleased to announce the release of ub, a PHP microbenchmarking framework. You can download it from http://github.com/ning/ub.

The goal is to make it as easy as possible to compare the runtime of alternative approaches to the same problem, such as different regular expressions, or different methods for string or array manipulation.

The source distribution contains a README with some documentation and a bunch of sample benchmarks.

For normal use, it is rare that two similar, but different approaches produce appreciable differences in runtime. (Inefficient regexes and bloated call stacks aside.) The payoff from this kind of benchmarking is really on operations that happen hundreds or thousands of times in a request, or are happening on hundreds or thousands of servers. At that point, shaving off small amounts of runtime performance can really make a difference.

I am looking forward to beef up the set of included benchmarks -- contributions are welcome!

ZendCon 2008: Static and Dynamic Analysis at Ning

I presented my Static and Dynamic Analysis at Ning talk today at the 2008 Zend/PHP Conference. The conference is much bigger this year than last, very exciting to see all the different things people are doing.

The slides from my talk are available at http://www.sklar.com/files/static-dynamic-analysis-zendcon-2008.pdf .

PHP + Emacs

Here are some links that are related to the Editor/IDE panel I'm on today at the 2008 DC PHP Conference:

GEBEN (interactive debugging)

nxhtml-mode (php-mode, multiple major modes)

"Little Bobby Tables" vs. the US Government

Chris often cites this xkcd cartoon in security talks, since it's a) funny and b) a good example of SQL Injection.

I was curious to see what sorts of shenanigans one can get away with in a legal name. I'm still waiting to hear back from the NYC agency that issues birth certificates but here's what the US Social Security Agency told me:

The maximum number of characters to be shown on the Social Security number (SSN) card for the first and middle name is 26; the maximum number of characters for the last name is 26. Full names will not be reduced to initials unless the combination of first and middle names exceeds 26 characters. The only acceptable characters are alphas, hyphens, and apostrophes. The SSN card will be printed as entered into the enumeration system, but the SSN record will not display the hyphens/apostrophes.

So with hyphens and apostrophes you might be able to get away with a little syntax error mischief, I suppose.

Turns out the SSA has really detailed public documentation of all their procedures.

Prolific Name-Sources

I had never heard of Haskell Curry before I read the preface to SICP just now, but it occurs to me that his distinction of having each of his names (first and last) turned into a term used in the discipline he studied (Haskell the programming language and Curry the operation) is sort of like how Glenn Seaborg, working at LBL, could have a letter addressed to him using element names (seaborgium / lawrencium berkelium / californium / americium.)

Who else can you think of that falls into this admittedly fuzzy-edged bucket?

Let a thousand string concatenations bloom

How many different ways (in PHP) are there to concatenate the string values in two variables and put the result in a third variable?

Here are a few to start:

$alice = $bob . $charlie;
$alice = "$bob$charlie";
$alice = sprintf('%s%s', $bob, $charlie);
ob_start(); echo $bob, $charlie; $alice = ob_get_clean();
$alice = implode('', array($bob,$charlie));

I would say something like "Of course, this entire exercise is just for fun and in practice is totally useless," but whenever I start out thinking that something actually productive eventually emerges. So perhaps this is totally useless, perhaps not!

DC PHP 2007 Slides

The slides from my "API Design in PHP" talk at the 2007 DC PHP Conference are up at http://www.sklar.com/files/dcphp-2007-api-design.pdf. Enjoy!

Subversion Time Machine

Jon Aquino, my talented cow-orker built a time-lapse view for Subversion. This is fantastic.

We switched from Perforce to Subversion recently (at Ning), and one of the things that I miss about Perforce was its handy time-lapse-view feature. No longer!

API Design Talk Slides

The slides from my API design talk yesterday are available.

If you missed Zendcon, come to the DC PHP Conference in a few weeks. I'll be talking about API Design there on November 8.

Ning Turns Two

As Diego and Gina have pointed out, the Ning Platform has just celebrated its second birthday.

The past two years have been a lot of work and a lot of fun. It's been gratifying to see the crazy, heartwarming, innovative uses folks have come up with for the platform and also extremely rewarding to work with such talented people.

I wonder what I'll have to say in a blog post a year from now that links to this one?

Zendcon 2007

Zendcon (aka "The 2007 Zend/PHP Conference and Expo") draws near. I'll be there giving a talk about API Design in PHP and undoubtedly getting sucked into various conversations about how PHP's disorganized ugliness is its greatest asset.

E-Mail Address Validation and Mission Creep

+1 to what ndg said in response to Jacob's post about e-mail address validation.

I think Jacob sort of touches on this in his post but too often it's not explicit what people are looking for under the vague and broad umbrella of "e-mail validation".

If it's "Did the AOL user signing up for my site forget the '.com' part at the end of the address?" then some pretty simplistic text-based matching will do.

If it's "Is the address supplied a valid mailbox that is controlled by the person who is supplying it to me?" then all the string parsing in the world isn't going to save you. You'll need to roundtrip something out to the mailbox with confirmation instructions. And even then you're not out of the woods -- maybe the user's mailbox is full, or a transient DNS error is causing a problem.

I think e-mail address validation tends to be a bit of a tarpit for nerds (myself included) because the delicious complexity of RFCs 822 and 2822 makes writing code to handle the grammars they describe a fun challenge. That makes it easy to forget what the point of the address validation in the real world actually is: handholding well-meaning users that may have made a careless mistake, or robust protection against malicious users, or bot protection (in which case you probably want to ban RFC-valid addresses that have known-suspicious domain names or components), etc.

API Design in PHP at ZendCon 2007

I'm going to be giving a talk on "API Design in PHP" at ZendCon 2007 in October.

Our PHP API at Ning is a little over two years old at this point. There are some things I love about it that I think we've done really well, some things that we have revised, some things we can do better, and some interesting things planned for the future.

The talk is all about what we've learned developing, building, and supporting the API with both a ferocious devotion to backwards compatibility and a rapid new release cycle.

See you in October!

Runkit, "static", and inheritance

A PHP issue that comes up over and over again is how the static keyword doesn't know about inheritance. That is, code such as:



class Masons {

    static $where = "World-wide";

    static function show() {

        print self::$where;

    }

}



class Stonecutters extends Masons {

    static $where = "Springfield";

}



Stonecutters::show();

prints World-wide, not Springfield because the self inside Masons::show() is bound at compile time to the Masons class. This is different than how $this works in instances, so it can be unexpected.

There are plenty of good reasons why PHP 5 works this way and it seems that in PHP 6 the static keyword will be able to be used in place of self to get the dynamic behavior a lot of folks are looking for (that is, print static::$where; in Masons::show() would cause Stonecutters::show() to print Springfield.

All well and good once PHP 6 is done.

In the meantime, I was noodling around with runkit and came up with some glue that lets you do something like this:



class Model {

    public static function find($class, $filters) {

        // This method would actually do an SQL query or 

        // REST request to retrieve data

        print "I'm looking for a $class with ";

        $tmp = array();

        foreach ($filters as $k => $v) { $tmp[] = "$k=$v"; }

        print implode(', ', $tmp); 

        print "\n";

    }



    public static function findById($class, $id) {

        return self::find($class, array('id' => $id));

    }

}



class Monkey extends Model { }



class Elephant extends Model {

    public static function findByTrunkColor($color) {

        return self::find(array('color' => $color));

    }

}

And then after the mysterious (for a few seconds) glue:



MethodHelper::fixStaticMethods('Model');

you can do:



Monkey::find(array('name' => 'George', 'is' => 'curious'));

// prints: I'm looking for a Monkey with name=George, is=curious



Elephant::findById(1274);

// prints: I'm looking for a Elephant with id=1274



Elephant::findByTrunkColor('grey');

// prints: I'm looking for a Elephant with color=grey



Monkey::findById('abe');

// prints: I'm looking for a Monkey with id=abe

(The innards of MethodHelper after the jump...)
Continue reading "Runkit, "static", and inheritance"