php 5.4 - skin deep syntactic sugar

I was quite excited to see that php 5.4 was getting some sugar for array definitions. e.g. "array(1,2,3)" would finally become "[1,2,3]". I was also excited to see "array dereferencing" was fixed or accessing an array by index after it is returned from a function e.g. "func()[index]". However let us be clear in noting that this does NOT mean it has attained the flexibility one would anticipate. For instance you can not immediately access an array after declaring it such as "['name' => 'peter', 'age' => 30]['age']" results in a syntax error. Of course you could combine both of the new features to do "func(['name' => 'peter', 'age' => 03])['age']" - which probably covers the majority of ways in which it would be used. The other issue I have is the inability to access a function upon it's return from function application e.g. "func(1)(2)" will result in a syntax error. This is problematic for higher order techniques that benefit dynamic languages.

Why would anyone want features such as those I have outlined above? People designing "in language" domain specific languages benefit greatly from such flexibility. Particularly in a language which already limits facilities for creating such mythical beasts.

That being said - I will gladly let go of my personal hack for passing in name/value pairs to a function which I have lovingly/jokingly referred to as "appositive expressions" e.g. "func('key', 'value', 'key2', 'value2')" can now become "func(['key' => 'value', 'key2' => 'value2'])". It is not surprising at all that php finally gets this feature as ruby finally bit the bullet and now accepts the json esque ":" in place of the "hash rocket =>". '

I should mention that the addition of "traits" will go miles in cleaning up some otherwise convoluted approaches to attaining mixins in php! I am very excited to see how this is utilized by the different frameworks.

smalltalk, apl, soy lisp, and higher order operands

APL, Smalltalk, Lisp. The holy trinity of programming languages. In this post I aim to better explain the Soy Lisp syntax which allows for all three to coexist.

Consider first the ability to drop parens when an expression is terminated via ".":

(func arg1 arg2)
//becomes
func arg1 arg2.

This creates an immediate problem w/ disambiguation when dealing with arguments, well thankfully we do not lose the ability to return to s-expressions when required e.g.:

func arg1 (func2 arg1 arg2).

What else can we glean from smalltalk? Perhaps the ability to cascade via ;, e.g.:

(((func arg1) arg2) arg3)
//becomes
func arg1; arg2; arg3.

In prior posts I have shown my preference for "square lambdas" e.g: [x | x y]. So I will not continue to rant about the benefits here. The last thing I will borrow from smalltalk is keyword constants (whilst keeping lisp keywords) e.g.:

(func :keyword1 var1 :keyword2 (lambda (x) (x :says 'peter))
//becomes
func keyword1: var1 keyword2: [x | x says: 'peter].

The above assumes func is a higher order function which knows how to dispatch keyword selector arguments, which is essentially how I would implement a smalltalk esque language on top of soy.

I think it is clear that functional programming in lisp can benefit from the smalltalk syntax whilst not losing the power of macro expansion and s-expressions.

This brings us to APL the other language I have toyed w/ writing interpreters for. The real crux of apl is working with higher order operators. e.g. reduce which takes on the left hand an operator which it then uses to reduce a set of numbers on the right. In traditional prefix notation this would end up like:

((/ +) 1 2 3 4)
//not bad but noisier than
+ / 1 2 3 4.
//which becomes
(+ / 1 2 3 4)

Which works just fine because + is a higher order operator which knows if its first argument is another operator it should return a function which does a + reduction.

(- (+ 5 5) 5)
//becomes
5 + 5 - 5.

In soy the latter works because numbers are "higher order operands" which can consume operators and return functions which consume more operators/operands as expected. Stay tuned for a web repl for those interested in playing with this. Bare in mind that both forms in all of these examples work with in the same environment as this is all syntactic sugar! Lisp wins again.

Keyword Functions in PHP

So, as spurned by a comment on hacker news relating to the lack of ability for keyword arguments in php I figured I would expound more upon the Phutility library, in particular the Func module.

Check it out:

#say you have a function (perhaps defined by you, perhaps not)

namespace SomeNamespace;
use Phutility\Func; 

function testFunc($name, $eyes, $hair = 'blonde', $age = 0, $weight = 200, $goatFace = 'definitely') {
    return array(
        'name' => $name, 
        'eyes' => $eyes, 
        'hair' => $hair, 
        'age' => $age, 
        'weight' => $weight,
        'goatFace' => $goatFace);
}

#if you want a keyword wrapper for it:

$testFuncWrapper = Func::keyword('\SomeNamespace\testFunc'); 

$result = $testFuncWrapper(
    age, 30,
    name, 'peter'
);

/*would yield a $result of
array(
    'name' => 'peter',
    'eyes' => 'blue',
    'hair' => 'blonde',
    'age' => 30,
    'weight' => 1200,
    'goatFace' => 'definitely'
)
*/

# If you want to not have a variable func you would need to do something like:
function TestFuncWrapper() {
    static $wrapper; 
    if(!$wrapper) {
        $wrapper = Func::keyword('\SomeNamespace\TestFunc');
    }

    return call_user_func_array($wrapper, func_get_args());
}

#a little gnarlier but this is the price we pay to call it like:

$result = TestFuncWrapper(
    goatFace, 'NoWay'
);

#oops this would yield an exception because name and age are required!

If this is at all interesting to you and you'd like to use it or contribute, the source and tests are all up on github.

On unit testing

When ever I am confronted with a problem with a code base, my first reaction is: "wtfutfts" meaning "where's the fucking unit tests for that shit". No unit tests? Now you know your first move in providing resolution.

Alternative s-expression syntax for Soy Lisp

Soy is the name of my lisp dialect (previously called lisphp, and before that phlisp). This is quite a radical departure from standard lisp s-expression but not so far as to, say have infix notation or "sweet expressions". Rather this is a simple inverse of s-expression e.g.:

(if (predicate arg1 arg2)
    (thenFunc arg2)
    (elseFunc arg3))

Becomes:

if(predicate(arg1 arg2)
   thenFunc(arg2)
   elseFunc(arg3))

Your first question should be, "but wait! what about homoiconicity and the ability to have quasiquote macro definitions!?". I'm glad you asked, I have two proposals. 1) allow for a s-expression macro syntax, 2) allow for quasiquote to start with an atom:

defmacro(eat-apple person number-of-apples
         `person-eat(,person 'apple ,number-of-apples))

You may have noticed the other drawback with this syntax is that it makes having macros which handle sublists a tad more difficult. As any atom followed by (, applies the atom to the arguments inside of the parens. This can be resolved by using rest args for forms which anticipate the last expression to be treated as a lambda. Also Soy allows for [ | ] lambda forms and { } vector/dicts. So it is also possible that would resolve any ambiguities.

One of my major ambitions with this is to have a very simple syntax for exploring alternative object models, similar to my previous post about an alternative object model for javascript.

;simple keyword selector
some-object(set-first-name: 'peter and-last-name: 'jenkum)
;chaining 
some-object(some-keyword: arg1 selector: arg2) 
           (chained-selector: arg3 takes-closure: [ a | a(message: arg4)])
           (another-chained-selector: [_(anon-var)])

I really dig how close this comes to snythesizing what I love about smalltalk w/ lisp w/o really forsaking either.

One of the real "wins" with this is when you have function returning functions .. n you don't end up w/ a massive amount of nesting.

(def adder [x | [y | [z | + x  y z]]])
(((adder 6) 7) 8)

def(adder [x | [y | [z | +(x y z)]]])
adder(6)(7)(8)

I think the latter reads in a much more obvious "left to right" fashion.

simple keyword message selectors for javascript

As I have been messing with alternative object systems for php one of the ones I came up with was ab(using) the __invoke magic method to take a keyword argument list and find a given selector based upon it. I realized this could be done in an even more trivial manner in javascript using the object syntax and returning a new object w/ the dispatch mechanism rather than "return this".

var Klasses = {
    'Person': {
        'selectorPart1:selectorPart2': function(a, b) {
            console.log('called with ' + a + ' and ' + b);
            return obj(this);
        },
        'subSelector': function(a) {
            console.log('subselector with:' + a + ' for ' + this.name);
            return obj(this);
        },
        'get': function(field) {
             console.log(this.name + ' over-ride of get with ' + field);   
        }
    },
    '*': {
        'get': function(field) {
            if(this[field]) {
                return this[field]
            } else {
                throw new Exception('No field ' + field);
            }
        }
    }
};

var obj = function(self) {
    return function(selector) {
        var sel = [];
        var args = [];
        $.each(selector, function(k, v) {
           sel.push(k);
           args.push(v);
        });
    
        var selectorName = sel.join(':');
        var message = false;
        if(Klasses[self.klass][selectorName]) {
            message = Klasses[self.klass][selectorName];
        }
        else if(Klasses['*'][selectorName]) {
            message = Klasses['*'][selectorName];
        }            
        
        if(message) {
            return message.apply(self, args);
        }
    };
}
    
var guy = obj({klass: 'Person', name: 'peter'});
guy({selectorPart1: 'apple', selectorPart2: 'candy'})
   ({subSelector: 'rotten'})

var otherGuy = obj({klass: 'Person', name: 'samuel'})({subSelector: 'cotton'})
console.log(otherGuy({get: 'name'}))

To see it live: jsfiddle running the above

On Phutility

Phutility is a collection of tools which I utilize in many of my other projects. For example the Node class is utilized by both covenants (sql builder) and also phmop (meta object protocol). Today though I am going to be discussing some of the work that I have been doing in Func to lay the framework for "higher order object oriented programming" with phmop. To give you an idea of what I am driving at, consider:

defineClass('Person', 
    slot('someMethod', function($self) {
        $args = func_get_args();
        $self = array_shift($args);
        $options = appos($args);
        if($self->{$options[ifField]} == $options[isValue]) {
            $self->call($options[callMethod], $options[withValues]);
        }
    })->accessor('someMethod'));

$person = new Person;
$person->someMethod(
    ifField, 'age',
    isValue, 30, 
    callMethod, 'someMethod2', 
    withValues, array(300));

There are a few things we can cleanup. Firstly lets create a function "method" which wraps up simple functions which utilize an accessor which is the same as their internal name. Then lets utilize a higher order function "Func\options" which returns a function which has the last param defined as an options hash to clean things up:

defineClass('Person', 
    method('someMethod', Func\options(function($self, $options) {
        if($self->{$options[ifField]} == $options[isValue]) {
            $self->call($options[callMethod], $options[withValues]);}})));

But we can go further by creation an optionsMethod which wraps up the need for the explicit Func\options call. Also we can have it pass a param which gives us back a std class for options rather than an array.

defineClass('Person', 
    optionsMethod('someMethod', function($self, $options) {
        if($self->{$options->ifField} == $options->isValue) {
            $self->call($options->callMethod, $options->withValues);}}));

For the sake of completeness let me show off another way this could be defined more explicitly taking advantage of Func\keyword which evaluates a closure and then returns a function which responds to an appositive style call:

$keywordFunc = Func\keyword(function($age, $name, $color) {
    return array('age' => $age, 'name' => $name, 'color' => $color);
});

print_r($keywordFunc(
    age, 30, 
    color, 'blue', 
    name, 'peter'));

array(
    'age' => 30,
    'name' => 'peter',
    'color' => 'blue');

As you can see this allows us to pass the params in any order we please (which to be fair our Func\options implementation does as well but packaged into an $options var). So now we will skip to step 3 of what a keywordMethod would look like in use:

defineClass('Person', 
    keywordMethod('someMethod', 
        function($self, $ifField, $isValue, $callMethod, $withValues) {
            if($self->$ifField == $isValue) {
                $self->call($callMethod, $withValues);}}));

$person = new Person;
$person->someMethod(
    ifField, 'age',
    isValue, 30, 
    callMethod, 'someMethod2', 
    withValues, array(300));

I think this makes clear why both a meta object protocol and higher order functions are so awesome when they can play together. Rather than having only inheritence, interfaces, or even mixins, as the tool for making things more elegant we can easily add new features to our language with simple function composition.

Considering a MetaObject Protocol for PHP

The purpose of a "MetaObject Protocol" is to allow for defining the very nature of how an object system functions. This means the ability to add traits/mixins, aspects, multiple inheritance, "Generics" etc. w/o requiring compilation or low level tinkering. One major advantage is that this means not having to wait for a revision of your language to express certain design patterns.

As usual php is so close, yet so far away. However ab/utiliz/ing the magic methods you can actually get pretty close to something resembling the CLOS system. To accomplish this it requires a few building blocks which I have abstracted into my Phutility library. Primarily the MetaArg and Node which provide the building blocks for composing declarative domain specific languages.

The majority of the magic occurs via macro expansion in CLOS. We are going to make up for that via declaring an abstract syntax tree (AST) which we will expand and weave into our method dispatch mechanism.

Enough talk, let's code. First our goal is be able to declare classes and methods.

defineClass('Shape',
    slot('width'),
    slot('height'),
    method('getHeight', function($self) {
        return $self->height;
    }), 
    method('getArea', function($self) {
        return $self->width * $self->height;
    })
);

$shape = new Shape(
    width, 20, 
    height, 50
);

echo $shape->getHeight();
echo $shape->getArea();

defineGeneric('beforeGetArea', 
    before(getArea),
    arity(Shape), 
    function($self) {
        echo "Called before get area!\n";
    }
);

echo $shape->getArea(); //outputs "Called before get area!" and then area

Looks like a class and acts like a class. But most definitely not a raw class. How does this work? Principally we keep the meta information about each class in a Registry. Each class defined results in a simple declaration of "class X extends \phmop\StandardClass{}" which allows the usual "new Class" syntax. It writes this to a cache directory and does a require_once on it so as to avoid eval.

The base Obj class looks something like:

class StandardClass {
    public $slots = array();

    public function __call($slot, $args) {
        return Registry::dispatchMethod($this, $slot, $args);
    }

    public function &__get($slot) {
        return Registry::getSlot($this, $slot);
    }

    public function __set($slot, $value) {
        Registry::setSlot($this, $slot, $value);
    }

    public function __construct() {
        foreach(Appos(func_get_args()) as $key => $val) { 
            $this->$key = $val;
        }
    }
}

So pretty clearly the real magic is taking place in the way that defineClass, defineGeneric, and friends interoperate with the Registry.

I am going to end things here for now as I need to make a few posts about some of the inner workings of phutility to really get into the thick of things.

a look at the chet "appositive expression" orm dsl

Chet is a lightweight, minimal, php framework intended to make writing php not feel like a giant pile of array('a' => array('b' => array('c', 'd')). This is accomplished mainly by declaring appropriate constants and using what I am calling "appositive expressions". e.g. Person(name, 'shaun', age, 28);

While this started out as a joke it has actually begun to have merit and today I am going to demonstrate the latest portion of the project which is the sql dsl which in a nut shell looks like:

TodoItem::with(
    creationDate, between($start, $end), 
    either(
        owner, eq(25),
        status, in('active', 'pending')));

SCS: 26
When I am messing with snytax ideas I have come up with a system for scoring their visual complexity which comes down to counting any piece of non-word character i.e. parens, colons, commas, quotes etc. For the above example the SCS (syntactic-complexity-score) is 26.

The above in a raw php + sql approach would be:

safe_mysql_query("select * 
  from TodoItem
 where creationDate between(?, ?)
   and (owner = ?
        or status in(?, ?))", array($start, $end, 25, 'active', 'pending'));

SCS: 33

The equivalent in something like cake php would be (from with in a controller abusing the concept of $this to access a model... ugh)

$this->TodoItem->find('all', array('conditions' => array(
    'creationDate between ? and ?' => array($start, $end), 
    'OR' => array(
         'owner' => 25, 
         'status in(?, ?)' => array('active', 'pending')))));

SCS: 58

I am pretty sure that the above describes what I am trying to get at: if your "abstraction" is messier/harder to read than that which you are meant to be abstracting yourself "from" it is time to go back to the drawing board. While Chet started out as a joke (a reference to Sinatra for ruby but more fitting as Chet Baker was a horrible drug addict yet an incredible musician - thus php is a terrible language yet still capable of doing marvelous things in a non-classy manner even), I am slowly actually wanting to use it for real projects.

One of the design choices was to either bit the bullet and have methods called _and, _or etc. or find alternatives for them. For "and" I have chosen "with". For "or" I have chosen "either". For all of the other sql operators I have chosen the perl approach: "=" is "eq", ">" is "gt", ">=" is "gte" etc. between and in are now both just methods. The beauty is that because all of this is just sugar you can just as easily express the above as:

TodoItem::with(array(
    'creationDate' => array('between' => array($start, $end)), 
    'or' => array(
        'owner' => array('=', 25),
        'status' => array('in', array('active', 'pending')))));

SCS: 56

Still not as gnarly as the cake php but clearly noisier than the "real chet" approach. My point is that you can drop the sugar in favor of the array noise if one is so inclined.

I am using this same approach for the html/css which yields some very similar gains in readability and "self-similarity" across the entire application.

All of the above being said this is purely an exercise in "how far" I can push php to gain something that resembles a non-array/class-abusing DSL with a small amount of code generation and utilization of everything php 5.3 has to offer.