templating dsl

So I am evaluating different ways of using my new "view adapter" templating engine. Sparing the details I would like to try out a few different versions of syntax. Straight php, s-expressions and then let/smalltalk.

#PHP
$layout = Array('border_layout',   
  'center' => 'this is the center',
  'south' => Array('border_layout',
    'north' => 'this is the north of the south',
    'south' => 'this is the south of the south'),
  'north' => Array('form', 'method'=>'POST', 'with'=>Array(
    Array('input', 'name'=>'first_name', 'label'=>'First Name'), 
    Array('submit', 'value'=>'Save')));

One nice thing about the php approach is that I can combine integer indexed with associative indexed arrays thus Array('submit', 'value' => 'Save') becomes possible as opposed to Array('type' => 'submit', 'value' => 'Save'). Clearly the adapter has to know that [0] is the type and thus shift that off before rendering the rest of the component.

#LET
$layout = [border_layout
  center:'this is the center'
  south:[border_layout
    north:'this is the north of the south'
    south:'this is the south of the south']
  north:[form method:post with:[
    [input name:first_name label:'First Name]
    [submit value:Save]]];

As let sits atop php the same combination of integer indexed and associative indexes applies here too. Clearly replacing => with : and dropping quotes around single word keys and value makes things nicer as does making the commas optional as lists are white space seperated. As a note the reason I am dropping quotes is that within let there are no constants unless they are class constants. Thus there would never be a time you "accidentally" used a constant instead of a key. Also [] is not used for indexes or pushing values onto array in LET as it is more "smalltalk esque" in that you use methods for all things. And the reason I am not using [$var|$body] for code blocks is that I am using {$var|$body} as they are not used in smalltalk and coming from c/php { } feels right to me to denote a block of code.

#S-EXPR
$layout = (border_layout
  (center 'this is the center')
  (south (border_layout
    (north 'this is the north of the south')
    (south 'this is the south of the south'))
  (north (form method:post with:(
    (input name:first_name label:'First Name')
    (submit value:Save))))

S-expressions will always have a certain element of "balance" about them not afforded by any other syntax, next to let I prefer this for sure. Except this is not strictly "lisp"y in that I am using key:value pairs intead of :key value or (key value) groupings.

#JSON
$layout = {border_layout:{
  center: 'this is the center', 
  south: {border_layout:{
    north: 'this is the north of the south', 
    south: 'this is the south of the south'}},
  north: {type:'form', method:'post', with:{
    {type:'input', name:'first_name', label:'First Name'}, 
    {type:'submit', value:'Save'}}};

Json is again consistent but there is no way of doing the {control, key:value} trick here except for maybe {submit:1, key:value} but that feels pretty hackish. Still at least you don't have to write Array() all over the place!

nodes relating to nodes to relate to nodes

A design decision which I made early on was to have the simple concept of nodes which can relate to nodes to describe their very relation by more node relations. haha sound confusing? The main point of actually using a node itself to describe a relationship is for that reason exactly: to be able to store node content to describe that particular relationship. i.e. if you were to model a recipe perhaps you would want to give a more meaningful description of each ingredient in the recipe beyond just what the ingredient is and it's quantity. thus you end up with

apple pie (recipe)
  -> (recipe ingredient) -> apple (fruit)
                         -> 1/2 cup (measurement)

  -> (recipe ingredient) -> margarine (food)
                         -> 1 pound (weight)

I think that makes sense the trick is that each word in parentheses is indicating what type of node it is. The generalization of this relation is:

(recipe) -> (recipe ingredient) -> ((food) (measurement))

So there is a relation between recipe and recipe ingredient types. Then there are two more relations between recipe ingredient and food and measurement. You will notice that this means the relation is valid for anything descending from the specified type i.e. weight extends measurement and fruit extends food thus both beig valid relations.

Another example would be a "record"

(record) -> (pressing) -> ((format) (color) (pressing number) (number pressed))
         -> (relased by) -> (band)

the point is that the structure of the node types is by nature hierarchical merely to allow the inheritence/cascading of relations. But the way the instances of nodes relate is not hierarchical at all in that any node can relate to any other node so long as the relation is established and in the main I think that is going to remain entirely open except for in the case of "system structures" i.e. want lists and "collection space collections".

collections of collections and relations of relations

I am constantly torn between the necessity of hierarchy in classifying data and the equal necessity to allow all things to remain miscellaneous. With the new collectionspace data structure I feel I am finding that middle ground at last. The key is allowing relationships between nodes to be freely described and extended. The one node which I feel is going to provide the most interesting use is the collection - which is sort of funny as it makes the name of our site relevant again in the context of our new knowledge base.

• node
  
  
  
  • person
  
  
  • place
  
  
  • thing
    
    
    
    • record
    
    
    • shirt
    
    
    • book
    
    
    
    
  • concept
    
    
    
    • belief
    
    
    • date
    
    
    • number
    
    
    • measurement
      
      
      
      • mass
      
      
      • weight
      
      
      • volume
      
      
      • pressure
      
      
      • force
      
      
      • distance
      
      
      • time
      
      
    
    
    • color
    
    
  
  
  • collection
    
    
    
    • group of people
      
      • band
    
    
    • group of things
      
      • want list
      • have list
    
    
  
  

I feel this is a good start

On Authors and Hackers

I strive to be an author. If I were to write a piece of prose like people write "production" and "enterprise" code you would think I was trying to convey tourettes onto paper. This is not programming. It is a reflection of the neurosis created by capital.

initial impressions of seaside

So this is the second time I have tried to "get into" seaside and my initial impression is "man thats a lot of classes" it just feels like total overload but I know that is because I am browsing everything available within squeak as well as smalltalk - still I am just recording my impression so I can come back later and reflect upon it.

I really like the idea of storing class definitions within a db including hierarchy, methods, properties etc. and then fwriting them as actual php class files when necessary. I see this as a sort of compromise/middle ground between the apache and smalltalk image approach. I think that is definitely the direction I will be going with this eventual framework. I want to provide the minimal class hierarchy necessary to create powerful web applications with out including all of the stuff necessary for operating upon the kitchen sink.

The ability to quickly fly through classes/methods and make and commit changes is awesome. It would be pretty easy to include version control this way too!

The other idea based on this structure would be actually storing session instances of given classes.... that's something I am chewing on though in the back of my mind - but for editing/developing I think this idea has wings. Already got a basic interface together using extjs. It will be sweet when things are to the point that I can re-write it in the new language/framework but sometimes a bootstrap is necessary.

Another thing I am for sure wanting to start messing with is the idea of render/adapter-context-free views in that I can define how an interface works with out worrying about the actual display code wether that is html/css or pure extjs etc it should still function with out extra work on my part once the initial adapter is written. So the first step is coming up with what the "interface" for such an adapter would be. An interesting thing to consider is how would one action/view be rendered differently based upon extjs or plain html.

trying to put my finger on it

So I am constantly trying to entirely re-evaluate the entire way that I/we currently develop web "applications". There are a few things which profoundly bother me.

Lack of object persistence (I go to a page which is deep with in a business app which requires several "models" to be loaded into memory in order to generate a given interface. Now I want to populate something via ajax so I pass in the relative IDs (or if I am lucky the IDs are in a session, regardless) the page then has to again load those SAME models into memory (temporarily) to fetch some sort of related json tidbit).

Too much coupling to one particular "display stack" i.e. html, css, xhtml, javascript (extjs) - surely my "view" should have the intended layout recorded in a meaningful and timeless manner including callbacks/events and then have it be implemented by what ever adapter/rendering method I choose i.e. html or extjs or svg or what ever comes down the pipeline! this way there is a simple "plugging in" of a new adapter which meets the protocol for rendering and away I go.

My new framework/language/paradigm is going to tackle these problems absolutely.

lisp+smalltalk+php ?

So I am closing on a a complete set of rules for my syntax:

#use . to denote end of statement
#list
:(1 2 3 4).

#boolean
($this == $that).

#block
{$x $y | $x*$y}.

#message
[$object method_call: $arg with:[object method:$sub]].

#use ; to denote "with" 
[$object methoda;
         methodb;
         methodc].

#to emulate int objects anytime a square bracket encounters an int
#it will "cast" as intobject thus:
[1 to: 20 do: {$n | echo $n}].

#transmogrifies to 
$intObject(1)->to(20)->apply('lambda_block_n');

#it may be the case that the int methods could be factored into more
#procedural form even such as for loops etc. It all depends on performance.
#but pre-optimization is definitely NOT what I am going for here

The real beauty to this is largely that we can overload operators and not have to use s-expressions to do so i.e. if I decided I wanted / to diff arrays I could easily add a method to the ArrayObject called / which accepts an array as an argument and then calls array_diff between self and the passed array:

[:(1 2 3 4 5) / :(2 4 7)]. #yields (1 3 5 7)

#transmogrified to 
arrayObj(1,2,3,4,5)->__SYMB_DIV(arrayObj(1,2,3,4,5));

#hmm may get ugly with nested hash
$hash = :(a:(b:(c:true))).

#it feels unbalanced but 
$hash = :(a::(b::(c:true))). #is far worse

#or I could stop being an idiot and use #
[#(1 2 3 4) each {$n | echo $n}].

#my only hesitation is as illustrated here, # is used for comments in c-alikes.

more on syntax

The more I think about it the more it becomes apparent that logically using square brackets for codeblocks/lambdas does NOT make sense for what I am trying to accomplish with let. Using { } for anon blocks and ( ) for lists as well as "grouping" and boolean statements makes much more sense thus saving square brackets for sub-statements and removing [ ] from arrays entirely as they can use dot notation or accessor methods. Consider:

$list = (1 2 3);
$list.each {$x | echo $x}; 

$number = let ($x = 10){
  (add:{$y | $x += $y}
   sub:{$y | $x -= $y});
  #note if we wanted the method to be named add: we would do 'add:':{$y | $x += $y}
}
$number.add 10; #20
$number.sub 5;  #15

#And from last time:
$decide = {$char | $char == a ? sum : mul};

$num = 10;
$letter = a;
$result = [$decide char:$letter] 6 6 ($num + 50);

Notice in the last one that this makes square brackets the equivelant of "apply" and keeps open brackets for grouping and or boolean operation which is what people coming from php would expect. The { } also feels more natural for anon code blocks for php-centric people as well.

Another approach would be using square brackets for "messages" sort of like obj c but more like sugar for dropping dots in objects i.e.

$range = (1..5); 
[$list reverse each {echo _}]
#becomes
function block_name($arg0){echo $arg0;}
$list->reverse->each('block_name'); 

philosophy of duality of syntax

Now for a more text-ful approach to describing the problem and my approach towards a solution. It's interesting that as I have been slowly changing php to suit my needs I have been grappling with the issue of consistency much like the developers of py3k have. It seems that in an attempt to "make things consistent" they have tried to make everything work like a function i.e. no more print 'string', now it is print('string') - this feels, to me, like they took the wrong fork in the road. Instead of making print work like a function why not make all functions work like keywords/in built functions?

I guess I am in my ivory tower but really if I am going to create a language/dialect which I ENJOY coding in day in/day out I am not worried about backward compatibility and or what people are used to - I am worried about what makes things consistent and beautiful. The obvious choice for the most consistent approach would be straight s-expressions but there is something to be said for the readability of declaration and assignment. My approach is basically s expressions with out the initial brackets i.e.

#LET
$x = func arg:val arg1:val2 arg2:(func1 arga:vara); 
 
#s-expr
(setf x (func arg:val arg1:var arg2:(func1 arga:vara))); 

Now I can fully see how the s-expression is more perfectly recursively consistent but I personally find the prior approach more readable and for "in the main" programming it would be the most maintainable.

Something I have been toying with is the idea of how to transmogrify dynamic function calls in s-expressions:

$decide = [$char | $char == a ? sum : mul];

$num = 10;
$letter = a;
$result = ($decide char:$letter) 6 6 ($num + 50);

#should become
$result = sum 6 6 ($num + 50); 

#which becomes
$result = sum(6,6,($num + 50));

duality of syntax

So I have hit a bit of a conondrum with my current approach to introducing a new syntax for lambdas in LET/php.

The problem comes from the current use of square brackets to access array elements mixed with my desire for them to be lambda expressions. Consider the following:

$array = (a b c d);
$array [0]; #a

#the function expecting the yield block is passed a block which returns 0
$expecting_yield [0];

#one solution as we have dropped ( and ) for calling functions would be using 
#that for array indexes, this would sort of fit with the fact that it is used 
#for defining arrays. 

$array = (a b c d); 
$array(0); #a

$array (a:(b:(c:cat))); 
$array(a)(b)(c); #cat

#what about methods stored in arrays?
$number = let($num = 5){
  (mul:[$x | $x * $num]
   div:[$num/_])
}

#note that _ is for anon vars and _1 _2 is if there are many

$number (mul) 10; #50 
#that feels odd. 

#using magic __get and keyword param
$number.mul x:10;

#that works well for keys which are strings but what of numeric index?
$array = (1 2 3 4);
$array.0 #1 

#perhaps we could be clever enough to turn numeric dots into
$array[0] 

#if all arrays are objects then why would you not use an accessor method?
$array = (1 2 cat 4); 

#or using an accessor method - thats only one extra char (the .)
$array.at 2; #cat 

#maybe removing the [ ] notation will encourage functional programming
#instead of iterating over for loops anyway? 

#but what about look ups in a hash table?
$actions = (cat:walks rat:crawls bat:flies);
(cat rat bat).each [$creature | 
  echo "the $creature {$actions.$creature}";
];

#normal php 
$actions = Array('cat'=>'walks', 'rat'=>'crawls', 'bat'=>'flies'); 
foreach(Array('cat','rat','bat') as $creature){
  echo "the $creature {$actions[$creature]}";
}

more on let blocks

So as an aside the two are equivalent:

$func = let($x = 10){
  [$y | $x = $x*$y];
}

$func = [$x | [$y | $x = $x*$y]] x:10; 

The let block is obviously more readable as you understand you are creating a binding for $x and then using that binding in the let block (in this case to return a function which closes over $x).

So to leverage the power of multiple-transmogrifications the system first translates let blocks to be square lambdas and then the square lambda transmogrification can just handle everything else (including closures). I think it's dope.

closures in php

So here is the implementation of closures which my "language" (LET) makes use of. This would normally be another boring attempt at closures in php but I believe it is quite clean. First let me show you the code before transmogrification:

$func = let($x = 10){[$y | $x = $x*$y]};
echo $func y:2; #20
echo $func y:2; #40 (as x is now 20 having been modified by prior call to closure)

Notice the dropped brackets and keyword parameters? Also note the fact that a lambda by default returns the value of it's last statement, the let block is a form of lambda and thus also returns it's last value which is in this case the name of the generated function. Note that this is actually parsing the square lambda to determine if it makes use of any free variables, if so it creates a closure otherwise it will just create a "normal" function. Because of the fact I am parsing and then writing the new php file I can take my time instead of just dumping in get_defined_vars (holy waste of memory batman). Also note that if the square lambda were to have passed in $x as a variable "before the pipe" than it would NOT consider $x a free variable and thus not create a closure.

First the simple class for handling closures

class __Closure{
        private $method;
        private $env;
        function __construct($method, $env){
                $this->method = $method;
                $this->env = $env;
        }

        function call(){
                $args = func_get_args();
                $args[] = $this->env;
                return call_user_func_array($this->method,$args);
        }
}

And now the transmogrified version of the let statement:

function lambda_closure_0($y,$env){
        return $env['x'] * $y;
}

function lambda_func_0($y){
        return $GLOBALS['lambda_closure_0']->call($y);
}

function lambda_let_block_0($x){
        $GLOBALS['lambda_closure_0'] = new __Closure('lambda_func_0',Array('x'=>&$x));
        return 'lambda_func_0';
}

$func = lambda_let_block_0(10);
echo $func(2); #20
echo $func(2); #40

The clever part of this whole thing is determining at parse time which "free" variable are being accessed by the square lambda and then passing them to a new instance of __Closure class by reference (thus: Array('x' => &$x)).

Notice that it is a LOT more code once the let block has been expanded by the tansmogrifier. This is the entire point of the project -expand to verbose and yet simple code which tries to make minimal use of any create_function calls.

When php 5.3 becomes "acceptable" and on all the servers I use then I will make the transmogrifier use the new "uses" clause with the inline function syntax but that's precisely the beauty of the transmogrifier - my syntax stays the same and the underlying php can be optimized instead. This is the art and beauty of meta programming.