Rating: 1 Star2 Stars3 Stars4 Stars5 Stars

Next big programming language feature after closures

Closures are the current hot feature for programming languages. The inclusion of closures in Java appears to be around the corner, and the C++ committee has recently voted on closures in the upcoming C++ 0x standard.

The introduction of closures into many mainstream languages indicate to me that the logicial next big features is going to be primitive aggregate operations. Those are operations that operate on collections such as lists or arrays.

Closures, are anonymous functions created at run-time which can refer to variables that are visible where the function is declared.

Closures are especially useful when performing operations over elements in a collection. The most common collection operations (aggregate operations) in functional programming are map, filterfold, and unfold operations.

A map operation transforms a list into a new same-sized list by applying a transformation function (such as a closure) to each element in the original list. A common example of a map operation is when performing a vector scaling operation.

A filter operation creates a list from another list using only those items for which a predicate function returns true.

The fold operation combines values in a list using a binary function and an initial value. A summation function is a simple example of a fold operation.

The unfold operation creates a list from an initial value and by successively applying a generator function, until a terimination predicate returns true.

The introduction of closures into C++ and Java make aggregate operations (operations that operate on collections) easier to write and will make their usage more frequent.

Aggregate operations like unfold, fold, filter, and “map” have the characteristic that they can be combined allowing significant reduction in the number of intermediate operations and data-structures created. This technique is called deforestation and is well-known when applied to pure functional programming languages such as Haskell.

In order for a C++ or Java compiler to take full-effect of deforestation, the compiler would need to know when a particular aggregate operation is occuring and whether or not there are side-effects. This is a very hard task for a compiler, unless the language has a predefined notion of aggregate operation.

Providing aggregate operations directly in programming languages have the additional advantage that it is easier for compilers to generate code that exploits multiple cores.

A testament of how effective aggregate operations is demonstrated by the success of the array-oriented languages APL, J, and K. These are usually implemented with interpreters which have excellent performance characteristics.

There are already some basic predefined operations on arrays in the Java virtual machine (JVM) and Common Intermediate Language (CIL) [Edit: replaced CLR with CIL, I meant to refer to .NET byte-code] for indexing and computing the size. The introduction of more sophisticated aggregate operations like map, filter, fold and unfold would be a logical extension to the CLR and the JVM. The performance benefits would not only be significant for single processors but there would also be benefit for code running on multiple processirs.