Grad School Application Advice:

• CS Professor Mor Harchol-Balter at Carnegie Mellon
• Phillip Guo (a Stanford CS grad student)
• CS Professor Mike Gleicher at Wisconsin
• CS Professor Matt Might at Utah

It also may be useful to take a look at a few forums. The quality of information here is far less consistent, but there are certainly useful facts to be gleaned.

Forums:

• The Grad Cafe
• Urch CS Admission Forums
• Graduate School Forum on College Confidential

I’ll resist editorializing the insights I’ve gained form all these resources, but the links I’ve provided should give anyone interested in CS graduate school a good starting point for developing their own opinions.

Although I’ve lived in Charlottesville for quite a while now, I haven’t really taken advantage of the numerous and various local farms. This is perhaps odd, given my health-obsessive nature, and our local prevalence of natural, grass-fed animal products. But whatever the impediment to my action — let us suppose schoolwork, research, or entrepreneurial activity — I finally got around to patronizing Avery’s Branch Farms last week.

So what does this have to do with asymmetry?  Well, I’ll be short and banal and to the point: I found that the “natural” Avery eggs look nicer than their industrial-farm begotten counterparts. Consider Avery eggs:

Against that of the typical no-name brand:

At least to my eye, the local farm’s eggs look quite a bit more appealing, and not simply because of differences in color or lighting. To my unartistic perception, I would suggest that the appeal stems directly from asymmetry. If you look closely at the farm eggs, you might see that there are small but noticeable deviations in size across the carton, a slight variance in color, and a psuedo-random sprinkling of freckles across shell exteriors. This is all in comparison to the bleached look of the industrial eggs, with little or no variation in egg-size across the carton.

Rest assured, lest I be accused of inane ramblings on this wonderful subject, that there is a larger point here. I am guessing that a slight asymmetry plays nicer with human perception. It might draw our eyes to important details, and make it easier for us to perceive the totality of written information. In short, asymmetrical perception may may implications for how we write code.

Consider the comparison of a C function (taken sort-of randomly from Git) and a lisp function (taken from Gajure). First let’s look at add_files_to_cache:

Reading from the left, this function is actually quite symmetrical. The eye is not overly drawn to any particular piece of code within the curly braces — perhaps the longer lines, if anything — and functional properties are not clearly conveyed through it’s structure. Much of this, although not all of it, has to do with the imperative nature of C.  Now, consider a second function, list-crossover:

Here we see a nesting of sorts. Although this is a simpler function, to be sure, we can see by way of indentation certain functional qualities of the code (say, that take and drop are applied in the context of concat). The functional nature of clojure (or any lisp) lends itself well to this kind of visual deconstruction, and at least to my eye, meaning is more readily conveyed through such asymmetrical properties.

In closing, I don’t mean to imply that one language is better than another at conveying meaning through syntax and symmetry, merely to suggest that the way code is written (surprise, surprise, no?) has quite a bit to do with how easily it can be understood. To that end, I think symmetry, or a lack thereof, plays an important role.

To do this, I drew a bit of help from Enlive, which functions primarily as a templating library, but has a few features useful for screen-scraping. This was perhaps a bit of overkill, as I only ended up using one of it’s functions, html-resource, which takes an URL as input, and outputs an hash that nicely represents a web-page’s structure.

What I ended up is wordy, which at the moment can do a simple word-count frequency analysis on a given page. That is, it counts how often words used, filtering (if desired) on word length. In just a bit, I’ll get into some of the more interesting aspects of coding it  up, but first,  here is a simple use case.

Running the following in slime…
(graph-words "http://ethanjfast.com" 5 5 1)

Where the parameters correspond to:

• ethanjfast.com -> web page to look at
• 5 -> minimum length (letter count) of word for first anaylsis
• 5 -> minimum length of word for last anaylsis
• 1 -> the amount of word length to increment by between the first and last anaylsis

To make this a bit clearer, consider a different run:
(graph-words "http://ycombinator.posterous.com" 3 10 3)

Here wordy does three analyses, with minimum word lengths of 3, 6, and 9 respectively. Clearly, I have some work to do insofar as these graphs look rather pathetic, but it was nice to get incanter working.

Now, onto some implementation details. Most of the code is quite simple, so I’ll just go through a few functions that may have some value to someone learning Clojure. For instance, here is rec-map, a function which recursively traverses the map/list structure returned by html-resource.

Basically, this function filters out all page content that doesn’t match specific tags (getting rid of links, css, javascript, ect.) But at first glance, you might wonder why I used trampoline rather than recur. After all, trampoline is used to recurse between two different functions, and it looks very much like rec-map is calling itself. Well, the trick is that I am calling trampoline inside the function passed to map, so recur will fail spectacularly (and in a very confusing manner). So watch out for recursion within anonymous functions!

Here is another bit of code, where I create the graph with Incanter.

The :group-by parameter is slightly unintuitive. To use it, you make a new vector of labels, each label mapping to a counterpart in the data vector. All data with the same label are then put into the same group (e.g for data ["You" "Me" "I"] [3 2 4] one might use the label vector [0 1 1] to group “Me” and “I” together). The rest is fairly self-explanatory, but I’ll mention one thing that I didn’t know until this morning. You can’t nest the # function shortcut. For instance, the following would not work:

(map #(map #(first %1) %1) lst)

It’s rather obvious in retrospect, I know. But I was dumb enough to try it. That’s all for now, and the code is available on github.

As always, the code open source and available at github.

Now, I doubt that it will come as a surprise, but I am rather a newbie at this whole R thing, and as I later found out, loops in R should be avoided at all costs. After hacking around with my code, I found that its critical path looked something like this:

total <- 0
for(i in 1:nrow(dataset)){
total <- total + sum( dataset[i,1:25]*coef )
}

Now this is very simple loop, and it seemed to me somewhat less than obvious that it would beget a significant performance bottleneck. Ever so naturally, then, it did.

Ironically, the solution here is to use code more along the lines of the map-reduce paradigm, something I would have loved to do in the first place, were not I overcome by the cryptic nature of R’s documentation. After all, my favorite languages are all variants of lisp, and I am no stranger to functional programming. After some digging, I stumbled across apply, which more-or-less functions along the lines of map in scheme or clojure. So I tried:

my_sum <- function(x){ sum( x[1:25]*coef ) }
sum( apply( dataset, my_sum ) )

In addition to being more elegant, this is much, much faster. What was taking hours, now takes tens of seconds. Apparently, R has a fast backend implementation for this sort of thing.  So, this post is dedicated to as a warning to my fellow inexperienced users: avoid iterative loops in R!

• Linear Algebra (Jim Heffron)
• Statistics (Michael Lavine)
• A Field Guide to Genetic Programming (Riccardo Poli, William B Langdon, Nicholas Freitag McPhee)
• Neural Networks (Raul Rojas)
• Introduction to Computing (David Evans)