I, for one, whole heartedly support the writers in their strike. In fact, I hope they remain on strike. Forever. The internet has never before had such a great range of funny videos to watch. However after a while it gets a little boring to continually watch videos about the strike. So, writers, we get your point but please move on to some other topics. You have so much talent not to waste it all as one trick ponies. Or LoLcats.

Books I have read in the past few weeks:

1. Twinkie Deconstructed. Really boring. Last time I trust Tyler Cowen for a book recommendation.
2. The Electric Kool Aid Acid Test - Great. The closest thing fiction that I have read in a while, without actually being fiction. It’s the story of Ken Kesey and his merry pranksters who suffered heavily from the burden of being some of the earliest LSD addicts. And as such weren’t enlightened by the now commonplace knowledge that _everything_ you do on acid seems really interesting and important.
3. Moneyball From the author of Liars Poker comes the story of Billy Beane and how he used refined statistical analysis to lead the Oakland A’s to a world series victory. Or at least thats what I thought until I got to the end chapter. Turns out that they don’t win, but they get close. Either way, its a good read. I walked away even more convinced of the fact that most established knowledge is right, but every now and then its very wrong.
4. The Informant. My dad left this book with me when he was over in New York last week. On the 600-odd pages of this book we learn the story of international price fixing in the lysine market and the development of a huge white collar crime case by the FBI and DoJ, centering on the Archer Daniels Midland company. I had dinner on the weekend with some FBI agents [[Why is it that a statistically significant portion of my friends work in law enforcement in America, yet most of my Australian friends are criminals?]] and they had never heard of the case. Supposedly they are making a film of this book, which doesn’t surprise me as it reads like a script. Should be a great film, very exciting story.

Book(s) I am currently reading:

1. Judgement under Uncertainty. After my disappointment with Behavioural Finance, by James Montier, I decided to go straight to the horses mouth and finally read the works of Kahnemann and Tversky. So far the book is confirming my prior belief that behavioral finance/economics is simply plain old economics with more complex utility functions than the analytically pleasing forms that are often assumed to describe rationality.

Books I probably won’t ever read:

1. The Tipping Point, by Malcolm Gladwell
2. Anything by Richard Dawkins

Machines I am currently building:

1. Soup! - I’m not quite sure what it does yet. But its fun to be building machines again. I really suck at drilling holes in a straight line. I want a drill press for Christmas. And a lab in which to use it.

Jerry just wrote an interesting post about the impact of behavioral targeting on media prices. Half of our ads work, we just don’t know which half, so the adage goes. Jerry asks:

If advertisers could put their ad only in front of the fifty people who are likely to buy the product rather than the hundred that might or might not, those other fifty ad slots go begging. Half the advertising inventory will not be bought. This 50% drop in demand should drastically cut prices.

On the other hand, if an advertiser can target better, she should be willing to spend more to get in front of that targeted audience. But how much more? Twice as much? Does the advertiser end up paying less, resulting in lower overall media revenues? Or does media benefit by being able to charge premium prices for all of their inventory?

Ceteris paribus, net revenues to the publisher should remain the same. They are selling half the units at twice the cost. And the advertiser ends up paying the same amount for the same number of customers. Of course, if this were the case, behavioral targeting companies would not exist - the industry would be indifferent as to whether to target or not. Behavioral targeting companies market their technology with the premise that there is no need to pay $50 CPM’s for premium placements when the same eyeballs also see remnant inventory at $0.50 CPM. The targeting platform makes sure that the right eyeballs see the right ad at the right time, regardless of placement. That said, as Jerry recognizes, if everyone switches to behavioral targeting, then it all works out in the wash, and pricing per placement becomes a thing of the past. But net, the cash flows remain the same.

“Nobody counts the number of ads you run;they just remember the impression you make.”
–
William Bernbach

Where things get interesting is if we allow for the possibility that premium placements really deserve higher CPM’s on the basis of placement alone. In which case, we can no longer lump together the n units of premium inventory with the N units of remnant/run of site, into a fungible (n+N) size pool. If there is anything fundamentally different about premium inventory then ceteris stops looking paribus. There are plenty of smart cookies, with more PhD’s than you can poke a University of Phoenix marketing diploma at, who continue to pay a premium for front page ad runs. I reckon they might be onto something, behavioral targeting be damned.

I’ve been following the 20×200 blog / store since inception. I love the idea and and many of the works. A few weeks ago Shianling and I decided to buy a print (above). Unfortunately I’m not terribly happy with the print quality. I don’t know much about printing, but am familiar with the problems of viewing art online, where the gamut of a monitor is greatly limited compared with what can be printed. However, the print we received lacks much of the vibrancy of the online picture. Oh welp. We supported an artist and have a shiny new thing to hang on our rapidly crowding walls.

Complaints about the print quality aside, I am very tempted to buy a new work that became available today (below) as I seem to have a deep emotional connection to custom bound books.

One of the best ways to improve your Page Rank is to slavishly follow other blogs and provide witty responses to the posts of more credible bloggers. In that spirit, let me one-up Felix Salmon and raise his correlation analysis with a p-value. It turns out that Felix holds annual Pinot parties. Sorry, not parties, but scientific evaluations of Pinot Noir. Eschewing issues arising from normality, rank scoring, sample size, and underrepresentation of Australian wines, I present the following confirmation of Felix’s guesstimate that Pinot rankings and price are uncorrelated. Rather than rely on the unwieldy crutch of an R-square, I prefer to look at the p-value on the regression of score against price:

 > score=c(115,196,137,146,175,212,193,184,180,167,154,143); > price=c(18,23,29,52,33,12,26,18,10,50,13,12); > l<-lm(score ~ price); > summary(l) Call: lm(formula = score ~ price) Residuals: Min 1Q Median 3Q Max -53.982 -19.423 8.385 17.913 41.085 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 174.7818 17.4850 9.996 1.60e-06 *** price -0.3222 0.6200 -0.520 0.615 — Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 29.36 on 10 degrees of freedom Multiple R-Squared: 0.0263, Adjusted R-squared: -0.07107 F-statistic: 0.2701 on 1 and 10 DF, p-value: 0.6146 

The above analysis, done in my BFF language - R, demonstrates that price is not a significant predictor of wine score. With a p-value of 0.615, the null hypothesis is looking a lot like a Pinot Party. A place where we can all be comfortable.

For bonus points, I leave it to my good readers to perform a contingency table analysis.

One of the best things about being crowned a romance buff by the prestigious New York Post is that I now can get away with pretty much anything at home.

“Honey, can you do the dishes?”
“But I’m busy knitting you a scarf”
“Yes, but doing the dishes would be romantic. I declare it so.”

I now have a Frank Bruni-esque[1] power to declare anything romantic and it must be so. Anything else would be denying the Post’s role as an arbiter of New York cultural icons. Of which I am one.

—
[1]: Or whomever the NY Post’s equivalent may be.

Update: According to one of Shianling’s patients, we also made it to The Gothamist. Now I’m truly a cultural icon.

Postgres:

 think=# select 'hello' = 'hello'; ?column? ---------- t (1 row) think=# select 'hello' = 'goodbye'; ?column? ---------- f (1 row) think=# select 'hello' = 'HELLO'; ?column? ---------- f (1 row) 

MySQL:

 mysql> select 'hello' = 'hello'; +-------------------+ | 'hello' = 'hello' | +-------------------+ | 1 | +-------------------+ 1 row in set (0.00 sec) mysql> select 'hello' = 'goodbye'; +---------------------+ | 'hello' = 'goodbye' | +---------------------+ | 0 | +---------------------+ 1 row in set (0.00 sec) mysql> select 'hello' = 'HELLO'; +-------------------+ | 'hello' = 'HELLO' | +-------------------+ | 1 | +-------------------+ 1 row in set (0.00 sec) 

This is not kindergarten people, this is UNIX. Do what I say, not what you think I mean.

Since appearing on the popular section of del.icio.us, a number of people have asked me to explain how it is, in fact, possible to replicate what we see in the following video:

Whats going on here is really quite simple. All you need to do is hold the video camera with one hand and wrap the foil etc. using your free hand. It definitely helps if you have a small, hand-held video camera. If you have one of these it might be good to have a friend around while attempting the above.

Mike called my bluff. The book I refer to in this post was actually from 1988, not the 1960’s. The book is Dynamic Graphics for Statistics and can be had for a princely sum of $149 from Amazon. I’m certain that I paid approximately 1/100th of that for it.

In a semi-unrelated meme, I was chatting with my friend Aaron a few weeks back about why I disliked 3D graphics, especially in games. To his credit, I don’t play computer games so I miss out any much of the emotional nuance that comes from interacting with simulated gore and I probably shroud my argument with rationales purely to hide my irrational aesthetic bias. But that said, I don’t like 3D graphics for two interrelated reasons. Firstly, I assume that the designers of these graphics curse themselves at the lack of rendering power at their fingertips. Sure, they think their blood spatter looks better than any game that came before it, but if only they had 64 GPU pipelines instead of 32, it would look so much better. George Suerat never complained about the number of lines per inch in his Gillot paper. The paper was chosen for its capabilities and its characteristics were exploited to promote the aesthetic that Seurat was seeking. I have long felt that art and constraints go hand in hand and successful creators express themselves within boundaries. For the most part, game graphics are not timeless. Today people look back on games from 2003 and chuckle at the lack of realism. In 2009 the creators of Halo will secretly wish that they could go back and remake their games with 2009 era technology. Who knows if the makers of Galaga would prefer to recreate their game using today’s technology, my guess is not.

Fundamentally, I find it a little offensive when people make graphics and aren’t happy with the outcome. Sure, game creators (and players) probably really enjoy their work. But whenever they admit that they need a better computer to ‘get the full experience’, they are stating that they are not happy with what they have. Computers are immensely powerful. But I fear that much of that power is being dedicated to doing things very uncomputer-like. If people adapted to and adopted the medium, we may have graphics that look very different to reality but in the abstraction away from reality I think we can find a fluid, informative, and medium appropriate representations. People don’t have an innate understanding of what data looks like, and in the world of statistical analysis and visualization we can afford to break the mold. Perhaps not in many games, but I have seen some games that take this approach and invent new visual realities that are (to my untrained eyes) as emotionally engaging and poorly rendered trees and bump-mapped blood spatter.

I won’t dwell too much on my second concern with contemporary computer graphics as it is well covered by Tufte. Namely, the core concern is that too many pixels and graphic elements are used for non-informative purposes. Shading a chart, or purposelessly extruding a bar chart in 3D adds no new information to a graphic. This is why on any new install of Excel I spend a good hour or so setting up new chart defaults to get rid of all the junk it tends to decorate my charts with. (Yes, at times, I use Excel.) Going beyond the world of images, the same principle can be extended in interface design. Just as one needs fewer distractions from the information, interfaces need fewer mouse clicks and menu selections to play with the data. In this age of computer power, people often neglect the value of exploratory data analysis. More, now than ever, is EDA important. I have terabytes of data at my fingertips, and most visualization tools just don’t keep up. I want to be able to slice, dice, plot and histogram data at a breeze. Data retrieval isn’t the bottleneck, but rather poor interface design.

What this world needs is a nice, simple, stand alone, does-one-thing-and-does-it-well exploratory data analysis tool. Maybe I should make one.

A movie about exactly that. A documentary about the world we live in.
Thee spirit ov thee times.

I watched this movie recently, it is a melding of many different themes that are current in todays society and prevelant in everyones mind. It centres around Money , War, Terrorism and how we are all fooled by the mass media. On the surface this kind of description may seem to be rather conspiracy thoerist and alarmist , but infact i found this movie to be a rather sobering roller coaster journey back to reality. It puts together alot of factual information and presents it in a way which just makes you think “this is whats going on” and it indeed is.

Although the movie is focussed largely on America and its historical impercial capitalist value system, it is equally relevant for Australia , since as the famous old addage goes “When America sneeze’s the rest of the world gets a cold” , seems particularly true in light of the information presented. Particularly regarding some of the more notable “Black flag” operations , such as 911 , 7/7, and bali bombings ,the date of which , sadly escapes my current frame of mental reference; and the strikingly similar circumstances in which those so called “terrorist” attacks occured. I have always had my doubts on the validity of the information the media presented on these generic operations. This movie has served only to assist me in strengthening my resolve to think for myself and question authority. I must be getting old.

sense