I did the 10 km running + 20 km cycling thing.

I was the last-but-one guy to finish and I did take twice the amount of time as the first guy to finish.

But I didn’t care about that. I expected to finish in 3 hours and I completed before that. And I finished strongly, not crawling to the end as I used to. I enjoyed the run, I enjoyed the cycling and I was satisfied.

Photos by Vikram:

It reminded me of the book “What I talk about when I talk about running” by Haruki Murakami that I read recently (borrowed from Varun).

I really liked the book, because Murakami puts into words the things I have felt as a runner but is almost impossible to truly explain it to somebody else.

Just to put things into perspective – Murakami started running in 1982 at the age of 30, running everyday since then for nearly 23 years. He has run at least one marathon every year, i.e., 23 marathons till date [when the book was published], and many more long-distance runs.

Some of my favorite passages from the book are below.

About the rhythm:

As long as I can run a certain distance, that’s all I care about. Sometimes I run fast when I feel like it, but if I increase the pace I shorten the amount of time I run, the point being to let the exhilaration I feel at the end of each run carry over to the next day. This is the same sort of tack I find necessary when writing a novel. I stop every day right at the ponit where I feel I can write more. Do that, and the next day’s work goes surprisingly smoothly. I think Ernest Hemingway did something like that. To keep on going, you have to keep up the rhythm. This is the important thing for long-term projects. Once you set the pace, the rest will follow. The problem is getting the flywheel to spin at a set speed – and to get to that point takes as much concentration and effort as you can manage.

About why we run:

Most ordinary runners are motivated by an individual goal, more than anything: namely, a time they want to beat. As long as he can beat that time, a runner will feel he’s accomplished what he set out to do, and if he can’t, then he’ll feel he hasn’t. Even if he doesn’t break the time he’d hoped for, as long as he has the sense of satisfaction at having done his very best – and, possibly, having made some significant discovery about himself in the process – then that in itself is an accomplishment, a positive feeling he can carry over to the next race.

… Marathon runners will understand what I mean. We don’t really care whether we beat any other particular runner. World-class runners, of course, want to outdo their closest rivals, but for your average, everyday runner, individual rivalry isn’t a major issue. I’m sure there are garden-variety runners whose desire to beat a particular rival spurs them on to train harder. But what happens if their rival, for whatever reason, drops out of the competition? Their motivation for running would disappear or at least diminish, and it’d be hard for them to remain runners for long.

For me, running is both exercise and a metaphor. Running day after day, piling up the races, bit by bit I raise the bar, and by clearing each level I elevate myself. At least that’s why I’ve put in the effort day after day: to raise my own level. I’m no great runner, by any means. I’m at an ordinary – or perhaps more like mediocre – level. But that’s not the point. The point is whether or not I improved over yesterday. In long-distance running the only opponent you have to beat is yourself, the way you used to be.

About running as an outlet:

When I’m criticized unjustly (from my viewpoint, at least), or when someone I’m sure will understand me doesn’t, I go running for a little longer than usual. By running longer it’s like I can physically exhaust that portion of my discontent. It also makes me realize again how weak I am, how limited by abilities are. I become aware, physically, of these low points. And one of the results of running a little farther than usual is that I become that much stronger. If I’m angry, I direct that anger towards myself. If I have a frustrating experience, I used that to improve myself. That’s the way I’ve always lived. I quietly absorb the things I’m able to, releasing them later, and in as changed a form as possible, as part of the story line in a novel.

What do we think about when we run:

… The hour or so I spend running, maintaining my own silent, private time, is important to help me keep my mental well-being. When I’m running I don’t have to talk to anybody and don’t have to listen to anybody. All I need to do is gaze at the scenery passing by. This is a part of my day I can’t do without.

I’m often asked what I think about as I run. Usually the people who ask this have never run long distances themselves. I always ponder the question. What exactly do I think about when I’m running? I don’t have a clue.

On cold days I guess I think a little about how cold it is. And about the heat on hot days. When I’m sad I think a little about sadness. When I’m happy I think a little about happiness. As I mentioned before, random memories come to me too. And occasionally, hardly ever, really, I get an idea to use in a novel. But really as I run, I don’t think much of anything worth mentioning.

I just run. I run in a void. Or maybe I should put it the other way: I run in order to acquire a void. But as you might expect, an occasional thought will slip into this void. People’s minds can’t be a complete blank. Human beings’ emotions are not strong or consistent enough to sustain a vacuum. What I mean is, the kinds of thoughts and ideas that invade my emotions as I run remain subordinate to that void. Lacking content, they are just random thoughts that gather around that central void.

The thoughts that occur to me while I’m running are like clouds in the sky. Clouds of all different sizes. They come and they go, while the sky remains the same sky as always. The clouds are mere guests in the sky that pass away and vanish, leaving behind the sky. They sky both exists and doesn’t exist. It has substance and at the same time doesn’t. And we merely accept that vast expanse and drink it in.

About the last 20%:

… Rereading the article I wrote at the time of this run in Greece, I’ve discovered that after twenty-some years, and as many marathons later, the feelings I have when I run twenty-six miles are the same as back then. Even now, whenever I run a marathon my mind goes through the same exact process. Up to nineteen miles I’m sure I can run a good time, but past twenty-two miles I run out of fuel and start to get upset at everything. And at the end I feel like a car that’s run out of gas. But after I finish and some time has passed, I forget all the pain and misery and am already planning how I can run an even better time in the next race. The funny thing is, no matter how much experience I have under my belt, no matter how old I get, it’s all just a repeat of what came before.

I think certain types of processes don’t allow for any variation. If you have to be part of that process, all you can do is transform – or perhaps distort – yourself through that persistent repetition, and make that process a part of your own personality.

About pain:

… Of course it was painful, and there were times when, emotionally, I just wanted to chuck it all. But pain seems to be a precondition for this kind of sport. If pain weren’t involved, who in the world would ever go to the trouble of taking part in sports like the triathlon or the marathon, which demand such an investment of time and energy? It’s precisely because of the pain, precisely because we want to overcome that pain, that we can get the feeling, through this process, of really being alive – or at least a partial sense of it. Your quality of experience is based not on standards such as time or ranking, but on finally awakening to an awareness of the fluidity within action itself. If things go well, that is.

About the life lessons from running:

Most of what I know about writing I’ve learned through running every day. These are practical, physical lessons. How much can I push myself? How much rest is appropriate – and how much is too much? How far can I take something and still keep it decent and consistent? When does it become narrow-minded and inflexible? How much should I be aware of the world outside, and how much should I focus on my inner world? To what extent should I be confident in my abilities, and when should I start doubting myself? I know that if I hadn’t become a long-distance runner when I became a novelist, my work would have been vastly different. How different? Hard to say. But something would have definitely been different.

My time, the rank I attain, my outward appearance – all of these are secondary. For a runner like me, what’s really important is reaching the goal I set myself, under my own power. I give it everything I have, endure what needs enduring, and am able, in my own way, to be satisfied. From out of the failures and joys I always try to come away having grasped a concrete lesson. (It’s got to be concrete, no matter how small it is.) And I hope that, over time, as one race follows another, in the end I’ll reach a place I’m content with. Or maybe just catch a glimpse of it. (Yes, that’s a more appropriate way of putting it.)

Murakami explains beautifully on how running and life lessons are so intertwined. I like to think on the same lines too.

For example, if you find the concept of running boring – I can argue the same applies for life – get up, go to work, come back, watch TV for a few hours, repeat 5-6 times, then go out for a movie on the weekend. Most people follow the same routine every week. How boring, isn’t it?

If you find getting started with running difficult – the same applies for getting started with any new activity. Whether it is learning a new subject in school, learning a new technology or process at work, or learning how to become a good husband or wife – getting started is always difficult. But it doesn’t mean you give up, right?

If you think running is too much effort for nothing, the same applies for life – you struggle so hard to survive and keep moving forward in life… for what?

Running requires a certain rhythm to be achieved, only then you can truly enjoy it. Same applies for life. If you’re going fully strong, only then will there be a stride in your walk.

The last 20% is the really difficult part of any long-distance run. Same for any long-term project. Really taking things to completion will give you a high like nothing else will.

Running is about competing with yourself, not with others. Same applies for life. If all you care about is being in the top ten, then you’re not truly enjoying the run itself, your focus is on the finish. But every wise man has already said that it is about the journey, not the destination.

Running is most fun when you cross the finish line strongly and knowing that you could’ve done more. Same applies for life. It is depressing to have given up in life and just waiting for the end, it is much more invigorating and worthwhile to be moving strongly.

“I do not run to add days to my life — I run to add life to my days.”
— Ronald Rook

• Shows me what are the movies that are popular in theatres right now
• Shows what new movies are releasing this week
• The first item on every movie page is the average user rating
• It links to every individual artiste involved (actor, director, etc.) so that if I like to follow a particular actor like Russell Crowe, then I just have to visit his page and I have all the details right there in a compact list.
• Trivia and Quotes from the movie – the amusing/fun aspect.
• The information is not cluttered with random reviews, that is on a separate page if you are so inclined.

But IMDB is for movies only.

What about an IMDB for music?

Yes, there is last.fm but it concentrates on the actual playing of the existing music content that they have (which it is very good at), but not about the people who make the music or their discographies. For example, I don’t think they list albums/songs that are upcoming or are not part of their playlists.

What about an IMDB for books?

Yes, there is Amazon and there is WeRead.com but I notice the same problems as last.fm.

What I’m looking for is something like the burrp of music and books: showcasing (1) the latest, (2) the best and (3) the right kind of information, nothing else.

Also, are there similar sites for India? For example, I’d love to see the list of the top rated movie soundtracks in 2009 by “Shankar, Ehsaan & Loy” or the top songs ever by Pentagram (I just love “Bad Man”), and so on. Are there such lists and information already out there?

I think IMDB-equivalents for music and for books can be successful online businesses and communities.

Update : “Good Reads” seems to be a good option for books.

Update 2 : Looks like “MovieDB.in” wants to be the IMDB of Indian movies.

Note: This blog post is part of the “Ideas are Cheap. Execution is Everything.” series where I pen down thoughts on what I see is a need in the market or what could be a successful idea/business.

Just a few days back, a friend was telling me that the famous Utility building Kamat restaurant in Bangalore no longer has quality food and hence no longer a popular place. I read this book and it gave the background to this situation – it is no longer being run by the Kamats for whom hospitality is everything, it is now being run by the Kamat that usurped the properties. At least, that’s what the book says.

But that’s not what the book is about. The book is about the entrepreneur’s journey. What I liked about the book was that it was written in plain and simple English, and Vithal writes about his life and the hard work he put in, the mistakes made and the lessons learned from it. It sounds familiar like any other entrepreneur’s autobiography, but what made it special for me was that this was an Indian and almost everyone has heard about the famous Kamat restaurants! It was good to read the story of the restaurants and the people who make the place what it is.

During the story, some good traits of entrepreneurs were demonstrated:

• Having knowledge, great ideas and executing them. For example, when Vithal was a kid, his uncle’s son was getting married and in that event, the soft drinks were not cooled and there was just 15-20 min before the guests started arriving. Young Vithal then used his knowledge of how kulfis are made, took fistfuls of salt and threw it on the ice which made it drastically go down in temperature and hence all the soft drinks were chilled in 15 min. The same goes for many of his tactics such as putting free buses to and fro the airport to his hotel, the Kamat Plaza, to make waiting less stressful for travellers and that became an instant hit. He said that brought in more customers than any amount of advertising could have done. Eventually, the airways people would suggest travellers to rest at Kamat so as to make them less annoyed about delayed flights, etc. A win-win-win situation indeed.
• Doing a lot of networking. Vithal proves time and again how his networking and at the same time being known for their hospitality and credibility helped him in many a situation.
• The importance of preparation. This is everything in the hotel business, he says. For example, that’s how you get your food so quickly when you order (instead of the hours that it would take if you cooked at home yourself).
• Having a great dream, a great passion. Vithal has lost a lot while trying to make his dream ‘The Orchid’ come true, especially after all the property was usurped by his younger brother, and he had taken many high-interest loans so that he could build his dream hotel while his father was alive (who was dying of cancer). And yet, all the goodwill that he had generated and his will power slowly helped him eke out of the pit and the dream came true. This part of the story was heart-wrenching and inspiring at the same time. These are the kinds of stories that we see in movies but this is a real true story.

The only downside to the book is that you have to read the parts about the perfect character/attitude with a pinch of salt, because it sounds preachy at times and frankly, sounds too good to be true.

If you ever wanted to know what entrepreneurship is about, don’t read MBA sites, just read this book, if you can find it*. And then decide whether you are prepared for it. At the same time, you’ll finish the book feeling inspired.

^* It is such a tragedy that this book is not available in any online Indian book store that I know of.

Here’s an excerpt:

Cultural legacies matter, and once we’ve seen the surprising effects of such things as power distance and numbers that can be said in a quarter as opposed to a third of a second, it’s hard not to wonder how many other cultural legacies have an impact on our twenty-first-century intellectual tasks.

What redeemed the life of a rice farmer, however, was the nature of the work. It was a lot like the garment work done by the Jewish immigrants to New York. It was meaningful.

First of all, there is a clear relationship in rice farming between effort and reward. The harder you work a rice field, the more it yields.

Second, it’s complex work. The rice farmer isn’t simply planting in the spring and harvesting in the fall. He or she effectively runs a small business, juggling a family workforce, hedging uncertainty through seed selection, building and managing a sophisticated irrigation system, and coordinating the complicated process of harvesting the first crop while simultaneously preparing the second crop.

And, most of all, it’s autonomous. The peasants of Europe worked essentially as low-paid slaves of an aristocratic landlord, with little control over their own destinies. But China and Japan never developed that kind of oppressive feudal system, because feudalism simply can’t work in a rice economy. Growing rice is too complicated and intricate for a system that requires farmers to be coerced and bullied into going out into the fields each morning. By the fourteenth and fifteenth centuries, landlords in central and Southern China had an almost completely hands-off relationship with their tenants: they would collect a fixed rent and let farmers go about their business.

Here’s a second excerpt:

Every four years, an international group of educators administers a comprehensive mathematics and science test to elementary and junior high students around the world called TIMMS. The point is to compare the educational achievement of one country with another’s.

When students sit down to take the TIMSS exam, they also have to fill out a questionnaire. It asks them all kinds of things, such as what their parents’ level of education is, and what their views about math are, and what their friendss are like. It’s not a trivial exercise. It’s about 120 questions long. In fact, it is so tedious and demanding that many students leave as many as ten or twenty questions blank.

Now, here’s the interesting part. As it turns out, the average number of items answered on that questionnaire varies from country to country. It is possible, in fact, to rank all the participating countries according to how many items their students answer on the questionnaire. Now, what do you think happens if you compare the questionnaire rankings with the math rankings on the TIMSS? They are exactly the same. In other words, countries whose students are willing to concentrate and sit still long enough and focus on answering every question in an endless questionnaire are the same countries whose students do the best job of solving math problems.

Think about this another way. Imagine that every year, there was a Math Olympics in some fabulous city in the world. And every country in the world sent its own team of one thousand eighth graders. Boe’s point is that we could predict precisely the order in which every country would finish in the Math Olympics without asking a single math question. All we would have to do is give them some task measuring how hard they are willing to work. In fact, we wouldn’t even have to give them a task. We should be able to predict which countries are best at math simply by looking at which national cultures place the highest emphasis on effort and hard work.

So, which places are at the top of both lists? The answer shouldn’t surprise you: Singapore, South Korea, China (Taiwan), Hong Kong, and Japan. What those five have in common, of course, is that they are all cultures shaped by the tradition of wet-rice agriculture and meaningful work. They are the kinds of places where, for hundreds of years, penniless peasants, slaving away in the rice paddies three thousand hours a year, said things to one another like “No one who can rise before dawn three hundred sixty days a year fails to make his family rich.”

See how the two excerpts are related? This explains how your cultural legacies matter (and don’t worry, maths is not the criterion for success, this is just one example in the book). Another example is how cultural legacies are related to plane crashes of the respective national airlines.

There’s a lot more in the book like the Matthew Effect, the 10,000-Hour Rule, why “practical intelligence” matters, why “concerted cultivation” matters, about the KIPP schools, and so on.

The book is a must-read IMHO, just for the thought-provocativeness, even if not how to learn to be “successful.”

(1) First of all, I did try to talk to publishers, hoping that I would convince them to release the book simultaneously under a free license as well as a printed version (which is true for many technical books these days). All the publishers I spoke to said there is no market for such a book and said no to the idea. But that didn’t deter me, because I really wanted to see such a book out there, so I wrote it anyway.

(2) Technical books readership is on the decline. It seems very few techies buy and read books, they just google it and solve their immediate problems vs. reading a whole book.

If you don’t believe me, see what John Resig, Charles Petzold, Jeff Atwood and Eric Sink have to say on the subject.

(3) I had a concern bigger than not getting it published, it was that nobody would get to know about the book and hence the book would go in vain. Since money was not a motivating factor in this particular case, I was far more interested in seeing lots of readers and widespread usage than to see fewer readers with the published book although the latter would make me more money.

Tim O’Reilly’s words remained stuck in my mind:

“Obscurity is a far greater threat to authors and creative artists than piracy.”

Of course, I did have a printed books option, so I still could have made money just like 37 Signals did with their “Getting Real” book which was free to read online plus available as a paid PDF download. Unfortunately, it seems I lack their marketing pizzazz.

(4) The book was intended to be a contribution back to the open source community. We constantly keep taking and taking – whether it is using Linux, Vim, Firefox, or countless other software, so it felt great to be useful to the community in return.

As Steve Jobs said:

You know, we don’t grow most of the food we eat. We wear clothes other people make. We speak a language that other people developed. We use a mathematics that other people evolved… I mean, we’re constantly taking things. It’s a wonderful, ecstatic feeling to create something that puts it back in the pool of human experience and knowledge.

(5) My experience has been that a lot of people would like to translate such books to their native languages to help more people use the software. So, I’m happy to see volunteers now translating the new Vim book to Chinese, Russian and Swedish languages!

I needed a balanced approach to what I was trying to achieve, and all the above reasons led me to use a Creative Commons license.

It has been a revelatory book for me, a developer who considers himself to be the last person to know about “design.” The book mainly focuses on the lessons learned from their experiences in working with clients to design and create products and services.

Design

They define design as an activity, as opposed to a look and feel that is added later on. The activity incorporates:

• Empathy – Design must serve a human purpose, and so design requires an understanding of how people will interact with whatever you’re designing.
• Problem Solving – Design really shines when it’s used to address complex problems where the outcome is clear, many stakeholders are involved, and the boundaries are fuzzy.
• Ideation and prototyping – Design produces things, whether they’re abstract (schematics, blueprints, wireframes, conceptual models) or concrete (prototypes, physical models). Design is a creative activity and thus requires actually creating something.
• Finding alternatives – Design is less about the analysis of existing options than the creation of new options. Sometimes that means looking at existing options in new ways, and at other times that means creating from scratch. An effective design process typically offers many solutions to a problem.

They repeatedly explain that the experience is what matters to the end-user and that’s the real product rather than how it is delivered.

Experience

The qualities of a person’s experience and how it affects whether they think of a product or service as a “good” or “bad” experience depends on these qualities:

• Motivations – why they are engaged with your offering, and what they hope to get out of it.
• Expectations – the preconceptions they bring to how something works
• Perceptions – the ways in which your offering affects their senses (see, hear, touch, smell, taste)
• Abilities – how they are able to cognitively and physically interact with your offering
• Flow – how they engage with your offering over time
• Culture – the framework of codes (manners, language, rituals), behavioral norms, and systems of belief wiwthin which the person operates.

Empathy

They take great pains and length to explain Empathy, and how it is important and central to getting key insights which lead to pathbreaking products.

Empathy is an understanding of a person or group’s subjective experience by sharing that experience vicariously. Empathy is something like a balanced curiosity that can lead to a deeper understanding of another person. After all, customers lives are messy and complex, not always rational or task-oriented or follow perfect paths that product-makers expect.

They talk about what’s missing in our understanding are

• The emotional aspect
• The cultural aspect
• The user’s context

One of the best insights I read was their description of organizations evolving their product strategy from a:

Focus on Technology → Focus on Features → Focus on Experience

This is the exactly the kind of transition that we can relate to.

Parallelly, a company’s view of people also shifts from:

“What users?” → Their tasks, goals and preferences → Their behaviors and motivations.

Experience, not Products

They stress to not design products, but to design the system as part of the larger experience, whether the experience is end-to-end your system, or part of a larger ecosystem i.e. understand the context.

• Organizations find it tough to coordinate, it has to become a competency, doing cheaper/faster is not a long-term strategy in today’s world.
• Design the product in the context of systems and strategy.
• “Design gives form to ideas so that their value can be communicated, evaluated and improved.”
• Storyboards can be better to do first than Prototyping.
• Strategy and Prototyping should go hand in hand.
• “Wow!” factor ⇒ Loyalty of customers

This chapter is where I had most of my moments of epiphany.

My Takeaways

• Customer’s experience + context/system > product/technology
• Empathy > Throwing ideas
• Design artifacts important
• Design as an organizational competency → Multiple solutions, Focus, Constraints
• The “Wow!” Factor
• Give up control to the user

Bottom line: Agile technical implementation + Design competency + Empathy ⇒ Critical for a successful company

The downside was that it is a short book and I wish it was more concrete in its explanations and descriptions.

Overall, I highly recommend the book for anyone who is involved in the creation of products and services in their companies, especially entrepreneurs.

On a related note, Rands in Repose has some great points on his expectations of a product:

1. Make it look and feel like magic.
2. Work flawlessly in the first 10 minutes. If you can’t survive 10 minutes of critical analysis, I’m gone.
3. Provide additional, unexpected awesomeness.

I was so glad he did. I ended up spending hours reading Pavlina’s articles. Reflecting upon the ideas in these articles was very beneficial. When I read that Steve was releasing a new book, I jumped at the chance to get it.

The book was different from most self-improvement books because it didn’t focus on productivity or time management. Steve claimed that he has discovered the essential principles of life!

According to Steve, there are just three core principles – truth, love and power. The secondary principles are:

• Oneness = Truth + Love
• Authority = Truth + Power
• Courage = Love + Power
• Intelligence = Truth + Love + Power

I found it incredulous to see someone make such a claim. So I started reading the book with a sense of disbelief.

While I started reading the book, I didn’t appreciate its brevity but the upside was that I got through the book more quickly. The basic concepts were things I understood but concepts like ‘oneness’ was something I couldn’t fathom.

Eventually, a friend called me up and was describing a personal problem, I started to test whether Pavlina’s principles were applicable, and voila, I was amazed to pinpoint to something which I was convinced was the root cause. It was at that moment that I started thinking that Steve might be on to something.

I had a hard time reading through the book, not because it was bad but because for every other page I would stop and reflect upon the concept being described and I would do some journaling to help me clarify my thoughts. In the process, I realized I was applying the ‘Truth’ principle and finally accepting some things that I “delayed thinking about” (read as “avoid”).

Eventually, I started reflecting upon the past ups and downs of life and see if the good things were as a result of cohesion of the three core principles. Well, it did. And at the same time, I could place a lot of my faults into the categories under “Blocks to Love” and “Blocks to Power” sections.

Strangely, I felt like I was reading one of those Linda Goodman books which claim to know every detail of the character of a person just based on the date on which they were born. The logical portion of my brain simply refuses to accept something like that is possible. Similarly, I have a hard time believing that someone can boil down the psychology and well-being of humans to such a simple list of things.

Nevertheless, the true impact of a self-improvement book is only felt months later, so I’m still in the process of applying some of the concepts and thinking to my daily habits. I find myself aligned with the principle of truth, but not with the principles of love and power. I hope some of the 30-day trials (as described in the book) in applying these concepts will pay off.

All in all, I would highly recommend Steve Pavlina’s book “Personal Development for Smart People”. It will make you think and hopefully make you grow as well.

The best part is that the hard-working translators can also publish their translations and sell the printed copies, benefiting both the readers and the translators.

I had received many requests from readers for hard copies of the book and I’m glad to finally get this working. Interestingly, I was previously trying to get the book printed via CreateSpace because the book would automatically get listed on Amazon.com (since CreateSpace is owned by Amazon). However, their process was not streamlined and confusing. Worse, I couldn’t get the PDF in their required size formats because of a bug with mwlib.rl.

I got tired and decided to try Lulu and I was very surprised. They are miles ahead in terms of usability of their service as well as wide range of options and sensible defaults. For example, it was a pain waiting for manual approval of the book by the CreateSpace staff and it is an unnecessary delay every time I upload a new version. On the other hand, Lulu made it very easy to design a rudimentary cover using their process. Overall, I was able to make the printed copy available for purchase in a single evening.

Of course, all this is possible because of the ability to generate PDFs from a wiki, thanks to the nice people at PediaPress.

Update: For Indian readers, the book is now available via pothi.com.

The interesting news is that it is updated for the upcoming Python 3.0 language making it probably the first book to be released for Python 3.0.

The book is now a wiki too at www.swaroopch.com/notes/Python which means you can contribute too!

The book and wiki are now under the Creative Commons Attribution-Share Alike 3.0 Unported license. The Non-Commercial clause present in the previous edition of the book has been removed. It was becoming a hurdle for translators as well as people who wanted to use the book for genuinely good activities, so I decided it to drop the clause.

Since it is a wiki, volunteers can directly create their translations on the wiki. This eliminates the need to learn DocBook XML and its tools which had become a hindrance for many translators, and I’m glad to see this already bearing fruit with Eirik Vågeskar starting off a Norwegian translation at www.swaroopch.com/notes/Python_nb-no:Forord.

I will soon be making a printed version of the book available as I have had many requests for this.

So back to the main question: Why an update after nearly 4 years? Two reasons.

First, because of foss.in. I dedicate this new release to the foss.in community for their spirit and enthusiasm over the years which have rubbed off on me and kept me working on the update of the book.

Second, Over the past few years, the readers’ reactions have been simply splendid:

Neil (bigdealneil-at-yahoo-dot-com) said:

“(I) got an if else to work and I can follow your tutorial, which I have never been able to do no matter who wrote the thing! you’re a genius Swaroop!”

Gao shuai (ejwjvh-at-126-dot-com) took the effort taken to write an email to me in English:

dear swaroop: I am a chinese student.My name is gao shuai,”gao”is my family name. Although your book is easy to understand,but my english is bad,so what I read is the chinese edition. I have made some programs now.It is interesting.I like it very much.

I emailed back and he replied:

Mr Swaroop: I am exciting to read your back. _(Editor’s note: I think he means ‘reply’)_ Tt is the first time that I talk to foreigner though the internet. I saw that you have your own mail ab.I think You’re a great man. Thanks for your back!(*^_^*) regards, gaoshuai

The interesting part is that this student somewhere in China was being benefited by this book and he “talked to a foreigner through the internet for the first time” and that person was me. It was truly humbling.

People are even putting ads for it, and I had no clue about it until I chanced upon it myself:

If that wasn’t enough, I found out that there are 8-9 university courses officially using the book, including Harvard and other institutions. And apparently even NASA is using the book in their Jet Propulsion Laboratory.

Users have suggested that it should replace the official tutorial but I really wouldn’t go as far as that

Recently, I had sent a sneak peek for the book’s group of readers and within a day, I had the first 10$ donation by Horst JENS. I remembered seeing that name somewhere, so I searched my emails and found this:

On Mar 4, 2007:

“Hello Swaroop, i teach a class of (3) Children how to program in Python. Just want to thank you because without your ‘a byte of python’ (that i read one year ago) i would maybe never have begun to code in python and consequently would never leaved my old job to become a Python teacher.”

A person in Vienna, Austria changed his career from a sys-admin job which he didn’t like, to teaching children about programming, a job he loves. Wow! Again, this is so humbling. I could have never imagined that a small book can make such a difference.

The point is that I’m grateful for all these people writing to me and sharing their delight and stories. The book is still alive and kicking thanks to all these people.

Happy programming!

So what is supercrunching?

Now something is changing. Business and government professionals are relying more and more on databases to guide their decisions. The story of hedge funds is really the story of a new breed of number crunchers – call them Super Crunchers – who have analyzed large datasets to discover empirical correlations between seemingly unrelated things. Want to hedge a large purchase of euros? Turns out you should sell a carefully balanced portfolio of twenty-six other stocks and commodities that might include Wal-Mart stock.

What is Super Crunching? It is statistical analysis that impacts real-world decisions. Super Crunching predictions usually bring together some combination of size, speed and scale. The sizes of datasets are really big – both in the number of observations and in the number of variables. The speed of the analysis is increasing. We often witness the real-time crunching of numbers as the data come hot off the press. And the scale of the impact is sometimes truly huge. This isn’t a bunch of egghead academics cranking out provocative journal articles. Super Crunching is done by or for decision makers who are looking for a better way to do things.

This is best explained by the chess example:

We tend to think that the chess grandmaster Garry Kasparov lost to the Deep Blue computer because of IBM’s smarter software. That software is really a gigantic database that ranks the power of different positions. The speed of the computer is important, but in large part it was the computer’s ability to access a database of 700,000 grandmaster chess games that was decisive. Kasparov’s intuitions lost out to data-based decision making.

(emphasis mine)

The book starts off with the example of Orley Ashenfelter, a Princeton economics professor as well as founder and editor of the Journal of Wine Economics who wanted to apply supercrunching techniques to predict whether a wine from a particular year would be a good wine or not. He ended up with the following equation:

Wine quality = 12.145 + 0.00117 winter rainfall + 0.0614 average growing season temperature – 0.00386 harvest rainfall

You can imagine the commotion that followed. The wine experts brushed off this theory and that numbers can predict the wine quality better than they can. After all, “Just as it’s more accurate to see the movie, shouldn’t it be more accurate to actually taste the wine?”

And yet, the equation did indeed make better predictions, especially with the prediction that 1989 and 1990 wines would be bestsellers.

Orley was able to make this analysis because he had access to data about the weather and the wine quality. Ian explains that there are two ways to get the data – it already exists (like surveys and census or simply transaction logs of companies) or you create it using randomized trials.

The latter idea of creating data with the “flip of a coin” is such a simple yet powerful concept. Techies would be familiar with this already under a different name – “A/B testing”.

Let’s take the example of JoAnn sewing machines:

So when JoAnn.com was optimizing their website, they decided to take a gamble and include in their testing an unlikely promotion for sewing machines: “Buy two machines and save 10 percent.” They didn’t expect this test to pan out. After all, how many people need to buy two sewing machines? Much to their amazement, the promotion generated by far the highest returns. “People were pulling their friends together,” says Linsly Donnelly, JoAnn.com’s chief operating officer. The discount was turning their customers into sales agents. Overall, randomized testing increased its revenue per visitor by a whopping 209 percent.

The key is that:

Randomization also frees the researcher to take control of the questions being asked and to create the information that he/she wants. Data mining on the historic record is limited by what people have actually done.

To realize how valuable this methodology is, let’s take the case of Progresa:

But by far the most important recent randomized social experiment of development policy is the Progresa Program for Education Health and Nutrition.

(paraphrased) Zedillo, the Mexican President in 1995, decided that he wanted to have a major effect on Mexico’s poverty and together with the members of his administration, he came up with a very unique poverty alleviation program, which is Progresa.

Progresa is a conditional transfer of cash to poor people. “To get cash,” Gertler said, “you had to keep your kids in school. To get the cash you had to get prenatal care if you are pregnant. You had to go for nutrition monitoring. The idea was to break the intergenerational transfer of poverty because children who typically grow up in poverty tend to remain poor.”

…

Zedillo’s biggest problem was to try to structure Progresa so that it might outlive his presidency… Gertler said “If you have a five-year administration and it takes three years to get a program up and running, then it doesn’t have much time to have an impact before the new government comes and closes it”.

So starting in 1997, Mexico began a randomized experiment on more than 24,000 households in 506 villages.

…

The Progresa villages almost immediately showed substantial improvements in education and health. Progresa boys attended school 10 percent more than their non-Progresa counterparts. And school enrollment for Progresa girls was 20 percent higher than for the control group.

…

The improvements in health were even more dramatic. The program produced a 12 percent lower incidence of serious illness and a 12.7 percent reduction in hemoglobin measures of anemia. Children in the treated villages were nearly a centimeter taller than their non-Progresa peers. A centimeter of additional growth in such a short time is a big deal as a measure of increased health.

(emphasis mine)

Best of all, the evidence of Progresa being a good thing was so convincing that the new government kept it going but under a different name for political reasons. Zedillo’s idea worked. And beautifully.

Ian goes on to demonstrate similarly how Don Berwick’s campaign prevented an estimated 1,22,342 hospital deaths in eighteen months. The campaign was just a few simple suggestions that were determined based on statistics of how deaths occurred and these suggestions were implemented by the participating hospitals. The suggestions included regular washing of hands.

Ian quotes several real-world examples throughout the book and the number of times that number crunching and data crunching beat human expertise is staggering. But Ian says that this does not mean the end of need for human intervention. Supercrunching can validate ideas but the ideas and hypotheses themselves have to be formulated by us humans.

He goes on to explain the 2SD rule and the Bayes’ theorem in layman terms. Just understanding these two concepts would go a long way in helping anyone decipher statistics.

All in all, the book was a good inspiring read. I would highly recommend the book for anyone (even non-techies) interested in how computers and databases are changing how decisions are made. These decisions are not limited to websites. As we have seen above, it is changing everything from how government policy decisions are made to how movie scripts are being written.

The key takeaway for me is that data insights are hard and so is intuition. People who can straddle both will be important people in future. Learning to read the data will mean getting comfortable with statistics, models and even neural networks (as explained in the book).

If you’re not patient enough to read the book, you can watch the Google Tech Talk by Ian Ayres. You can also read more of Ian Ayres’ supercrunching stories on the Freakonomics blog.

We are drowning in information, while starving for wisdom. The world henceforth will be run by synthesizers, people able to put together the right information at the right time, think critically about it, and make important choices wisely.

– E. O. Wilson (entomologist and biologist)