Update on 30 Jul, 2011: I have switched to another paid option now – Rackspace Email.

Update on 24 Apr, 2011: I was using Zoho Mail exclusively for a long time, but I got tired of my email landing in spam folders of Yahoo! Mail and Gmail users. Going back to Yahoo! Mail was not an option (IMAP support is only in an expensive paid option and I don’t like the Yahoo! Mail UI any more), so the Hobson’s choice was to get back to Gmail. Sigh.

A series of incidents and thoughts led me to try an experiment – to be “100% Google Free”. This turned out to be a lot harder than I thought, and ended up admiring Google a lot, and at the same time, worried and curious about what they do with all that data they have.

First things first, since I no longer use Google’s Feedburner, please kindly update your RSS readers to use http://www.swaroopch.com/feed/ instead of the earlier Feedburner link. For those 140+ people who are subscribed via email, I have migrated to MailChimp (emails were also being sent by Feedburner earlier), so emails will continue to be delivered to you from this post onwards. You can subscribe or unsubscribe for email delivery on this page.

Back to the main topic… there were a few reasons that led me to this experiment:

• Security : I have been bitten in the past due to flaws in certain intentional security designs by Google products, and this was a put-off for me.
• Privacy : Google seems to be blind to the importance of privacy – the recent Google Buzz fiasco was just a confirmation of my fears. Sure, they did do a quick turnaround immediately because it was a publicly released product. What about the data they have and are using privately? For example:
  • Did you realize that if you “log out” of Google, it doesn’t matter – they still associate your activities with your “log in” ID!
  • In order to make “improvements” to their ads matching, they will use the searches you just made “for a few hours” to show ads if Google cannot find good ads for a random web page that you are visiting which is displaying Google ads.
  • And what about the long-term damage they might be inflicting? Especially the kids? Are they getting a bit… arrogant? Is it just “good business”?
  • The worst part is that user’s don’t care about their own privacy. They just won’t do anything about it.
  • Before you stereotype me as a luddite, let me be clear – I do not think that online == unsafe (I trust LastPass with notes on my bank details), I am not against data collection (that’s the only way online businesses can improve their products), I am not against data mining (shopping recommendations FTW), and I am not against ad-supported business models (otherwise, freemium products will go away). It’s just that I am worried about Google’s intentions, so I’m moving on. Simple.
• Profiling : I’m okay with them mining the data that I choose to make public, not my private data – that’s why I’m weary of Gmail. That’s why I’m weary of very-cool but data-mining note-taking tools such as Springpad.
• Lock-in : In spite of good steps they have taken, I have still not been able to take a full backup of my email from Gmail, why is that, Google? I have tried Thunderbird, Evolution and Outlook to do a full IMAP sync, but the sync always fails. Their IMAP server just won’t let me download more than a hundred emails, and will completely error out after a few tries. If you know how to export a Gmail mailbox containing around 15 folders and 14,210 emails, please let me know. Even Backupify.com hasn’t been able to do it!
• Longevity : I think I’ll prefer a for-fee business for my critical data – because I know the company will be around and it is in their interest to continuously improve the actual product vs. the profiling algorithms to show better ads. For example, I have shifted to Zoho Mail because they show NO ADS even in the free version! Their business model is simple – the basic product is forever free to use, you can pay for more storage and users.
• Overaching : It’s handling of Arunachal Pradesh in Google Maps, it’s deal with the NSA, it’s handling of deletion of music blogs, etc. is just plain bad.
• Boring : If you ask me, there is nothing interesting coming out of Google these days. They’re just expanding ways (Chrome, Android) to make people search more and buying up companies that can be potentially threatening. Even Buzz seems like a love child of Twitter and Facebook. Sure it might be a game changer because of Gmail’s user base, but seriously, what’s interesting or new about it?
• Monoculture : A huge problem I have observed is that people are not willing to adopt any new tool unless it is by Google. And there is a current startup hype in India. I find this direct conflict amusing because if even an average techie in India (e.g. the common person in the service industry, for example) is not willing to give non-Google products a try, how exactly will people be open to use tools made by startups? Pop Quiz: “How many of you have started using a new tool or new service (esp. by a startup) in the past one year and have used it for more than 2 months?” I wanted to switch to give other companies and startups a try.

Phew. I think those were enough reasons to move away from Google, at least for a while.

And, boy, it has been tough. Let’s face it, it’s hard for companies to beat Google when Google makes slick products and gives it away for free.

Here is what my transition looks like:

1. Search – The funny thing is I used Google Search only in 2004-2005, started using Yahoo! Search since 2006, and have moved to Bing exclusively since the past 6 months. (free)
2. Analytics – Moved to Mint ($30) + Piwik (open source)
3. Reader – Moved to Tiny Tiny RSS (open source)
4. Feedburner – Moved to the default WordPress feed link + MailChimp for emails (freemium)
5. Google Apps – Moved to Zoho for Business ($5 per month)
6. Docs – Moved to Zoho Docs which turned out to be way more powerful (free)
7. GTalk – Stopped using IM, it was a distraction anyway. (zero)
8. Contacts – Exported from Google, stored only on iPhone (free)
9. Calendar – Zoho Calendar (free)
10. Google Groups – subscribe to RSS feeds of the group (free)
11. Maps – Since the map application on iPhone uses Google Maps, no alternative
12. Google Alerts – no alternative
13. Google Adsense – This is still a todo item, haven’t looked into it yet. I have heard about Komli, Chitika, etc. but yet to investigate.
14. Phone – My next phone is probably going to be an Android phone, looks like there is no alternative (I’m tired of having to use Windows just for iTunes, only because I have an iPhone)

As I’m sure you have deciphered, this took some installation of server-side software and some money to make this transition. These were the best alternatives that I came across that suited me.

So far I’ve been very happy about this experiment, because I got to discover and try out new tools and realized that there is so much more cool functionality available out there that I would have never discovered otherwise!

And at the same time, I admire Google even more now (from a startupper’s perspective) because they discovered a business model because of which they are able to give away so much functionality for free, and hence brought more people online.

Update: Thanks to Helen (in the comments below), got to know that Leo Babauta (Zen Habits) wrote about the exact same topic just 2 days ago. Good to know that I’m not alone in my concern!

Ever wondered who wrote it? It was Eric Schmidt, along with Mike Lesk.

That’s right, the CEO of Google.

I can imagine the CTO of a company to have such a background, but I never would have expected the CEO.

But then again, it’s Google. They can do anything.

Not only did I get money, but later in the day, Google (Mail) gave me career advice as well:

Heh.

#!/usr/bin/env perl
use warnings;
use strict;

=head1 INTRODUCTION

Checks if there are new unread messages in your GMail Inbox.

=head1 USAGE

    $ perl check_gmail.pl
    1       Swaroop C H     Looks like check_gmail.pl works

=cut

############## Configuration ##############

# Change this to your correct username.
use constant GMAIL_USERNAME => "username";
# Change this to your correct password.
use constant GMAIL_PASSWORD => "password";

########## Don't change anything below this. ##########

use LWP::UserAgent;
use XML::Atom::Feed;

my $fetcher = LWP::UserAgent->new();
$fetcher->agent("check_gmail.pl/0.01");

my $request = HTTP::Request->new(
    'GET'   => "https://mail.google.com/gmail/feed/atom",
);
$request->authorization_basic(GMAIL_USERNAME, GMAIL_PASSWORD);

my $response = $fetcher->request($request);

if (! $response->is_success())
{
    die("Unsuccessful in trying to talk to GMail");
}

my $content = $response->content;
my $feed = XML::Atom::Feed->new(\$content);
my @new_messages = $feed->entries();

my $i = 1;
foreach my $message(@new_messages)
{
    print join("\t", $i, $message->author->name,
                    $message->title), "\n";
    $i++;
}

# The End

Update : Baishampayan Ghose quickly jotted down a Python version of this script.

I just finished reading ‘The Search’ by John Battelle. What an amazing story to read. Learning about Bill Gross and his IdeaLab alone was worth reading the book, and he still keeps ideating, like Snap.com. Heck, even Picasa came from IdeaLab.

There are many tidbits from the book that were interesting, such as about Louis Monier:

It was Louis Monier who took AltaVista from concept to executable code … “I’ve always been interested in big, nasty problems,” Monier told me. Search provided one of the nastiest. Not only do the numbers scale to the near infinite, there was a very real need for good search in 1994. “Search engines at the time were just terrible,” Monier recalls. “Yahoo was a great catalog, but it had no search. So I set about to work on the crawl.”

About Stanford:

Stanford’s 6,200-acre patch of rolling California woodlands is the most productive incubator of technology companies the world has ever seen. Nestled between the silicon factories of Intel and Apple on one end and Sand Hill Road’s venture capitalists on the other, Stanford is a place where students have already dreamed of starting their own companies or going to work for a pre-IPO start-up. And Stanford’s computer science department, where Yang and Filo hung their hats, is perhaps the most prodigious start-up incubator of them all.

About Yahoo:

Another reason Yahoo succeeded was its sense of fun – a characteristic that would come to define not only Yahoo, but nearly every Internet company seeking the fickle approval of the Web public. Yahoo pioneered some of the Web’s earliest social mores – including, for example, links to competitors’ sites in case a searcher could not find what he or she was looking for, and listing “what’s hot” prominently on its home page, thereby driving extraordinary amounts of traffic to otherwise obscure sites.

Thanks to practices like these, the company captured the public’s imagination early and often, garnering a slew of adoring press notices familiar to anyone watching Google’s rise to prominence over the past few years.

About how a mathematical curiosity led to PageRank:

Page didn’t land on the idea of Web-based search at the outset; far from it. Despite the fact that Stanford alumni were getting rich starting Internet companies, Page found the Web interesting primarily for its mathematical characteristics. Each computer was a node, and each link on a Web page was a connection between nodes – a classic graph structure. “Computer scientists love graphs,” Page tells me, referring to the mathematical definition of the term. The World Wide Web, Page theorized, may have been the largest graph ever created, and it was growing at a breakneck pace. One could reasonably argue that many useful insights lurked in its vertices, awaiting discovery by inquiring graduate students. Winograd agreed, and Page set about pondering the link structure of the Web.

About Google’s geeky sense of humor and control:

On April 29, 2004, Google filed what certainly had to be the most unusual S1 – the formal public offering document – in recent memory. At filing, Google declared it would sell $2,718,281,828 worth of its shares – a seemingly random number, which was, in fact, the mathematical equivalent of e, a concept not unlike pi that has unique characteristics and is well known to serious math geeks. By manipulating the actual offering to provide this knowing wink to nerd humor, Google was in effect declaring: the geeks are in control.

Perhaps, the most interesting part of the book for me was the last chapter – ‘Perfect Search’. Battelle profiles what could be the future of Search.

When it comes to search, as with the Internet itself, the most interesting stuff is yet to come. As every engineer in the search field loves to tell you, search is at best 5 percent solved – we’re not even into the double digits of its potential. And search itself is changing at such a rapid pace – in the past year important innovations have rolled out once a week, if not faster – that attempts to predict the near future are almost certainly doomed.

I’ve been working on the Yahoo! Buzz Index for the past 2 years, and many a time I’ve been asked (by friends and colleagues) why I haven’t changed teams yet. But I often ponder to myself – change to what? Being a rabidly information-hungry internet user (well, I’ve calmed down off late), I always found search engines remarkable and Buzz does a lot of analysis on search, it’s quite fascinating, and the sheer volume of data is equally interesting. I’ve had my share of ups and downs (and some very steep downs), but it has been interesting.

We do a lot more than what Google Trends does, however Buzz has a more practical business model in which the interesting insights are kept for the paid customers and the interesting stories are written for the public.

Isn’t it striking on how the ability to search for what you want (and relatedly, blogs have become an important factor in this) has made us more knowledgeable? All the articles on productivity that I came across have really helped me improve in my professional and personal life. My problem has always been converting thoughts into actions, but GTD and the ready-fire-aim approach are helping me improve, but that’s a whole different story. Search has helped me find lyrics of so many songs. Search helped me research about the places I travelled to and want to travel to. Search has helped me discover some hands-on activities that could help me teach the Parikrma kids to explore and think about science, the earth and the human body. I could go on, but just count the number of times you use a search engine in a day, and you’ll know what I mean.

All this has got me more interested in the Search-related domain as a field to work in. And all the money that Search makes is interesting as well Will I ever move on to working on something more in this direction, or will I find something more interesting tomorrow, or will I simply remain yet-another-software-coolie (which seems more likely)? I honestly don’t know, but don’t we all have a search(!) for “that one thing that drives us”?

I’m not saying that Search is the important problem out there in the world, far from it, there are more real world problems like our community, our economy, and quality of life. I believe that knowledge is an important way of empowering people, and Search is a step in helping people to keep finding and gaining knowledge.

I’ve been thinking about all of this, especially, because of Richard Hamming’s words:

Over on the other side of the dining hall was a chemistry table. I had worked with one of the fellows, Dave McCall. I went over and said, “Do you mind if I join you?” They can’t say no, so I started eating with them for a while. And I started asking, “What are the important problems of your field?” And after a week or so, “What important problems are you working on?” And after some more time I came in one day and said, “If what you are doing is not important, and if you don’t think it is going to lead to something important, why are you at Bell Labs working on it?” I wasn’t welcomed after that; I had to find somebody else to eat with! That was in the spring.

In the fall, Dave McCall stopped me in the hall and said, “Hamming, that remark of yours got underneath my skin. I thought about it all summer, i.e. what were the important problems in my field. I haven’t changed my research,” he says, “but I think it was well worthwhile.” And I said, “Thank you Dave,” and went on. I noticed a couple of months later he was made the head of the department. I noticed the other day he was a Member of the National Academy of Engineering. I noticed he has succeeded. I have never heard the names of any of the other fellows at that table mentioned in science and scientific circles. They were unable to ask themselves, “What are the important problems in my field?”

If you do not work on an important problem, it’s unlikely you’ll do important work. It’s perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them. Let me warn you, ‘important problem’ must be phrased carefully. The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. We didn’t work on (1) time travel, (2) teleportation, and (3) antigravity. They are not important problems because we do not have an attack. It’s not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don’t work on important problems, I mean it in that sense. The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn’t believe that they will lead to important problems.

I spoke earlier about planting acorns so that oaks will grow. You can’t always know exactly where to be, but you can keep active in places where something might happen. And even if you believe that great science is a matter of luck, you can stand on a mountain top where lightning strikes; you don’t have to hide in the valley where you’re safe. But the average scientist does routine safe work almost all the time and so he (or she) doesn’t produce much. It’s that simple. If you want to do great work, you clearly must work on important problems, and you should have an idea.

Back to the book … As a colleague said, this book should be made compulsory reading for anyone working in this field.

Also, special thanks to Rajiv Vyas for sending the book across the ocean to me just because he thought “I might want to read it”! … come to think about it, how did Rajiv and me become friends? Well, he searched for something and chanced upon my blog …

Google’s chief executive Eric Schmidt has predicted that India and not China will become the world’s biggest Internet market in “about five or ten years from now, based on current trends.”

And what’s more, Schmidt’s other futurist view is that Hindi, not Hispanic, could become one of the world’s three Internet languages, in conjunction with English and Chinese.

Schmidt’s remarkable predictions have left techies and the Internet market analysts scrambling to ask what Google was doing to prepare for a tomorrow when India is virtually the Internet and Internet is India?

That’s an interesting question. What does have Google in mind?

On a related note, the Yahoo! India Summer Special website is interesting as well.

Then again, I’m more interested in whether iTunes is coming to India.

The past few years many people have sponsored my work on Vim. Now version 7 is nearing completion, beta testing will start soon, very soon. A big thanks to all who supported me! I would not have been able to do this without your help.

But things are going to change. I have accepted a job offer and will go back to a full time job in a few months. To avoid speculation and rumours: I am going to work for Google in Zurich. Fortunately I can spend part of my time on Vim. But it will obviously be less than the past few years when I was working 150% of my time on Vim.

I will no longer need sponsorship to survive. Therefore, starting the end of March, all money given for Vim sponsorship and registration will go to the project in Kibaale, Uganda. And no, this is not an April fools joke.

The voting will continue as before. And seeing people give money will motivate me to keep working on Vim. The children need the help much more than me anyway. Thus please keep sending money!

P.S. Yes, I have donated to Vim before.

Update on 14th April : It has arrived.

I (Sean) have been hired by Google, moved to Seattle, and have been working on the Google Talk team for about a month and a half. The goal of Google Talk is to make real-time communication as open as possible, and in that regard, I’ve been working to offer all of Google Talk’s features into other clients. Currently, I’m working on making it as easy as possible for other clients to use Google Talk’s voice features. You can expect Gaim and other clients to be interoperable with Google Talk’s voice features in the near future.

On a related note, the gaim-vv project—which aimed to offer a framework for voice and video support in Gaim—is being merged back into Gaim proper for hopeful incorporation into Gaim 2.0.0. This will be used to support Google Talk’s voice as well as MSN and Yahoo! webcams.

Wow! I was praying for something like this to happen.

(Click on the image to view it in full size)

It’s nice to see the media showing that cool stuff does get done in Bangalore. As you can see, the Y! Bangalore R&D team is 400 strong now… that’s a population explosion considering we were only 80-90 when I started out. Aah, the good old days…

One thing to notice is that they have words from the CEO of Amazon Bangalore and from the CTO of Yahoo! Bangalore, but no one from Google Bangalore! Relatedly, I hear about people joining Yahoo! and about people joining Amazon – how come I never hear about people joining Google in Bangalore? Is something happening in Google at Bangalore at all?