Swaroop C H

blog books about contact subscribe

Yahoo Search Python API Howto

01 Mar 2005

I guess most of you would've heard the announcement by now that a Yahoo! Search Developer Network a.k.a. YSDN has been started. The idea is to build a community around Yahoo! Search by opening up search webservices (with documentation) along with mailing lists, a blog and even a wiki. I haven't had much time to explore it, but here's what I've explored so far. This is more of a 'how i did it' rather than 'howto' but oh well, let's get on with it.

First, I got an application ID. You need a Yahoo! ID to get a appID. Since I had already logged into my mail, I didn't need to sign in. I gave a very unimaginative name 'qpicsearch' - I was thinking of creating an image search GUI using PyQt. I clicked on 'Submit Registration' and done.

I downloaded the SDK and read the FAQ. Python is my language of choice and as you can see from the title of this post, I am concentrating on that. Then, I installed the Python API:

python setup.py build
sudo python setup.py install

It first gave me an error that /usr/lib/python2.3/config/Makefile was not found, so I installed the python-dev debian package, the installation then proceeded smoothly.

The pYsearch-1.0/docs/ directory has some nifty pydoc-generated documentation. It helped me get started with my first program:

#!/usr/bin/env python

import sys
from yahoo.search.webservices import ImageSearch

app_id = 'qpicsearch' # your appid here!
searcher = ImageSearch(app_id)
searcher.query = sys.argv[1]
searcher.results = 1

results = searcher.parse_results()
for result in results:

for key, value in result.items():
    print key, '->', value

and I ran it like this:

$ python qpicsearch1.py bangalore
Thumbnail -> {'Url': u'http://re2.mm-c.yimg.com/image/548038611', 'Width': 120, 'Height': 81}
Publisher ->
RefererUrl -> http://www.zonevoyage.com/photos/inde/imagepages/image1.htm
Copyright ->
Title -> IND Bangalore VBangal1
Url -> http://www.zonevoyage.com/photos/inde/images/IND-Bangalore-VBangal1.jpg
ClickUrl -> http://www.zonevoyage.com/photos/inde/images/IND-Bangalore-VBangal1.jpg
Summary ->
Width -> 450
FileSize -> 43602
Height -> 304
FileFormat -> jpeg
Restrictions ->

That's an amazingly small program and does so much cool stuff. Leif has designed a very intuitive API. Hats off to him!

Next, I wanted to create a GUI using PyQt. So, I opened the Qt Designer and drag-and-dropped a line input, button and iconview widgets.

Qt Designer

I used urllib2 to download the actual images and then used QIconViewItem to display each of the images. I even put in a nifty progress bar that indicates how many images have been downloaded. So, here's the result!

QPicView GUI to search for images using Yahoo! Search

Pretty neat, eh? There's actually much much more - local search, news search, video search and of course, the web search. I wonder what kind of applications will start using the Y! Search API. There's a whole range of possibilities. As the Chinese proverb goes, 'May you live in interesting times!'

Note:

Comments

srid says:

Firefox toolbar, and then the SDK with official support for Python! Yahoo! beats google in this regard. Really great! I would like to see Jabber (with seemless integration with yahoo messenger) next :P

Anand, there is a python binding for the google api - http://pygoogle.sourceforge.net/

sundar says:

great work. and am happy that yahoo is doing this.

Avinash says:

Cool, really cool! Looks like I need to do a lot more work to get python-dev and the other stuff working on fBSD :(

Swaroop C H says:

Thanks guys. The Yahoos that have made all this possible deserve all the wows :)

Swaroop C H says:

geekdreams: Corrected that. Thanks :)

Pramod says:

wow!!

geekdreams says:

"I gave a very unimaginative name (which I'm not telling)..."
But you left it in the source. Oops! :)

Kiran says:

Really Cool!!

Anand says:

Google also offers a similar API as a free download. Using it you can query Google like you normally do. However, it does not have a Python interface which I think is a sore point since Google uses so much Python internally.

You can download the API SDK at http://www.google.com/apis/ .

Swaroop C H says:

Sriram: Jeffrey Friedl said the same thing to me, so I've put it back. It actually makes sense to me now that it is a *appid*.

Sriram says:

Hmm..the whole point of the app id is that you distribute it with your apps (unlike Google's).I suggest that you leave it in - as otherwise people will assume it's the same as Google's

Swaroop C H says:

Anand: Yes, Google has had it since nearly 2 years. However, the Y! API is more comprehensive - local search, news search, image search, etc. and also AFAIK allows more searches per day than google.

Pradeep Kishore G says:

Hey! dude,
the story is on slashdot. I've submitted this post in the replies..
Oops! watch out for your server. ;)

Migs says:

Nice work! And Sushmita Sen... won Miss Universe here in Manila back in 1994 (same year we got hooked up to the Internet...) :)

Leif Hedstrom says:

Sweet, very cool. I've never played with Qt, gotta check this out to see if it is as good as they say. :-) I'm working on a similar, but different, tool using GTK/GDK, it's a plugin for the Gimp to do Image search. It should be ready in another couple of weekends. It's not using Python though, but C. Not that Python doesn't rock (because it really does!), but there were fewer dependencies writing it in C.

-- Leif

Swaroop C H says:

Leif:

> similar, but different

Now, there's an oxymoron I haven't heard in a while ;)

> but there were fewer dependencies writing it in C.

I don't think there is a C API! Unless you're working on one.

ac says:

Note that it is not necessary for you to GPL your code with PyQt as long as you use a GPL-compatible license on your code. BSD (without advertisement) clause or LGPL are compatible licenses for example.

Anyway, nice work Swaroop!

Leif Hedstrom says:

Swaroop:
Yeah, no APIs for C as far as I'm aware. I wrote a custom HTTP "enginer" and XML parser using the APIs provided by GTK/GDK. I use the asynchronous I/O "event" engine for all HTTP requests, which is pretty neat, allows for easy concurrency (this is also on my wish list for the Python APIs, to support at least Medusa/async-core and possibly Twisted). GTK also provides a very simple event driven XML parser framework, and I built a "state" machine around it.

It's actually a lot more complicated than I thought it would be, but I've got it to do the searches, parse the XML, retrieve the Thumbnail images, and show those. I'm sure I would have been done by now had I used Python and the PyGtk bindings! :-)

Swaroop C H says:

Hey Julian, yes, I _am_ using Qt Designer (see the part which says "... _So, I opened the Qt Designer and drag-and-dropped a line input_ ..."). That's the tricky yet cool part to using PyQt - write Python code inside C++ declarations! :)

Swaroop C H says:

Hey Julian, that's a pretty neat way of using PyQt; although, one concern is that if you make changes to your GUI, how is it affected?

Julian says:

Swaroop, did you use the internal QtDesigner C++ editor to write your code?
I asked someone elses approach to coding in QtDesigner/PyQt which I think you might find interesting:
- Set up a connection in QDesigner to the clicked signal, and then create a python function with the same name in the implementation.
- Use pyuic to generate an implementation file (which inherits from the .py generated from the .ui): pyuic -o classimpl.py -subimpl
- Then fill in the code for the slot methods in there.

That way you don't lose code if updating the .ui file (since you're not coding in QtDesigner).

Swaroop C H says:

Julian, that is interesting. I'll try it for my next PyQt hack :)

Julian says:

I wrote a post about how you can use your own favourite Python editor (in my case IDLE) with PyQt instead of the Qt Designer C++ editor once you have designed the GUI: http://julianyap.blogspot.com/2005/03/technical-pyqt-creating-gui-with-qt.html

Julian says:

Yes, I thought of that too initially but then I looked at how simple it is to add a new function.

If you make changes to your GUI, all you will need to do is re-run the step which generates the Python code from the .ui file:
pyuic -o formmain.py formmain.ui

If there are any new connections, you just create the function in the subclass implementation file.

Makes a lot of sense to me as you're spending as little time in QtDesigner as possible and can test /run/debug your code in a proper Python IDE.

For your application, try the following:
1. Copy formmain.ui to a new directory.
2. Remove the 'includes' block which points to formmain.ui.h.
3. Run 'pyuic -o formmain.py formmain.ui'.
4. Run 'pyuic -x -o qpicsearch.py -subimpl main formmain.ui'.
5. Open up qpicsearch.py in your favourite Python IDE.
6. Add in the code for the doSearch function. You'll see how clean and simple it is to add more functions here. If you need to change the GUI, just re-run step 3 and add the function in qpicsearch.py.

I think you'll agree that the least amount of time spent in the Qt Designer environment, the better.

Feedback

There's no comment box, but please do email me or tweet me your thoughts and criticisms, and I will publish the relevant ones here.