search
Twitter Rss Feeds
MicrosoftArticlesForumsGroups
C# .NET
VB.NET
Visual Studio .NET
ADO.NET
Xml/Xslt
VB 6.0
.NET CF
GDI+
LINQ
Deployment
Security
FoxPro
Silverlight / WPF
Entity Framework
RIA Services

Web ProgrammingArticlesForumsGroups
JavaScript
ASP
ASP.NET
Web Services

Non-MicrosoftArticlesForumsGroups
NHibernate
Perl
PHP
Ruby
Java
Linux / Unix
Apple
Open Source

DatabasesArticlesForumsGroups
SQL Server
Access
Oracle
MySQL
Other Databases

OfficeArticlesForumsGroups
Microsoft Excel
Microsoft Word
Microsoft Powerpoint
Publisher
Money

Operating SystemsArticlesForumsGroups
Windows 7
Windows Server
Windows Vista
Windows XP
Windows Update
MAC
Linux / UNIX

Server PlatformsArticlesForumsGroups
Share Point
BizTalk
Site Server
Exhange Server
IIS
Transaction Server

Graphic DesignArticlesForumsGroups
Macromedia Flash
Adobe PhotoShop
Microsoft Expression

OtherArticlesForumsGroups
Subversion / CVS
Ask Dr. Dotnetsky
Active Directory
Networking
Uninstall Virus
Job Openings
Reviews
Search Engines
Resumes

 

Grokking the Google API with VS.NET
(or - Is ScreenScraping Dead?)

Peter A. Bromberg, Ph.D.
Printer - Friendly Version

Peter Bromberg

Google has absolutely become the developer's favorite search engine. Its just that in a couple of years, they've put together a massive cadre of computers, a fast crawler/indexer, and a very intuitive search engine that offers newsgroup content going back some 20 years.

Now the Google people are gonna let us use their service (for non-commercial applications and limited to 1,000 searches per day) without having to screen-scrape their content. They've published a web service, WSDL - compliant API that includes all the stuff in their regular search engine (including Base64 encoded cached pages) as well as their spell checker and other features. You can get the developer "kit" and a free license key here.

The easiest way to get into the Google API is to either include the WSDL-generated proxy class from their Developer Kit into your project, or run WSDL.EXE on their WSDL file externally and include the generated class file in your project. In this manner you can avoid the potential problems involved in generating a proxy as their API does not directly expose the familiar "?WSDL" interface that VS.NET developers have become accustomed to.

Once that's done, it's easy to look at Class View or the Object Browser and see what's available and how to use it. Theirs is the typical cross platform webservice API and it is very object - oriented:

As can be seen from the Class View screen shot above, the API exposes a GoogleSearchService Class, a ResultElement, a DirectoryCategory and a GoogleSearchResult class. To use it, you simply create a GoogleSearchService instance and call the doGoogleSearch method for synchronous operation, passing in the license key, textual query, start page for results, maximum number of results (limited to 10 in the Beta), any filter, a string search restriction and some optional other junk which frankly I haven't even bothered to look at yet.

You get back a GoogleSearch result instance which you can then use to build a DataSet which can be DataBound to your favorite DataGrid or other ASP.NET control. The whole exercise, including a couple of helper methods I used to create the DataSet looks like the following:

string lic = "You key here";
GoogleSearchService s = new GoogleSearchService();

try
{
int start = ( Convert.ToInt32(TextBox2.Text)-1) *10;

GoogleSearchResult r= s.doGoogleSearch(lic, TextBox1.Text ,
start, 10, false, "", false, "", "", "");
// Extract the estimated number of results for the search and display it
int estResults = r.estimatedTotalResultsCount;
Label1.Text = Convert.ToString(estResults) + " Results found";
DataSet ds1 = new DataSet();
DataSet ds =FillGoogleDS(ds1,r);
DataGrid1.DataSource=ds;
DataGrid1.DataBind();
}
catch (Exception ex)
{
Label1.Text =ex.Message;
return;
}

return;

}

private DataSet CreateGoogleDS(DataSet ds )
{
ds.Tables.Add("GoogleSearch");
ds.Tables[0].Columns.Add("Title");
ds.Tables[0].Columns.Add("Snippet");
ds.Tables[0].Columns.Add("Summary");
ds.Tables[0].Columns.Add("URL");
return ds;
}

private DataSet FillGoogleDS(DataSet ds, GoogleSearchResult srchResult)
{
try
{
ds = CreateGoogleDS(ds);
int i =0;
DataRow dr;
string strURL=null;
for (i = 0;i<srchResult.resultElements.Length ;i++)
{
dr = ds.Tables["GoogleSearch"].NewRow();
dr["Title"] = srchResult.resultElements[i].title.ToString();
dr["Snippet"] = srchResult.resultElements[i].snippet.ToString();
dr["Summary"] =srchResult.resultElements[i].summary.ToString();
strURL=srchResult.resultElements[i].URL.ToString();
dr["URL"] = "<a href=" +strURL + ">"+strURL+"</a>";
ds.Tables["GoogleSearch"].Rows.Add(dr);
}

}
catch(Exception e)
{
Label1.Text=e.Message;
return null;
}
return ds;
}

If you wanted to, you could get pretty sophisticated with this. One idea is to create a usercontrol that can simply be plugged into a web page at the location of your choice. You can also use the tool to restrict the search to only a particular site (like their "put Google Search on your site" feature) so that it only searches on your site.

If you'd like to try my live sample of the above code, check this out. To get a full VS.NET Solution in C#, download the file here.

 


Peter Bromberg is a C# MVP, MCP, and .NET consultant who has worked in the banking and financial industry for 20 years. He has architected and developed web - based corporate distributed application solutions since 1995, and focuses exclusively on the .NET Platform. Pete's samples at GotDotNet.com have been downloaded over 41,000 times. You can read Peter's UnBlog Here.  --><--NOTE: Post QUESTIONS on FORUMS!

 
Do you have a question or comment about this article? Have a programming problem you need to solve? Post it at eggheadcafe.com forums and receive immediate email notification of responses.


Pete's Blog   |    Pete's Resume   |    Robbe's Blog   |    Robbe's Resume   |    Archive #2   |    Archive #3   |    Dotnetslackers   |    XmlPitStop   |    Advertise   |   Contact Us   |   Privacy   |   Copyright (c) 2000 - 2009 eggheadcafe.com  All rights reserved.