Lounge - How Can I Extract Website "Store Locator" Data

Asked By Gary Anderson on 06-Oct-11 05:46 PM

Hi - I am looking to extract locator data from this website into a flat file (csv for example). The web page says there are 28,000 locations (records).

The on-screen locator does not allow just selecting a state (that would be fairly easy with 50 states) and limits the max results to 100. The biggest chunk of data I can get at one time is by zip code, but I would have to know every US zip code, which I don't and even doing it that way would probably result in thousands of duplicates as I would be over-lapping zip codes.

Is there any web scraper or similar free utility that would be able to extract this data... with minimal or no duplicates?

Thanks for any ideas!

Kirtan Patel replied to Gary Anderson on 16-Oct-11 10:43 AM
it depends on the site how site is designed if site is designed with flash like thing it will be impossible to scrap it . there are some tools available in the market about data scrapping but they are not working in all the site at all .

better you should code your self by learning about HttpWebRequest and WebResponse + RegularExpressions to do the task programatically..

thanks
help
Hi Every One. . ! i need to scrape website data(specific input fields, span, td, title). . please help me. thanks. . Follow this example- using System Find(s)) { Debug .WriteLine(i); } } } Try this and let em know. Try this example of data Scrapping: using System; using System.Text; using System.Text.RegularExpressions; using System.Net; using System login dialog page. StreamReader reader = new StreamReader (response.GetResponseStream()); string loginDlgPage = reader.ReadToEnd(); reader.Close(); / / extract the viewstate value from the login dialog page. / / We need to post this back, / / along btnOK = OK" , viewState, username, password); / / our second request is the POST of the username / password data. HttpWebRequest request2 = ( HttpWebRequest ) WebRequest .Create(loginDlgUri); request2.Method = "POST" ; request2.ContentType = "application / x-www-form-urlencoded" ; request2.CookieContainer = cookies; / / write our postback data into the request stream StreamWriter writer = new StreamWriter (request2.GetRequestStream()); writer.Write(postback); writer.Close
I am creating a screen scraping app that will extract data from a website. The screen scraping is pretty straightforward using .NET 2.0, but stripping out all extraneous characters is proving to be more difficult. I am basically trying to extract the team, quarter, score for the quarter, and score for the entire game from this Hello JP, I posted a regex a while back that did almost this. this will extract all the rows with the info you need. It will store the respective values in pack. It allows you to do XPath queries over HTML. A very powerful way to extract data from HTML files. http: / / www.codeplex.com / htmlagilitypack - - Jesse Houwing jesse.houwing at sogeti.nl Jesse, That's exactly what I am trying to do with the data within the HTML. The expressions and code you listed don't apply to the HTML
I was looking through this discussion group, found something close to Jan \ file name.csv)". I need to search within the group of csv files and extract data into an Excel file. What I would like to do is if Cell A1 in my active wb matches the data to the right of the 76 comma in the csv file extract the 'text' value (must be specified as text as this data is a 19 digit number and cannot have it tuncated), in cell A2. Then in cell B2, extract the data that is to the right of the 109 comma. Continue searching the current
Hi! Group Need to extract data from Exchange Server and loading it in a datawarehouse (Microsoft Analysis service). The datamart will text file. In that case I can use an ETL tool and quickly import the data to a relational database. I guess there are different approaches to this and would appreciate Another option is WebDAV (http: / / msdn2.microsoft.com / en-us / library / aa486282.aspx and my website http: / / www.infinitec.de) Kind regards, Henning Krause keywords: Loading, data, from, Exchange, server, to, Datawarehouse description: Hi! Group Need to extract data from Exchange Server and loading it in a datawarehouse (Microsoft Analysis service). The datamart will
Hi, I want to extract data from website. The website is password protected. I can log in into website using i macro add on of Firefox. Now i want to extract data from that web page. I had tried vb script and i macros both but
Hello, could anyone please help. I have a macro which will pull the URL address from the excel sheet and open the website then extract the data from it. (using for loop) I am running into a problem, where a pop up you for your reply, but I have hundreds URL that macro has go through and extract data out of each website. could you please clarify by going to the website first and leaving it up. thanks
Hope I am not irritating anyone, but I indeed want to know how to extract the information contained in the tagname "HTML" of an IE page to a cell. My 1 next itm end with keywords: New, Q, to, old, Post:, Stuck, at, Trying, to, Extract, Data, from, a, Website, using, JSP description: Hope I am not irritating anyone, but I indeed want to know how to extract the information contained in the tagname HTML of an IE page to a cell. My
I tried to web query but it only give me a whole table of data while I only need the latest data only. The data one I want to get from this page is the "Last Trade" http: / / quotes.ino SalesAid (1) FA47E8114034 (1) Dguillett1 (1) NYBOT (1) B12 (1) On another sheet, IF? the data is ALWAYS at cell b12, use = sheet1!b12 For a series use a looping macro an email to my address below I will send you a file that uses Yahoo data. - - Don Guillett Microsoft MVP Excel SalesAid Software dguillett1@austin.rr.com keywords: How, to, extract
May i know is it possible to use Visual Studio.net 2008 to generate a website , and then extract data from Microsoft project server 2007, For example , i want to display the task haven complete (<100%) in my website??Is it possible , or i need to use another tool to do that ?Should i You can either use the PSI API's for Project Server, or much simpler read data directly from Project Server's Reporting Database (schema in Project SDK). I am not sure May i know is it possible to use Visual Studio.net 2008 to generate a website , and then extract data from Microsoft project server 2007, For example , i