ASP.NET - getting info from website

Asked By asif hameed on 19-May-12 01:53 PM
Earn up to 30 extra points for answering this tough question.

 have an XML with following structure:

<table name="tblsiccodes"> 
    <row> 
        <sSICCode>0888</sSICCode> 
    </row> 
    <row> 
        <sSICCode>0900</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1000</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1040</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1044</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1090</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1220</sSICCode> 
    </row> 
    <row> 
        <sSICCode>1221</sSICCode> 
    </row> 
</table>

and I need to pick the description for each of SIC code from this site

http://www.epa.gov/enviro/html/sic_lkup.html

or this site

http://www.sec.gov/info/edgar/siccodes.htm

What will be easiest way using vb.net to get description for each SIC from the website? I can not modify this XML file. Do I need to create a collection with SIC code and description manually copied to it and then get description of SIC code in XMl by matching SIC code with collection ? Final collection will be used to populate a repeater with each item/row having SIC code and Industry Name.

Do I need to do screen scrapping ? I don't think that these websites have some API/service using which I can get industry names

I can not use HTMl agility pack or any other third party library except Jquery.

Please suggest.

Jitendra Faye replied to asif hameed on 21-May-12 12:22 AM
I have gone through links which you have given in question,  Here I have found that these links are showing information about SIC code.

But first tell me are you using any web service to get this data, if yea then you can formulate object to maintain details in your desired format.

If not then it is difficult to create object as you want.

Because if you have web service for this then you can write common code to formulate object otherwise every time you have to copy the result from that si9de and
write different code.

Somesh Yadav replied to asif hameed on 21-May-12 12:33 AM
I think the easiest way to do that is to use Yahoo Query Languag (YQL) This javascript lib allows you to retrieve pages from other websites in different formats. One of the being JSON. Once you've done that you can manipulate and display the data.

http://developer.yahoo.com/yql/

Hope this helps
http://www.webdeveloper.com/forum/newreply.php?s=ac1c640b60309feeeafdac809ad70408&do=newreply&p=1118326