Search EggHeadCafe's Job Board
EggHeadCafe Silverlight WPF ASP.NET VB.NET C# Excel SQL Server SharePoint
search
Windows 2003 Server GroupsView
Windows Server Active_Directory
Windows Server Dfs_Frs
Windows Server Dns
Windows Server General
Windows Server Migration
Windows Server Networking
Windows Server Sbs
Windows Server Scripting
Windows Server Security
Windows Server Setup
Windows Server Clustering
Windows Server Update_Services

Group SummariesView
.NET Framework
Access
BizTalk
Certifications
CRM
DDK
Exchange Server
FoxPro
French
French .NET
Games
German
German .NET
Graphic Design
IIS
Internet
ISA Server
Italian
Italian .NET
Maps
MCIS
Miscellaneous
Mobile Apps
Money
MSN
Networking
Office
Ops Mgr
Publisher
Security
SharePoint
Small Business
Spanish
Spanish .NET
SQL Server
Systems Management Server
Transaction Server
Virtual PC / Virtual Server
Visual Studio
Win32
Windows 2000
Windows 2003 Server
Windows 7
Windows Live
Windows Media
Windows Update
Windows Vista
Windows XP
 

View All Microsoft Windows Server General Posts  Ask A New Question 

Is it possible to use windows file system as a database system?

ZhiguoYoun posted on Wednesday, May 30, 2007 1:08 PM

Is it possible to use windows file system as a database system?

I am working on an image crawler now.
One the most challenging problems I am facing is to judge whether a
certain URL has already been crawled.
I cannot keep all the visited URL strings in memory, and currently I
am using MySQL to manage them.
But as the number of URL increases, access to the db become slow.
Very likely I am not getting the most out of MySQL.
Does anyone know how to tune it?

An alternative method I have thought of is to use the file system as a
database system.
After crawling URL _current_url, I create a file _current_url.txt and
write some information in it.
So when _new_url comes, I only need to check the existence of file
_new_url.txt to decide whether it is already visited.
Is this method sound enough?
How does the system decide whether a file exist?

Any help will be appreciated.
Thanks in advance.
reply


Previous Microsoft Windows Server General conversation.