Windows 7 Forums

Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: Check URL using DOS script

24 Nov 2014   #11
Tookeri

Windows 7 Pro 32
 
 

Sorry, that's the best I can do. The problem is that your PC will have to access every URL and wait for the web site to finish loading it until you know if the picture exist or not.

Normally these kind of things are done with web crawler tools from a server but that's another issue and one I can't help you with.


My System SpecsSystem Spec
.
24 Nov 2014   #12
born2achieve

windows7 64 bit
 
 

Thanks Tookeri,

Can you please suggest the best web crawler tool? how can i achieve this functionality in a better way and faster way
My System SpecsSystem Spec
24 Nov 2014   #13
Kaktussoft

Microsoft Community Contributor Award Recipient

Microsoft Windows 7 Home Premium 64-bits 7601 Multiprocessor Free Service Pack 1
 
 

Quote   Quote: Originally Posted by born2achieve View Post
Hey Dude,

Finally i was able to install the GetGnuwin32 and i did all the installation specified in the document. Now i could see the Wget exe and i tried to

D:\GnuWin32\GetGnuWin32\bin> wget.exe Error 404 (Not Found)!!1 ... 4530698365

I could see the result on the command prompt and i can see the image downloaded on the root folder. Now Could you please help me on reading the URL.txt has 200000 urls and need to output which URL has doesn't have image on output.txt.
then i tried to achieve the ecat output by below code,

Code:
@echo off
(for /f "usebackq delims=" %%a in ("url-list.txt") do (
    "D:\GnuWin32\GetGnuWin32\bin\wget.exe" --spider "%%a" || echo missing %%a
))>url.log
pause
i am good now. but seems to be this is not fastest way i am looking for as it takes plenty of time to output the result. please guide me with the fastest way to achieve this
use wget -S --spider

post results
My System SpecsSystem Spec
.

24 Nov 2014   #14
born2achieve

windows7 64 bit
 
 

Hi Kaktusoft,

Yes, i am using wget -S --spider for this purpose. But it take huge time(hours and hours) and still it didn't finish yet.

So what is the best way to achieve this . please guide me
My System SpecsSystem Spec
24 Nov 2014   #15
Tookeri

Windows 7 Pro 32
 
 

Here's a very simple solution that will do it 10 times faster:

Split the file with URLs to 10 separate files and create 10 versions of the batch file that reads each URL file, and create 10 different output files. Then run all 10 simultaneously. When all have finished you add all output files into one single file.
My System SpecsSystem Spec
26 Nov 2014   #16
born2achieve

windows7 64 bit
 
 

Hi Tookri,

good solution. I tried splitting as 10000 of 20 times and i could finish with in 3-4 hours. I like this idea.

thanks for the great tip. Thanks a lot to everyone who participated and helped me.
My System SpecsSystem Spec
26 Nov 2014   #17
Tookeri

Windows 7 Pro 32
 
 

Happy to hear it worked out for you
My System SpecsSystem Spec
Reply

 Check URL using DOS script




Thread Tools Search this Thread
Search this Thread:

Advanced Search




Similar help and support threads
Thread Forum
Dos script help
Hi, Below is my folder path, D:/Sample I have .sql files in that folder and i need to get the file names based on modified/created date i pass. for ex : i need to get the file names created/modified on or after 10/01/2014(mm/dd/yyyy). Can any please give me some sample script to make this...
General Discussion
Check Internet connectivity automatically via script and reconnect
Hi all I have a few computers that connect via rasdial to a bridged modem. They are dropping off randomly and frequently enough that its a pain to have to reset them manually everytime. Does anyone know how I could write a script or something that would check for internet connectivity and if...
Network & Sharing
Script to Check Processes then Launch
I would like a script that does the following: If ProcessA.exe NOT Running? Open ProgramA (Access link file) End If If ProcessB.exe NOT Running? Open ProgramB (Access link file) End If
General Discussion
In need of a script...
I am looking for a script which can delete cookies, Windows + browser cache automatically on a regular (scheduled) basis. Can this be done through Windows itself or would i have to implement a custom script or something? I am looking to replace CCleaner and similar software package(s). Just now...
Performance & Maintenance


Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd

All times are GMT -5. The time now is 14:14.

Twitter Facebook Google+



Windows 7 Forums

Seven Forums Android App Seven Forums IOS App