Check URL using DOS script

born2achieve · 21 Nov 2014

Hi,

I am using windows7 64 bit. I have a text file "sample.txt" and text file has 200000 image url's as like below

http://www.imagesup.net/img/icon_index1.png
http://www.imagesup.net/img/icon_index2.png
http://www.imagesup.net/img/icon_index3.png
http://www.imagesup.net/img/icon_index3.png
http://www.imagesup.net/img/icon_index5.png
http://www.imagesup.net/img/icon_index6.png
http://www.imagesup.net/img/icon_index7.png
http://www.imagesup.net/img/icon_index8.png
........

I wanted to check the image existson the directory, All my images are hosted remotely.Is it possible to achive using the batch script? if i executhe batch, it should output the url which doesn't have the image on output.txt file. Is it possible to achieve using DOS script. any sampel code please

Thanks

Kaktussoft · 21 Nov 2014

Can you post the txt file somewhere (preferable zipped) and post a link to it?

you just want to check if http://www.imagesup.net/img/icon_index6.png exists? (and that very a lot of files). So the files have to be checked on the webserver. is that a linux or windows machine?

born2achieve · 21 Nov 2014

Hi,

thanks for your reply and it's windows server. but i don't have access to the server. Is it possible to check from m y local machine? could you please assist me

Kaktussoft · 22 Nov 2014

Is it just a large number of url's you want to check?
each line in txt file has 1 url?

Post the txt file zipped please

Kaktussoft · 22 Nov 2014

Step 1 :create an empty file named results.log
Step 2:
WGET http://www.somehost.com/folder/subfolder/filename.png > null:
IF %ERRORLEVEL% echo http://www.somehost.com/folder/subfolder/filename.png >> results.log

of course you have to install wget

born2achieve · 22 Nov 2014

Hey Dude,

thanks for the reply and Finally i was able to install the GetGnuwin32 and i did all the installation specified in the document. Now i could see the Wget exe and i tried to

D:\GnuWin32\GetGnuWin32\bin> wget.exe http://pagead2.googlesyndication.com...32194530698365

I could see the result on the command prompt and i can see the image downloaded on the root folder. Now Could you please help me on reading the URL.txt has 200000 urls and need to output which URL has doesn't have image on output.txt.

[Note: I don't want to download the image to my folder. I just need
get the URL which doesn't have image]

could you please help on making this process as batch script

Any help please.

Kaktussoft · 23 Nov 2014

Can you post the txt file. Best to zip it first.
I want to see the format and test. Or you don't want that due to privacy???
I that case post only a small part

Tookeri · 23 Nov 2014

Code:

@ECHO OFF

REM Only URLs that failed will be printed

REM Create Vbscript to check URL:
 >"TestURL.vbs" echo URL = wscript.arguments(0)
>>"TestURL.vbs" echo Set objXMLHTTP = CreateObject("MSXML2.XMLHTTP")
>>"TestURL.vbs" echo objXMLHTTP.open "GET", URL, false
>>"TestURL.vbs" echo objXMLHTTP.send()
>>"TestURL.vbs" echo While Not objXMLHTTP.ReadyState = 4
>>"TestURL.vbs" echo     Sleep 10
>>"TestURL.vbs" echo Wend
>>"TestURL.vbs" echo If Not objXMLHTTP.Status = 200 Then
>>"TestURL.vbs" echo     WScript.Echo(URL)
>>"TestURL.vbs" echo End if
>>"TestURL.vbs" echo Set objXMLHTTP = Nothing

REM Read each line in txt file and call the Vbscript with the line as parameter:
for /F "delims=" %%a in (sample.txt) do (
    cscript /nologo TestURL.vbs "%%a"
)

Output when I run the batch file:

http://www.imagesup.net/img/icon_index1.png
http://www.imagesup.net/img/icon_index2.png
http://www.imagesup.net/img/icon_index3.png
http://www.imagesup.net/img/icon_index3.png
http://www.imagesup.net/img/icon_index5.png
http://www.imagesup.net/img/icon_index6.png
http://www.imagesup.net/img/icon_index7.png
http://www.imagesup.net/img/icon_index8.png

Only the failed URLs were listed. Not the 4 URLs I added.

For testing purposes the batch file won't write to output.txt but when you've tested it and want to create the output.txt file you run the batch file like this that will print to the file instead of the command window:
nameOfBatchFile > output.txt

born2achieve · 23 Nov 2014

Hey Dude,

Finally i was able to install the GetGnuwin32 and i did all the installation specified in the document. Now i could see the Wget exe and i tried to

D:\GnuWin32\GetGnuWin32\bin> wget.exe http://pagead2.googlesyndication.com/si ... 4530698365

I could see the result on the command prompt and i can see the image downloaded on the root folder. Now Could you please help me on reading the URL.txt has 200000 urls and need to output which URL has doesn't have image on output.txt.
then i tried to achieve the ecat output by below code,

Code:

@echo off
(for /f "usebackq delims=" %%a in ("url-list.txt") do (
    "D:\GnuWin32\GetGnuWin32\bin\wget.exe" --spider "%%a" || echo missing %%a
))>url.log
pause

i am good now. but seems to be this is not fastest way i am looking for as it takes plenty of time to output the result. please guide me with the fastest way to achieve this

born2achieve · 24 Nov 2014

Hey Guyz,

thakns for all your timely help and could you please help on achieving this faster. Any suggestions please