Windows 7 Forums

Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: VirusTotal + HerdProtect - Check Files with Simultaneously

15 Feb 2015   #10
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Quote   Quote: Originally Posted by Tookeri View Post
Herdprotect seem to have different HTML codes for different status types. I noticed that for Adware detections the Explanation info is missing but a Description info is there instead. But not always: sometimes there's none of them but a "What does it do?" section instead. So very confusing!
Could you tell me which files you scanned for which shows up a 'Description info' instead of 'Explanation' or nothing at all.
If i have that sample, i can make a fix for that.
For the empty space, if 'Explanation:' does not exist in the html page, it will simply use 'None' as Discription.

Quote   Quote: Originally Posted by Tookeri View Post
I'll have to take a closer look at the Explanation info for different detections as well as zero ones and see if I maybe can switch to that.
But maybe the best approach is to only get the Status(Clean, Adware etc) and ignore the rest including Explanation and Description info. After all, if it's a non-zero detection the report will open in a browser where you'll get all additional information. Luckily the Virustotal info is processed before this error might occur for Herdprotect
Use my script (or parts of it), i did use Delayed Expansions (!) on every variable (because why not?) in the module ':HRDPT', so you may need to adjust it if not enabling EnableDelayedExpansion.

Do keep in mind i added a real tab instead of a vbs file generating one for the var 'TAB', so if the script errors because tab suddenly dissapeared to re-add it.

The script should be easily to read, every section is a portable module(function) which gets CALLed when needed and i nested whatever is a subjob (do keep in mind you need to use EnableDelayedExpansion or else changes in vars in nested will not show up inside the same nested).


My System SpecsSystem Spec
.
15 Feb 2015   #11
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Hello Tookeri,

I found a new bug (which i fixed ^^).
  • Bug:
    For the detection status, if the detected item is a 'Malware' (search EICAR for a sample), a html element is shown.
    If 'Clean' or 'Potential Unwanted', the Status shows up normal as expected...
  • Cause:
    When a Malware is found, there is no Span element used, so the text is not at the sixth, but the fifth '>' token.
Fix:
Code:
FOR /F "tokens=5,6 delims=^>" %%A in ('TYPE "!TEMP!\tempHerdProtect.html" ^| FIND "Status:"') do (
	SET "STSA=%%A"
	SET "STSB=%%B"
)

IF not "!STSA!"=="" (
	IF "!STSB!"=="!STSB:</span=!" (
		SET "STS=!STSA:</div=!"
	) else (
		SET "STS=!STSB:</span=!"
	)
) else (
	SET "STS=None"
)
Old code:
Code:
FOR /F "tokens=6 delims=^>" %%A in ('TYPE "!TEMP!\tempHerdProtect.html" ^| FIND "Status:"') do (
	SET "STS=%%A"
)

IF NOT "!STS!"=="" (
	SET "STS=!STS:</span=!"
)
My System SpecsSystem Spec
17 Feb 2015   #12
Tookeri

Windows 7 Pro 32
 
 

I didn't actually scan any files. I tried a few of the "Latest File Detections" on the KB page and then viewed the page source to confirm. Try these: Example1 Example2

Good catch on the Status=Malware missing the span tag! I thought I'd be safe from more bugs if I skipped the Explanation etc and only used the Status info, so I'm glad you found this bug. I've looked up some more KB samples and I can't see a true pattern for these different info sections, when they appear and not. So I think I'll take the safest approach possible and only use the Status info in my code + your latest fix.

Thanks again
My System SpecsSystem Spec
17 Feb 2015   #13
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Quote   Quote: Originally Posted by Tookeri View Post
I didn't actually scan any files. I tried a few of the "Latest File Detections" on the KB page and then viewed the page source to confirm. Try these: Example1 Example2
Thanks for the page, luckely it seems all detection description text follow the same html pattern so i can write a adaptable code for it:
Code:
<div class="keyvaluepair"><div class="key">Description text</div><div class="value" style="color: #777777;">Description info</div><br style="clear:both;"></div>
So i made a new code which fixes this issue for all the description texts i have found which is thanks to my skills also easily modifyable and will not execute last echo if none is found:
Code:
FOR %%A in (Explanation: Note:) do (
	FOR /F "tokens=5 delims=^>" %%B in ('TYPE "!TEMP!\tempHerdProtect.html" ^| FIND "%%A"') do (
		IF not "%%B"=="" (
			SET "STS_INFO=%%B"
			SET "STS_INFO=ECHO:        %%A    !STS_INFO:</div=!"
		)
	)
)

!STS_INFO!
As you see it also adapts the description header based on what it found in "%%A", of course i made it use Delayedexpansion so don't forget to enable it.
Quote   Quote: Originally Posted by Tookeri View Post
Good catch on the Status=Malware missing the span tag! I thought I'd be safe from more bugs if I skipped the Explanation etc and only used the Status info, so I'm glad you found this bug. I've looked up some more KB samples and I can't see a true pattern for these different info sections, when they appear and not. So I think I'll take the safest approach possible and only use the Status info in my code + your latest fix.

Thanks again
No problem, i like this script so i will 'try' perfect it.
Please don't skip the Explanation text, it is useful ^^, instead add all different description headers to the script and they will be found.
My System SpecsSystem Spec
.

18 Feb 2015   #14
Tookeri

Windows 7 Pro 32
 
 

Nice!

Well, I decided to simplify my version even more and only get the detection rate for HerdProtect, and skip all other info. That way it works like sigcheck does for Virustotal and more importantly less things might break if HerdProtect decides to change things. Your version will provide more information and look better but that's not my priority. Besides, all that extra info will still be available in the browser report so I don't see the point in risking future parsing errors that might break things only to get that extra info in the command window as well.

Thanks again for all input, much appreciated!

A tip for you that I didn't want to have in the public version:
If you change the -vr parameter for the call to sigcheck, to -vrs then sigcheck will automatically upload any unknown files to VirusTotal.
My System SpecsSystem Spec
19 Feb 2015   #15
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Quote   Quote: Originally Posted by Tookeri View Post
Nice!

Well, I decided to simplify my version even more and only get the detection rate for HerdProtect, and skip all other info. That way it works like sigcheck does for Virustotal and more importantly less things might break if HerdProtect decides to change things. Your version will provide more information and look better but that's not my priority. Besides, all that extra info will still be available in the browser report so I don't see the point in risking future parsing errors that might break things only to get that extra info in the command window as well.
Of course, it is your script, you may do as you wish with it. But if it gives out not enough info though it feels less perfect.
I thought of something though, if Herdprotect's layout does not change for a while
the script could collect all sha1 and filenames for later in the script for Virustotal and Hardprotect, thus asking the user at the end of the script if they want to open all page or not, for systems that are a bit slow on browsers.
The script would then be slice of the pages itself before.

I also made some new code based on what you told me:
Code:
FOR %%A in (Explanation: Note:) do (
	FOR /F "tokens=5 delims=^>" %%B in ('TYPE "!TEMP!\tempHerdProtect.html" ^| FIND "%%A"') do (
		IF not "%%B"=="!STS_INFO:</div=!" (
			SET "STS_INFO=%%B"
			SET "STS_INFO=ECHO:        %%A    !STS_INFO:</div=!"
		)
	)
)
Less breakages for now ^^, cause if empty or not contain '</div', it will not parse it and thus skip detection info, making it very less likely to create breakages.
Quote   Quote: Originally Posted by Tookeri View Post
Thanks again for all input, much appreciated!

A tip for you that I didn't want to have in the public version:
If you change the -vr parameter for the call to sigcheck, to -vrs then sigcheck will automatically upload any unknown files to VirusTotal.
Thanks for the info, i actually already knew cause i looked at the command (/?) page of 'sigcheck', i didn't really like it cause i like the website better and (dumb) people could spam too many files at once it get blocked from virustotal.

But it may come in handy incase i have alot of unknown executables and decide to help the world by sending them all at once, for example the lastest Uniblue(ADW) software which i found was not on Virustotal yet.
My System SpecsSystem Spec
19 Feb 2015   #16
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Hello Tookeri,
I have noticed (1 out of 15) the script could not reach Herdprotect and fails to download the HTML page, breaking the script.

So i advice adding a code to check if the page is downloaded or not before parsing it.
Code:
IF exist "!TEMP!\tempHerdProtect.html" (
	*CODE*
) else (
	ECHO(Was Not able to connect to:
	ECHO(http://www.herdprotect.com/!SHA1FILE!-!SHA1!.aspx
)
I have added this in Post 6 of my version of your script, along with many other optimalizations.

Also, i noticed this change in your script is not present, just the lastest decision (post #14) minus 3 Herdprotect data's.
Quote   Quote: Originally Posted by Tookeri View Post
Feb 18, 2015: New version of the script code and .bat file. Fixed a bug that caused the script to fail and stop further executing if Herdprotect detected a file as type "Malware". The Herdprotect info would in those cases be missing from the command window, but the Virustotal info for that file would still be displayed.
Just here to let you know ^^.
My System SpecsSystem Spec
20 Feb 2015   #17
Tookeri

Windows 7 Pro 32
 
 

Quote   Quote: Originally Posted by Midori View Post
Hello Tookeri,
I have noticed (1 out of 15) the script could not reach Herdprotect and fails to download the HTML page, breaking the script.

So i advice adding a code to check if the page is downloaded or not before parsing it.
I've never noticed that problem, but a code check for it is a good advice!

Quote   Quote: Originally Posted by Midori View Post
Also, i noticed this change in your script is not present, just the lastest decision (post #14) minus 3 Herdprotect data's.
Quote   Quote: Originally Posted by Tookeri View Post
Feb 18, 2015: New version of the script code and .bat file. Fixed a bug that caused the script to fail and stop further executing if Herdprotect detected a file as type "Malware". The Herdprotect info would in those cases be missing from the command window, but the Virustotal info for that file would still be displayed.
Just here to let you know ^^.
Either you've misunderstood or I don't get what you mean. I changed the script to only parse detections info and set the DT variable. No other variables. The bug occurred for STS which is not in the script anymore and therefor can't break it. A very simple solution I think
My System SpecsSystem Spec
20 Feb 2015   #18
Midori

Primary OS: Archlinux with Kde-Plasma5 x86-64. Secondary OS: Windows 8.1 x64. UEFI Setup.
 
 

Quote   Quote: Originally Posted by Tookeri View Post
Quote   Quote: Originally Posted by Midori View Post
Also, i noticed this change in your script is not present, just the lastest decision (post #14) minus 3 Herdprotect data's.
Quote   Quote: Originally Posted by Tookeri View Post
Feb 18, 2015: New version of the script code and .bat file. Fixed a bug that caused the script to fail and stop further executing if Herdprotect detected a file as type "Malware". The Herdprotect info would in those cases be missing from the command window, but the Virustotal info for that file would still be displayed.
Just here to let you know ^^.
Either you've misunderstood or I don't get what you mean. I changed the script to only parse detections info and set the DT variable. No other variables. The bug occurred for STS which is not in the script anymore and therefor can't break it. A very simple solution I think
Excuse me, i should have been more clear.

What i meant was the notice said, the status section was fixed but instead you decided (#14) to remove all information except detections.
Although i could be misunderstanding (#11), that the notice meant the bug fix is done by removing all the other info's.

Hope that clears it up.

Btw, my version of the script now hides (No echo) any sections which cannot be found or are invalid instead of defaulting to None.
Also some small code optimalizations and Ui corrections.
My System SpecsSystem Spec
22 Feb 2015   #19
Tookeri

Windows 7 Pro 32
 
 

Well ok it wasn't a traditional bug fix. The code for the bug was removed instead of changed/fixed. Either way the bug can't happen again
My System SpecsSystem Spec
Comment

 VirusTotal + HerdProtect - Check Files with Simultaneously




Tutorial Tools Search this Tutorial
Search this Tutorial:

Advanced Search




Similar help and support threads
Windows 7 Tutorial Category
Process Explorer + VirusTotal (to check all processes with 50+ AV's)
Process Explorer by Sysinternals(Microsoft) is a more advanced alternative to Windows Task Manager VirusTotal.com is a multi-engined scanner service from Google with more than 50 different anti-virus products including: AVG, Avast, Avira, BitDefender, ESET, F-Secure, GData, Kaspersky,...
Tutorials
VirusTotal getting annoying cause of FPs
I use sigcheck from Sysinternals to once a month or so check all executable images in system32\drivers and system32 on VirusTotal. Usually there's only a few false positives, mostly from AegisLab and sometimes from ByteHero. I don't know these two engines but from what I've seen so far, I'm not...
System Security
herdProtect: Malware Detection
herdProtect by Reason Malware Detection: herdProtect, a multi-engine cloud based scanner ! Warning ! Effective malware remediation often requires specialized knowledge and tools. You can use this tutorial on your own, but it is best to have the guidance of a SevenForums (SF) member...
Tutorials
VirusTotal Uploader
VirusTotal Uploader VirusTotal Uploader (VTup) adds an Explorer context menu that allows you to right click on a file detected as suspicious by any malware scanner or Anti-Virus (AV) application and send it to VirusTotal (VT) for further analysis. . 1. Read the VTup online documentation. ...
Tutorials
HerdProtect getting stopped by AVG
Hi, I just downloaded and tried to run Herd Protect. Then AVG popped up with a security threat. The first time i clicked the "allow threat" The second and third time I clicked "remove threat." Then I turned off AVG in the task manager. But Herd Protect stayd stuck and the AVG window kept returning....
System Security
Open more than 15 files simultaneously from explorer
I often have a need to open many files simultaneously from explorer (e.g. file1.txt, file2.txt, file3.txt, etc.). In Windows XP, I can select, say 30 files, and open them all by selecting them in an explorer window and either hitting "enter" or selecting "open" from the right mouse context dialog...
General Discussion


Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd

All times are GMT -5. The time now is 19:40.

Twitter Facebook Google+



Windows 7 Forums

Seven Forums Android App Seven Forums IOS App