Solved Can I automate downloading numerous MP3 files?

[PSB]

Hi,
I'm not sure if this is the correct location to post this thread. If not, I apologize. I am trying to download hundreds of legal MP3 files. They are old radio shows that are in the public domain. One of the webs site is:

https://archive.org/details/OTRR_Suspense_Singles

This is an example of one radio program, there are many more I'd like to download. All the web sites are set up in this format. I'm wondering if there is a Windows scripting program that I can use/write that will iterate through the column that contains the link to the MP3 file and save each file to a specific folder on my laptop? To keep things simple I could navigate to the correct web location before invoking the script. In so doing, the script could simply act on the current screen and wouldn't have to dynamically find each web site. In addition, the script could have a static "save to" location that I would change before running it on a new radio program (I want to save each radio program in its own folder - i.e. Suspense radio in a folder, Night Beat in a folder, etc..). Again, thinking of ways to keep this simple and doable.

Anything along these lines would be a great help. Thanks in advance.

 

[Gator]

You could download something like gnuwin32 and/or cygwin and use a command like wget -r www. example.com
I have heard mixed reviews of this though, you may need to do some research on it because some have reported that it will download all files on said page so you will need to make sure you're downloading from a directory and not a full dynamic webpage.

This may help a little
http://pebblesinthesand.wordpress.c...et-to-download-multiple-files-from-a-website/

Remember, this isn't a native Windows command so you have to have a linux based CMD or find a Windows equivalent command.

 

[Gator]

Also, I'm not familiar with the site, but does this option on the left side download the whole lot of the files?

 

[bigmck]

I would be surprised if there was anything like what you want. I frequent a site that has thousands of different "Desktop Wallpapers". To download them individually they are free. If you want to save time and download them in bulk, you must pay. If it were possible to download them in bulk for free, as you want to, I am sure someone would have figured this out. I just get mine in bulk and pay. (I understand that you site is free regardless)

 

[Britton30]

There is Down Them All (DTA) which can do multiple downloads of file type you choose from a webpage. It is only for Firefox though.

DownThemAll!

 

[Gator]

There is Down Them All (DTA) which can do multiple downloads of file type you choose from a webpage. It is only for Firefox though.

DownThemAll!

Nice suggestion, I'll look into this one.

 

[Callender]

Another method

If you're running Firefox browser or a Firefox variant you can try:

FoxySpider

Just selectively enable the add on when you want to use it and disable it at other times when it's not needed.

Right click the FoxySpider icon to choose your options (there's a lot of them).

The icon is the red one to the left of the browser's address bar.

Screenshots:

Configured to crawl for mp3 files:



FoxySpider opens a list of files in a new tab. Select files to download and choose download folder:



Download in progress:



Haven't tried DownThemAll so can't comment on it. FoxySpider has plenty of options and content filters!

 

[UsernameIssues]

All of the suggestions mentioned above would probably work, but I'm not a big fan of using browser extensions or add-ons. They tend to break when the browser is updated or they need updating - then features change and you have to learn how to use the new version.

Also, the link that you provided has 909 MP3 files to be downloaded.
(https://ia600604.us.archive.org/13/items/OTRR_Suspense_Singles/)
If you opt not to use the script that I've provided below, then I would suggesting picking an option that automatically crawls the webpage so that you don't have to select each file to be downloaded. There are several tools that will crawl websites and download the items that you want... but they can be hard to learn how to use.



Here is how I would get the MP3 files that you want:

1) Make the folder where you want the files to go.
2) Copy/paste/run the script file below into that folder.
3) Click on the HTTPS link that Gator mentioned:

4) Paste the URL from that HTTPS page into the script's prompt.
5) Click OK to start the downloads.

Opt("TrayIconDebug", 1)

Global $current_handle, $element, $file, $FileArray, $link, $position, $running, $total, $track, $URL

HotKeySet("^+a", "_end_this")

$URL = InputBox("URL entry", "Paste the URL.", "Paste URL here.", "", 600, 100, 0, 0)
If @error = 1 Then Exit ;Cancel was selected

;FYI URL
;https://archive.org/details/OTRR_Suspense_Singles

;sample URL that can be pasted - 900+ files (20+GB)
;$URL = "https://ia600604.us.archive.org/13/items/OTRR_Suspense_Singles/"


$running = "AutoIt - " & $URL
If WinExists($running) Then
    MsgBox(0, "AutoIt", "This script is already pulling files from:" & @CR & $URL)
    Exit
EndIf
AutoItWinSetTitle($running)


$FileArray = StringSplit(BinaryToString(InetRead($URL)), @CRLF, 1)

$total = 0
For $element In $FileArray
    $position = StringInStr($element, '.mp3">')
    If $position Then $total = $total + 1
Next

$track = 0
For $element In $FileArray
    $position = StringInStr($element, '.mp3">')
    If $position Then
        $track = $track + 1
        $file = StringTrimLeft($element, 9)
        $file = StringLeft($file, $position - 6)
        $link = $URL & $file
        $current_handle = InetGet($link, $file, 1, 1)
        Do
            ToolTip("Downloading " & $track & " of " & $total & "  Press Ctrl + Shift + a to abort.  " & InetGetInfo($current_handle, 0) & " bytes", 10, 10)
            ;TrayTip("Downloading " & $track & " of " & $total, "Press Ctrl + Shift + a to abort." & @CR & InetGetInfo($current_handle, 0) & " bytes", 100)
            Sleep(333)
        Until InetGetInfo($current_handle, 2)
        InetClose($current_handle)
    EndIf
Next

Func _end_this()
    InetClose($current_handle)
    FileDelete($file) ;delete current file in case it had not completed the download
    Exit
EndFunc

After you install AutoIt* from here...
AutoIt Downloads - AutoItScript
...copy/paste the code above into a new script file.
You can sort of see that being done in second video in this post.
You do not need to compile the code...
...just run it as a text file.
You don't need to put it in the start up folder either.

*Right after the AutoIt installation completes, you might need to select an empty spot on the desktop and press F5. Doing that should rebuild/refresh the options that are available to you via Explorer's context menu (right click > New...).


The script has two download progress indicator. You can keep bot, pick one or remove both. Place a semicolon in front of ToolTip and/or TrayTip to turn them off. You can also move the ToolTip to another location if you want.


You can run as many copies of this script at the same time as you want. Just create as many folders as you want and copy/paste/run the script from that folder. The downloaded files go into that same folder that the script is in. If you want to run lots of these at the same time, either turn off the ToolTip and TrayTip progress indicators or just turn off the TrayTip and change the location of each ToolTip so that they do not overlap.

For example, you can change this part of the ToolTip line so that each copy of the script is painting the ToolTip in a different location:
....bytes", 10, 10)
For another copy that is running from another folder, you could use 20:
....bytes", 20, 10)
For another copy that is running from another folder, you could use 40:
....bytes", 40, 10)
For another copy that is running from another folder, you could use 60:
....bytes", 60, 10)

I've not tested those numbers - you might need a larger increment like 10, 50, 100, 150.....


edit: Minor change to the script:
It now prevents multiple copies of the script from downloading files from the same URL at the same time.

 

[MilesAhead]

WGET for Windows - Utilities - AutoHotkey Community

 

[PSB]

Thanks everyone - there are some great suggestions. I am going to try them out next week when I once again have access to high speed internet. I'll let you know.