[Help] Batch File move every 15.000 files to another folders

[SevenDP]

Hi guys
I'm new here, please help

I have about 2 million email files that i want to convert into mbox files.
but the tool i used only can convert every 15.000 files.
so i need a batch file that can move this 2 million files into another folder (15.000 files each folder)
and the folder name is automatically created (counter).
example:

folder 1 contain 15.000 files
folder 2 contain next 15.000 files
.....
folder 134 contain the rest of files ...

note:
2.000.000 / 15.000 = 133,333 so there will be 134 folders

if can, i hope the files will be renamed too ... (adding ".eml")
example: file name 12345678.123456.abcde moved and renamed to 12345678.123456.abcde.eml

i already have the solution for linux, but i do not have any idea in windows

please help
and sorry for my english

thanks

 

[Slartybart]

Are you going to do this on a regular basis?

Do you want only 15,000 files in each folder or would you prefer to have all of the files in one folder?

Are the files currently in different folders?

I think you can just copy the files to a new location renaming them in the copy process
You have to use copy in Command Prompt, you cannot do this in Windows.

Try this as a test:
Make two new folders to test the process. They can be any name in any location, but I'll use C:\DirA_Src and C:\DirB_Dest as examples
Copy a few (20?) files that you want to rename to C:\DirA_Src folder. If the files have different extensions, that would be a good test.

Launch Command Prompt
Navigate to C:\DirA_Src and list the contents

cd C:\DirA_Src
dir​

The try a rename with copy

copy *.* C:\DirB_Dest\*.*.eml​

That might be difficult to decipher so I'll put the strings on separate lines
copy
*.*
C:\DirB_Dest\*.*.eml

The command has to be entered on one line, just know that there is a space between each string I show on separate lines. Also be aware that a space in the new folder names will require parenthesis around the folder name, so I recommend not having any spaces in the new folder names (note that I used the underscore character)

If that does what you want, then you can use the process to copy all of your files to any folder you choose.

Try it an let me know if it works or if you have other questions.

 

[SevenDP]

Are you going to do this on a regular basis?

Do you want only 15,000 files in each folder or would you prefer to have all of the files in one folder?

Are the files currently in different folders?

I think you can just copy the files to a new location renaming them in the copy process
You have to use copy in Command Prompt, you cannot do this in Windows.

Try this as a test:
Make two new folders to test the process. They can be any name in any location, but I'll use C:\DirA_Src and C:\DirB_Dest as examples
Copy a few (20?) files that you want to rename to C:\DirA_Src folder. If the files have different extensions, that would be a good test.

Launch Command Prompt
Navigate to C:\DirA_Src and list the contents

cd C:\DirA_Src
dir​

The try a rename with copy

copy *.* C:\DirB_Dest\*.*.eml​

That might be difficult to decipher so I'll put the strings on separate lines
copy
*.*
C:\DirB_Dest\*.*.eml

The command has to be entered on one line, just know that there is a space between each string I show on separate lines. Also be aware that a space in the new folder names will require parenthesis around the folder name, so I recommend not having any spaces in the new folder names (note that I used the underscore character)

If that does what you want, then you can use the process to copy all of your files to any folder you choose.

Try it an let me know if it works or if you have other questions.

Hi Slartybart
renaming file was ok. thanks for your information.

About moving file,
It is not regular basis, i just want to do that if the email files > 1 million files.

example:
I have directory "c:\all" contains 2 million files
I want files in "c:\all" moved in new folder every 15.000 files

I need a batch files, so the result:

"c:\result\1" contain 15.000 files
"c:\result\2" contain next 15.000 files
etc

thanks

 

[Slartybart]

Ok, let me think about this. I thought the 15,000 limit was due to the tool you were using.

But I see now that you only want 15,000 files in each sub-folder.

It's going to be simple using a for statement, I just need to brush up on that and then I'll ask you to run another test before doing the entire 2 million.

Do you want to copy the files or move them. Same idea, copy/rename... move/rename - different command that's all.
Oh - I see... you did say move - ok.

Bill

 

[Slartybart]

I'm rustier than I thought.

The basic move batch is easy, but I'm having a hard time figuring out the loop control (limit to 15000). Programming it would be easy (rusty there too though ) - batch command files aren't really meant for complex logic.

It will take me a little longer to find and test - I'm sure it can be done, I'm probably just looking at it the wrong way.
while x is less than 15000
move the file (rename in move)
simple enough - but I keep moving all of the files in my test folder (I lowered the count to 3 instead of 15000)

Anyway, I'll keep trying different approaches to the loop control.

 

[SevenDP]

Hi Slartybart
Yes i want 15.000 files in each folder because the tool i use only can convert about 15.000 files in one process.
the idea is simple ... but i do not know how to implemented on windows batch ...
maybe i should mount the hdd into a linux computer and process there



thanks

 

[Slartybart]

That's where I'm getting confused - the tool. I thought you needed a batch file to replace the tool due to the 15,000 file limit. It sounds as though you still will be using the tool after the "conversion" by copy.

Should I spend any more time on the batch file or are you going to use Linux?

 

[SevenDP]

Hi Slartybart
The tool i used is Thunderbird add-on : Import email from maildir format into Mbox format.
I will going to use Linux's script then ... thanks for your help
regards,
denny

 

[Slartybart]

Thanks Denny,

I can continue trying to write the batch file, but didn't want to spend any time if you didn't need it.
I have a fairly basic bat that works for all files, but I'm hitting a snag when it comes to breaking out of the loop (stopping at 15,000). I'm sure there's an easy way to do it, but I'm rustier than I thought

Bill
.

 

[edmscan]

It will be fun .. I was going to suggest Robocopy but that is more for fast copying of files.

Hmm .. will have to think on this one.

 

[Slartybart]

Well, I gave up... sort of.

I finished fooling around with batch scripts, but still could not find a clean way to break the loop (a problem with command)

I thought that PowerShell might be a better option and did some research, but did not write anything for you to use. I spent too much time with batch and thought "Hey maybe if I just point SevenDP to PowerShell, he/she might be able to figure it out from there - the OP seems intelligent enough".

Here are some links that might get you started. I don't think PowerShell has the same problem with breaking out of loops that the command shell has.

Windows PowerShell Owner's Manual
Scripting with Windows PowerShell
Windows PowerShell Scripting

Loop examples
ForEach | PowerShell | SS64.com
powershell - Loop X number of times - Stack Overflow
Hey, Scripting Guy! How Can I Loop Through Collections with Windows PowerShell? - Hey, Scripting Guy! Blog - Site Home - TechNet Blogs

The attached batch file is something you can paly with if you wish, but you'll be better off spending the time on Powershell.

If you don't understand something in the batch file (not well documented) ask and I'll try to explain what I wanted it to do. It's somewhat convoluted in the recursion call and I had to just exit the batch file to break the loop.

Run some tests (it only moves 3 files but there is a variable you can change) I think I left all of the echos on, so the output isn't' pretty.

Bill
.