Robocopy mass nesting bug

BunBun · 01 Feb 2010

Didn't know where else to put this so I hope this is good.

Twice now I have experienced a issue with robocopy where while copying a user folder it mass nests the Application Data folder within Local. It replicates this to both \User\AppData\Local and \User\Local Settings. Weird thing is Local Settings is just a place holder to point legacy programs to the new location so why robo copy created a folder from this I am not to sure. Both times I have had this happen it has replicated the Application Data folder so many times within itself that it fills the HDD down to the last byte. And since it is nesting it the path is far too long for any standard windows command to do anything with this structure. For example my last instance of it replicated about 15,000 files into over 200,000 from ~9GB to over 80GB and over 30 directories deep. Not sure if there is any end to it or if it does just keep repeating until the drive is full. Some loop issue with the local settings place holder?

Anyway to delete this massively nested directory structure I have found two methods.

1. Boot into Ubuntu (or any distro of your chooseing... I like the Ubuntu live CD) and delete the folder from there. In Ubuntu it moves into the trash so you have to delete it from there as well.

2. Use robocopy again to copy over a nul directory. For instance I created a empty folder called empty on the root of C and then as follows ROBOCOPY C:\EMPTY "X:\USERS\ADMINISTRATOR\APPDATA\LOCAL\APPLICATION DATA" /NOCOPY /PURGE which then deletes all extra files that are in the directory that are not in empty. Works very quickly and effectively.

Before discovering the last technicque I formatted my drive the last time to correct this issue as the drive was empty anyway.

To prevent this from happening again I temporarily moved the local settings place holder out of the source directory and created the files local settings and application data with no extensions in there appropriate places to prevent robocopy from creating those directories in the first place. However while watching I didn't even see it attempt to do this (would have seen a cannot write destination or some error to that extent) so it seems removing the local settings place holder corrected it.

Just thought I would share and see if anyone could give some insight into this issue/bug.

Dwarf · 01 Feb 2010

This sounds as though it could have something to do with junctions. Have you tried the command with the /XJ switch? You might need to use /XJD (Directory) or /XJF (File) switch instead. This switch tells Robocopy to ignore or eXclude the specified type of Junction point. These are included by default.

BunBun · 01 Feb 2010

Dwarf said:

This sounds as though it could have something to do with junctions. Have you tried the command with the /XJ switch? You might need to use /XJD (Directory) or /XJF (File) switch instead. This switch tells Robocopy to ignore or eXclude the specified type of Junction point. These are included by default.

What I was thinking was that everytime it got to the Local Settings thing it would loop back and then nest since the directory already existed...

I will look into junctions and try those switches. Thanks.

pparks1 · 01 Feb 2010

Hey Guys,

Thanks for posting that response Dwarf. I was having a problem trying to use robocopy to backup C:\users\profilename\documents using the /MIR switch to a backup disk as it was getting an access denied trying to get to MY Music and such.

What I find funny is robocopy /? says that /XJ (normally included by default). I thought that meant that it was the default behavior, but

robocopy C:\users\profilename\Documents D:\backup\Document /MIR (fails)

robocopy C:\users\profilename\Documents D:\backup\Document /MIR /XJ (works)

Wierd

Dwarf · 01 Feb 2010

You have interpretted it wrongly. What it means is that adding the switch /XJ (or one of the alternatives) means that the relevant type of Junction will be eXcluded. By default, i.e. without the switch, all junctions are included.

pparks1 · 01 Feb 2010

Yes, after testing it and reading it again...i get it. Thanks again.....It resolved my issue where I used a handful of /XD switches on my backup command.

Dwarf · 01 Feb 2010

You're welcome. :)

Dwarf · 01 Feb 2010

To further explain this, let's use an illustrative example.

<Directory>
File#1
File#2
<Junction> (points to <Directory> i.e. <Junction> resolves to <Directory>)
File#3
File#4

Executing the command without the /XJ switch (i.e. the default) gives the following result:

<Directory>
File#1
File#2
<Directory> (redirected from <Junction>)
File#1
File#2
<Directory>
File#1
File#2
etc

As you can see, when a Junction is encountered, it is resolved and the program continues from the location where the Junction is referencing. This creates an endless loop.

Now with the /XJ switch:

<Directory>
File#1
File#2
(the switch /XJ tells the program to skip <Junction>)
File#3
File#4
end

Here the /XJ switch is specified. As a result, Junctions are ignored and not resolved. The program simply skips over them and continues from the next location.

Dwarf · 01 Feb 2010

Incidentally, there is not a bug in Robocopy. Robocopy was included in earlier versions of Windows prior to the introduction of Junctions. Instead of rewriting the routine, the developers added code to detect the use of the /XJ and associated switches. This resulted in the default behaviour that we have seen described here.

BunBun · 01 Feb 2010

Sorta of a bug as it was not originally intended to operate like that. The bug has a fix which is not common knowledge... but then again neither is robocopy really so :)

I could have sworn I have done this before without issue though...

Thank you for the information.