View Single Post
13 Mar 2012  
TanaT

Windows 7 Ultimate x64
 
 
Archive attribute trouble with some Robocopy version

It looks like robocopy sometimes do not handle Archive attribute correctly with /M option.

I tried many options without success. Piping robocopy to attrib.exe is not solving it because it returns destination files (not source files), so here is my work around to have partial backups really partial !

I launch attrib.exe -A just after robocopy full backup. The only problem with this is that if the backup takes a lot of time and you change a file during robocopy execution, it might not be copied and then unset archive flag by attrib. So be careful with the first backup : you should launch it and give it some time...

Here is version 0.1 corrected :
EDIT (19/03/2012) : Version 0.2.

Code:
# -----------------------------------------------------------------
# ------------------------ MVBACKUP (0.2) ------------------------
# -----------------------------------------------------------------
#
# Laptop backups management, by TanaT.
# Created 2012-03-03.
# Version 0.2 (2012-03-19) : Corrected following trouble : Function
RemoveItem doesn't work properly in  recursive mode and -force option
is not enough to get rid of  confirmation prompt.
# Version 0.1 (2012-03-13) : archive atttribute handled despite a bug
# in some robocopy versions.
#
# 1. Complete SETTINGS part with the folders you want to backup.
# 2. Launch the script manually anytime you want or automatically with
# control panel/Admin tools/Tasks scheduler.
#
# Any time the script is launched, it looks for available Destination
# directories in any hard drive available. If it finds full backup
# directories as set in BckFold it launches a mirroring to it and set
# the archive flags of sources files. If it finds partial backup
# directories in any hard drive available (like USB Key for instance)
# it launches a mirroring to it restricted to files without any archive
# flag set (those who hasn't been backed up yet).

# Example :
# BckFold = "_mvBackup_" and PrtFold = "_partBack_"
# Create a folder named _mvBackup_ in a big external hard drive (500Go ?)
# Create a folder named _partBack_ in a USB Key (8Go ?)
# Keep your big hard drive home and bring your USB Key with your laptop.
# Connect your hard drive and launch mvbackup.ps1 when your home.
# Connect your USB key and launch mvbackup.ps1 when your out and you've
# just worked hours on any kind of stuff you want to keep safe.
# When you're back home, launch it again on your external hard drive.
# You can have as many as you want drives (big or small). Just create the
# appropriate folder at the root of your drive.

# Note : If some files are marked with system attribute, they can't be
# handled by attrib.exe. You should launch powershell as administrator and
# enter following command over the folder containing these marked files :
#
# attrib.exe -A /S /D /L C:\Users\TanaT\FolderWithMarkedFiles\*
# (it removes system attribute on all files, folders, including subfolders.)

# -----------------------------------------------------------------
# ------------------------ SETTINGS -------------------------------
# -----------------------------------------------------------------

# List of "folders to backup" = "Names of folders to give in archive" separated with ;
$SrcDirs = @{
"C:\Users\TanaT\Pictures" = "Images_TanaT";
"C:\Users\TanaT\Documents" = "MesDocs_TanaT"
}

# Name of full backup folder we should look for in "big" hard drives
$BckFold = "mvBackup_AT"

# Name of partial backup folder we should look for in "small" usb keys / SD cards
$PrtFold = "partBack_AT"

# Timestamps when a backup action (full) is complete
# are stored in $SrcDirs_items\$LastFTSFile (source side)
# Timestamps when a backup action (partial) is complete
# are stored in $PrtFold\$BckDirs_items\$LastPTSFile (destination side)
#   (Creation timestamps of files are used.)
$LastFTSFile = ".lastFullTimestamp.mvb"
$LastPTSFile = ".lastPartTimestamp.mvb"
$LogFile  = "C:\users\TanaT\logs\mvbackup.log"
$LogFileP = "C:\users\TanaT\logs\mvbackupP.log"

# -----------------------------------------------------------------
# ------------------------ ROUTINES -------------------------------
# -----------------------------------------------------------------

# Touch equivalent except that it works with Creation timestamp
function TouchFile ()
{ 
  param([string] $file)
  if (test-path $file)
  { $now = Get-Date
    (Get-Item $file).CreationTime = $now }
  else
  { echo $null > $file }
}

# MvMirror full|part srcDir destDir
# case full :
#   * Mirror srcDir to destDir
#   * Set Archive flag of every saved files and dirs
#   * Set creation time of $args[1] \ $LastFTSFile = now 
# case part :
#   * mirror srcDir to destDir ignoring files with Archive flag set
#   * Set creation time of $args[2] \ $LastPTSFile = now
function MvMirror ()
{
  If($args.Count -ne 3)
  {
    write-host "Function MvMirror must have 3 arguments."
    return
  }

  $action = $args[0]
  If($action -eq "full")
  {
    # Full backup : Mirror all and unset Archive flag of source files/folders
    robocopy $args[1] $args[2] /MIR /XJ /R:1 /W:2 /NP /MT:16 /log:$LogFile /XF $LastFTSFile $LastPTSFile
    if ( $? -le 1 )
    { # Success unset archive flag of all files and dirs
      #write-host attrib -A /S /D /L "$($args[1])\*"
      attrib -A /S /D /L "$($args[1])\*"
      # Success set new timestamp of full backup
      #write-host TouchFile ("$($args[1])\$LastFTSFile")
      TouchFile ("$($args[1])\$LastFTSFile")
    }
  }
  elseIf($action -eq "part")
  {
    # Partial backup : Mirror files with Archive flag set
    # (those that hasn't been backed up yet)
    robocopy $args[1] $args[2] /S /PURGE /IA:A /XJ /R:1 /W:2 /NP /MT:16 /log:$LogFileP /XF $LastFTSFile $LastPTSFile
    if ( $? -le 1 )
    { # Success set new timestamp of partial backup
      #write-host TouchFile ("$($args[2])\$LastPTSFile")
      TouchFile ("$($args[2])\$LastPTSFile")
    }
  }
  else
  {
    write-host "Function MvMirror: 1st arg must be full or part."
  }
}

# -----------------------------------------------------------------
# ------------------------ MAIN -----------------------------------
# -----------------------------------------------------------------

# ------------------------ GLOBAL CONTROLS ------------------------

# Invalid filename chars pattern
$pattern = "[{0}]" -f ([Regex]::Escape([String] `
[System.IO.Path]::GetInvalidFilenameChars()))

foreach ($dir in ($SrcDirs.clone()).Keys)
{
  # If a source folder doesn't exist, remove it from the list
  if ( ! (Test-Path $dir -PathType container) )
  {    
    write-host "Removing $dir from the list (unreadable dir)"
    $SrcDirs.Remove($dir)
  }
  else
  {
    # If a destination dir name is illegal, remove it from the list
    if ($SrcDirs[$dir] -match $pattern)
    {
      write-host "Removing $dir from the list (illegal foldername $SrcDirs[$dir]"
      $SrcDirs.Remove($dir)
    }
  }
}


# $SrcDirs

# ------------------------ DIRS SEARCH ----------------------------
$FullBDirs = @() # To list any full    backup drive + path found
$PartBDirs = @() # To list any partial backup drive + path found

# Look for any hard drive letter and search for mvbackup folders in them
foreach ($drive in (Get-WmiObject Win32_LogicalDisk -filter "DriveType = 2 or DriveType=3"))
{
  if (Test-Path "$($drive.DeviceID)\$BckFold")
  {
    $FullBDirs += "$($drive.DeviceID)\$BckFold" # Add a full backup dir
  }  
  if (Test-Path "$($drive.DeviceID)\$PrtFold")
  {
    $PartBDirs += "$($drive.DeviceID)\$PrtFold" # Add a partial backup dir
  }    
}

write-host "`n----------------------------------------------"
write-host "------------- MVBACKUP 0.2 -------------------"
write-host "----------------------------------------------`n`n"

write-host "------------- Full backups to do -------------`n"
$FullBDirs

write-host "`n----------- Partial backups to do ------------`n"
$PartBDirs
write-host "`n`n"

# ------------------------ BEGIN ! --------------------------------

# -- If full backup dir available, do it
if ($FullBDirs.length -gt 0)
{
  write-host "------- FULL BACKUPS PROCESS STARTED... ------" -foreground "cyan"
  write-host "----------------------------------------------`n" -foreground "cyan"

  foreach ($currFullDestPath in $FullBDirs)
  {
    # loop over any full destination dirs available
    foreach ( $currSrcDir in ($SrcDirs.Keys) )
    {
      write-host "`n ***" -foreground "cyan"
      write-host " *** Processing $currFullDestPath\$($SrcDirs[$currSrcDir])" -foreground "cyan"
      write-host " ***`n" -foreground "cyan"
      MvMirror full $currSrcDir $currFullDestPath\$($SrcDirs[$currSrcDir])
    }
  }
}

# -- If partial backup dir available, do it
if ($PartBDirs.length -gt 0)
{
  write-host "`n`n----- PARTIAL BACKUPS PROCESS STARTED... -----" -foreground "green"
  write-host "----------------------------------------------`n" -foreground "green"

  foreach ($currPartDestPath in $PartBDirs)
  {
    # loop over any partial destination dirs available
    foreach ( $currSrcDir in ($SrcDirs.Keys) )
    {
      # If a full backup occurred more recently than last partial backup, empty partial dir before.
      if ( (Test-Path "$currPartDestPath\$($SrcDirs[$currSrcDir])\$LastPTSFile") -and (Test-Path "$currSrcDir\$LastFTSFile") )
      {
        if ((Get-Item "$currPartDestPath\$($SrcDirs[$currSrcDir])\$LastPTSFile").CreationTime -lt (Get-Item "$currSrcDir\$LastFTSFile").CreationTime)
        {
          write-host "Delete partial folder first"
#          Remove-Item doesn't work properly in recurse mode (even written in detailed help of it !) so here is proper way :
          Get-ChildItem $currPartDestPath\$($SrcDirs[$currSrcDir]) -recurse | Remove-Item -recurse -force
#          Note that second -recurse parameter is mandatory if you don't want to be prompted.
        }
      }        
      write-host "`n ***" -foreground "green"
      write-host " *** Processing partial $currPartDestPath\$($SrcDirs[$currSrcDir])" -foreground "green"
      write-host " ***`n" -foreground "green"
      MvMirror part $currSrcDir $currPartDestPath\$($SrcDirs[$currSrcDir])
    }
  }
}

write-host "`n`n             END - SUCCESS"
write-host "----------------------------------------------"
start-sleep -s 4
return

# ------------------------ EOF ------------------------------------
My System SpecsSystem Spec