Is Libraries Useful; a Debilitating Crutch--Or Both?

Hi there
IMO Libraries weren't a good implementation in any case.
They are going to disappear in the latest version of W8 (W8.1). - They will actually still exist but be hard to find so IMO no point in using them.

A much better way is to either SPAN DISKS (risk though if any disk in the spanned group fails then data on the whole lot fails) or use Storage Pools (W8.1).

If you back up data regularly then I wouldn't worry about problems with using Spanned volumes. With these a group of volumes say D,E.F.G for example is treated by Windows as a single "Logical volume". So you don't have to worry or care WHERE your data is physically located. Also good if you have a few lower capacity disks - you can "Aggregate" these into a larger disk space.

I like this a lot for multi-media - particularly Music and Video collections. These days people often have several TB of music / video - and most music organising programs get into problems if your music collection say fills up a standard disk. This method avoids that - the music collection can be as large as the entire 4 disks in my example and the program still thinks it's all stored on a single disk.

The newer storage pool system (W8.1) works much the same -- but more robustly. When the W8.1 preview appears it's worth testing this feature.

Spanned disks - sceenshot enc.

You'll see in one screenshot TWO entries for Disk F (the physical configuration)
In the other you'll see the aggregate - Disk F is now ONE volume as far as Windows is concerned.

Cheers

jimbo

How do you back up and restore a drive in spanned drives?

Hi there
Backup just assumes a Single volume as does restore -- however if you have several TB assigned to a spanned volume set it doesn't make a lot of sense to back up BY VOLUME as you will need several pieces of EXTERNAL media to back this up on -- for example I might need THREE external USB drives. Acronis simply asks for a new volume when one is full.

What I do is Back up by FOLDERS as I have just DATA on the spanned volume. Usually I'll backup by "Music I", "Music II" , Video I, Video II and then everything else (Docs etc).

I don't currently have any folder that's over 1.2 TB in size so 2 TB passport USB drives are what I use for backup. That will certainly change in future though.

Backing up large volumes of data though might be better using a Lower tech solution - TAPES - but in the past I found these slow, expensive and unreliable.

I'd suggest perhaps a new topic should be opened -- about Backup when large volumes of data are involved as this is a totally different problem from backing up say a 45 GB OS partition -- and already 3 TB disks are cheaply available.

Anybody reading this -- How do Banks / the CIA / Google etc back up their (or is it really OUR !!) DATA.

Cheers
jimbo
 

My Computer My Computer

At a glance

Linux CENTOS 7 / various Windows OS'es and se...Intel i7 Intel i58GB, 16GBOn Motherboard
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Custom built, several laptops HP/ASUS
OS
Linux CENTOS 7 / various Windows OS'es and servers
CPU
Intel i7 Intel i5
Memory
8GB, 16GB
Graphics Card(s)
On Motherboard
Sound Card
Realtek HD audio
Monitor(s) Displays
Apple Cinema display, Samsung LCD
Screen Resolution
1920 X 1080
Hard Drives
4 X 1TB SATA
Mouse
Toshiba wireless laser
Internet Speed
> 20MB up
Hi there
IMO Libraries weren't a good implementation in any case.
They are going to disappear in the latest version of W8 (W8.1). - They will actually still exist but be hard to find so IMO no point in using them.

A much better way is to either SPAN DISKS (risk though if any disk in the spanned group fails then data on the whole lot fails) or use Storage Pools (W8.1).

If you back up data regularly then I wouldn't worry about problems with using Spanned volumes. With these a group of volumes say D,E.F.G for example is treated by Windows as a single "Logical volume". So you don't have to worry or care WHERE your data is physically located. Also good if you have a few lower capacity disks - you can "Aggregate" these into a larger disk space.

I like this a lot for multi-media - particularly Music and Video collections. These days people often have several TB of music / video - and most music organising programs get into problems if your music collection say fills up a standard disk. This method avoids that - the music collection can be as large as the entire 4 disks in my example and the program still thinks it's all stored on a single disk.

The newer storage pool system (W8.1) works much the same -- but more robustly. When the W8.1 preview appears it's worth testing this feature.

Spanned disks - sceenshot enc.

You'll see in one screenshot TWO entries for Disk F (the physical configuration)
In the other you'll see the aggregate - Disk F is now ONE volume as far as Windows is concerned.

Cheers

jimbo

How do you back up and restore a drive in spanned drives?

Hi there
Backup just assumes a Single volume as does restore -- however if you have several TB assigned to a spanned volume set it doesn't make a lot of sense to back up BY VOLUME as you will need several pieces of EXTERNAL media to back this up on -- for example I might need THREE external USB drives. Acronis simply asks for a new volume when one is full.

What I do is Back up by FOLDERS as I have just DATA on the spanned volume. Usually I'll backup by "Music I", "Music II" , Video I, Video II and then everything else (Docs etc).

I don't currently have any folder that's over 1.2 TB in size so 2 TB passport USB drives are what I use for backup. That will certainly change in future though.

Backing up large volumes of data though might be better using a Lower tech solution - TAPES - but in the past I found these slow, expensive and unreliable.

I'd suggest perhaps a new topic should be opened -- about Backup when large volumes of data are involved as this is a totally different problem from backing up say a 45 GB OS partition -- and already 3 TB disks are cheaply available.

Anybody reading this -- How do Banks / the CIA / Google etc back up their (or is it really OUR !!) DATA.

Cheers
jimbo

The problem I see with your backup scheme is, if you lose one drive, you would have to restore data to all of your drives, not just the replacement drive. I still feel using Win 7 Libraries to get a single directory is safer than spanning drives. At least, you are wise enough to realize RAID is not a reliable backup solution and keep backups on HDDs separate HDDs.

I have too many folders to mess with folder by folder backups. It's much simpler, more reliable, and faster to back up an entire drive. The catch is, except for my C:/ drive, my drives are 2TB. I use Macrium Reflect to make images of the C:/ drive to 160GB spinners (three of them; I believe in redundancy). I use three 2 TB spinners to make redundant backups of the one 2 TB drive I'm using inside my machine right now (I have a second one installed right now, although I'm not using it yet, and have room for four more). I prefer to clone that drive rather than make a backup; it takes longer but I just set Macrium Reflect to mak the clone, then let run while I do something else (watch TV, read, run errands, party, sleep, etc.; stuff I would be doing anyway). Two of the backups of each HDD in use are kept at home and the third is kept offsite in a safe deposit box at my credit union (and gets swapped out at least once a month). I also back up data only to a paid cloud backup service—Carbonite—to cover the gap between backups to the spinners (I do weekly backups and, if I dump a large amount of data on my machine in a short period of time, I'll do an additional backup; I also do a full backup of the C:/ drive before making any changes).

Banks, CIA, Google, and other big business (you missed the NSA! ;)) use massive servers to store their data and additional servers in different locations to back that data up. Each location has backup power sources to avoid loss from loss of the power grid. If a location loses data to a disaster, the other locations will still have it. I suspect the different locations operate like massive RAID 10s, 50s, 60s, or something similar; each location uses some kind of RAID (other than just 0) and that RAID is mirrored by the other locations.
 

My Computer My Computer

At a glance

Win 7 Ultimate 64 bitIntel i7-3930KKingston HyperX Genesis 32GB Kit (8x4GB Modul...MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2...
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Custom Build
OS
Win 7 Ultimate 64 bit
CPU
Intel i7-3930K
Motherboard
ASUS P9X79 WS
Memory
Kingston HyperX Genesis 32GB Kit (8x4GB Modules) 1600MHz DDR
Graphics Card(s)
MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2GB 256-bit GDDR
Sound Card
Asus Xonar Essence STX
Monitor(s) Displays
3x Asus VG248QE 24", Vizio 32" TV
Screen Resolution
1920 x 1080, ?
Hard Drives
Samsung 128GB 840 Pro SSD (1),
Samsung 4TB 850 EVO SSDs (4)
Samsung 4TB 850 EVO SSDs (16) external backup drives used in 2.5" hot swap bays in the computer.
PSU
Corsair HX750w
Case
Antec Two Hundred v2 (modified)
Cooling
Cooler Master GeminII S524 120mm (fan replaced with a 140mm)
Keyboard
Logitech G510s
Mouse
Logitech M525 (two in use)
Internet Speed
=< 32Mbps down, 8Mbps up
Antivirus
AVAST!, MBAM, SAS, Spybot S&D (all but MBAM free) Glary Util
Browser
IE11
Other Info
LSI 9211-8i HBA card (8 SATA III ports), 2.5" & 3.5" Hot Swap Bays, HooToo HT-CR001 PCI-E to USB 3.0 Internal Hub + 6 Slot Card Reader, and LG Model CH12LS28 BD-ROM Optical Drive. Also, ScanSnap S1500 ADF duplexing scanner, Canon 9000F flat bed scanner, Corsair SP2500 2.1 speakers, Samsung CLP 415nw laser color printer, Cyberpower PP2200SW UPS
I remapped the standard explorer view to My Computer (click on the "folder" icon on taskbar and it opens My Computer... isn't it a luxury?) and use my own folders and folder structure to keep my stuff, thank you very much.

I suspect the different locations operate like massive RAID 10s, 50s, 60s, or something similar; each location uses some kind of RAID (other than just 0) and that RAID is mirrored by the other locations.
That's a lot of overhead over a lot of distance. They probably just hash all the stuff (MD5 or SHA1 or whatever) and send around the hashes for checking if every server has the right files, if not some server asks a ultra-secure encrypted top secret download. Most cloud services do this to avoid ridiculous bandwitdth usage.
 

My Computer My Computer

At a glance

Win 7 Pro 64-bit 7601AMD Phenom 9650 QuadCore, revision DR-B35 GB yes I run 2x 2GB and 1x 1GB, different b...NVIDIA GeForce 9800 GT 512 Mb, unknown manufa...
Computer type
PC/Desktop
Computer Manufacturer/Model Number
custom built
OS
Win 7 Pro 64-bit 7601
CPU
AMD Phenom 9650 QuadCore, revision DR-B3
Motherboard
ASUS M4A78
Memory
5 GB yes I run 2x 2GB and 1x 1GB, different brand, spank me.
Graphics Card(s)
NVIDIA GeForce 9800 GT 512 Mb, unknown manufacturer.
Sound Card
Crappy Realtek Integrated Audio
Monitor(s) Displays
Fujitsu Siemens P19-3P
Screen Resolution
1280 x 1024 x 32 bits @ 60 Hz Oh yeah, 4:3 rocks!
Hard Drives
(1) MAXTOR S TM3320613AS SATA Disk Device (2) STM35004 18AS SATA Disk Device (3) TOSHIBA USB 2.5"-HDD
PSU
whatever, around 450w
Case
Scavenged from old company PC, 10+ years old
Cooling
CPU fan, GPU fan, case fan, nothing fancy
Keyboard
Microsoft, PS/2, white.
Mouse
Optical, logitec.
Internet Speed
effective max speeds: 70-ish kB/s down 30-ish kB/s up
Antivirus
Avira, free edition.
Browser
Firefox with FXChrome to make it look like Google Chrome :P
Other Info
Was discarded by previous owner due to "horrible performance".
Was running Win Xp from a IDE drive. Yeah. Was a pain.
SATA II drive and Win7 and it zips away! Yay!
I remapped the standard explorer view to My Computer (click on the "folder" icon on taskbar and it opens My Computer... isn't it a luxury?) and use my own folders and folder structure to keep my stuff, thank you very much.

I suspect the different locations operate like massive RAID 10s, 50s, 60s, or something similar; each location uses some kind of RAID (other than just 0) and that RAID is mirrored by the other locations.
That's a lot of overhead over a lot of distance. They probably just hash all the stuff (MD5 or SHA1 or whatever) and send around the hashes for checking if every server has the right files, if not some server asks a ultra-secure encrypted top secret download. Most cloud services do this to avoid ridiculous bandwitdth usage.

I said like those RAIDs, not actually long distance RAIDS. Each site would naturally use RAIDed servers but the "mirroring" that would occur would just be a duplicate saved simultaneously to more than one location. That way, the communication connection would be needed only for as long as needed to initially transmit the data. A lot of cloud servers probably compress data to cut down on storage requirements (I know Carbonite does). Some throttle data transmission after a certain threshold (again, Carbonite is one that does; I'm thinking 205-300GB is the threshold, which I've exceeded) to reduce bandwidth requirements. It also discourages people, such as businesses, from abusing unlimited storage plans by continuously uploading and downloading massive amounts of data (business plans with tiered rates are more appropriate for that kind of use).
 

My Computer My Computer

At a glance

Win 7 Ultimate 64 bitIntel i7-3930KKingston HyperX Genesis 32GB Kit (8x4GB Modul...MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2...
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Custom Build
OS
Win 7 Ultimate 64 bit
CPU
Intel i7-3930K
Motherboard
ASUS P9X79 WS
Memory
Kingston HyperX Genesis 32GB Kit (8x4GB Modules) 1600MHz DDR
Graphics Card(s)
MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2GB 256-bit GDDR
Sound Card
Asus Xonar Essence STX
Monitor(s) Displays
3x Asus VG248QE 24", Vizio 32" TV
Screen Resolution
1920 x 1080, ?
Hard Drives
Samsung 128GB 840 Pro SSD (1),
Samsung 4TB 850 EVO SSDs (4)
Samsung 4TB 850 EVO SSDs (16) external backup drives used in 2.5" hot swap bays in the computer.
PSU
Corsair HX750w
Case
Antec Two Hundred v2 (modified)
Cooling
Cooler Master GeminII S524 120mm (fan replaced with a 140mm)
Keyboard
Logitech G510s
Mouse
Logitech M525 (two in use)
Internet Speed
=< 32Mbps down, 8Mbps up
Antivirus
AVAST!, MBAM, SAS, Spybot S&D (all but MBAM free) Glary Util
Browser
IE11
Other Info
LSI 9211-8i HBA card (8 SATA III ports), 2.5" & 3.5" Hot Swap Bays, HooToo HT-CR001 PCI-E to USB 3.0 Internal Hub + 6 Slot Card Reader, and LG Model CH12LS28 BD-ROM Optical Drive. Also, ScanSnap S1500 ADF duplexing scanner, Canon 9000F flat bed scanner, Corsair SP2500 2.1 speakers, Samsung CLP 415nw laser color printer, Cyberpower PP2200SW UPS
Hi there
What we really need is some distinction between Backup and Archive -- for instance my Music Libraries don't change a lot so once backed up then the whole kybosh doesn't really need to be backed up again -- only the additions / changes (not many).

Data that is USED regularly and changes a bit needs to be regularly BACKED UP. My copies of scanned Tax documents etc again just need to be backed up ONCE (archive).

Cloud based solutions aren't really appropriate for ARCHIVE due to the large volumes of data needed in creating the initial archive - however perversely this would actually be the best use of Cloud services once you'd got the stuff uploaded. If you needed to recover something from the archive it would only be a small amount of data anyway.

For instant retrieval etc you need LOCAL data rather than Cloud based solutions. Most Cloud data plans only give you say 100 - 200 GB in any case -- that's much too small for my ARCHIVAL needs.

The other problem becomes in managing data you've actually archived / backed up -- you usually need to know What's on the archive and where it is. Until now most home solutions work reasonably well with just "Ad Hoc" backing up entire volumes -- most people don't have enough data to make this a problem - but already with ever increasing HDD sizes and home servers being used more I think data backup and retrieval needs a more professional approach - but not as extreme as fully blown Corporate or Enterprise solutions.

In our house back home we have about 7 Machines depending on whose staying at home at the time (not all mine -- I'm not that "Geeky") of various sorts -- and the poor old W2K3 server where I try and back the lot up to is getting a bit over worked and I am definitely in need of some decent solutions.

At least I haven't had any problems of where kids have lost their music collections etc. At least not yet !!! although I've had to spend a few hours on recovering them at times. The older ones BTW seem far worse at losing data than the younger one's --suppose that's a part of "growing Up" process. !!

Cheers
jimbo
 

My Computer My Computer

At a glance

Linux CENTOS 7 / various Windows OS'es and se...Intel i7 Intel i58GB, 16GBOn Motherboard
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Custom built, several laptops HP/ASUS
OS
Linux CENTOS 7 / various Windows OS'es and servers
CPU
Intel i7 Intel i5
Memory
8GB, 16GB
Graphics Card(s)
On Motherboard
Sound Card
Realtek HD audio
Monitor(s) Displays
Apple Cinema display, Samsung LCD
Screen Resolution
1920 X 1080
Hard Drives
4 X 1TB SATA
Mouse
Toshiba wireless laser
Internet Speed
> 20MB up
Actually, archival is just remote storage of data. A backup is redundant storage of data. Archival is when you don't store data on your computer due to, say, a lack of space. Data stored solely on a NAS, for example, is archived. A backup is data, either on your computer or archived, that is duplicated and stored elsewhere, sometimes in a different format, so the duplicate can be used restore the original data should it ever be lost or corrupted.

Carbonite is a "cloud solution" that can meet both jimbo's definition of archival and backup. It has unlimited backup capacity (the only caveat being the throttling that I mentioned before above a certain volume). It is not a single solution, however, nor should anyone expect any solution to be a single solution. The initial backup to Carbonite can take days to weeks, depending on the initial volume, the speed of one's internet connection, and how long one's computer runs each day (mine runs 24/7 unless I'm out of town). As long as a file remains on your computer, Carbonite will retain it. Even if the computer is offline for long periods of time, Carbonite will continue to retain the file (assuming you keep paying the bill; my last desktop was offline for several months and I didn't lose any data). Once the initial upload has been done, the data remains until it Carbonite detects that it has been changed or deleted; then, thirty days later, the changed or deleted data will be deleted from Carbonite's servers.

As jimbo pointed out, cloud solutions with huge amounts of data will take huge amounts of time to recover, which is one of many reasons why cloud backups should never be relied on as a sole backup or for sole archival (archival meaning the only place where one's data is stored). Another reason cloud archival or backups should not be a sole solution is they can fail. Free cloud sites have a history of being ephemeral; they can disappear, usually with little or no warning. Paid plans are usually more reliable. Business plans are fastest and are the most reliable but are also expensive. they are geared toward businesses that need large, daily uploads and fast retrieval if local data is lost. Usually, business plans will store one's data on redundant servers whereas home plans will store data on only one server. Business cloud backup plans are geared for new businesses and ones too small to cost effectively maintain offsite servers. Home plans are usually slower and don't have the server redundancy that the more expensive business plans do. But even paid cloud backup solutions still can fail, same as any other media one can use to store backups on.

Jimbo is correct in that static information needs to be backed up but once but redundancy is still vital and the backups still needs to be maintained to ensure the media the backup is kept on hasn't failed. Even though the main HDD (as I call it) in my computer has static information that will rarely, if ever, change, I re-back it up on a weekly (or more frequent) basis simply because it is easier for me to do so. I simply plug a back up HDD into the 3.5" hot swap bay in my computer, set Macrium Reflect to clone the main drive, then walk away. That way, I don't have to keep track of what files in what folders need backing up (a nightmare since the changed or new files are distributed across the drive). When that clone is completed, I plug in the second backup drive into the swap bay and repeat the process. I don't even have to do it on the same day although I prefer they be as close to each other as possible. Cloning is fast and easy to implement, is reliable, the data is easily accessed, and it requires no effort on my part during the process.

On the other hand, once I start putting my videos onto my computer, keeping track of new additions to be backed up will be easy so I will back them up by copying the new additions to the backup HDDs as I add them to the video drive (again, my term since I will use HDDs in my computer that are dedicated to just that purpose to save my videos to). I'll keep new additions in a temporary folder on my main HDD until a video drive in my computer and all three backup HDDs have received a copy (one back up HDD is kept offsite but gets swapped out at least once a month), then the temporary folder can be emptied. Carbonite, by default, will not upload large files, such as movies, unless manually instructed to (it takes all of three mouse clicks to do so), so I will only tell Carbonite to backup only the video saved to a video drive, not the copy in the temporary folder.

As stated before, I use three HDDs to back up each HDD I have in use. Two are stored locally for convenience in updating and recovery. One is kept offsite to avoid it being lost should a disaster take out my originals and my local backups. Since the offsite backup will be incomplete due to it not being updated as frequently as the local backups, I can use Carbonite to recover the data that is missing from the offsite backup. Since Carbonite is allowed to backup new and changed data 24/7, it also can fill the gap between local backups. I can also access my data from Carbonite from another computer, such as my notebook when I'm out of town (safer and more reliable than accessing my computer directly over the internet). However, I would really hate to have to recover all my data from online due to the time it would take to do so (hence, the offsite backup HDD). Still, if something really catastrophic were to happen, such as a meteor strike that takes out my home and my credit union six miles away while I'm lucky enough to be out of town, it is an option that will be available to me.

Reasons I've decided against data in a home server instead of keeping all my data inside my computer include the expense of the server itself, the amount of space it would take up (I just don't have room for a second box), and the difficulty of backing up large volumes (the only solution I can think of would be multiple servers which are too large to put in a safe deposit box). It is far more convenient and faster for me to keep all my data in my computer on discrete HDDs (as opposed to a RAID), then use matching discrete HDDs to put the backups on (the backup HDDs are easily stored in a dresser drawer). I don't need the assurance of uninterrupted operation a RAID would provide (the primary reason for a RAID, other than RAID 0). Having a data HDD die on me would cause an inconvenient interruption but one that can be easily and fairly quickly recovered from (restoring the system drive would take much longer but I have my notebook to keep me going until a replacement SSD can be obtained) and I can get to my data by using the backup HDDs in a dock), unlike a business that would suffer severe financial loss should data become temporarily inaccessible.
 

My Computer My Computer

At a glance

Win 7 Ultimate 64 bitIntel i7-3930KKingston HyperX Genesis 32GB Kit (8x4GB Modul...MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2...
Computer type
PC/Desktop
Computer Manufacturer/Model Number
Custom Build
OS
Win 7 Ultimate 64 bit
CPU
Intel i7-3930K
Motherboard
ASUS P9X79 WS
Memory
Kingston HyperX Genesis 32GB Kit (8x4GB Modules) 1600MHz DDR
Graphics Card(s)
MSI R7850 Twin Frozr 2GD5/OC Radeon HD 7850 2GB 256-bit GDDR
Sound Card
Asus Xonar Essence STX
Monitor(s) Displays
3x Asus VG248QE 24", Vizio 32" TV
Screen Resolution
1920 x 1080, ?
Hard Drives
Samsung 128GB 840 Pro SSD (1),
Samsung 4TB 850 EVO SSDs (4)
Samsung 4TB 850 EVO SSDs (16) external backup drives used in 2.5" hot swap bays in the computer.
PSU
Corsair HX750w
Case
Antec Two Hundred v2 (modified)
Cooling
Cooler Master GeminII S524 120mm (fan replaced with a 140mm)
Keyboard
Logitech G510s
Mouse
Logitech M525 (two in use)
Internet Speed
=< 32Mbps down, 8Mbps up
Antivirus
AVAST!, MBAM, SAS, Spybot S&D (all but MBAM free) Glary Util
Browser
IE11
Other Info
LSI 9211-8i HBA card (8 SATA III ports), 2.5" & 3.5" Hot Swap Bays, HooToo HT-CR001 PCI-E to USB 3.0 Internal Hub + 6 Slot Card Reader, and LG Model CH12LS28 BD-ROM Optical Drive. Also, ScanSnap S1500 ADF duplexing scanner, Canon 9000F flat bed scanner, Corsair SP2500 2.1 speakers, Samsung CLP 415nw laser color printer, Cyberpower PP2200SW UPS
Back
Top