Do we still need drive letters...?

[zzz2496]

I think, for now, "pool" data store is the most sensible way of managing storage space. This is year 2011, we should not care where we put something, how much free space does a particular storage device has, etc... When we use pool(s), all we need to know is this: "My total storage space is 10TB, free space is 2TB, used 8TB". With S.M.A.R.T and many other sensors we can more of less predicts which storage device in the storage pool that's going to fail in near future, and to use the volume manager to "move" the data off that dying storage device to another, and then remove the said storage device. When we add another storage device, we can add it to an existing pool, or we can make a new pool altogether...

The problem is that Windows 7 can't even remember where I told it to install to install programs (to be fair I have Indexing turned off).
Every month I have to manually search for a program, because Windows has forgotten the shortcut details!
If I let Windows look for the program, it takes stupendous amounts of time to find it (i.e. I can navigate to the location much faster, unless it is a system file).
The same is true with documents and pictures.

I have a hierarchical file structure (like a library) so I can usually go straight to the file I want.

You seem to be suggesting that a library would be more efficient, if the books were fitted with RFID tags and then strewn about at random.
To find a given book you just enter the RFID code into a scanner and then wander around the library waving it around, until you locate what you want.
I guess it would work if every shelf had a reader.

SMART doesn't provide 100% protection from failure.
Allegedly ~50% of failures are due to controller failures (which also affect SSDs).

"All SSDs do is replace a hard drive’s head disk assembly - the platters and heads - with a lot of flash chips. The rest of the stuff is the same - and that stuff accounts for about half of all drive failures."
SSDs no more reliable than hard drives | ZDNet

"The most common failure cause for the 7200.11 was faulty firmware, which would issue the error code "000000CC." This led the Storelab engineers to coin the term "CC fly" (tsetse fly). Faulty drives would slow down and subsequently die after rebooting."
Typical Failures And Data Losses : Study: A Look At Hard Drive Reliability In Russia

zzz2496, That is what RAID does. I have several rack storage slices, put a series of HDDs in, becomes one big glob of space. A drive goes bad, pull it out and put a new one in, without turning the machine off. The data gets reconstructed on the new drive. Automatically.

If you are using RAID with data protection (e.g. RAID 5) you have a reasonable chance of getting your data back.

The best part? It doesn't matter what OS you run. Its all in the hardware.

I still see debates about which is better (hardware or software RAID).

Funny, but if I want to take stuff with me I certainly care. I can't just "hope" that data appears magically on my USB drive. If I want to access files on a CD-Rom i just put in, I damn well be care because otherwise i won't be able to find the files.

I agree.

In the early pages of this thread, I still thread between Filesystems and data store silos, with fuzzy definition between where the data is, and how to manage it. Now, my data store part should act like a SAN, but with minimal management overhead. As for filesystem part, I'd go with Apple's way to manage their OS. Although HFS+ is inferior compared to NTFS, it has "Spotlight" database on top of it which appended it's inferiority...

I'll post again latter, have to go...

zzz2496

 

[carwiz]

I'm from the age of Floppy Disks so it's hard for me to comprehend what folks do with 8Tb of data. That's roughly half of all printed material in the Library of Congress. I couldn't read that in a lifetime.

I think the move is toward moving away large/multiple drives in favor of "The Cloud". Or "Star System" in the future. I don't see the alphabet scheme going away for the average consumer systems in the very near future. But in the close future, I see drives going away almost completely. Systems of the future will have an on-board SSD for the OS, indexes and keys then everything else will be to/from the net. The desktop PC will only be as large as the display you want to view. Files as we know them will be virtual so there would be no need to store them locally. And, all the files in the World would take up less space because there would be one copy that everyone could access. (Book, movies, music, data, etc.) Your "copy" might be called "zzz2496-Favorite-Pic1" on your PC but the virtual name might be "00019670-28436142-99018956-50891423". You would see the file as what ever your called it but your PC would only store the key. "Files" on your PC would be grouped or organized how ever you want and there would be no drive letter unless you grouped it that way.

Have fun with that kids.

 

[zzz2496]

I'm from the age of Floppy Disks so it's hard for me to comprehend what folks do with 8Tb of data. That's roughly half of all printed material in the Library of Congress. I couldn't read that in a lifetime.

I think the move is toward moving away large/multiple drives in favor of "The Cloud". Or "Star System" in the future. I don't see the alphabet scheme going away for the average consumer systems in the very near future. But in the close future, I see drives going away almost completely. Systems of the future will have an on-board SSD for the OS, indexes and keys then everything else will be to/from the net. The desktop PC will only be as large as the display you want to view. Files as we know them will be virtual so there would be no need to store them locally. And, all the files in the World would take up less space because there would be one copy that everyone could access. (Book, movies, music, data, etc.) Your "copy" might be called "zzz2496-Favorite-Pic1" on your PC but the virtual name might be "00019670-28436142-99018956-50891423". You would see the file as what ever your called it but your PC would only store the key. "Files" on your PC would be grouped or organized how ever you want and there would be no drive letter unless you grouped it that way.

Have fun with that kids.

That system I have running on my little lab over here... I have a small "Cloud" on my lab, running NexentaStor with deduplication running on the volume. It works by deduplication at block level, so same blocks form different files will be deduped all the same, the space reduction is close to "holy crap" levels (I store media files on it, like audio files, video files, etc). I haven't had the time to move it to my production system...

zzz2496

 

[lehnerus2000]

The flip side

That's great for saving space.

What happens if 30 files share a block, so the block is only saved once and that block gets corrupted?
You lose 30 files (instead of one).

"Deduplication ultimately reduces redundancy."

 

[zzz2496]

That's great for saving space.

What happens if 30 files share a block, so the block is only saved once and that block gets corrupted?
You lose 30 files (instead of one).

"Deduplication ultimately reduces redundancy."

That's when the ZFS's block checksum comes to the rescue, but then again, those data always rsynced to another SAN (in production scenarios). So it's VERY safe...

zzz2496

Edit: There are many reasons why I use ZFS for my storage volume. ZFS is currently the MOST ADVANCED File system to date (outside commercial SAN equipment). It can check for silent corruption, the throughput is beyond amazing, the FS can manage it's own "cache" of high speed storage devices (think of using SSD as a cache for your RAID array) as second tier cache - and use your high speed RAM as the first tier cache. Not to mention the ability to remove the journal log off to a SSD, increasing write speed to another level, and many more other features...

Have a read on http://www.anandtech.com/show/3963/zfs-building-testing-and-benchmarking

zzz2496

 

[lehnerus2000]

Thanks zzz2496

Thanks for the link.