Loading huge files - fast

if it's not a secret, can you tell what are you trying to do (i.e. what are those big files for, and why you need so fast transfer rates)? maybe there is a better way around the issue.

JerometheGiraff said:

Able to load 25 gig files in up to 5-10 seconds. At a rate of 160 megs per second, is still slow.

No. It's just the limit of the technology.

JerometheGiraff said:

I may have to live with it

Best answer

JerometheGiraff said:

and find a way to down size the files, so I don't run into this, although even if I down size the file sizes are still 2-5 gigs max, no way around it.

Many video programs will compress video streams into .avi, .mp4, etc. to save disk space. However, I have found it takes more time to do the conversion that to wait for the file to transfer.

I would also suggest meditation!

There really is no other solution, besides trying to find the fastest SSD on the market, which probably isn't fast enough, or accepting the wait times. I'll have to accept the wait times :)

You checked out RAID? Add moar disks and run them as one to improve speed. Raid 0 is the fastest, but if a drive dies you lose everything, raid 10 and you have moar speed and enough fault tolerance to survive some losses.

If you have a crappy PC laying around (and an excessive amount of hard drives) you can add that at the software level with linux. Then you just need to find a cable with enough bandwidth to move data to/from it.

OP, yeah you are in the technology limitation, 160 megs a second is SLOW?

What are you doing, that it's a problem? [someone needs perspective, here you try a 1200 baud modem!! ]

JerometheGiraff said:

There really is no other solution, besides trying to find the fastest SSD on the market, which probably isn't fast enough, or accepting the wait times. I'll have to accept the wait times :)

Until 2237 when we'll have bio gel packs as storage medium. No waiting at all and they hold exabytes of data.

Well, that's from Star Trek Voyager isn't it?
Not a show known for scientific accuracy.

Although yeah, bio-stuff has a ridiculously high data density (DNA mostly), the issue is that reading speed tends to suck a bit as it's a loooooooooooong tape.

bobafetthotmail said:

Well, that's from Star Trek Voyager isn't it?
Not a show known for scientific accuracy.

The original Star Trek predicted the flip phone!

We are waiting for Quantum Computing, where (as the breathless protagonists tell us) the computer will have the answer before you ask it. (Verbally of course!)

We have already quantum computers. They are one-trick ponies though. That is, they can do stuff normal computers cannot, but cannot do what a normal computer can. So they do just some equation solving for the moment.
Btw, Google bought one recently for its brand new Quantum Artificial Intelligence Lab. Likely to make smarter search algorithms, and not at all a world-ruling Skynet. Googlenet.

the computer will have the answer before you ask it. (Verbally of course!)

Any smartphone equipped with Siri or one of its android clones can do that. Wolfram Alpha is another similar project (answers questions written in plain english) but does not listen/speak (not that it's hard to add anyway)