Solved Torrent writes and SSD

[M18XR2]

I wanted to confirm just how much more writes torrents do.

my options are these
-torrent directly onto a SSD
-torrent onto a 8-20gb of ramdisk, and then move the file onto SSD upon completing the file

imho both have same amount of writes because of file size so for both options it should not matter which one I choose, thus former is better cause easier to set up and less work required. however I can't help to think torrenting actually writes more than just direct file transfer, and quite more at that too. I do not understand how torrent works and even if torrent writes more, how much more and why does it write more?

thanks for your help!

 

[Alejandro85]

Normally, I would expect the client to just write the size of the download file only, and hardly anything else. It just download the pieces in any order it wants and builds the file from there, but overall I don't think it does much more than that. After finished downloading, it will still be reading for seeding until the torrent is no longer valid or you stop it. Clients also keep statistics of how much you upload and download and writes those to disk (this is negligible however, comparable to the cost of keeping history of a web browser).

Some clients does what is known as "preallocation", that is create a dummy file large enough to hold the whole file, but filled with garbage, just to speed up and reserve the space for the incoming data. I'm not sure (and maybe it's client specific), but those techniques may zero the file before saving it, effectively writing all the data twice (one for the initial allocation, the second for the real data when it arrives).

Technically, torrents work by connecting you and all people that want to download a file and people that already have that file. By connecting to all those at the same time instead of a central server that gives everything, a lot of bandwidth is saved and generally greater speeds and availability is achieved. After you finished downloading, you begin to upload and share to others what you have. More here: BitTorrent - Wikipedia, the free encyclopedia

Anyway, why are you concerned about how much does it write to disk?

 

[M18XR2]

Normally, I would expect the client to just write the size of the download file only, and hardly anything else. It just download the pieces in any order it wants and builds the file from there, but overall I don't think it does much more than that. After finished downloading, it will still be reading for seeding until the torrent is no longer valid or you stop it. Clients also keep statistics of how much you upload and download and writes those to disk (this is negligible however, comparable to the cost of keeping history of a web browser).

Some clients does what is known as "preallocation", that is create a dummy file large enough to hold the whole file, but filled with garbage, just to speed up and reserve the space for the incoming data. I'm not sure (and maybe it's client specific), but those techniques may zero the file before saving it, effectively writing all the data twice (one for the initial allocation, the second for the real data when it arrives).

Technically, torrents work by connecting you and all people that want to download a file and people that already have that file. By connecting to all those at the same time instead of a central server that gives everything, a lot of bandwidth is saved and generally greater speeds and availability is achieved. After you finished downloading, you begin to upload and share to others what you have. More here: BitTorrent - Wikipedia, the free encyclopedia

Anyway, why are you concerned about how much does it write to disk?

thanks for the detail info and I think the biggest impact here is preallocation and I have thought of that before. I use both Vuze and bittorrent to download most things nowadays, sometimes browser direct download, ffox and rarely IE.

reason is SSD gets killed with mass downloads, especially with newer SSD with lower 2*nm size when compared to 3* size doesn't last as long even with better SSD controller. currently I have two SSD in raid0 for my laptop as boot drive, storage is two blackscorpio HDD in raid 0 for speed/storage but I'd like to move those two HDD to SSD for solely performance sake. instead of having transferring file to external backup at 100-150MB/s I would like to see 250-500MB/s since my externals are all raid setups. it certainly is a waste of money lol

that aside, a 32/34? nm crucial 256gb SSD known for its durability died after 1 year of straight torrent nearly at 24/7, I won't do as much as 24/7 but something along the line. I'm looking at intel's latest HET-MLC SSD which is said to have at least 14 petabyte of endurance from just a 800gb drive and if thats the case it'll last me a good while but torrent always gets me worried though.

edit: i do about at least 2.5-3TB of downloads and file sorting and bunch of other work in less than half a year..

 

[Alejandro85]

I would then worry about what disk you buy then. If it can't support a simple download without dying I think it's a very bad disk to begin with. What's more problematic, the OS itself writes far more than any download regularly to disk, due to swap file, temporals, hibernation, logs and a few other things, so any unreliable unit as yours would die anyway.

But back to the point, the torrent isn't different in what it writes to disk as a regular HTTP or FTP download. It just writes the data as it becomes available (difference is that it's in a random order) but no much more than a normal download. Then after it begins seeding, which can produce a very high number of reads (not writes) which aren't present in normal downloads of course.
If preallocation worries you, look at the options of your torrent client, it should have an option to disable it and only create a file as it becomes available. No idea how to do that exactly in Vuze or Bittrorrent, as I use uTorrent myself (which has that option).

With so crappy disks, why not simply download the files to a normal HD instead? It should not have any performance difference in the download itself, as local hard disks are much faster than internet connections, and then move to SSD if you want thereafter.

 

[M18XR2]

I would then worry about what disk you buy then. If it can't support a simple download without dying I think it's a very bad disk to begin with. What's more problematic, the OS itself writes far more than any download regularly to disk, due to swap file, temporals, hibernation, logs and a few other things, so any unreliable unit as yours would die anyway.

But back to the point, the torrent isn't different in what it writes to disk as a regular HTTP or FTP download. It just writes the data as it becomes available (difference is that it's in a random order) but no much more than a normal download. Then after it begins seeding, which can produce a very high number of reads (not writes) which aren't present in normal downloads of course.
If preallocation worries you, look at the options of your torrent client, it should have an option to disable it and only create a file as it becomes available. No idea how to do that exactly in Vuze or Bittrorrent, as I use uTorrent myself (which has that option).

With so crappy disks, why not simply download the files to a normal HD instead? It should not have any performance difference in the download itself, as local hard disks are much faster than internet connections, and then move to SSD if you want thereafter.

I have most of my writes disabled as much as possible includes log files, temp files, hibernation and bunch of others. whats left are bunch of other background writes which I probably have no control of, or disabling them would means not using OS, this includes Appdata since I still want to benefit from fast snappy storage I leave Appdata on my SSD.

bolded text is what I needed to know and confirm it so I dont fear SSD dying on me. also I got confused and thought bittorrent is utorrent. I use utorrent so how do I turn off preallocation? as for downloading to HD thing, which is what I am using right now and trying to change it, so not an option lol

 

[Alejandro85]

In uTorrent you disable preallocation by going to Options menu => Preferences => General => untick Pre-allocate files. This should not create anything on disk until the data actually arrives from another peer.

Aren't you putting your downloads in the SSD directly? Or are you using a normal spinner disk for that? With the normal HD, the bulk of activity will go to that disk of course and no torrent client (or any other download type) would ever touch the SSD for anything. I always tough that you throw your downloads to the SSD directly.

Particularly, uTorrent writes to any folder you download to (on that particular disk only) but also a little on AppData to keep its statistics about upload/download, torrent list, activity and ratio. This should be negligible even for the most unreliable disks.

 

[Duzzy]

Here's another screen shot from Utorrent's options that may benefit this discussion. Vuze and other client's may have similar options.

 

[smarteyeball]

Torrents aren't going to kill your SSD

The guys already have you covered on the pre-allocation side of things.

The only reason I don't use SSD's for torrenting is that of space.

 

[M18XR2]

@Alejandro & Duzzy, thanks for screenshot those really help. my current setup is two SSD as OS, two HDD where torrent downloads to and would like to change those two HDD to two SSD. My OS drives are both SLC SSD but its only 256gb togther and not enough, so I have to have eMLC or HET-MLC SSD for storage which s3700 comes in in about couple of months.

just a question though in Duzzy's screenshot I left it on default cause I was told it's best to let the client manage itself. default was at 4MB iirc and I see yours as 256MB, is it better? or how does it work?


@Smarteyball torrent may not kill SSD but "killing it" isn't really correct. as I mentioned earlier SSD may not die after 2 years of straight torrent, but as it gets closer to being dead, it slows down considerably. average transfer rate could go from 300-350MB/s for a high end SSD to 150 when it's nearly dead. so to get real good performance even after couple of years I'd need real durable SSD with enterprise flash, ie 32nm+ size or at least 10k P/E cycle, ontop of that bigger the better.

 

[kbrady1979]

Many torture tests have shown some higher end SSD's to outlive their specifications 10x over. If you set the directory to one of the HDD's, the SSD will never see any of the torrent.....there is no reason why movies, music, pictures should be on an SSD because they don't benefit from the performance increase over a HDD. Also, when you talk about average transfer rate, if you are referring to those "awesome" Sequential read/write speeds, just remember you can only transfer as fast as your slowest component, which will be your HDD. To utilize those fast sequential read/write speeds, you need to transfer from SSD to SSD. Quite a few people don't realize that.

 

[smarteyeball]

@Smarteyball torrent may not kill SSD but "killing it" isn't really correct. as I mentioned earlier SSD may not die after 2 years of straight torrent, but as it gets closer to being dead, it slows down considerably. average transfer rate could go from 300-350MB/s for a high end SSD to 150 when it's nearly dead. so to get real good performance even after couple of years I'd need real durable SSD with enterprise flash, ie 32nm+ size or at least 10k P/E cycle, ontop of that bigger the better.

Of course the SSD will slow down, but TRIM and GC have been improving each gen and the performance degradation not as rapid. The drive will still outperform mech drives. As for life span, it's component lotto. 24/7 HDD use is equally as prone to failure as

Otherwise, if you have the room (assuming it's not a laptop) - Set up a mech RAID volume.


Transfers, unzipping etc is still pretty damn quick.

 

[M18XR2]

Many torture tests have shown some higher end SSD's to outlive their specifications 10x over. If you set the directory to one of the HDD's, the SSD will never see any of the torrent.....there is no reason why movies, music, pictures should be on an SSD because they don't benefit from the performance increase over a HDD. Also, when you talk about average transfer rate, if you are referring to those "awesome" Sequential read/write speeds, just remember you can only transfer as fast as your slowest component, which will be your HDD. To utilize those fast sequential read/write speeds, you need to transfer from SSD to SSD. Quite a few people don't realize that.

As my first few post indicates, I have external raid HDDs, to be exact, three barracuda 3TB in raid 0 each one capable of 200-90MB/s read and write, thus giving me 600-250MB/s, and slowest thing would be my raid0 HDD inside my laptop, which is at 240-100MB/s via esata.

Of course the SSD will slow down, but TRIM and GC have been improving each gen and the performance degradation not as rapid. The drive will still outperform mech drives. As for life span, it's component lotto. 24/7 HDD use is equally as prone to failure as

Otherwise, if you have the room (assuming it's not a laptop) - Set up a mech RAID volume.

yes SSD tends to last longer than its actual life. the 3k-10k P/E cycle is likely for 1% of the cells to first start to die while rest are all still fine, however, dying flash loses speed.. and not a Trim/GC issue.

for 2nd part, I do have laptop m18x R2 holds four 2.5" drive bay and a msata, total of 5 storage bays lol, lappy GOOO!

 

[jimbo45]

Hi there
In Torrents remember you are also concurrently UPLOADING parts of the file to other peers while downloading so the torrent client (usually) but not always Utorrent will probably do quite a few WRITES to store things like index / address markers to show what pieces of the file it's got, who's connected in the swarm etc etc apart from the actual retrieval of the data itself.

If you specify a target drive other than the SSD for the torrent you won't use the SSD for downloading the torrent - and in any case unless you have a really fast internet connection and zillions of seeders in the torrent the I/O would be slow anyway even if it is on the SSD.

Given the drop in price of SSD's and the vast improvement in their design - I wouldn't worry to much about their longevity --these days they seem just as good if not better than HDD's and this will continue to improve.

Cheers
jimbo

 

[M18XR2]

Hi there
In Torrents remember you are also concurrently UPLOADING parts of the file to other peers while downloading so the torrent client (usually) but not always Utorrent will probably do quite a few WRITES to store things like index / address markers to show what pieces of the file it's got, who's connected in the swarm etc etc apart from the actual retrieval of the data itself.o

so removing the complete torrent also help reduce reads/writes ummm well seeding isnt my thing but good to know

 

[Duzzy]

just a question though in Duzzy's screenshot I left it on default cause I was told it's best to let the client manage itself. default was at 4MB iirc and I see yours as 256MB, is it better? or how does it work?

uTorrent has a default of 32MB but is 256MB better, I really don't know. For speed I'm 99.9% sure it makes no difference. If you check my screen shot below you can see uTorrent makes a lot more read/writes to the cache than it does to the hard drive but then the size of the chunks are different and the amount of data is similar. Also I don't think I've seen it use more than 50MB of the write cache not that I look very often, so I just figured give it more than it needs if you have memory to spare but I really don't know if 256MB makes any difference.

Then there are those other setting in my previous screenshot which could also affect how many read/writes are made to the hard drive but I havn't played with them. The main thing is if you read the first sentence it makes you think bigger the cache the better although too big may be a waste and just the size may not make much difference.

I just basically put up the image so you and others could see there is a little control over the read/writes to the drive not really knowing how much difference the setting make.

When I took this screenshot uTorrent had been running for around 1 and half hours to 2hrs downloading at 100kB\s - 500kB\s, uploading at 50kB\s - 90kB\s and I had downloaded roughly 1.2GB and upload 140MB.

 

[M18XR2]

alright sorry guys, thread will still be marked as solved by I have gotten couple more info from a few SSD reviewers. in general holds true for all storage devices not just SSD or HDD but random writes are much harsher on storage devices than sequential, as to why I'm not sure yet and will find out tomorrow, or the day after tomorrow when I get my answer from them.

this somewhat make sense because even when writing to drive 4k or w/e block size, random QD1 speed is much slower than QD1 sequential for a reason and this maybe it. as for preallocation it COULD be a method used by torrent software to prevent poor space management on users/ small storage devices or to make it sequential instead of random, then again I don't write the program and I dont design the storage drives so I do not know.

in the end, best way to torrent to a SSD and save its life span would be using ramdisk. have like 4 to 20gb of ramdisk drive for large amount of torrents at once then transfer the file to SSD with sequential write instead of random writes thus saving SSD life.


with that in mind, there are only 2 major issues/ disadvantage using ramdisk instead of directly torrenting onto SSD.

- Ram is expensive, uses up RAM, 32gb would work nicely have 20 for RAMDisk size and 12 for VMWARE/ media editing.
- powerloss, will kill off your torrent files instantly so if you're at 99%, good luck LOL. I don't really have to worry about this because I use laptop and has battery..!!!!

 

[M18XR2]

storage in pages on the NAND. 8K blocks make a page, new 19nm and 20nm flash uses 16K blocks to make a page. So when you write a 4K size bit the data has to be held somewhere else (read), then that 4K added to the data and written to the page. Then it does it again to fill an 8K block. Data can be read at the block level but must be written to at the page level. So if writing single 128K data piece (sequential data) each block is only updated once (and you use more of them at the same time). So you are writing the whole page once instead of twice with an 8K page, four times with a 16K page.

my thanks to a friend whom is very knowledgeable in regards to solid state drives on why sequential writes does prolong life rather than random write, which in this case torrenting, browsing net and other stuff.. anyway that is to say with under 30nm size SSD and above 21nm have 8K page, sequential does it once to the drive, pre alloaction makes it twice while rest are all random which is twice more, total of 4 writes instead of once. if Ramdisk is used here I can save random writes and pre allocation, making it writing only once to the drive in sequential transfer.