Ugly Hedgehog - Photography
Home Active Topics Newest Pictures Search Login Register
General Chit-Chat (non-photography talk)
FYI warning - are you having slow backups ?
Page 1 of 2 next>
Mar 18, 2023 06:04:47   #
chrissybabe Loc: New Zealand
 
If you notice that your backups are slow then check what drives you are using. This is probably more relevant if you use a program like I do where an entire disk is copied over, and it is a big disk like 4TB, and the backup is maybe 3.5GB in size, and you use the same disk. My backups check the source and destination and only copy over files that have been changed or added (and deletes files from the destination that are no longer present on the source).
If you are noticing that a backup is taking a long time find out how your drive writes (and this may be difficult). A conventional hard drive uses what is called CMR and will have multiple platters etc. The slow drive will use something called SMR and the writes overlap with adjacent channels but the read head is smaller and centered. This is done to squeeze larger capacity from a drive and maybe uses less platters so manufacturing costs are reduced. This affects all manufacturers as far as I am aware and usually on drives over 1TB. The extra delay is caused during writes because of overhead to ensure that data isn't lost. I have a couple of backups like I described at the beginning (USB attached) so the first write to a clean drive might take a half day but the next backup can be extended out to 3 days especially if the source has a lot of changes. Using the wrong drives in a raid array can cause a failure or exceptionally long rebuild time also. My current 4TB backup 2.5" drives (SMR) are all being returned and replaced with 3.5" CMR drives.
An example of how prevalent these drives are is all WD Red drives from 2TB to 6TB are SMR but WD Red Pro are CMR. I was using Adata 4TB 2.5" drives (they use Toshiba 4TB drives inside) and they are SMR. The SMR drives are okay for use for casual backups or backing up while travelling, or archival purposes, etc but they will slow down eventually. Apparently a 'clear drive' can be done which will restore original speed but on my 4TB this took 8 hours - results to be checked overnight tonight. This is a real effect. Another way to tell if they are SMR drives is check the description - if they say not suitable for Commercial use then they are SMR. This does not apply to SSDs.
If you don't think this is a real effect then Google 'CMR cf SMR hard drives'. It might explain some things that have been bothering you over the last 2-3 years. Personally I won't touch another SMR drive ever (although I might use them for backing up while travelling because of the smaller size and when I can rebuild them after the trip).

Reply
Mar 18, 2023 08:06:39   #
Longshadow Loc: Audubon, PA, United States
 
I don't like Shingled recording (SMR), simply because the track data literally overlaps the adjacent track data. They can squeeze more data in that way. I just don't have a lot of faith in it, just by design (methodology). I too will shoot for CMR drives. I don't know if SMR drives do a write verify or not, but yea, that could almost double the time to do a backup. The best test for that would be backup the SAME data to a CMR and an SMR drive, both being "empty", and compare the times.

My system, and cloned system backup drive, are WD Gold drives, which are CMR (Conventional).

I've no idea what format my old recycled 1 Tb Seagate Barracudas backup drives use, but I don't have that much data to back up. It only takes about 3+ hours over USB.

Reply
Mar 18, 2023 08:29:59   #
ELNikkor
 
Never had slow back-ups. Plenty of fruit & veggies...

Reply
 
 
Mar 18, 2023 15:23:48   #
chrissybabe Loc: New Zealand
 
Longshadow wrote:
......The best test for that would be backup the SAME data to a CMR and an SMR drive, both being "empty", and compare the times.
..........I've no idea what format my old recycled 1 Tb Seagate Barracudas backup drives use, but I don't have that much data to back up. It only takes about 3+ hours over USB.

I just zeroed out my 4TB SMR drive and it backed up 34,000 files of 125,000 files and it took 8 hours. That would be almost a day and a half to copy everything. I will check a CMR drive tomorrow when I pick a couple up.
Apparently SMR are creeping in everywhere and you won't always know because they will change from CMR to SMR but keep the part number the same. And it certainly isn't usually listed as part of the drives specifications. As far as I am concerned this is a disaster for drive usage at least for some types of operations.
I have a couple of 4TB USB drives that don't do this and they are older drives before SMR came in I suspect.
Users who are not having any bother probably have older drives as I think this has only been going on for the last 2-3 years.

Reply
Mar 18, 2023 21:20:09   #
TriX Loc: Raleigh, NC
 
Another answer is SSDs. Then the backup will take minutes instead of hours.

Reply
Mar 18, 2023 21:40:37   #
chrissybabe Loc: New Zealand
 
TriX wrote:
Another answer is SSDs. Then the backup will take minutes instead of hours.

Let me know where you can source 4TB portable SSD's from at a reasonable price, actually at any price. And note that they are connected via USB and that, in my case, I am potentially backing up to 4TB at a time (at least 1st time around).
Believe me I would use SSDs if they were available. I would need 4 of them.
I have a specific need to use these as backups this way and in the fullness of time this need will go away but not for a year or so.

Reply
Mar 18, 2023 21:40:37   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
Another answer is SSDs. Then the backup will take minutes instead of hours.

Even via USB?

Reply
 
 
Mar 18, 2023 22:14:21   #
TriX Loc: Raleigh, NC
 
chrissybabe wrote:
Let me know where you can source 4TB portable SSD's from at a reasonable price, actually at any price. And note that they are connected via USB and that, in my case, I am potentially backing up to 4TB at a time (at least 1st time around).
Believe me I would use SSDs if they were available. I would need 4 of them.
I have a specific need to use these as backups this way and in the fullness of time this need will go away but not for a year or so.


Both Samsung and SanDisk make USB connected 4TB SSDs in the $300 range that will run ~1,000 MB/sec over USB 3.2. So that is probably 10x the actual benchmarked speed of a conventional HD, BUT the most important spec which kills your backup time is the random access time, and with 34,000 files (or 178,000 for a full backup), that’s a LOT of accesses. The random access time of a conventional HD is typically 12-16 milliseconds, while the SSD is in the 40-50 microsecond range - hundreds of times faster. Plus that kind of continuous head seeks over hours is the type of activity that really wears a conventional disk.

True, that’s 2-3x the price of an enterprise class HD, BUT the backup window shrinks dramatically and the reliability increases as well. If you have 10-15 TB to back up, then the cost may be prohibitive, but for 4TB, it’s a solid alternative. And at the rate SSD prices are dropping, in a year or so, you’ll likely be able to buy 8TB for the same price or 4TB for half what it is today. I’m now doing my local backup to SSD - no more spinning disk in my house.

Reply
Mar 18, 2023 22:22:08   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
Both Samsung and SanDisk make USB connected 4TB SSDs in the $300 range that will run ~1,000 MB/sec over USB 3.2. So that is probably 10x the actual benchmarked speed of a conventional HD, BUT the most important spec which kills your backup time is the random access time, and with 34,000 files (or 178,000 for a full backup), that’s a LOT of accesses. The random access time of a conventional HD is typically 12-16 milliseconds, while the SSD is in the 40-50 microsecond range - hundreds of times faster. Plus that kind of continuous head seeks over hours is the type of activity that really wears a conventional disk.
Both Samsung and SanDisk make USB connected 4TB SS... (show quote)

I would hope that the HD accesses don't "wear" the disks at all. The heads actually float on a cushion of air, 5 nanometers thick. The heads only touch the disk when they park (a special area of the disk designed for parking) and when they crash.

Reply
Mar 18, 2023 23:06:40   #
TriX Loc: Raleigh, NC
 
Longshadow wrote:
I would hope that the HD accesses don't "wear" the disks at all. The heads actually float on a cushion of air, 5 nanometers thick. The heads only touch the disk when they park (a special area of the disk designed for parking) and when they crash.


It wears the actuators which move the heads plus that generates heat. Plus, as actuators and their axis wear, their chance of the head contacting the platter (a head “crash”) increases.



Reply
Mar 19, 2023 00:57:15   #
chrissybabe Loc: New Zealand
 
You guys living in the land of milk and honey sometimes make the mistake that the way you live and the access you have to tech is universal. Well it isn't. 4TB SSD's are either not in stock or cost an arm and a leg. I can source them here although not at a reasonable price considering I need 4 of them. None of them are the same price as the US ! None of them are USB - they are either M2 or sata 2.5" so you then have to add the cost of a GOOD housing to them. As I said in a year or two things might change.
I am also waiting until US citizens publishing recipes stop using local brand name products that can only be sourced in the US. You are only 4% of the worlds population.

Reply
 
 
Mar 19, 2023 08:40:05   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
It wears the actuators which move the heads plus that generates heat. Plus, as actuators and their axis wear, their chance of the head contacting the platter (a head “crash”) increases.

Sorry, you said "wears a conventional disk.", so, I thought you meant "disk"..... Which I'm sure others would also interpret the same way. I didn't want people to think that the heads actually run ON the disk.

You do know that when drives (the whole unit) are life tested, the actuators are in constant motion for the duration of the life test, and they are run at an abnormal elevated temperature?
I'm not worried about hard drives at all. Yes, some will fail over time, statistics (so will SSDs), but I'm not going to let that be a black cloud over my head waiting for a pending storm.
I keep my computers a LONG time, the last ones are 10+ years each. I have had ONE drive "fail" since about 1985(?), and that one I replaced because the "computer said it was going to fail".
Thanks HAL.....

Reply
Mar 19, 2023 09:08:14   #
Canisdirus
 
I use ssd's for backups...and all programs.

Disk drives only for image storage...but still use SSD's there as well...always have at least 2 copies...three if you like the images.

Reply
Mar 19, 2023 10:48:21   #
TriX Loc: Raleigh, NC
 
chrissybabe wrote:
You guys living in the land of milk and honey sometimes make the mistake that the way you live and the access you have to tech is universal. Well it isn't. 4TB SSD's are either not in stock or cost an arm and a leg. I can source them here although not at a reasonable price considering I need 4 of them. None of them are the same price as the US ! None of them are USB - they are either M2 or sata 2.5" so you then have to add the cost of a GOOD housing to them. As I said in a year or two things might change.
I am also waiting until US citizens publishing recipes stop using local brand name products that can only be sourced in the US. You are only 4% of the worlds population.
You guys living in the land of milk and honey some... (show quote)


Here are two 4TB USB connected SSDs - one Samsung and one SanDisk. I would be surprised if these two popular brands can’t be bought in New Zealand or that New Zealand has some sort of embargo on SSDs and not HDs, but only you know that. And only you know the prices there, but I’m proposing this solution for the maybe 90+% of the readership who do have access to these products. Since Amazon carries both, here’s a link on how to get Amazon to ship anything to New Zealand:

https://www.howitravel.co/amazon-new-zealand/

But since B&H sells it as well, here’s the Samsung: https://www.bhphotovideo.com/c/product/1742972-REG/samsung_mu_pe4t0s_am_4tb_t7_shield_portable.html/?ap=y&ap=y&smp=ba_f1_lar&smp=y&lsft=BI%3A514&gbraid=0AAAAAD7yMh2KZDj2bbFmG1uItvAahe8zV&gclid=CjwKCAjw5dqgBhBNEiwA7PryaOkhWnIFs8BtAuC2suZXGAronX42ZRe3Mk21YY2Gr4dy1fCt3H7gghoCP-sQAvD_BwE

And here’s the SanDisk: https://www.westerndigital.com/products/portable-drives/sandisk-extreme-usb-3-2-ssd?ef_id=CjwKCAjw5dqgBhBNEiwA7PryaFguj6M7i4bnEEulFFReb5tJGxJA6dOHJtP2EcOamCuWvD-HtMqVhRoC-EgQAvD_BwE:G:s&s_kwcid=AL!15012!3!594788379493!!!g!295207995776!!17058314342!134663304934&utm_medium=pdsh2&utm_source=gads&utm_campaign=GOOG-SD-NA-US-SKUS-LowPriority-en-Shopping&utm_content=295207995776&utm_term=SDSSDE61-4T00-G25#SDSSDE61-4T00-G25

Just another valid solution to your issue - whether it makes sense to you is a personal decision. And yes, both ARE USB connected.

Cheers

Reply
Mar 19, 2023 11:02:40   #
TriX Loc: Raleigh, NC
 
Longshadow wrote:
Sorry, you said "wears a conventional disk.", so, I thought you meant "disk"..... Which I'm sure others would also interpret the same way. I didn't want people to think that the heads actually run ON the disk.

You do know that when drives (the whole unit) are life tested, the actuators are in constant motion for the duration of the life test, and they are run at an abnormal elevated temperature?
I'm not worried about hard drives at all. Yes, some will fail over time, statistics (so will SSDs), but I'm not going to let that be a black cloud over my head waiting for a pending storm.
I keep my computers a LONG time, the last ones are 10+ years each. I have had ONE drive "fail" since about 1985(?), and that one I replaced because the "computer said it was going to fail".
Thanks HAL.....
Sorry, you said "wears a conventional disk.&q... (show quote)


As SSDs come into wide usage, statistics show them to be more reliable than spinning disk - period. And it makes sense that devices with less mechanical moving parts have a lower number of failure modes - we stopped using mechanical computers after WW2 because of speed, complexity, reliability and cost/size, and now the same thing is happening to storage. You can stick with last century’s technology as long as you can get parts, but you’re depriving yourself of a huge increase in performance for very few extra $ unless you have many, many TBs of data and the cost is an issue - there is no other valid reason, but hey, some people (at least one or two on this forum) are still running DOS. Time and technology moves quickly in the compute space and investing in last Gen technology is like investing in meat if you don’t have a refrigerator.

Cheers

Reply
Page 1 of 2 next>
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
General Chit-Chat (non-photography talk)
UglyHedgehog.com
Copyright 2011-2024 Ugly Hedgehog, Inc.