Ugly Hedgehog - Photography
Home Active Topics Newest Pictures Search Login Register
General Chit-Chat (non-photography talk)
FYI warning - are you having slow backups ?
Page <prev 2 of 2
Mar 19, 2023 12:00:02   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
As SSDs come into wide usage, statistics show them to be more reliable than spinning disk - period. And it makes sense that devices with less mechanical moving parts have a lower number of failure modes - we stopped using mechanical computers after WW2 because of speed, complexity, reliability and cost/size, and now the same thing is happening to storage. You can stick with last century’s technology as long as you can get parts, but you’re depriving yourself of a huge increase in performance for very few extra $ unless you have many, many TBs of data and the cost is an issue - there is no other valid reason, but hey, some people (at least one or two on this forum) are still running DOS. Time and technology moves quickly in the compute space and investing in last Gen technology is like investing in meat if you don’t have a refrigerator.

Cheers
As SSDs come into wide usage, statistics show them... (show quote)

I suppose I'm depriving myself of something I don't require or need...
Just because some people need something doesn't mean that I should also.
Funny, that some people cannot understand that.
Just like I don't need my computers to be cabled to get more speed, WIFI is fine.

Reply
Mar 19, 2023 12:45:29   #
TriX Loc: Raleigh, NC
 
Longshadow wrote:
I suppose I'm depriving myself of something I don't require or need...
Just because some people need something doesn't mean that I should also.
Funny, that some people cannot understand that.
Just like I don't need my computers to be cabled to get more speed, WIFI is fine.


No one is suggesting that you should do anything computer related - I’m describing best practices. Whether you embrace them or not or rationalize not embracing them is entirely a personal decision, just like everything photography related, or automobiles or TV or…. And there is no need to defend your own desires. At the same time, that doesn’t make them the best decisions for others.

Reply
Mar 19, 2023 12:47:41   #
jerryc41 Loc: Catskill Mts of NY
 
chrissybabe wrote:
If you notice that your backups are slow then check what drives you are using. This is probably more relevant if you use a program like I do where an entire disk is copied over, and it is a big disk like 4TB, and the backup is maybe 3.5GB in size, and you use the same disk. My backups check the source and destination and only copy over files that have been changed or added (and deletes files from the destination that are no longer present on the source).
If you are noticing that a backup is taking a long time find out how your drive writes (and this may be difficult). A conventional hard drive uses what is called CMR and will have multiple platters etc. The slow drive will use something called SMR and the writes overlap with adjacent channels but the read head is smaller and centered. This is done to squeeze larger capacity from a drive and maybe uses less platters so manufacturing costs are reduced. This affects all manufacturers as far as I am aware and usually on drives over 1TB. The extra delay is caused during writes because of overhead to ensure that data isn't lost. I have a couple of backups like I described at the beginning (USB attached) so the first write to a clean drive might take a half day but the next backup can be extended out to 3 days especially if the source has a lot of changes. Using the wrong drives in a raid array can cause a failure or exceptionally long rebuild time also. My current 4TB backup 2.5" drives (SMR) are all being returned and replaced with 3.5" CMR drives.
An example of how prevalent these drives are is all WD Red drives from 2TB to 6TB are SMR but WD Red Pro are CMR. I was using Adata 4TB 2.5" drives (they use Toshiba 4TB drives inside) and they are SMR. The SMR drives are okay for use for casual backups or backing up while travelling, or archival purposes, etc but they will slow down eventually. Apparently a 'clear drive' can be done which will restore original speed but on my 4TB this took 8 hours - results to be checked overnight tonight. This is a real effect. Another way to tell if they are SMR drives is check the description - if they say not suitable for Commercial use then they are SMR. This does not apply to SSDs.
If you don't think this is a real effect then Google 'CMR cf SMR hard drives'. It might explain some things that have been bothering you over the last 2-3 years. Personally I won't touch another SMR drive ever (although I might use them for backing up while travelling because of the smaller size and when I can rebuild them after the trip).
If you notice that your backups are slow then chec... (show quote)


Yeah, I read about that a while ago. Speed of backing up isn't a big concern for me, not as much as the cost of replacing whatever disks I have. When - if - I buy a new HDD, I'll be sure to check the specs.

Reply
 
 
Mar 19, 2023 12:50:57   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
No one is suggesting that you should do anything computer related - I’m describing best practices. Whether you embrace them or not or rationalize not embracing them is entirely a personal decision, just like everything photography related, or automobiles or TV or…. And there is no need to defend your own desires. At the same time, that doesn’t make them the best decisions for others.

Exactly!

I've been using/programming computers since 1972.
I know my needs and desires.

Reply
Mar 19, 2023 13:38:50   #
Toby
 
chrissybabe wrote:
If you notice that your backups are slow then check what drives you are using. This is probably more relevant if you use a program like I do where an entire disk is copied over, and it is a big disk like 4TB, and the backup is maybe 3.5GB in size, and you use the same disk. My backups check the source and destination and only copy over files that have been changed or added (and deletes files from the destination that are no longer present on the source).
If you are noticing that a backup is taking a long time find out how your drive writes (and this may be difficult). A conventional hard drive uses what is called CMR and will have multiple platters etc. The slow drive will use something called SMR and the writes overlap with adjacent channels but the read head is smaller and centered. This is done to squeeze larger capacity from a drive and maybe uses less platters so manufacturing costs are reduced. This affects all manufacturers as far as I am aware and usually on drives over 1TB. The extra delay is caused during writes because of overhead to ensure that data isn't lost. I have a couple of backups like I described at the beginning (USB attached) so the first write to a clean drive might take a half day but the next backup can be extended out to 3 days especially if the source has a lot of changes. Using the wrong drives in a raid array can cause a failure or exceptionally long rebuild time also. My current 4TB backup 2.5" drives (SMR) are all being returned and replaced with 3.5" CMR drives.
An example of how prevalent these drives are is all WD Red drives from 2TB to 6TB are SMR but WD Red Pro are CMR. I was using Adata 4TB 2.5" drives (they use Toshiba 4TB drives inside) and they are SMR. The SMR drives are okay for use for casual backups or backing up while travelling, or archival purposes, etc but they will slow down eventually. Apparently a 'clear drive' can be done which will restore original speed but on my 4TB this took 8 hours - results to be checked overnight tonight. This is a real effect. Another way to tell if they are SMR drives is check the description - if they say not suitable for Commercial use then they are SMR. This does not apply to SSDs.
If you don't think this is a real effect then Google 'CMR cf SMR hard drives'. It might explain some things that have been bothering you over the last 2-3 years. Personally I won't touch another SMR drive ever (although I might use them for backing up while travelling because of the smaller size and when I can rebuild them after the trip).
If you notice that your backups are slow then chec... (show quote)


Thanks for the info

Reply
Mar 19, 2023 15:56:10   #
chrissybabe Loc: New Zealand
 
I don't think my backups are unique. My wifes data disk is 4TB. She uses Adobe extensively both for Indesign and Photoshop. The disk is constantly being changed with deletions, file changes and file additions. There can be odd times when up to 1TB is changed between backups. I use bvckup2 which backs up the entire disk by creating a table of whats on the source, another table of the current destination, compares the two, backs up any change, any new file and deals with any deletions on either disk. So lots of changes within the disk. I think it is the way SMR recording deals with this that causes the problem. A big backup can take up to 3 days which is patently ridiculous. I have a similar issue with one of my storage disks also a 4TB. So both these disks are being changed to CMR disks later today.
Re buying new stuff in NZ. We do use Amazon a lot here. Only problem is we end up subsidising Amazons, and their couriers, shareholders retirement funds with horrendous freight bills. Plus we get stung with 15% GST on top of it all. You only buy from Amazon if you cannot buy it here in NZ and MUST have it. Local business importers add generous markups onto imports claiming they also get big freight bills. The retailers do the same so stuff ends up costing a fortune. Hence demand drops and without demand they just don't import useful stuff like 4TD SSD's (any style) until demand increases. A catch22 situation. Now if NZ was considered a 'developing nation' like China then I am sure our freight costs would be a lot less. One of the justifications for 15% GST (other than shoveling money into the governments pockets) is to give an incentive to the population to buy locally. Which doesn't work however because the locals prices are either TOO high or they don't import an item. I buy more from overseas than locally because of this. NZ importers also have a habit of signing up exclusive deals with overseas manufacturers but out of say a range of items of 400 only import 3 of them. Guess which 3 I don't want. Efforts to go direct end up with emails from the manufacturer saying to see their local agent. Who will sometimes order the odd item for you but it takes 3 months. Stuff from China on slow mail only takes a month. Things do change but it takes 5 years. The word competition is not even understood by most importers or retailers here. All this is not helped by our low population of 5 odd million.

Reply
Mar 19, 2023 18:20:44   #
TriX Loc: Raleigh, NC
 
chrissybabe wrote:
I don't think my backups are unique. My wifes data disk is 4TB. She uses Adobe extensively both for Indesign and Photoshop. The disk is constantly being changed with deletions, file changes and file additions. There can be odd times when up to 1TB is changed between backups. I use bvckup2 which backs up the entire disk by creating a table of whats on the source, another table of the current destination, compares the two, backs up any change, any new file and deals with any deletions on either disk. So lots of changes within the disk. I think it is the way SMR recording deals with this that causes the problem. A big backup can take up to 3 days which is patently ridiculous. I have a similar issue with one of my storage disks also a 4TB. So both these disks are being changed to CMR disks later today.
Re buying new stuff in NZ. We do use Amazon a lot here. Only problem is we end up subsidising Amazons, and their couriers, shareholders retirement funds with horrendous freight bills. Plus we get stung with 15% GST on top of it all. You only buy from Amazon if you cannot buy it here in NZ and MUST have it. Local business importers add generous markups onto imports claiming they also get big freight bills. The retailers do the same so stuff ends up costing a fortune. Hence demand drops and without demand they just don't import useful stuff like 4TD SSD's (any style) until demand increases. A catch22 situation. Now if NZ was considered a 'developing nation' like China then I am sure our freight costs would be a lot less. One of the justifications for 15% GST (other than shoveling money into the governments pockets) is to give an incentive to the population to buy locally. Which doesn't work however because the locals prices are either TOO high or they don't import an item. I buy more from overseas than locally because of this. NZ importers also have a habit of signing up exclusive deals with overseas manufacturers but out of say a range of items of 400 only import 3 of them. Guess which 3 I don't want. Efforts to go direct end up with emails from the manufacturer saying to see their local agent. Who will sometimes order the odd item for you but it takes 3 months. Stuff from China on slow mail only takes a month. Things do change but it takes 5 years. The word competition is not even understood by most importers or retailers here. All this is not helped by our low population of 5 odd million.
I don't think my backups are unique. My wifes data... (show quote)

Wow! That is indeed a painful situation!

Reply
 
 
Mar 19, 2023 20:15:30   #
tcthome Loc: NJ
 
ELNikkor wrote:
Never had slow back-ups. Plenty of fruit & veggies...



Reply
Mar 19, 2023 21:51:46   #
Bret P Loc: California
 
TriX wrote:
Another answer is SSDs. Then the backup will take minutes instead of hours.


Yep. SSDs great for frequent, always-connected backups.

If used for archival backups, SSDs need to be connected and refreshed every few months or more. I lost a drive because it wasn't connected for 6 months, tech said to connect it once a month (best) or at least every 3 months.

Reply
Mar 20, 2023 07:57:15   #
Longshadow Loc: Audubon, PA, United States
 
Bret P wrote:
Yep. SSDs great for frequent, always-connected backups.

If used for archival backups, SSDs need to be connected and refreshed every few months or more. I lost a drive because it wasn't connected for 6 months, tech said to connect it once a month (best) or at least every 3 months.

SSDs are built with CCDs (Charge Coupled Devices), same as fobs. The charges in the memory cells will eventually bleed off unless the charge is replaced. How long? Don't know. When the charge in cell(s) bleeds off, the data becomes corrupt. I've had fobs last many months, but they wind up being connected periodically for use at some point. Ones that have been on the shelf for a year have not been used yet. Best to connect the device every month or three to be safe. It allows the charge to "rebuild".
I don't consider them as "long term" storage as HHDs would be.

Reply
Mar 20, 2023 11:18:36   #
TriX Loc: Raleigh, NC
 
The worry about charge “bleed off” was a consideration with older devices when SSDs were introduced, but not any more. Modern drives are spec’d anywhere from 2-50 years, but 2-5 years would be very conservative and 5-10 years is probably safe. Never the less, SSDs are not intended as an archive device. For that, the currently most reliable and long lived device is an MDisk, provided you don’t burn or break it. I’m any event, we’re talking about backup, and it’s hoped you backup more often than every 2-5 years

Reply
 
 
Mar 20, 2023 12:00:18   #
Longshadow Loc: Audubon, PA, United States
 
TriX wrote:
The worry about charge “bleed off” was a consideration with older devices when SSDs were introduced, but not any more. Modern drives are spec’d anywhere from 2-50 years, but 2-5 years would be very conservative and 5-10 years is probably safe. Never the less, SSDs are not intended as an archive device. For that, the currently most reliable and long lived device is an MDisk, provided you don’t burn or break it. I’m any event, we’re talking about backup, and it’s hoped you backup more often than every 2-5 years
The worry about charge “bleed off” was a considera... (show quote)

Good to know about the bleed off improvement. I don't follow technology enough to have seen that.

I find out the hard way that certain tags in HTML have been depreciated.
They no longer work....

Reply
Page <prev 2 of 2
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
General Chit-Chat (non-photography talk)
UglyHedgehog.com
Copyright 2011-2024 Ugly Hedgehog, Inc.