Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Thumb drive question
Page <<first <prev 3 of 4 next>
May 20, 2021 16:46:40   #
JimH123 Loc: Morgan Hill, CA
 
jackm1943 wrote:
Can you enlighten me as to what the issues are? I was seriously considering a couple of 2T flash drives for archival storage but will hold off if there are major issues. Thanks.


I'm retired now, but I worked for a company that made the testers that are used in manufacturing the flash memories, and I used to write the test programs to completely test them.

All Flash memories are not good for archival storage. They all are made up of small cells (FETs) that hold a charge on a floating gate or no charge on that gate to represent a '1' or '0'. Older flash memories actually were better at long term storage than newer flash memory. And this is because there is such a push for denser (more) memory per chip and faster speeds. The result is that individual cells have shrunken in size to the point that only a few electrons are needed on the gate of a FET for it to be in a charged state. And if that charge is lost, the cell becomes the opposite state. To compensate for that, elaborate Error Correction Schemes have been added so that it takes multiple bad cells for the error to be seen.

There are only a small number of manufacturers that make flash memory. The manufacturing equipment is very expensive and the process is difficult.

As for thumb drives, they may be manufactured by a company that also makes the flash memory chips. But they are also manufactured by companies that do not manufacture memory chips. In this case, they buy what is called known good die from one of the companies that do manufacture the memory chips and do their own packaging which includes a memory controller chip that interfaces the memory chips to a format that your computer can talk to. I have seen more problems with these memory controller chips than I have actually seen from the memory chips.

When memory chips are manufactured, the test equipment can grade them for speed, number of bad sectors, etc., so that they can be sold at premium or non premium prices.

Bottom line is that flash memory storage is not forever. Failures tend to accumulate, with error correction hiding failures until some threshold is passed. The major failure mode of memory chips is corruption of data. The major failure mode the memory manager chips is total failure of the entire package.

Also, SSDs are much more expensive than hard drives. And they also aren't forever.

Reply
May 20, 2021 16:55:30   #
jackm1943 Loc: Omaha, Nebraska
 
DirtFarmer wrote:
Hold off. Forever. There are MAJOR issues.

There have been scam advertisements of 2TB flash drives out there for a couple years now. They DO NOT hold 2 TB, no matter what the computer might tell you about the space available. If you're lucky you can get a couple GB onto them before they start erasing old files to make room for new ones.

I would not say it's impossible to build a 2TB flash drive, but it won't be cheap. External hard drives are cheaper and more reliable for that volume of data.

And whatever you decide to do, follow the main rules for backup: Duplicate, Distribute, Validate.
Hold off. Forever. There are MAJOR issues. br br... (show quote)


Thanks for the info Dirt Farmer.
JackM

Reply
May 20, 2021 16:57:45   #
jackm1943 Loc: Omaha, Nebraska
 
JimH123 wrote:
I'm retired now, but I worked for a company that made the testers that are used in manufacturing the flash memories, and I used to write the test programs to completely test them.

All Flash memories are not good for archival storage. They all are made up of small cells (FETs) that hold a charge on a floating gate or no charge on that gate to represent a '1' or '0'. Older flash memories actually were better at long term storage than newer flash memory. And this is because there is such a push for denser (more) memory per chip and faster speeds. The result is that individual cells have shrunken in size to the point that only a few electrons are needed on the gate of a FET for it to be in a charged state. And if that charge is lost, the cell becomes the opposite state. To compensate for that, elaborate Error Correction Schemes have been added so that it takes multiple bad cells for the error to be seen.

There are only a small number of manufacturers that make flash memory. The manufacturing equipment is very expensive and the process is difficult.

As for thumb drives, they may be manufactured by a company that also makes the flash memory chips. But they are also manufactured by companies that do not manufacture memory chips. In this case, they buy what is called known good die from one of the companies that do manufacture the memory chips and do their own packaging which includes a memory controller chip that interfaces the memory chips to a format that your computer can talk to. I have seen more problems with these memory controller chips than I have actually seen from the memory chips.

When memory chips are manufactured, the test equipment can grade them for speed, number of bad sectors, etc., so that they can be sold at premium or non premium prices.

Bottom line is that flash memory storage is not forever. Failures tend to accumulate, with error correction hiding failures until some threshold is passed. The major failure mode of memory chips is corruption of data. The major failure mode the memory manager chips is total failure of the entire package.

Also, SSDs are much more expensive than hard drives. And they also aren't forever.
I'm retired now, but I worked for a company that m... (show quote)


Thanks for this info Jim. What would be your recommendation for archival stoage?
JackM

Reply
 
 
May 20, 2021 17:00:10   #
FotoHog Loc: on Cloud 9
 
DirtFarmer wrote:
Hold off. Forever. There are MAJOR issues.

There have been scam advertisements of 2TB flash drives out there for a couple years now. They DO NOT hold 2 TB, no matter what the computer might tell you about the space available. If you're lucky you can get a couple GB onto them before they start erasing old files to make room for new ones.

I would not say it's impossible to build a 2TB flash drive, but it won't be cheap. External hard drives are cheaper and more reliable for that volume of data.

And whatever you decide to do, follow the main rules for backup: Duplicate, Distribute, Validate.
Hold off. Forever. There are MAJOR issues. br br... (show quote)

Seems to me that capacity was not raised as the major issue with flash drives. . . .

Reply
May 20, 2021 19:24:18   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
FotoHog wrote:
Seems to me that capacity was not raised as the major issue with flash drives. . . .


It was not raised by the OP, but another poster was considering some 2TB thumb drives. I should have made it clear that I was addressing that peripheral concern and not the OP's question.

Reply
May 20, 2021 19:37:40   #
JimH123 Loc: Morgan Hill, CA
 
jackm1943 wrote:
Thanks for this info Jim. What would be your recommendation for archival stoage?
JackM


Multiple hard drives and as technology changes, to transfer it onto new technology as it is proven dependable so that you don't end up with old technology storage that can't be read any longer.

Reply
May 20, 2021 19:55:36   #
FotoHog Loc: on Cloud 9
 
DirtFarmer wrote:
It was not raised by the OP, but another poster was considering some 2TB thumb drives. I should have made it clear that I was addressing that peripheral concern and not the OP's question.

Fair enough . . .

Reply
 
 
May 20, 2021 21:03:24   #
jackm1943 Loc: Omaha, Nebraska
 
JimH123 wrote:
Multiple hard drives and as technology changes, to transfer it onto new technology as it is proven dependable so that you don't end up with old technology storage that can't be read any longer.


Thanks JimH. The "old technology storage" issue is a concern, that's why I'm not considering any discs. I use a 2T external drive for primary storage (I don't have any images in the actual computer) and have another 2T external drive not in current use. Looks like I need to put it into use.
JackM

Reply
May 20, 2021 22:21:32   #
TriX Loc: Raleigh, NC
 
jackm1943 wrote:
Thanks JimH. The "old technology storage" issue is a concern, that's why I'm not considering any discs. I use a 2T external drive for primary storage (I don't have any images in the actual computer) and have another 2T external drive not in current use. Looks like I need to put it into use.
JackM


Hard drives for archive storage? Backup, yes - archive, no. I’f you leave them running, it’s not if they’ll fail, it’s when; and if you don’t they may or may not start (a high percentage of drive failures occur on start up).

Reply
May 21, 2021 01:23:05   #
kemert
 
I've had one go through the washing machine and still work... But you are correct you should have another backup of important things. Example: I had a 4GB hard drive that I rarely used just fail for no reason. I was able to retrieve some things, but not all. Also, I've been hit by ransomware attacks - twice.

BTW, the only thumb/flash drive that has failed on me was physical damage when I had it in my laptop and my laptop was in the passenger seat and I stepped on the brakes. (I know there's some type of grammatical error in that sentence. heh...) Fortunately, only the flash drive broke when the laptop went flying.

Reply
May 21, 2021 10:25:22   #
RWR Loc: La Mesa, CA
 
TriX wrote:
The common sense would be to do what data storage professionals working for professional IT organizations do and that is NOT use Flash drives for backup or archive.

My real images are transparencies and negatives, which I digitize to print (I do not care for darkroom work). The digital imitations are stored on flash drives and two auxiliary hard drives, one of which is kept in my bank deposit box and is updated weekly. I’ve never had a failure, but I do replace the off-site drive every three years or so.

Reply
 
 
May 21, 2021 12:53:00   #
trapper1 Loc: Southside Virginia
 
Wow, when I posed my original question I never even began to expect the outpouring of information that has taken place in response to my question. Thanks to all who responded with such accurate and helpful comments and technical expertise. It was a real learning experience for me, and I hope for others. Again, thanks to all who responded.

Reply
May 21, 2021 22:46:23   #
trapper1 Loc: Southside Virginia
 
Back again. I sent SanDisk (all of my cards are SanDisk) an e-mail and asked them point-blank about the long-term storage reliability of their thumb drives. The answer came right away, sort of, hemming and hawing, and sort of tap-dancing around the question, but their answer was their thumb drives have a five-warranty and that the user should always have a back-up for the thumb drive. I guess that this means CD of DVD discs but the question comes up why not use a thumb drive and every four to five years copy the data/images to a new thumb drive with a further five years life, and so ad infinitum? You can copy the contents of one thumb drive to another, can't you?

Reply
May 21, 2021 22:52:43   #
TriX Loc: Raleigh, NC
 
trapper1 wrote:
Back again. I sent SanDisk (all of my cards are SanDisk) an e-mail and asked them point-blank about the long-term storage reliability of their thumb drives. The answer came right away, sort of, hemming and hawing, and sort of tap-dancing around the question, but their answer was their thumb drives have a five-warranty and that the user should always have a back-up for the thumb drive. I guess that this means CD of DVD discs but the question comes up why not use a thumb drive and every four to five years copy the data/images to a new thumb drive with a further five years life, and so ad infinitum? You can copy the contents of one thumb drive to another, can't you?
Back again. I sent SanDisk (all of my cards are Sa... (show quote)


I think Sandisk has answered the question for you. Just because they will replace it if it fails doesn’t mean they will replace the lost data.

And a general comment on warranties in general. A long warranty is not necessarily a reflection of the quality of a product. More often, it’s a marketing tool.

Reply
May 22, 2021 00:11:21   #
TriX Loc: Raleigh, NC
 
Here are 10 rules for preserving your most valuable photographic asset - your data.

1) you need 3 copies of your data - working, local backup and off-site disaster recovery
2) cheap consumer grade HDs, flash drives and CDs/DVDs are not reliable storage
3) the storage of choice for working storage is SSDs for the applications and OS as a minimum. It’s also fine for data unless you have multiple TBs and the cost per TB is prohibitive. In that case an enterprise class drive from HGST such as the UltraStar is the preferred choice
4) for external drives, either USB connected SSDs or an enterprise class HD that you choose and package in a fan cooled enclosure is preferred.
5) for backup, an enterprise drive, ideally on a separate system. You can also use MDisks, which are very robust, but are slower and limited to 100GB each.
6) for Archive, MDisks are the answer.
7) for the off-site disaster recovery copy, the cloud from a MAJOR cloud provider is the best choice. HDs and MDisks in a safe deposit box or another physical location is the second choice.
8) RAID systems for either working or backup can increase capacity and/or reliability over a single drive. RAID groups created by the OS on a JBOD (just a bunch of drives) enclosure with fan cooling is preferable to a HW stripped RAID.
9) a mirror is preferred over a compressed copy for backup. You must test your backup
10) you need to understand the deletion policy on your backup SW and cloud storage. Either the backup or DR copy should support versioning.

Reply
Page <<first <prev 3 of 4 next>
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.