Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Check out Panorama section of our forum.
General Chit-Chat (non-photography talk)
External Solid State Drives
Page <<first <prev 3 of 4 next>
Nov 23, 2019 21:12:46   #
TriX Loc: Raleigh, NC
 
GENorkus wrote:
Rereading articles, it seems that I was incorrect, in a way.

Being that few "regular people" read and write as many times as the SSD's are rated, I guess the SSD do last. HDD are suppose to last forever but I would guess that the mechanical parts will go bad after a very long time.

For those who nearly fill their SSD, that tells me the unused portion will be used lots over and over again. That will wear the unfilled portion out much faster than is a much larger unused portion.

So to me that means either will store well enough for "us regular people", but not is the SSD is nearly filled to begin with.


Here are some quotes from the following articles:


1) https://www.exittechnologies.com/blog/it-tips/hard-drive-vs-ssd-vs-storage-system/

"HDD’s are more vulnerable to mechanical failure since they contain moving parts...
Usually, they will warn the user before failure as they start to make a lot of noise or load data slowly."

"Solid-state drives do not suffer from mechanical problems...
However, the electrical circuits that contain the data can wear out and fail. Writing data is harder on the drive than reading data.
One unique issue that SSDs have that hard drives don’t is electron wear.
When writing a 1 or a 0 to the same location repeatedly, that cell can begin to wear out. This is mitigated through a concept called wear leveling or provisioning.
Normally, an SSD fills up to 60 % and that data doesn’t change much, but the remaining 40% changes often as the user creates and deletes files.
Once a failure occurs in that overused 40%, the whole drive fails. The provisioning function reorganizes the information on the drive as needed. That way all sectors are being worked evenly, prolonging the life of the drive."


2) https://www.howtogeek.com/322856/how-long-do-solid-state-drives-really-last/

"A joint study between Google and the University of Toronto covering drive failure rates on data servers. The study concluded that the physical age of the SSD, rather than the amount or frequency of data written, is the prime determiner in probability of data retention errors. It also determined that SSD drives were replaced at Google data centers far less often than conventional hard drives, at about a one to four ratio. But it wasn’t all positive in favor of SSDs: they experienced higher uncorrectable errors and bad blocks at a much higher rate than hard drives over the four-year testing period. Conclusion: in a high-stress, fast-read environment, SSDs will last longer than hard drives, but be more susceptible to non-catastrophic data errors. Older SSDs are more prone to total failure regardless of TBW or DWPD."
Rereading articles, it seems that I was incorrect,... (show quote)


Good information. As an added data point, all modern versions of Windows issues the TRIM command with each write, which accomplishes wear leveling. I can’t speak for Macs.

Btw, I am now on my 6th year with eight Intel 500 series SSDs spread across several machines. I routinely run at least one drive (usually the system drive) >90% capacity, but it, like most consumer machines, doesn’t get nearly the type of access as a commercial server. It’s not good practice, but it seems that many of the aps and system files are constantly growing, and I get behind on reallocating files to other drives.

Reply
Nov 23, 2019 21:25:57   #
cjc2 Loc: Hellertown PA
 
I'm using a 2018 MBP with USB-C/TB ports and a Samsung T5 external SSD to store images. Very fast. Very Happy! Best of luck.

Reply
Nov 23, 2019 21:35:37   #
GENorkus Loc: Washington Twp, Michigan
 
photoman43 wrote:
My experience with external HDD drives, 2 TB and 4 TB models , is not to rely on them after two years as failures can occur and have incurred for me. I try not to bang mine around, but I still have failures. And I try and never have any drive more than 75% full.


I have 4tb Seagate units. They're used about five or so years and I fill them to about 90 to 95% before retiring them.

They have only been used for backup and never really been turned on for more than about 15 minutes at a time excwpt for when I need to reference something from the past. That rarely needs more than an hour and doesn't happen very often. I can probably count the times on one hand.

In my estimation, both my drives are equal to less than 100 days total, per year old. As I mentioned both 4TB drives only cover 5 to 7 years so that is about 5 to 7 hundred hours old. Everything seems to work well.

They are external drives not internal so I can do the low hour mark very easily. Only two internal drives have gone bad for me in over 45 years. (The older ones weren't 4tb. I was lucky to have 1 megabit of storage back then. LoL)

Most of my friends leave their external drives on 24/7, (some kind of raid number?), and have mentioned an occaisional drive going bad.

Reply
Check out The Dynamics of Photographic Lighting section of our forum.
Nov 23, 2019 21:49:29   #
GENorkus Loc: Washington Twp, Michigan
 
TriX wrote:
Good information. As an added data point, all modern versions of Windows issues the TRIM command with each write, which accomplishes wear leveling. I can’t speak for Macs.

Btw, I am now on my 6th year with eight Intel 500 series SSDs spread across several machines. I routinely run at least one drive (usually the system drive) >90% capacity, but it, like most consumer machines, doesn’t get nearly the type of access as a commercial server. It’s not good practice, but it seems that many of the aps and system files are constantly growing, and I get behind on reallocating files to other drives.
Good information. As an added data point, all mode... (show quote)


In my Intel desktop I added an Intel 545s, 500GB. Love my Intel!!!! The 545s is for temporary storage and small work only.

On average when it gets about a month's storage on it, 40% filled or so, I download to my main 4TB hhd and start over with the ssd.

Although not exact, that is about every two weeks. (That keeps low hours on my main backup too!)

Reply
Nov 23, 2019 22:29:13   #
TriX Loc: Raleigh, NC
 
GENorkus wrote:
In my Intel desktop I added an Intel 545s, 500GB. Love my Intel!!!! The 545s is for temporary storage and small work only.

On average when it gets about a month's storage on it, 40% filled or so, I download to my main 4TB hhd and start over with the ssd.

Although not exact, that is about every two weeks. (That keeps low hours on my main backup too!)


Yep, I have several 545s. When I first started moving to SSDs when they were relatively new (and some companies were having high failure rates), from our studies at the HPC storage company I worked for, Intels were the most reliable SSD we tested, so I have been using them ever since with excellent results (so far).

Reply
Nov 24, 2019 00:46:02   #
nadelewitz Loc: Ithaca NY
 
lamiaceae wrote:


To me in ideal system would be a "tower" PC with a SSD and two paired (likely RAID configured) internal HDDs for data use and archiving. Long term archiving / back-up can be to paired duplicated External HDDs. This should provide data safety without breaking the bank.


You have a common misconception about what RAID is for. RAID is not for backing-up or archiving. People erroneously think that having a RAID mirror setup is the same thing as one drive backing up another. It isn't so. That's what external drives would be for. RAID is for multiple drives acting as ONE DRIVE, or redundancy, meaning if a drive in a RAID array fails, it gets replaced and the RAID controller configures the new drive to take its place in the array. A RAID array should be thought of as ONE DRIVE, not drives backing up one another.

A drive out of a RAID array cannot be read by another computer by, say, connecting it via USB. A RAID drive can only be read by the controller (hardware or software) that created it. A proprietary formatting system is used in a RAID system. I learned this when I first set up my NAS. I used two drives as a RAID mirror thinking just as you are...the primary drive fails, just read the second drive on another machine. The drive COULD NOT BE READ on another machine. I re-built my NAS with just extra USB drives, formatted normally, and the NAS operating system syncs the extra drives with the primary.

What you CAN do is have two or more drives in your computer, and use a drive synchronization program to duplicate the primary drive to other internal drives. Or, have just one data drive in the computer, and sync it with as many externals as you like for archival/backup purposes.

Reply
Nov 24, 2019 02:12:57   #
GENorkus Loc: Washington Twp, Michigan
 
nadelewitz wrote:
You have a common misconception about what RAID is for. RAID is not for backing-up or archiving. People erroneously think that having a RAID mirror setup is the same thing as one drive backing up another. It isn't so. That's what external drives would be for. RAID is for multiple drives acting as ONE DRIVE, or redundancy, meaning if a drive in a RAID array fails, it gets replaced and the RAID controller configures the new drive to take its place in the array. A RAID array should be thought of as ONE DRIVE, not drives backing up one another.

A drive out of a RAID array cannot be read by another computer by, say, connecting it via USB. A RAID drive can only be read by the controller (hardware or software) that created it. A proprietary formatting system is used in a RAID system. I learned this when I first set up my NAS. I used two drives as a RAID mirror thinking just as you are...the primary drive fails, just read the second drive on another machine. The drive COULD NOT BE READ on another machine. I re-built my NAS with just extra USB drives, formatted normally, and the NAS operating system syncs the extra drives with the primary.

What you CAN do is have two or more drives in your computer, and use a drive synchronization program to duplicate the primary drive to other internal drives. Or, have just one data drive in the computer, and sync it with as many externals as you like for archival/backup purposes.
You have a common misconception about what RAID is... (show quote)


I don't use raid anything but was told some raids can by read by other similar computers.
Raid 6 comes to mind.

Reply
Check out Digital Artistry section of our forum.
Nov 24, 2019 02:29:35   #
nadelewitz Loc: Ithaca NY
 
GENorkus wrote:
I don't use raid anything but was told some raids can by read by other similar computers.
Raid 6 comes to mind.


Here's one discussion of this:

https://superuser.com/questions/83923/can-a-mirrored-raid-1-disk-be-plugged-into-another-system-to-be-read

You can find a lot more with a web search for "read RAID drive on another computer".

There's a lot of "it depends" involved. Not a simple question or answers. Safe bet is to not assume you can do it.

Reply
Nov 24, 2019 08:57:47   #
TriX Loc: Raleigh, NC
 
nadelewitz wrote:
Here's one discussion of this:

https://superuser.com/questions/83923/can-a-mirrored-raid-1-disk-be-plugged-into-another-system-to-be-read

You can find a lot more with a web search for "read RAID drive on another computer".

There's a lot of "it depends" involved. Not a simple question or answers. Safe bet is to not assume you can do it.


In general, I agree (and thanks for pointing this out). One exception is a SW RAID 1 mirror managed by Windows. If both drives are NTFS (for example) and are written by Windows, the DATA should be readable by another Windows machine. That doesn’t mean though, that you can plug the drive into a different Windows machine and boot the OS from it (assuming you mirrored the OS as well).

Another common RAID misconception is that you can remove one drive from RAID set (redundancy group), such as a RAID 5, periodically, store it off-site as a DR copy (occasionally swapping it for a 2nd spare drive which the RAID will then rebuild). Except for RAID 1 (simple mirroring), the removed drive only contains a portion of your data. I have had this point argued to the death by a RAID user who believed that somehow, magically, all the data from the other drives was somehow contained on this single drive, which could then be used to restore all the data in the event of a RAID failure. He was dutifully swapping out one drive and storing it off site (essentially creating a drive failure in the RAID set) and repeating this operation regularly, alternating the drives. Fortunately, he had not had to test this method in the event of a real failure.

Reply
Nov 24, 2019 13:54:10   #
nadelewitz Loc: Ithaca NY
 
TriX wrote:
In general, I agree (and thanks for pointing this out). One exception is a SW RAID 1 mirror managed by Windows. If both drives are NTFS (for example) and are written by Windows, the DATA should be readable by another Windows machine. That doesn’t mean though, that you can plug the drive into a different Windows machine and boot the OS from it (assuming you mirrored the OS as well).

Another common RAID misconception is that you can remove one drive from RAID set (redundancy group), such as a RAID 5, periodically, store it off-site as a DR copy (occasionally swapping it for a 2nd spare drive which the RAID will then rebuild). Except for RAID 1 (simple mirroring), the removed drive only contains a portion of your data. I have had this point argued to the death by a RAID user who believed that somehow, magically, all the data from the other drives was somehow contained on this single drive, which could then be used to restore all the data in the event of a RAID failure. He was dutifully swapping out one drive and storing it off site (essentially creating a drive failure in the RAID set) and repeating this operation regularly, alternating the drives. Fortunately, he had not had to test this method in the event of a real failure.
In general, I agree (and thanks for pointing this ... (show quote)


An example the opposite of your Windows-managed RAID 1 array experience:

My NAS, based on a Linux OS (OpenMediaVault), configured the RAID drives (software RAID) with a proprietary file system, NOT with a standard Linux file system (ext3 or 4). The resulting drives could not separately be read on a Linux machine. They were only usable in the array on the NAS.

I researched this phenomenon and learned RAID was not the solution for plain-old backing up, which is what the uninformed (like I was) think they are doing.

I rebuilt the NAS with the network-accessed drive in Linux format (which OMV recommends), and multiple USB-connected drives formatted as NTFS, which are the backups that are synced with the main drive. The backup drives can be read on any computer...Windows, Linux, or Mac.

Reply
Nov 24, 2019 15:54:16   #
TriX Loc: Raleigh, NC
 
nadelewitz wrote:
An example the opposite of your Windows-managed RAID 1 array experience:

My NAS, based on a Linux OS (OpenMediaVault), configured the RAID drives (software RAID) with a proprietary file system, NOT with a standard Linux file system (ext3 or 4). The resulting drives could not separately be read on a Linux machine. They were only usable in the array on the NAS.

I researched this phenomenon and learned RAID was not the solution for plain-old backing up, which is what the uninformed (like I was) think they are doing.

I rebuilt the NAS with the network-accessed drive in Linux format (which OMV recommends), and multiple USB-connected drives formatted as NTFS, which are the backups that are synced with the main drive. The backup drives can be read on any computer...Windows, Linux, or Mac.
An example the opposite of your Windows-managed RA... (show quote)


Just to clarify. Let’s not confuse anyone with the two separate issues - the file system (NTFS in this case) and the manner in which a RAID system lays down data. The data on an NTFS formatted drive (or a FAT32 or...) can be read by any platform that supports that file system IF it is a plain, unstriped and unencoded file. The individual files can also be read in the same way IF they were written as a true mirror (RAID 1) of the original.

NAS and RAID arrays often use a proprietary file system, so only a machine using the same file system can read them. Secondly, since the data is byte or block striped as is true of other RAID levels, the RAID controller managing the array must use the same algorithm (and settings such as block size) as the controller that wrote the data. The net-net is that for levels other than RAID 1, the same type of RAID controller, using the same algorithm, settings and RAID level, must be used to read and reconstruct the original data.

Bottom line: simple mirroring (RAID1) is an exception to the above IF and ONLY IF the file system of the written data is capable of being read by the system doing the reads. NTFS is a relatively universal FS in that it can generally be read by Windows, Macs and Linux platforms.

Reply
Check out Bridge Camera Show Case section of our forum.
Nov 24, 2019 17:30:19   #
nadelewitz Loc: Ithaca NY
 
TriX wrote:
Just to clarify. Let’s not confuse anyone with the two separate issues - the file system (NTFS in this case) and the manner in which a RAID system lays down data. The data on an NTFS formatted drive (or a FAT32 or...) can be read by any platform that supports that file system IF it is a plain, unstriped and unencoded file. The individual files can also be read in the same way IF they were written as a true mirror (RAID 1) of the original.

NAS and RAID arrays often use a proprietary file system, so only a machine using the same file system can read them. Secondly, since the data is byte or block striped as is true of other RAID levels, the RAID controller managing the array must use the same algorithm (and settings such as block size) as the controller that wrote the data. The net-net is that for levels other than RAID 1, the same type of RAID controller, using the same algorithm, settings and RAID level, must be used to read and reconstruct the original data.

Bottom line: simple mirroring (RAID1) is an exception to the above IF and ONLY IF the file system of the written data is capable of being read by the system doing the reads. NTFS is a relatively universal FS in that it can generally be read by Windows, Macs and Linux platforms.
Just to clarify. Let’s not confuse anyone with the... (show quote)


Not sure what you are clarifying.

1. My RAID 1 system did not use NTFS. It used a proprietary format. Okay, some RAID 1 systems may use NTFS, but this is definitely NOT a standard practice. And that is the whole issue with attempting to use RAID 1 as a backup strategy.

2. Your bottom line is no different than what has already been said.

We aren't in disagreement here. My whole point is that for the person who maybe doesn't understand all this, and doesn't need to, the big takeaway is.....avoid RAID as a simple backup method. Use all drives with NTFS (with Windows), and backup the primary drive to others using a syncing program.

Reply
Nov 24, 2019 21:22:40   #
TriX Loc: Raleigh, NC
 
T
nadelewitz wrote:
Not sure what you are clarifying.

1. My RAID 1 system did not use NTFS. It used a proprietary format. Okay, some RAID 1 systems may use NTFS, but this is definitely NOT a standard practice. And that is the whole issue with attempting to use RAID 1 as a backup strategy.

2. Your bottom line is no different than what has already been said.

We aren't in disagreement here. My whole point is that for the person who maybe doesn't understand all this, and doesn't need to, the big takeaway is.....avoid RAID as a simple backup method. Use all drives with NTFS (with Windows), and backup the primary drive to others using a syncing program.
Not sure what you are clarifying. br br 1. My RAI... (show quote)


I believe if our members are sophisticated enough to understand digital photography, I think they are sophisticated enough to understand the basics of good data storage. I believe it’s important because all the time and money we spend on photographic skills and equipment has a single purpose - to produce data, and that data, if it’s worth producing, is worth preserving/protecting. My attempt was to clarify the difference between file systems and storage systems, which seem to be intertwined in the last few posts. To that end, i’ll simplify, and if you (or others) find this redundant, then ignore it.

1) A RAID system of any level other than RAID 0 is more robust than a single drive, and it can be used for primary storage or backup. In fact, if your backup is larger than the capacity of a single drive, you’re going to use multiple drives anyway, so why not add some protection against a drive failure.

2) Windows has built in RAID functionality, which is simple to use, so if you’re using Windows, as probably 80+% of our users do, then mirroring (RAID 1) is your first best line of protection against a drive failure, but it requires 2x the storage. It Can be your backup, but has several potential problems. First, if the platform crashes, you can’t access either drive, so backing up to a separate platform is an advantage in that respect. Secondly, depending on how you set it up, a corruption or virus or accidental deletion on the primary drive propagates to the secondary mirrored drive. This is true regardless of whether the drives are on the same or a different system (with some qualifications). The moral is to use a 3rd off-site disaster recovery (DR) copy.

3) a simple, unencrypted NTFS or FAT 32 file, whether produced by a mirror (RAID 1) or not can be read by a Windows, Mac or Linux system (if it supports NTFS or FAT 32) regardless of whether it’s the system that wrote the data or not.

4) if it’s any other RAID level or a NAS system with a proprietary file system (which most are), all bets are off, and you may need the same type and configuration storage system (and file system) to read the data.

Reply
Nov 25, 2019 08:31:09   #
tripsy76 Loc: Northshore, MA
 
I have a few Gnarbox 1T SSD’s and it has made a world of difference in my production work both for photo and video. They are super fast, and the ability to dump without the need for a computer helps a lot. On the down side they are expensive, but personally I wouldn’t go back to a spinning disk again. There are some great SSD’s out there and the prices are coming down. There may even be some good Black Friday deals!

Reply
Nov 25, 2019 08:45:24   #
Bison Bud
 
Man, I love it when the "Techies" come out like this! I started this thread wondering if a USB port would slow down the performance of a new SSD drive used in an external application and got good response on that subject. I certainly didn't know that writing to an SSD drive can actually wear it out and hadn't considered that data recovery of a crashed drive may not be possible with an SSD. Since my primary purpose is long term backup and data storage and speed is not the highest priority, I think I will stick to a mechanical drive for now and appreciate all the input. Also, the later discussion on Raid systems has been very interesting and informative as well and I've learned a lot more than I thought I would on drives in general due to these discussions. My thanks all that responded here for their input and good luck and good shooting to all.

Reply
Page <<first <prev 3 of 4 next>
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
General Chit-Chat (non-photography talk)
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.