Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Crash & burn
Page <<first <prev 4 of 4
Nov 25, 2016 09:37:34   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
Desert Gecko wrote:
In response to some concerns/comments I read in this thread:
1. Amazon offers a cloud backup and I'm pretty sure Amazon is going to be around for a while, so bankruptcy & losing a backup is not a concern. Amazon's backup is free for Prime members, and contrary to comments in another thread I read here recently, it does include backing up RAW files. The backup is not automatic, though, and it is very slow. Plan on it taking many days or even a week or more running 24 hours/day to complete an initial backup.
2. According to an objective, comprehensive review by cloud backup company Backblaze, Hitachi makes the most reliable drives, by far. Take a look at this data: https://www.backblaze.com/blog/hard-drive-reliability-stats-q1-2016/
3. An external backup drive is a good idea but nearly 100% effective only if the drive is kept in a location away from the computer to protect against theft or damage from fire, flood, rampaging elephants, etc. To guard only against a hard drive failure, an extra internal drive used for backup is faster and easier; but to also guard against gremlins and whatnot, either a cloud backup or an external drive kept at work or wherever is necessary.
4. I think a second backup drive, or backup of a backup is really unnecessary. Odds of a primary disk and a backup disk failing simultaneously are very low, and if you backup regularly to an external drive you will know it is in working order before you need it. This is especially true if also using a cloud backup.

Having considered all of this, IMHO the best solution is a combination of an external drive backup and cloud backup.
In response to some concerns/comments I read in th... (show quote)


Backblaze's sampling is full of errors and misrepresentations - so most in the industry have dismissed it as irrelevant.

http://www.theregister.co.uk/2014/02/17/backblaze_how_not_to_evaluate_disk_reliability/

Backblaze seems to be drawing many of it's conclusions from a study done by Carnegie Mellon University in 2006 that came to similar conclusions. While I don't have access to their raw data, I am suspicious of their having done due diligence in collecting and analyzing the data.

http://www.pdl.cmu.edu/PDL-FTP/Failure/CMU-PDL-06-111.pdf


The reason for a second backup is that when you are restoring from a single backup drive, you are most vulnerable. If the data is really important, then the second backup is a really good idea.

Cloud backup is not ideal for backup. However it is really good for archiving your data, mainly for the reasons you state - it takes forever to upload and anywhere from forever to half-of-forever to restore.

None of this addresses what happens when a system drive fails - which is why I suggested an image taken of the system drive. Some software claims to be able to restore an image taken from one system to another dissimilar system. I have had mixed results with that. Often, a simple mirror will be sufficient. If you upgrade a stolen computer, and you have a copy of everything on the system drive, you can rebuild the system drive to a new drive, on a new computer with a new OS, by reinstalling everything.

I always suggest and build with Hitachi Ultrastar or Western Digital RE (if doing RAID) or WD Black if just doing one drive per volume. I find all three to be outstanding performers, and rarely break. All are warrantied for 5 yrs. For the same reason you should take Backblaze's analysis with a grain of salt, I personally have built over 3000 computers since 1983 and found the drives above (and their ancestors) to be reliable.

Reply
Nov 25, 2016 10:39:25   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
whitewolfowner wrote:
Storing your photos on your main drive is not a good reason fro the following:

The photos will slow everything down
main drives are usually not the largest ones in the system, since they have everything else on them, makes them extremely limited
The main drive is the most used and is the most likely to go down
For anyone who has any amount of photos, their library will shut down the mani drive.

Remember; to always back up your photos on at least two different drives. It's not if your drive will go down; it's when will your drive go down!
Storing your photos on your main drive is not a go... (show quote)


Photos on a drive don't slow a system down. Using more than 70% of the available drive space will. You won't likely see a performance hit between a drive that has 70% free space or one that has only 30% free. But at 20% free, it will start to struggle, and there will be lots of milliseconds spent trying to find a place to write some data - and they add up to a sluggishly performing drive. At 5% free, your drive will be really struggling, and will likely fail prematurely.

Main drives can be a big as you want. Is an 8 tb drive big enough for you? Nothing stopping the installation of a huge drive. More than likely you would partition it into more efficient smaller logical volumes, but saying that a main or system drive has size or functional limitations is not exactly correct.

If you have lots of data, all of your drives will be highly used. Most of the activity on the main drive will be reading, however, which is not particularly stressful on a drive. The data drives will have lots of read and write activity, which does fragment a drive, resulting in more wear and tear. So while the system drive is not a good place to store data, it's not because system drives fail prematurely, but rather the act of storing data on a system drive may cause a drive to fail prematurely.
If you keep 30% of a disk's capacity free, you won't "shut down the main drive" - run any drive to within 3 mb of capacity, and it will shut down - main drive or not.

Reply
Nov 25, 2016 10:49:40   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
Desert Gecko wrote:
Good point, and I did not mean to imply that you should store photos on the OS drive. I have several internal drives, the largest of which is a 2TB Hitachi I use as my primary photo storage drive. I use my OS drive for apps, docs, and miscellany. I use other drives for other purposes, such as other boot drives including an SSD for LR & PS only.

I tried to keep my reply simple, but now that it isn't, I also recommend a dual-boot system. I've experienced a couple of hard disk failures over the years, but since XP I've always ran a dual (or multiple) boot and I've been fortunate the failure wasn't on the primary drive so I was able to boot to the other drive and retrieve all of my data before losing any of it (the boot loader should not be on the primary drive, but even if it is, it's simple to bypass and boot to the working drive.)

And regarding the SSD, I like the performance very much, but after setting it up I read that it's as good or better to use an SSD for the scratch drive when using LR or PS, and that it's not necessary to actually boot to and run the software from the SSD to realize the fast performance. I haven't been able to test this, though, because when I boot to my primary platter drive and run LR or PS, they won't show the SSD as an option for a scratch disk, and I haven't bothered to troubleshoot.

Finally, you are correct about the eventuality of an HDD failure. Even the most reliable drives, according to Backblaze (Hitachi drives), have a failure rate around 1% per year.
Good point, and I did not mean to imply that you s... (show quote)


DG, I used an ASUS motherboard that came with Intel's Smart Response technology to do just that. I put a 60 gb SSD in my system as a cache for the 1 Tb mechanical drive and boot times after POST fell from 2 minutes to about 25 secs. Now that I have upgraded to a 1 TB SSD for booting, the boot time has fallen to under 18 secs. Using an SSD as cache is a great idea. For Photoshop, the best performance enhancing approach is to load your system with as much ram as you can. Most users can get away with 32 gb, but if you routinely do HDR, Pano, Focus Stacking, and have a D800/810, or a Canon 5DS or 5DSR. 64 gb is the way to go. While cost per megabyte is higher with the ram approach, so are the benefits, making it ultimately more cost effective. I don't think that LR benefits much from cacheing.

Reply
 
 
Nov 25, 2016 11:04:32   #
Desert Gecko Loc: desert southwest, USA
 
Gene51 wrote:
DG, I used an ASUS motherboard that came with Intel's Smart Response technology to do just that. I put a 60 gb SSD in my system as a cache for the 1 Tb mechanical drive and boot times after POST fell from 2 minutes to about 25 secs. Now that I have upgraded to a 1 TB SSD for booting, the boot time has fallen to under 18 secs. Using an SSD as cache is a great idea. For Photoshop, the best performance enhancing approach is to load your system with as much ram as you can. Most users can get away with 32 gb, but if you routinely do HDR, Pano, Focus Stacking, and have a D800/810, or a Canon 5DS or 5DSR. 64 gb is the way to go. While cost per megabyte is higher with the ram approach, so are the benefits, making it ultimately more cost effective. I don't think that LR benefits much from cacheing.
DG, I used an ASUS motherboard that came with Inte... (show quote)


Gene - That much RAM? I'm running with 16GB at the moment, but I wonder if I would notice a significant performance increase with more.

Regarding LR and cache, I believe you're right for typical LR developing, but when editing video, synching, or when doing "edit in PS" then coming back, I think it would also benefit from using an SSD for the scratch.

And I like your solution of a large SSD for both OS and cache. I presume you keep most of your data on another drive. Do you find that 1TB is enough cache for both your OS and LR/PS? Often when I'm running PS I also run video tutorials, a browser, and even Word, plus all the background apps like antivirus, etc.

Reply
Nov 25, 2016 13:04:15   #
nelsond
 
I am in my late 70s and want my 2 adult children to have access to all my pictures when I pass, both own their own homes, married more than 25 years. I have a 3TB and a 1TB external HD on my computer at all times. Every time I finish a shoot(birthday, holiday or daily when on vacation etc) before any post processing I make a file with date and short name and copy that date/name to each picture in the file, put a copy on the 1TB and keep a copy on the computer. At the end of each year I make a file on the 3TB with the year as the name(2015, 2016 or ?) and copy all of that years pics to it and delete most of them from the computer. Then give the 1TB with that years pictures to one child one year and the other the next year, then go buy a new 1TB and and start over again. If there is room left I add the previous years pic on it. YOU ALWAYS HAVE AT LEAST ONE BACK UP SOMEWHERE. Still using the old Windows 7. Can find any picture quickly by knowing the approximate date.

Reply
Nov 25, 2016 21:49:31   #
27ftWhaler
 
I've been in IT most of my career so I'm pretty anal about backups. Here's what I do. I have RAID 1 drives on the computer which houses my photos. After I download my pictures to these drives, I copy them to my Synology NAS (5 drive NAS). Every month I do a 0 dump of the primary RAID 1 drives to a separate external hard drive (not the NAS). This external drive holds 3 dumps and I overwrite the oldest dump at every backup. Annually I swap out the external drive and put that in my safe deposit box along with the backup software and restore instructions. So far, in 10 years or so I've had 3 RAID 1 drives fail and they rebuilt automatically when a new replacement drive was installed and once during the Windows 10 migration I needed to rebuild from one of my 0 Dumps. Drives will fail but so far I haven't lost anything.

Reply
Nov 28, 2016 10:39:00   #
JackB
 
grampy26 wrote:
Just learned that my external hard drive which held my pics is unrecoverable. Unfortunately I did not have all of them backed up some where else. Hard lesson to learn and am looking into the best and least expensive way to have multiple back ups.


I use a Drobo 5N unit. This is a RAID system or (Redundant Array of Independent Disks). There are a number of different brands on the market but my research led me to purchase the Drobo brand. You can check them out at drobo.com. Expensive to get into but can depend on what size of drives you install in the bays. Besides the safety of the mirror imaging, it allows me to have all of my photos and files in one place, when I use 3 different computers. The drobo will work on PC and Mac or a combination of them.

Reply
 
 
Nov 28, 2016 14:02:53   #
whitewolfowner
 
Gene51 wrote:
Photos on a drive don't slow a system down. Using more than 70% of the available drive space will. You won't likely see a performance hit between a drive that has 70% free space or one that has only 30% free. But at 20% free, it will start to struggle, and there will be lots of milliseconds spent trying to find a place to write some data - and they add up to a sluggishly performing drive. At 5% free, your drive will be really struggling, and will likely fail prematurely.

Main drives can be a big as you want. Is an 8 tb drive big enough for you? Nothing stopping the installation of a huge drive. More than likely you would partition it into more efficient smaller logical volumes, but saying that a main or system drive has size or functional limitations is not exactly correct.

If you have lots of data, all of your drives will be highly used. Most of the activity on the main drive will be reading, however, which is not particularly stressful on a drive. The data drives will have lots of read and write activity, which does fragment a drive, resulting in more wear and tear. So while the system drive is not a good place to store data, it's not because system drives fail prematurely, but rather the act of storing data on a system drive may cause a drive to fail prematurely.
If you keep 30% of a disk's capacity free, you won't "shut down the main drive" - run any drive to within 3 mb of capacity, and it will shut down - main drive or not.
Photos on a drive don't slow a system down. Using ... (show quote)




I hear you and you are right; I just didn't get as far into it as you did. Most operating drives are not that big unless the operator is computer savvy and chooses to use a big one for that purpose; for what reason I do not know. Photos use up memory space quite quickly and that is where I was coming from, not to say that it's just not wise. I'm running 13 drives all together with about 35TB total and do not think that elaborate; just sufficient and hopefully will serve me for the next few years without need to expand. I personally hols no value in cloud storage. I have learned a long time ago if you want something done right; do it yourself. and try downloading all your work at once, or even worse uploading it; more than a nightmare.

Reply
Nov 28, 2016 14:30:26   #
MW
 
I have two external drives. One is managed by Acronis and does automatic back ups. But a paranoid sort of person so the second drive is for manual back ups I do myself every week or so.

Reply
Page <<first <prev 4 of 4
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.