bpulv Loc: Buena Park, CA

Although what you said is all true, the considerations regarding another entity having control over your data can be an issue and, as you said, you can't be too careful. Even megacorporations have gone bankrupt in recent history. When that happens, the first thing that happens is that their assets are often frozen until the court figures out what belongs to the corporation and what does not belong to the corporation. That can take a very long time and during that time you might not have access to your data.

Costs that are not under your direct control must also be a consideration. A cloud provider can change its prices with only a thirty day notice. That can pose two problems for the photographer. First is the obvious issue of the additional cost that the photographer did not anticipate and second is the fact that if you decide to switch to another cloud service and you have thousands of large files on the cloud, there may not be time to download all your photos and upload them to the new service before the notice period from your old provider expires.

Regarding filing systems. The problem with using titles first is one you have or will quickly discover. Each event can be identified by many different possible names. That leads to similar a lack of uniformity in naming files because you will not necessarily remember the title you previously used for a similar event and, therefor the titles are constantly expanding without any way to relate one to another. Have you thought of using a list of key words? If you had a program that can search on a combination of key words, you could use key words as the title of each file and have a way to efficiently search.

TriX Loc: Raleigh, NC

9

With all due respect, there are a number of inaccuracies in this post, and it would be a shame to let those influence others that might benefit from good data protection strategies.

First, let’s address what cloud storage actually is. A cloud data center consists of redundant storage, servers, networking and power housed in hardened, secure data center managed by professionals and more importantly, if you choose a major provider such as Amazon, Google, Microsoft Apple, IBM, etc., there will be 3-5 copies of your data at geographically separate data centers to protect against a disaster at any one site. You cannot possibly even come close to this level of professional management, enterprise hardware, redundancy, power conditioning, fire suppression, security or multiple geographically separate copies.

And let’s put a couple of other pieces misinformation concerning the cloud to rest. There has NEVER been a failure, bankruptcy, or “data held hostage” at ANY Major cloud provider. In fact, only ONE second tier provider (Nirvanix) has gone under, and they provided their customers months of notice to move their data. Now if you choose some unknown third tier “cloud” storage such as “Supersynch”, you’re on your own. Can your data be compromised (hacked) in the cloud? Sure, IF you use poor security practice on your end such as single authentication, using the same password at multiple sites, sloppy computing practices such as opening strange emails or attachments, visiting insecure sites, etc - anything that can allow your credentials to be stolen or malware to corrupt your data which is then mirrored to your backups. Are hackers likely to target your data in the cloud? No. Hackers want nice neat databases with social security or credit card data, not wasting time going through ten thousand photos looking for a single nugget of useful information. Finally, many of those that decry the cloud conveniently forget that their really sensitive data and that of at least 2/3 of businesses as well as the data from the nation’s intelligence services (NSA, CIA...) is already in the cloud. Your SSN, credit card, medical records, stock/retirement accounts, any military records banking and tax information is, guess where? In the cloud.

Now let’s address the speed limitation of the internet. If you have a dog slow internet connection AND a huge amount of data, then cloud storage may not be workable for you, but there is a workaround. A number of cloud providers will allow you to provide a disk with your information 5o upload and will provide you with a disk with your information if you need to download the entire contents. Note also that once you have “seeded” the cloud, future uploads consist only of new and changed files and that can be done automatically after hours so it has no impact on you. And the idea that the cloud will wear out your HD uploading is a red herring. Unless you choose not to have ANY backup, then the data has to be read from your HD at some point, regardless as to whether it’s the cloud or local backup.

In short, don’t let this inaccurate fear mongering keep you from keeping a DR copy of your data. Keeping a drive off-site (in a lock box for example) has a number of downsides including the bank (or your relative’s house) being destroyed by fire or flood, the drive not starting (a large percentage of drive failures occur on start up after being down for long periods), the data that is lost between those trips to the bank and the necessity of actually going through this time wasting routine every week or every month. That’s all your data in a single drive that may or may not start. And don’t forget the convenience of being able to access your data from anywhere with a computer and an internet connection. In the week after your home (and both your primary data storage and backup) are destroyed by fire or flood and you need that insurance information and inventory/pictures of your property while staying in a hotel (perhaps without your car), that data is at your fingertips. Finally, the idea of archiving your data on USB drives (as mentioned) or on DVDs is fraught with disaster - neither was ever intended as long term storage and can fail over time.

I also want to address the limitations of RAID storage. A good RAID array can be either faster or more reliable (or both) than a single drive, but they do have limitations. First, there is this idea that each drive in a RAiD array contains all the data or that you can remove one drive and store it as a complete backup for all your data. Neither is true. Unless we’re discussing simple mirroring (RAID 1 or 0+1), with all other RAID levels (0,3,4 or 5 or so called “6”/double parity schemes), each drive contains only a PIECE of your data and /or parity information. If you want to back up a RAID array, you need as much storage as the total of all the drives in the array. Now if you lose a drive AND you have a spare that you insert or comes online automatically, then the drive can be rebuilt as a background task from the other drives. You can continue to access your data, but the performance is typically degraded by ~50%. During that degraded period while a drive is being rebuilt (which can take days with large drives), a second drive failure during that time results in data loss. Because of this, many RAID arrays allow dual parity drives which can handle dual drive failure, BUT then you need two spare drives to rebuild, and double drive failures DO occur, usually as the result of power line issues, lightning strikes or controller failures. Finally, there is no RAID “standard” except mirroring although the RAID levels are somewhat defined. What that means is if your controller board dies, you’ll need another array of the same type/mfg. to get to your data - you can’t simply move the drives to a different array and expect then to be read.

Whew! Sorry for the long post, but after 25+ years specializing in data storage, i’ve seen too much lost data by too many causes, so I’m preaching 3 copies of your data - primary, backup and off-site/DR, and don’t let fear of not “having control of your data’ scare you away from the cloud, which for most users, is by far the safest method of off-site storage.

Longshadow Loc: Audubon, PA, United States

Filing by date and titling the images COULD result in two different images with the same name in two or more different directories if one is not careful. For example: If I went to Daniel Boone Homestead on four different occasions, took pictures of the dam, and each time named them DBH dam-01, DBH dam-02, etc., those names could also exist in three other directories. Unless one adds the date to the file name, but that is already in the file information.

Then, in order to find them, I would have to remember the date for each trip, or do a search....
If they are all in one directory by subject, no search required or remembering when I went there.

Myself, I like to keep all of one subject together.

Dngallagher Loc: Wilmington De.

DirtFarmer Loc: Escaped from the NYC area, back to MA

I view that scenario as unlikely, but my main point is that the cloud storage is secondary. You should be depending primarily on your local archive. The cloud storage is for cases where your local storage is destroyed. Then and only then will you need the cloud storage. So all your cloud files disappear suddenly. You have not lost anything but your backup backup because all your files are stored locally. The worst part of that scenario is uploading your files to a new cloud service.



If you decide to move to a new cloud provider, you do not need to download all your files (of course you are archiving everything, not just the photos) because you already have them on your local archive. You do have to spend some time uploading it all to a new provider, but as far as the old provider is concerned, you just walk away.



Actually I should have mentioned that all the descriptive named folders are subfolders of a folder named by the year. So the path is ../YYYY/Suzie's Birthday/RAW, etc. That takes care of duplication, and even if I put them all in the same folder, the date and time are on the file names so there's no chance of duplicating the file name, just the descriptive text portion of the file name. Placing the date and time on the file names allows me to be sure they are in chronologiical order. Even if I'm using multiple bodies on a shoot. The time is given to hundredths of a second because that takes care of 14 frame/second bursts (infrequent but they happen).

My file naming system is more fully described in Photo organization via folders

Fredrick Loc: Former NYC, now San Francisco Bay Area

Excellent article. Thank you.

bpulv Loc: Buena Park, CA

I read your post and looked at their website that did not give a lot of detailed information. There is nothing that indicates if there is a Mac version of the software available. Do you know?

yssirk123 Loc: New Jersey

Thanks Dirt Farmer for the excellent and informative post you linked to which I missed when you originally posted it. I presently use the YYYY-MM-DD plus a description of the images. I'm going to look into Downloader Pro now.

DirtFarmer Loc: Escaped from the NYC area, back to MA

I use YYYYMMDD, but YYYY-MM-DD follows ISO 8601

Longshadow Loc: Audubon, PA, United States

YYYY-MM-DD reads faster than YYYYMMDD.

DirtFarmer Loc: Escaped from the NYC area, back to MA

Not sure what you mean by "reads faster". The computer reads it faster? The viewer reads it faster?

I picked that format several years before I heard about ISO-8601. I chose the format for compactness.

Longshadow Loc: Audubon, PA, United States

Viewer.

Fredrick Loc: Former NYC, now San Francisco Bay Area

I’m not sure what product you’re referring to? If you let me know I will be more than happy to respond.

ELNikkor

Always have 2 identical back-up drive sets. If one fails, immediately get a new one and create another duplicate from the functioning drive. 2 won't fail on the same day...

DirtFarmer Loc: Escaped from the NYC area, back to MA

2 failing at the same time will depend on the failure mode. A line surge could do it.

Keep one unplugged.

A house fire could do it.

Keep one off-site.