gwilliams6

Still doing my search for the article on the international cloud service whose crash lost millions of files worldwide. But in the meantime I came across these articles of interest:

Is Cloud Storage Data Loss Possible? According to Microsoft, Yes It Is.

https://weareproactive.com/cloud-storage-data-loss-is-possible/

"Cloud crashes and outages are becoming increasingly common. And as more businesses rely more heavily on cloud computing, the cost of cloud crashes and outages to those businesses will continue to increase as well. Even the biggest cloud providers aren't immune to cloud crashes and outages. Sep 23, 2021 Reciprocity.com "


7 Biggest Cloud Outages Of The Past Year
By Twain Taylor / February 11, 2022
https://techgenix.com/7-biggest-cloud-outages-services-2021/


"Sometimes technology fails — computers freeze and backup copies are lost. Other times, servers crash and the information contained within is lost. These are all potential sources of data loss, and the cloud is not exempt from technology failure or human error.Jul 14, 2020"

TriX Loc: Raleigh, NC

Completely agree on moving to SSDs - I no longer use any spinning disk. I’m puzzled when professional photographers like yourself, whose data is the result of all your labor and expensive camera equipment, are willing to spend 6k$ for a camera body but feel that $75/TB is too much to spend for the most reliable storage to protect that investment, or trust it to teenagers whose experience is typically playing with computers in their basement. That is a decidedly unprofessional approach to your data. I reiterate that you need a PROFESSIONAL organization to manage your data if you don’t have the time or expertise, and the Geek Squad and Best Buy isn’t it, and Seagate consumer grade drives certainly aren’t.

If you took the time to look at the quarterly report from Backblaze that Jerry posted earlier in this thread, you’ll see how mistaken you are about Seagate and WD reliability. Who do you think knows more about drive reliability - the opinion of a kid working at Best Buy, or computer professionals managing Exabytes of storage and keeping accurate records and statistical analysis? That report and those from previous quarters should show you that HGST is making the most consistently reliable drives in the world, and that the new generations of WD drives aren’t bad either. Also, if you look at the whole report, you’ll see that SSDs are even more reliable. If you must use spinning disk, buy HGST enterprise class drives (NOT consumer class).

Now concerning cloud storage. You and everyone else who are constantly spreading horror stories about loss of data or data breaches in the cloud, need to check your facts. There has been ONE mid sized real cloud company that has gone belly up in the last 20 years, and even then, customers were given 60 days to move their data. A MAJOR cloud provider has NEVER gone belly up or lost customer data unless the customer did something VERY stupid. Remember three indisputable facts: (1) you cannot possibly implement a storage system that’s even remotely close to the reliability and redundancy of the cloud (2) your cloud storage is for disaster recovery when your primary storage AND local backup fail (3) cloud storage from major providers is so secure that >80% of companies use it, including all the 3 letter national intelligence agencies and the DOD. All your personal and financial data is already stored in the cloud. Why? Because IT professionals realize that they can’t implement storage this reliable and secure for anything near the price. So stop scaring people away from the cloud which supplies safe secure storage and versioning, which is critical. BUT, use a MAJOR cloud provider, not one of the 100+ mom and pop providers with a single data center.

You should have a primary copy, a local backup copy, and an off-site disaster recovery copy of your data. That is data storage industry best practice, and it should be stored on the most reliable media available, because all your work and all your equipment is worthless if your data is gone

So why am I so adamant about the above? Because the week doesn’t go by on UHH that some poor soul hasn’t lost their lifetime of data because of ignoring the above. This is the best free advice you’ll get - it is based on 50+ years of designing and selling storage systems for the largest companies in the world, starting with IBM and ending with Oracle and including the largest/fastest supercomputers in existence. You regularly remind us of your photography bona fides, so those are mine.

Cheers and best wishes to you as well.

TriX Loc: Raleigh, NC

You’ll be looking a long time.

Now let’s take a look at those “7 outages”. Was data lost or temporarily off line? BIG difference! Having storage inaccessible for a few minutes is one thing, losing data is another. And certainly small cloud services with a single data center can have their own disaster, but the major providers have multiple data centers and your data is stored at 3-5 geographically seperate data centers so if a natural disaster occurs, there are still other copies. Can your local storage do that? Is your data stored in professionally managed, hardened data centers with redundant storage, servers, networking and power? No way. Do you have versioning? Do you know what that is and why it’s important?

Again, cloud storage for the single user is a last resort if both primary storage and your local backup fail, are destroyed, infected with a virus or malware. It doesn’t make any difference if it’s off line for a few minutes or even hours. Your 7 biggest outages are just click bait without substance.

So let me tell you a story of a REAL cloud data loss and why it isn’t important. A SW development company was using Amazon AWS to host their PRIMARY data AND their compute platform. They were using single factor authentication and their password was hacked at their place of business. Their admin received a message on his management console (key) that if a ransom wasn’t paid, their data would be toast. They told the hacker to pack sand, and sure as chicken, he deleted their data and they went out of business. Now whose fault was that? The company not using 2 factor authentication and ignoring that the hacker was already in (denoted by the message on the admin’s console), or AWS? That’s the kind of story that becomes “fake news” about the danger of the cloud.

You are obviously a professional when it comes to photography, but you are not a data storage / IT professional. Just stop scaring people away from the cloud in the endeavor to prove you’re correct. You’re not, and it’s a disservice to the readers who really need robust data storage.

gwilliams6

I never once said here in UHH how I totally do all my backup, so you are assuming a lot. Every way of backup that you mentioned I have done for a long time, including some cloud storage. And you dont know how much of my files are on SSDs, HDDs , writeable discs , and clouds. And you have no idea of have many multiple copies I have of my work in different places. And I teach all storage and backup solution options in my university photography courses. Again you are wildly assuming ,and you know what assuming makes of you.

Actually with your background you should know that the most stable way for archival long term storage of digital files is gold-infused discs which are made to remain stable for hundred of years. But each disc has limited capacity. I do have many of my photos stored on gold discs also. This is the only storage medium guaranteed 100% uneditable, inerasable and tested to endure up to 300 years based on the National Institute of Standards and Technology.
https://processonephoto.com/archival-picture-storage-discs.htm

I have made millions of professional photographs over 50 years as a pro, and I am still making new ones, and I need to use all storage solutions.

And countless top pros around the world use large RAID storage systems which still uses multiple spinning drives, but I guess you knew that.

And my job here is not to answer to you or anyone about the facts and research in any article I didn't author, or that you didn't research or author either. LOL
You are free to dismiss them or believe them, it doesn't matter to me.

Your personal attacks on me just show your intolerance to any point of view other than yours, LOL
I wont stoop down to your level ever.

I did post about the cloud that went down a couple of years ago and lost millions of files. Whether I can find it again here in UHH or by searching google, i don't really care if you or anyone else doesn't think it happened.

The top pros I know and work with around the world, only use the cloud as one of their multiple storage solutions, and to a person they all will tell you, that it is a risk they won't take of only relying on the cloud, even as convenient as it is.

Cheers and best to you.

TriX Loc: Raleigh, NC

Here’s hoping you mean MDisks, not “gold disks”

There’s nothing personal in my comments to you. You have made incorrect statements, concerning Seagate ownership, the reliability of Seagate vs WD drives and especially concerning cloud storage. I am merely correcting them so your misinformation doesn’t cause others to make bad data storage decisions.

It would be to your advantage to accept the professional information regarding data storage that I’ve provided, just as I accept the information you provide as a photography professional, but it seems your ego is in the way, Your contention seems to be that taking “millions of photographs” makes you a authority on data storage. It doesn’t.

gwilliams6

"Archival Gold discs with Scratch Armor offer the most unique and trustworthy CD-R and DVD discs to safely store your most treasured digital data. This is the only storage medium guaranteed 100% uneditable, inerasable and tested to endure up to 300 years based on the National Institute of Standards and Technology. Photographers can conveniently refer back to their original data files and images when necessary and even pass them along to future generations"

Archival disc pricing.
Delkin Gold archival CD, 300 year life. 800mb $10.00 per disc

Delkin Gold archival DVD, 100 year life. 4.7gb $15.00 per disc

I have many of my photos stored on these archival gold discs. Just a fact.

I dont let any ego get in the way of facts, and that is how I roll and how I teach my photo students. Again you are assuming you know me, and you dont. Again your assuming makes you know what of you.

Cheers and best to you.

TriX Loc: Raleigh, NC

So yesterday and too small to be practical for images (you’d need a LOT of disks). Instead, the answer is MDIsks, the most reliable data storage media in existence (per DOD testing) available in 25, 50 and 100GB sizes.

https://en.m.wikipedia.org/wiki/M-DISC#Overview

gwilliams6

I did buy two LG MDisk drives and am having them installed in my new custom-built computer tower that will burn M-Discs. Along with other projects ,I will be doing some burning of them, but I have been using gold discs since they first came out. A 100GB M-Disc can go for around $100 USD. That will just scratch the surface of my extensive files.

Cheers and best to you.

Dikdik Loc: Winnipeg, Canada

I think CDs and DVDs are storage of the past. They are too small, if you have a 'bunch' of files. I think multiple copies on current 'best' media is the safe thing to do. Copying data on to current media ensures that in 200 years you don't have to search for a player.

I had an incident a couple of years back where a bunch of my wife's homeschooling files were on 5-1/4" diskettes. It was by luck that I located someone that had an old computer running DOS, I'd already purchased a 5-1/4" drive on eBay... it came from Hungary. I was going to purchase a controller when I found out about the person with the DOS machine. That was a lot of work and these diskettes were only about 30 years old... I have no idea of what 200 years or more would be like. They are now backed up on current media.

I had an incident a month back and nearly lost 30 years of engineering work. It was only luck that saved the day. All my files, including my 3 backups which were connected to my computer, were encrypted. It was only luck that I'd made a complete backup and the old drive was disconnected from the computer.

gwilliams6

I thought you might assume something again ,that I am not having M-Disc drives being installed in my custom tower computer being built, LOL

So here is a quick low res cellphone snap I made a few minutes ago .You can check the date and time code, LOL

Cheers and best to you

(Download)

gwilliams6

However photo data burned onto Gold discs and M-Discs are 100% uneditable and inerasable, whereas all other current popular forms of storage can be edited, erased and corrupted on SSDs, HDDs or in the cloud, and wont last hundreds of years as they will degrade over the years. That is a huge difference now and into the future.

Cheers and best to you.

Dikdik Loc: Winnipeg, Canada

So, how many Gold disks would I need for a 16 TB drive? and what would they cost? What if I have data that I need to edit? What if I cannot find a Gold disk player in 200 years?

Artcameraman Loc: Springfield NH

Happens all the time hopefully the SSD drives will be better.

DirtFarmer Loc: Escaped from the NYC area, back to MA

I think people are talking past each other.

Backup is important. For the individual. Each person has to decide what the value of their files is to them. The decision MAY hinge on what the value of their files is to others, but it's our own decision to make in the end.

I discount arguments that any given medium will preserve your files for 100 years or more as specious. Technology changes, and while wire recordings may still have audio on them from less than 100 years ago, they do not make a good archival medium. M-disks may be resistant to aging and deterioration, but will you be able to find a reader for one a century from now? Backup requires at least 3 things: Duplication, Distribution, and Maintenance. Maintenance includes updating the storage medium to current standards. Back in the '90s I saved a lot of stuff on CDs and DVDs. Those media are essentially dead now or at least well on their way out. I have moved the important stuff to hard disks. And the cloud.

One thing that has not been discussed is the value of one's files to others. My files are important to me, so I preserve them to the best of my ability. But probably most of those files are not important to my successors beyond the family photos. So why should I try to preserve my files for 100 years? I don't expect to care about my files when I'm 183 years old.

I believe this discussion started by someone dissing cloud services as a backup option. I think cloud services are an essential part of a backup system. Naturally there are all grades of cloud service out there from excellent to terrible. And they differ in what they can protect. Amazon photos protect your photos but not your word processing files, your email, your program configuration files, your important documents, etc., so not all cloud services are equal.

Your local backups are primary. They are right there by your side and you can access them and restore things quickly. And you can Duplicate them. Distribution is limited, because there is a significant convenience factor. You can distribute a backup to your neighbor but if your house burns down it could easily set your neighbor's house on fire too. You could distribute a backup to your relative in another town, but a regional disaster could get both houses. And regional disasters happen. Think about the Paradise fire in CA, where the entire town burned. Distribution needs to be coupled with distance, and the more distance, the more inconvenience, and the more inconvenience, the less frequently the backups will be synchronized. Then there's Maintenance. Your storage media are most likely maintained by an amateur.

Your cloud backups are secondary. Restoration can be slow, depending on available bandwidth. Yes, some cloud services will loan you a disk to restore a large volume of data, but that takes a couple weeks of your time. And there's a cost involved. But even though a cloud backup might be secondary, that does not mean it isn't important. Cloud backup is safer than local backup. The cloud media are maintained by professionals. The cloud data are duplicated to avoid loss by any one disk. The cloud data are distributed to avoid loss by any one site. The distribution is across the whole country, if not across continents. Physical data loss is extremely unlikely.

It has been noted above that a cloud service went bankrupt but allowed users 60 days to restore their stuff. That should not be all that important. You should already have local backup so if your cloud service disappears in a microsecond without allowing you to restore your stuff, your local backup becomes your backup and you can find another cloud service to take over the secondary role.

If your cloud service disappears suddenly without warning and deletes all your stuff, it is YOUR fault for not having a primary local backup available.

jerryc41 Loc: Catskill Mts of NY

I've been using "Renewed" HGST Ultrastar drives for years with not a single failure. I ordered two 3TB drives for $36 each. They'll arrive next week. I'm going to put them in my Buffalo 12TB four-drive box.