Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Links and Resources
How Topaz Sharpen AI uses deep learning
Nov 20, 2022 00:53:17   #
srsincary Loc: Cary, NC
 
An almost-four year old article. But it explains the basic differences between Photoshop's Shake Reduction implementation (an algorithmic solution using the mathematics of Deconvolution) and Sharpen AI's approach of using millions of blur-sharp Photo pairs to train a deep learning model how to transform a blurred image to a sharp one.

If anybody has seen a more recent article describing their secret sauce in more details, please share.

https://www.topazlabs.com/learn/how-we-use-ai-to-sharpen-your-photos

Reply
Nov 20, 2022 05:09:34   #
dpullum Loc: Tampa Florida
 
Thank you Srsincary, you have opened a subject that walks beyond photography. Elon Musk warned about AI and Robots, they deep learn and humans loose ability; a form of dementia. When I was in school, we had to learn, memorize, remember, connect and used our brain, now children ask their iPhone, "Hey Siri."
"I Tried Warning Them - Elon Musk on Superhuman AI"
https://www.youtube.com/watch?v=9LLmUKG9Toc

The Brain and Deep Learning: When young, we learn by our mistakes and quickly know not to touch a hot stove or shut our fingers in the car door; no math algorithm required. As we grow older, in my case dealing with refinement of industrial process, the process becomes intuitive and we are not quite sure how we "know" what to do; it just works or doesn't; continued "deep learning." In film days we learned to access a situation for a quick shot and perhaps with the help of Kodak short tutorials were educated to "Sunny Sweet 16"; motivated by how expensive photography was we learned quickly. Some compositions were made by staging and a steady hand... no digital back then; we used our brain and deep learning to express creativity.
https://www.slrlounge.com/photography-essentials-the-sunny-16-rule/

"Deep learning is understood as a form of machine learning which imitates how humans acquire knowledge. It first gained a lot of attention as it has enabled machines to approach and tackle cognitive tasks at which humans excel in a human-like fashion - including object recognition, speech processing, and cognitive planning. More recently, cognitive scientists have turned to deep learning models to study human cognition and its neural underpinnings." Scientific Reports is an open access journal publishing original research from across all areas of the natural sciences, psychology, medicine and engineering.
[Very Recent "Scientific Reports" ...calling for articles by December 31, 2022]
https://www.nature.com/collections/dbcdgiagfj/

DIY Photo Deep Learning complete with tool kit header:
https://www.mathworks.com/solutions/deep-learning/tutorials-examples.html#

Reply
Nov 20, 2022 06:37:22   #
srsincary Loc: Cary, NC
 
dpullum wrote:
Thank you Srsincary, you have opened a subject that walks beyond photography. Elon Musk warned about AI and Robots, they deep learn and humans loose ability; a form of dementia. When I was in school, we had to learn, memorize, remember, connect and used our brain, now children ask their iPhone, "Hey Siri."
"I Tried Warning Them - Elon Musk on Superhuman AI"
https://www.youtube.com/watch?v=9LLmUKG9Toc

The Brain and Deep Learning: When young, we learn by our mistakes and quickly know not to touch a hot stove or shut our fingers in the car door; no math algorithm required. As we grow older, in my case dealing with refinement of industrial process, the process becomes intuitive and we are not quite sure how we "know" what to do; it just works or doesn't; continued "deep learning." In film days we learned to access a situation for a quick shot and perhaps with the help of Kodak short tutorials were educated to "Sunny Sweet 16"; motivated by how expensive photography was we learned quickly. Some compositions were made by staging and a steady hand... no digital back then; we used our brain and deep learning to express creativity.
https://www.slrlounge.com/photography-essentials-the-sunny-16-rule/

"Deep learning is understood as a form of machine learning which imitates how humans acquire knowledge. It first gained a lot of attention as it has enabled machines to approach and tackle cognitive tasks at which humans excel in a human-like fashion - including object recognition, speech processing, and cognitive planning. More recently, cognitive scientists have turned to deep learning models to study human cognition and its neural underpinnings." Scientific Reports is an open access journal publishing original research from across all areas of the natural sciences, psychology, medicine and engineering.
[Very Recent "Scientific Reports" ...calling for articles by December 31, 2022]
https://www.nature.com/collections/dbcdgiagfj/

DIY Photo Deep Learning complete with tool kit header:
https://www.mathworks.com/solutions/deep-learning/tutorials-examples.html#
Thank you Srsincary, you have opened a subject tha... (show quote)


The dangers of AI is a complex topic. Even a brilliant physicist like Stephen Hawking started expressing his concerns about super-intelligent AI bots taking over society, stepping way out of his area of expertise. It's a complex topic, going well beyond the math of DL networks.

My interest is much narrower in scope - to understand how a photo-sharpening tool uses a tool/technique that has also been used to beat the world's best player in a game of Go.

The answer at a sufficiently high level of abstraction is "simple." Artificial neural networks at their core learn complex, nonlinear functions that take some input (typically a vector of real numbers) and produce some output (also a vector). Deep neural nets have more nodes and layers, so can learn more complex functions than traditional neural networks (invented decades ago). The magic of these techniques is that they can learn functions by simply looking at a sufficiently large number of input-output examples. The operative word here is "sufficient." 🙂

Traditionally, image sharpening has been achieved by trying to reverse the effects of blurring caused by Convolution, a well understood phenomenon which can be expressed mathematically. However, reversing the effects of Convolution, by applying Deconvolution, is complicated because the math (function) to apply to the image (pixels) requires *estimating* some parameters. And those parameters can be hard to estimate. So, DL to the rescue. Show enough examples to a DL network of the right "flavor,"

blurry-image --> sharp-image

and it will eventually learn "the correct Deconvolution function" to sharpen images. That is the theory anyway. Topaz Sharpen AI is getting better, but it is far from perfect.

For a more detailed technical explanation of the use of Deconvolution for image sharpening and different approaches (including DNNs), this NIST document seems to be a great source. Understanding the details is a challenge (I skimmed through it and it is pretty daunting)!

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8707587/

Reply
 
 
Nov 20, 2022 07:26:26   #
dpullum Loc: Tampa Florida
 
" input-output examples." or pairs of photos taken in differing conditions compared to a sharp photo... so when it sees a blur it understands it came from a better mage and it creates the "good" sharp image character. Our brain does that... as does a fly but they are more complex in nature with multi-receptors.
"A Novel Deep Learning Approach for the Removal of Speckle Noise from Optical Coherence Tomography Images Using Gated Convolution–Deconvolution Structure"
https://link.springer.com/chapter/10.1007/978-981-32-9291-8_10

"beat the world's best player in a game of Go" Well, Sophia messes the male mind because they are thinking she is a "Fully Functional" female Japanese robot.
https://sophia.play-asia.com/

From the article you cited: "Progress in modern imaging optics is still limited by the physical limitations of image resolution caused by the wave nature of light. The major fundamental constraint is the diffraction limit, but there are also limitations associated with the individual technical features of devices. Deconvolution can be used to work around these constraints; it is one of the possible approaches for obtaining super-resolution. It allows us to obtain images with a higher resolution than allowed by physical methods. "

Based on my education in Electronics, Circuit design, Symbolic Logic, and a course in Modern Algebra & Matrix Theory, I astringently figured that tackling how my CPU worked, humbled, I quickly decided that like religions myths, take the CPU as a mater of faith.

The use of Topaz is beyond "sharpen" their automate combo Photo AI is a miraculous plugin or stand [tall] alone. Witness the Photo AI work on a Cropped JPEG taken with a top of the line Kodak DC4800 first to achieve the record 3.1 mp raw image. Note the small block sampled in the eye glass nose area:

Reply
Nov 20, 2022 09:06:38   #
dpullum Loc: Tampa Florida
 
The image using Topaz Gigapixel was enlarged by 2000 time and can print a large image successfully. Original cropped face was 283 kb and the Gigapixel Enlarged is 5.9 mb or 5,900 kb.

Sorry for two posts, got a phone call and was timed out by UHH

Reply
Nov 20, 2022 11:48:59   #
abc1234 Loc: Elk Grove Village, Illinois
 
Interestingly, PS's shake reduction is gone from 2023. I used it a lot and wonder what has replaced it.

Reply
Nov 20, 2022 13:44:26   #
srsincary Loc: Cary, NC
 
abc1234 wrote:
Interestingly, PS's shake reduction is gone from 2023. I used it a lot and wonder what has replaced it.


Interesting. I did not use it much, found the results to be too "harsh."

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Links and Resources
UglyHedgehog.com - Forum
Copyright 2011-2022 Ugly Hedgehog, Inc.