Geez! The OP comes here, describes a situation in which he did nothing wrong, and asks a reasonable question. His behavior is called "disturbing" and he is referred to as a "pompous self-inflated ass". Nice.

For the record, as many others have responded, he was perfectly within his rights. The police were wrong. To assume that the police know the law is naive. There are too many links to attach. Just google "photographing children in public places" to see the many citations of the law.

I love all the Philadelphia Lawyers here who have stated the Law so convincingly but incorrect. First it does not matter what city or town you are in the law is the same in the US no matter what. The OP was within his rights and the Cops were 100% wrong. The tip off should have been when they said delete the photo or lose your camera. If they had confiscated the Camera the photos would have had to have been preserved for evidence. As some others have correctly stated any person place or thing in a public area maybe photographed. There is no expectation of privacy in a public area. This applies to everyone not just celebrities. You may in fact sell those images without a release. You do not need a release to share them post them sell them or do anything you want with them. You can print them and sell them as art if you want no release needed. You can sell them to a magazine no release needed. The only time you would need a release is if you are using them for some "commercial" purpose which is defined by law as using to promote or advertise. So you can sell them to a newspaper but not to Pepsi to be used in an advertisement with out a release. You would need a release if you post them on your own website for the purpose of promoting your own photography business. so the law gets blurry in cases where there is some cross over. Let's say the OP was going to put those photos on display at an art show which people can come and purchase them He has every right to do that without release However any photos he uses to advertise or promote the show he must have a release for.
Morally and Ethically the OP had every right to photograph that child. He did nothing wrong. For those that said they would have deleted it because it was the ethical thing to do STFU! You don't know what you are talking about! He did nothing unethical. This is how art is made and history is recorded what was unethical was the behavior of the police. What would have been unethical was destroying images taken in public because someone insisted. By the way The sailor kissing a girl in time square and the Girl from Afghanistan did not have model releases and those are two of the most published pictures in history. The girl from Afghanistan I believe was a minor at the time. The photographer made a lot of money off that one and won a Pulitzer Prize NO Model release.

I have taken many photos of kids in public places and at events, including kids dancing jumping or just being cute. Mostly as part of an event of some sort.
A few have gone into the news paper that I regularly shoot for.
It is a first amendment issue.
If any parent or person objected, I would IMMEDIATELY delete it.
This is just common courtesy.
If you use that photo for any commercial purpose with out a signed release you are looking for a lawsuit.
I generally go up to a parent and give them my card and offer to send them a picture via e-mail if they are around.

Absolutely on point. It's about subject isolation. Any thought of monetary gain, or other public use there of whether it be a building, a logo or in this case the child would constitute the need for a model/property release (this includes entering contest, whether prize based or not) from the owner/individual. The individual in this case happens to be a minor, the mother was 100% right. Just like hunting on private property, you need permission to be there or it's trespassing; isn't it? Same applies here, you'd need to have had the mother's permission to take the image in the first place if he/she (the child) were the intended subject.

Otherwise whose to say you weren't stalking these people, think about it. Would you want some stranger taking photos of your children without you knowing or okaying it? Who's to say what their actual intent is? The mother was completely within her right and in my opinion being a good mom and being aware of her child's surroundings. Think before you shot, it will go a long way to keeping you out of criminal or small claims court.