I have never noticed Latino women or really any type of person losing their charm after they start speaking their language. I believe that that would be a stereotype brought on by someone who has been turned down by women. Maybe that does not happen to everyone. Some cultures are different than others and people react and behave in different ways based on their culture. For example some cultures are known for being loud when they speak.
That does not mean they are angry with each other, but they just like to speak in a loud tone. That might be the case with Latino women. They don’t lose their charm because they just like to speak loud when they speak in their native language.