Microphone not working ONLY when speaker phone being used. iPhone 4

My Iphone 4 (GSM Model/ software version- 4.3.1)
I still have my warranty and my apple care warranty.
Say Im on a phone call phone to ear I can hear the person, the person can hear me.
Ill then push the speaker button so now i can still hear the person BUT! the person can not hear me through the microphone.
I tried the voice memos app and i recorded a few times and it works like its suppose to. so i know the Microphone isnt defective.
The issue only happens while on a phone call with a person.
Any word about this? any suggestions? I know its just speaker phone but at the same time if some small thing like this is defective it make me wonder what it could lead to.
New topic
Ive read a few post about cases causing Iphones proximity sensors to not work, and after removing the case the sensor does work. I have a custom skin on my iphone, my proximity sensor has not been working i cant remember if it was working right before i put the skin on, i put it on maybe 2 weeks ago, the skin is made of premium adhesive-backed cast vinyl so i was wondering if skins can cause the proximity sensor issue as well how cases did? its gonna suck if i gotta peel it off it gets ruined basically.