Home email setup on iPhone

Im having trouble setting up my iPhone to access my home email. I think it is related to the fact that my home email account is hosted by Gmail, but it is not a Gmail account. I have set up the account will all of the information provided by my ISP, and it is identical to whats set up on my home computer, but I keep getting a mail server pop.gmail.com is not responding error. However, once or twice a day it will all of a sudden download all/most of my email so I dont understand how it can work some of the time but not all of the time. Any suggestions?
Thanks.
iPhone 4 iOS 4