My heater in my 2003 Honda Civic LX Coupe is not working well. When my car is heated up, it begins to blow hot air, like normal but after about 2 minutes, it goes straight back to cold?

Sounds like it may be low on coolant? or the thermostat may be getting bad. One other thing it Might be is the HVAC Heater Control Valve in the heater hose may be causing the grief.

I have a new thermostat in it. Just put it in about a week ago and saw it open up myself. I'm still having this issue, it's not low on anything either. Also, when going up a mountain called Green Mountain here in town, my car over heats. I've gotten out of the car when it does and have noticed that the fan is only kicking in whenever it feels like it, or when I press the AC button on the dash. I've replaced the fan switch from having this problem before and it worked fine for a couple of months but is now doing the same thing as it was before.

Also did you purge the cooling system from air when refilling with coolant? With the cap off until thermostat opens?

Yes, I had help from a very good friend of mine that works at Auto Zone and he came out of the store to do it with me.

Does the temperature gauge stay at normal operating temp. on the dash? And is it only the heat that goes from hot to cold?

The temp gauge stays normal on the dash unless I use the car for awhile or if I'm driving up a mountain. Then it begins to climb into the red. And yes the heat goes from hot to cold, then back and forth.

Well, to me it still sounds like there is air in the system. And those fans have to work properly, but have nothing to do with the heat in the car, just the temp. of the motor. You can check by running the car, when it starts to blow cold air, check the heater hoses. I suspect one would be noticeably cooler than the other one, indicating that there is still air in the cooling system. Some cars are very difficult to purge. There are garages out there that have a machine to purge the cooling system.