The setup at the office is typically how I have set it up. The IP card needs a public IP address to work, but since it is outside the router it makes it difficult to have a router proritize traffic.

Ping of 81ms from end to end is pretty good, if it is like that all the time. Last time I was trying to troubleshoot call quality, I had very good ping results most of the time, then there would be spikes of like 500+ms. I was trobubleshooting on the IP phone side, but the delays were on the phone system side.I believe it was some server that would burst a large amount of data periodically, and that is when the calls sounded bad.

You can definately try using G729A to see if it works better. If delay is a problem and not bandwith, then it could sound worse.