Just to be clear are, you are using VOIP to connect the sites not QSIG over PRI right?
There is an issue with QSIG over PRI very similar to what you are having, samsung flew out here to trouble shoot this on one of our customers a few years ago and never figured out the problem. We ended up ditching the PRI cards for MGI/OAS cards and all is good now.