It should be a network issue. there is a Cisco router between AVAYA and NBN router, which might discard RTP/RTCP packets sometime.

It looks the issue is gone after bypass the Cisco router.