Look at the logs in the vm, they should shed some light on your issue.
From memory the sip dtmf settings are for outbound calls.

What forward do you have setup on the reception extension?
Or do calls ring to the operator group?
It's possible that when your reception is on the phone calls are going back into the vm and cause there is no mailbox for that extension callers are ending backup In the day aa menu