Valcom has a solution that would allow all-page and a button near each speaker. Press the button and it moves the conversation to just that speaker in talkback mode. Call Valcom - they'll help put it together.
Mike