Latency is a huge issue, but it goes away with the PTT model. I tried full duplex on initial prototyping but it was trash.

PTT forces solves this by simply forcing the listen, digest, then respond. You can expect about 2-3 seconds of delay from when you release the ptt, to when the other side hears it.