Apart from the common causes related to packet loss, the problem can also occur when a large number of audio codecs are enabled: this results in a large SIP packet size, which can be dropped by the receiving side. Try to leave only a few audio codecs (default value is opus,g722/16000,alaw,mulaw,g729).
Read more about codecs