ivozprovider
ivozprovider copied to clipboard
irontec
asterisk: IP address order may cause no audio
When I configure a virtual pbx with rtpproxy as media server there is no sound in both directions. These are the rtp streams from a capture.
where: xxx.xxx.xxx.195 is the kamailio Users proxy and media servers (rtpproxy, rtpengine) audio sock IP xxx.xxx.xxx.58 is the public IP of the NATed SIP phone xxx.xxx.xxx.91 is my trunk media server
We can see there are no rtp streams from ivozprovider rtpproxy media server to the device, nor to trunk media server. I have read rtpproxy is going to be removed soon but I included this test just in case it helps with rtpengine problem.
When using rtpengine on the other hand there is audio in both directions except one device that presents one way audio (it does not reproduce audio IN though I can confirm the audio reaches the device and on the correct port).
These are the rtp streams from the call through rtpengine.
We can see this time there are media streams from ivozprovider rtpengine media server to the device and to the trunk media server.
What I'm experimenting with rtpengine is one way audio in a Gigaset N510 DECT station, all other devices I have tested have audio in both directions. The only strange thing I can see on the RTP streams is that it looks they are comming from my trunk media server at a larger packetization size. I'm not familiarized with packetization but it all indicates they are comming from trunk media server at double payload size, 376 rtp packages, while the size of the packages we send is 216.
When I connect the trunk provider directly to an asterisk using chan_sip there is audio in both directions in all devices, Gigaset N510 DECT station included, and the RTP packages length is 216 both sides.
Maybe this has nothing to do with the problem but I don't see anything strange on SIP packages. All SDP looks good though I can be missing something.
~~By the way, when using rtpproxy there is no audio in both directions as the audio streams doesn't seem to be delivered to the other party. But the length of both the rtp stream from the device to the rtpproxy and from the trunk media server to the rtpproxy is 216~~.
Not true. Same sizes as when using rtpengine (376 length rtp packages from trunk media server, 216 from SIP device).
Hi @manfer ,
We made some tests and we cannot reproduce this problem. SDP negotiation is OK with both media-relays and the call has bidirectional audio.
Are the SDPs mangled correctly? If you analyze with sngrep rtp-call-flow feature you can see which part is not doing things right (not sending audio to corresponding port, etc.).
I'm afraid we cannot help you without a full SIP+RTP pcap.
Regards,
Well, I have seen it. There is a maxptime=150 in the SDP from the INVITE from ivozprovider kamailio users to the device and on the 200 OK SDP from ivozprovider kamailio trunks to the trunk SIP server. Can this be changed to 20 somewhere or removed?
I would like to test if with 20ms or without a maxptime value it works correctly using rtpengine.
This maxptime is not in the original INVITE from the trunk SIP server.
Probably has nothing to do with the issue. Sorry.
Hi @manfer ,
It seems that maxptime is hardcoded by codec in Asterisk source code to control RTP repacketization. As explained here, 150 is a reasonable value, in fact, the most tipical ptime value is 20ms that would be permited as it is lower than 150 (that is a maximum value).
Are you sure that this SDP param is causing the problem? :thinking:
Regards,
No. Probably not. I wanted to know if it was possible to change the value just to test.
I don't understand why the provider is sending the rtp packages with a different size than 216 (376 which is probably double payload). Maybe that's just perfectly valid anyway.
But I'm almost 100% sure the gigaset is not able to deal with those rtp packages because the packages reach the device and on the correct port.
The rest of devices of different brands I have tested can deal with the rtp packages of 376 length without any problem.
Now I'm trying to find out why rtpproxy is failing in this test installation.
I'm inspecting the SIP messages one by one. I'm comparing the ones from the call through rtpproxy with the ones in the call through rtpengine. The first SIP message, the invite from trunk provider to my system shows no difference at all. The second SIP message, the INVITE from kamailio trunks to asterisk has a difference on the last SDP header.
The IPs: xxx.xxx.xxx.194 kamailio trunks xxx.xxx.xxx.195 kamailio users / rtp proxies audio sock xxx.xxx.xxx.206 trunk provider SIP server xxx.xxx.xxx.91 trunk provider rtp proxy
While the call through rtpproxy sends a=sdpmangled:yes, the call through rtpengine sends a=rtcp:17071
I suppose that 17071 is the rtpengine port. Don't know if sdpmangled:yes is the expected header added by rtpproxy.
Hi @manfer ,
17071 is the rtcp port, that if not defined is the rtp port plus 1 (further info). And sdpmangled:yes is added by rtpproxy, yes. None of them should affect the SDP negotiation, though.
It sounds like a phone implementation bug.
Regards,
Yes, that's what I think about the problem with the Gigagset DECT station. A bug on the device itself. They must just be expecting 20ms PCMA and are not able to deal with other rtp size.
About the rtpproxy not working in my install this is what I see on sngrep. The arrows indicate the RTP packages only.
First a graph when the call goes through rtpengine.
It is a standalone install so all components are on same machine and all works correctly with rtp engine. Though intuitively I would expect some of the RTP packages source IP being 127.0.0.1 instead of the rtp proxy audio socket IP. Those marked on this graph in red. That doesn't seem to be a problem on standalone install but I suppose in a distributed install those packages source IP has to be the asterisk IP and not the rtp proxy IP.
When the call goes through rtpproxy the graph shows again same behaviour but in this case the audio from rtpproxy to the device and from rtpproxy to trunk provider is missing.
Or maybe I'm thinking of it the wrong way and the below graphs show the correct behaviour, some rtp packages going from rtp protocol proxy to rtp protocol proxy. Maybe it is just the proxy function itself.
Anyway, in my test, when the call goes through rtpproxy the audio from the device is not delivered to the trunk provider and the audio from the trunk provider is not delivered to the device.
Hi @manfer
The asterisk versión has not been (yet) changed in artemis, so the addresses used for rtp should be the same that the used in an oasis standalone installation. In 13.17.0 asterisk tries to detect the most suitable address for media, and sometimes it matches the media relay addresses. This should not be a problem because each software use different rtp port ranges. You could verify this behaviour in your oasis standalone production environment.
More testing on this will be requiered, focusing in rtpengine more than rtpproxy (although it would be nice if both works).
Is this problem related to 2.x or just introduced in bleeding? We are unable to reproduce this behaviour in our testing environment, so we can not yet determine this is a software flaw.
Thanks for the feedback and detailed data!
FTR, bleeding is quite experimental and some functionalities are merged in multiple steps, so don't expect everything will be working on every commit.
Anyway, all the feedback is always highly appreciated and lots of corner cases can only be tested by humans. Detecting bugs before merging into artemis grants more stability to releases (even if they are not prduction ready).
Regards!
Is this problem related to 2.x or just introduced in bleeding? We are unable to reproduce this behaviour in our testing environment, so we can not yet determine this is a software flaw.
No. The problem with Gigaset N510 happens to me on 1.x too.
My 1.x install uses rtpproxy if I'm not wrong. Though both rtpproxy and rtpengine services are up and running in the GUI media servers configuration only rtpproxy is added as media server. I can do some tcpdump on localhost port 22222 and port 22223 just to be sure which one is used.
About rtpproxy not working in my test not sure if artemis would work. I will try.
I have made a test that maybe is useless. Just things that comes to my mind to test that look interesting. I have tried changing rtp audio socket to kamailio trunks IP as I suppose the rtp proxy, could be anywhere, even on a different machine with different IP, if correctly configured (this is very important as I don't know if I fulfil this for the test I explain here), same as asterisk.
To do so, I edit /etc/default/rtpproxy and /etc/default/rtpengine. I only change the audio sock on rtpproxy and the bind address on rtpengine. Both continue listening in 127.0.0.1 port 22222 and port 22223. Though I could just restart the daemons I reboot the machine to be sure new changes take effect.
Then I configure the company to use rtpengine. I do external call and I have audio in both directions (except the N510 DECT which almost sure is a device bug) but I lost internal calls audio.
Am I missing something and my changes on /etc/default are not enough to change rtp proxy audio IP?
Sorry if this test is useless and incorrect.
I may know where the problem is.
Bleeding repos have packaged an experimental version of asterisk 15 with some custom patches that may be breaking everything. Please, verify your version by running:
asterisk -rx 'core show version'
If you're running asterisk 15.1.5, you're using an experimental package that won't be yet merged into artemis (at least not on next releases). I have removed that from bleeding, so downgrade by running
apt-get update
apt-get install --reinstall ivozprovider-asterisk=2.2~13.17.0~20180530.156~a7e080d+bleeding
asterisk -rx 'core restart now'
As mentioned in my previous note, bleeding is totally unstable and being the only project branch unlocked (so force pushes may also happen), can have testing things and development experiments than will be removed later.
Sorry if that caused you trouble!
My fault.
I downgraded to asterisk 13.17.0 and still no audio on external calls through rtprproxy.
I will do my tests with artemis better.
No problem!
Testing bleeding, as mentioned, always helps to detect problems and also to confirm fixes, but expect things to crash more often.
Thanks for you hard work @manfer !
I have installed an artemis and an oasis VM to have one of each version.
And I decided to proceed with my tests with oasis. Same result, rtpproxy no audio on external calls.
So, still don't know the source of the problem, but this is what I have found.
My virtual machines are installed on an OVH dedicated server. The server has a primary IP xxx.xxx.xxx.47 and gateway xxx.xxx.xxx.254. It has some secondary IPs two of them being yyy.yyy.yyy.194 and yyy.yyy.yyy.195 which are on other network range and that must be routed to the dedicated server gateway. The IPs are assigned to specific MAC so we only have to configure that MAC on the interface of our VM and configure the routing.
This is the network configuration /etc/network/interfaces from the debian VM where oasis is installed. Same one is used for artemis and bleeding. Only one of the machines is run at a time as all of them has same MAC on network card and use same IPs.
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address yyy.yyy.yyy.194
netmask 255.255.255.255
broadcast yyy.yyy.yyy.194
post-up /sbin/ip route add xxx.xxx.xxx.254 dev eth0
post-up /sbin/ip route add default via xxx.xxx.xxx.254
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 8.8.8.8 8.8.4.4
auto eth0:0
iface eth0:0 inet static
address yyy.yyy.yyy.195
netmask 255.255.255.255
broadcast yyy.yyy.yyy.195
where yyy.yyy.yyy.194 is kamailio trunks, yyy.yyy.yyy.195 is kamailio users, rtpproxy and rtpengine audio socket xxx.xxx.xxx.254 is the gateway
Which such a configuration and using rtpproxy there is audio on local calls but there is no audio on external calls.
If I change the network configuration to
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address yyy.yyy.yyy.195
netmask 255.255.255.255
broadcast yyy.yyy.yyy.195
post-up /sbin/ip route add xxx.xxx.xxx.254 dev eth0
post-up /sbin/ip route add default via xxx.xxx.xxx.254
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 8.8.8.8 8.8.4.4
auto eth0:0
iface eth0:0 inet static
address yyy.yyy.yyy.194
netmask 255.255.255.255
broadcast yyy.yyy.yyy.194
where again yyy.yyy.yyy.194 is kamailio trunks, yyy.yyy.yyy.195 is kamailio users, rtpproxy and rtpengine audio socket xxx.xxx.xxx.254 is the gateway
then I have audio with rtpproxy on internal and external calls.
In the first configuration -the one that has no audio on external calls- I can see how my sip device and my trunk provider are sending rtp packages to the correct requested rtp IP yyy.yyy.yyy.195 -the rtpproxy audio socket- but then rtpproxy sends those rtp packages to yyy.yyy.yyy.194.
In the second configuration -the one that works- I can see how my sip device and my trunk provider are again sending rtp packages to the correct requested rtp IP yyy.yyy.yyy.195 -the rtpproxy audio socket- and then rtpproxy sends those rtp packages to the correct IP too yyy.yyy.yyy.195.
Maybe I'm missing something in the routing. Some ip rule or something. But at first glance I don't see what that would be.
Hi @manfer
If I understand correctly your last note, the order of the addresses in configured interfaces may lead to no-audio scenarios. If that's the case, it may be a bug related to asterisk multihomed module, which has been reworked in the latest asterisk 13 versions. This will probably have less or no impact on distributed installations where proxies and applications servers don't share any IP address.
We won't change asterisk version for 2.3.0, but we're planning on upgrading for next releases, as soon as we have enough time to port custom patches and test everything works. If everything seems ok and this problem can not be reproduced, we can also backport the version to oasis.
Thanks for all the testing and feedback!
the order of the addresses in configured interfaces may lead to no-audio scenarios
Yes, that's the only change in network configuration.
Apologies to reply to an old ticket but considering it's still an open issue, I am going to ask the question as I experienced the same issue and the same solution to that.
@manfer I would appreciate your feedback if are you still using Ivoz with the setup mentioned above and if so have you encountered any issues with audio in production environment?
I am currently using the latest tempest version of Ivoz.
