Discussion:
[Freeswitch-users] RTP packet loss on outgoing streams
mbo
2015-04-17 15:01:01 UTC
Permalink
Hello,

we need to monitor the voice quality on our server. To test this, we setup a test environment with 2 freeswitches, originating calls from freeswitch 1 and then answer them on freeswitch 2 and play an announcement. To analyze the traffic, we us tshark with the following command:

tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap

The result is, that we see packet loss around 20% on outgoing RTP streams, when we have more then 20 calls. The load average shown with the top command is always < 0.3 with a 4 core CPU and CPU load is less then 10%.

========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722 800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722 599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722 798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722 713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722 633 165 (20.7%) 180.25 0.75 0.07 X

Instead of tshark, we also tried to capture SIP & RTP with tcpdump with the following command:

tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap

If we analyze this file with wireshark (TELEPHONY->RTP->Show all Streams), it shows almost always 0% for packet loss. Only if we increase the load to 50 calls we can see a similar scenario where inbound calls still have < 1% packet loss but outgoing RTP streams have up to 20 % packet loss. We also repeated the same test with freeswitch 1.4.15 with much more powerful hardware with almost the same results.

The question is what do we see here. Does tshark show us wrong result? Can we more trust on the results captured with tcpdump or is there finally an issue in freeswitch when sending outgoing packets? Or is the method to capture on the server in general the wrong approach?

Thanks for any comments and hints

Markus
Brian West
2015-04-17 15:47:27 UTC
Permalink
Where are you capturing this data? ON the system itself? or some other
point in the network?
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we setup
a test environment with 2 freeswitches, originating calls from freeswitch 1
and then answer them on freeswitch 2 and play an announcement. To analyze
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP streams,
when we have more then 20 calls. The load average shown with the top
command is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all Streams),
it shows almost always 0% for packet loss. Only if we increase the load to
50 calls we can see a similar scenario where inbound calls still have < 1%
packet loss but outgoing RTP streams have up to 20 % packet loss. We also
repeated the same test with freeswitch 1.4.15 with much more powerful
hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result? Can
we more trust on the results captured with tcpdump or is there finally an
issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
*Brian West*
***@freeswitch.org


*Twitter: @FreeSWITCH , @briankwest*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com

ClueCon 2015 Call for Speakers <https://www.cluecon.com/call-for-speakers/> |
Register <https://freeswitch.com/cart.php?gid=1> TODAY! | Reddit:
/r/freeswitch <https://www.reddit.com/r/freeswitch>

*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
mbo
2015-04-17 19:06:09 UTC
Permalink
On the system itself.
Where are you capturing this data? ON the system itself? or some other point in the network?
Hello,
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP streams, when we have more then 20 calls. The load average shown with the top command is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722 800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722 599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722 798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722 713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722 633 165 (20.7%) 180.25 0.75 0.07 X
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all Streams), it shows almost always 0% for packet loss. Only if we increase the load to 50 calls we can see a similar scenario where inbound calls still have < 1% packet loss but outgoing RTP streams have up to 20 % packet loss. We also repeated the same test with freeswitch 1.4.15 with much more powerful hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result? Can we more trust on the results captured with tcpdump or is there finally an issue in freeswitch when sending outgoing packets? Or is the method to capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Brian West
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com
ClueCon 2015 Call for Speakers | Register TODAY! | Reddit: /r/freeswitch
T:+19184209001 | F:+19184209002 | M:+1918424WEST (9378)
iNUM:+883 5100 1420 9001 | ISN:410*543 | Skype:briankwest
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
mbo
***@gmx.net
Brian West
2015-04-17 19:10:26 UTC
Permalink
What kind of network cards does this system have?
Post by mbo
On the system itself.
Where are you capturing this data? ON the system itself? or some other
point in the network?
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we
setup a test environment with 2 freeswitches, originating calls from
freeswitch 1 and then answer them on freeswitch 2 and play an announcement.
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP
streams, when we have more then 20 calls. The load average shown with the
top command is always < 0.3 with a 4 core CPU and CPU load is less then
10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all
Streams), it shows almost always 0% for packet loss. Only if we increase
the load to 50 calls we can see a similar scenario where inbound calls
still have < 1% packet loss but outgoing RTP streams have up to 20 % packet
loss. We also repeated the same test with freeswitch 1.4.15 with much more
powerful hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result?
Can we more trust on the results captured with tcpdump or is there finally
an issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
*Brian West*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com
ClueCon 2015 Call for Speakers
<https://www.cluecon.com/call-for-speakers/> | Register
<https://freeswitch.com/cart.php?gid=1> TODAY! | Reddit: /r/freeswitch
<https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
mbo
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
*Brian West*
***@freeswitch.org


*Twitter: @FreeSWITCH , @briankwest*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com

ClueCon 2015 Call for Speakers <https://www.cluecon.com/call-for-speakers/> |
Register <https://freeswitch.com/cart.php?gid=1> TODAY! | Reddit:
/r/freeswitch <https://www.reddit.com/r/freeswitch>

*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
Stanislav Sinyagin
2015-04-17 20:47:15 UTC
Permalink
you can try playing with the buffer size (-B option), but in general,
tshark is quite CPU-intensive when it tries to analyze the UDP packet
stream in real time. Also I'm not sure if it's multithreaded -- I
guess it's not, so it tries to analyze all the streams in a single
thread in real time.

I made this set of scripts to run on a low-power CPU (Atom and such):
https://github.com/voxserv/voip_qos_probe
What I do is run tcpdump during the call, store the capture, and then
analyze the output with tshark in low priority.

Also I noticed that on a Xen VM on an Intel CoreDuo CPU, you need to
increase the tcpdump's buffer size with -B option. With default
settings, sometimes it's not even able to capture a single SIP session
without dropping packets.
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we setup a
test environment with 2 freeswitches, originating calls from freeswitch 1
and then answer them on freeswitch 2 and play an announcement. To analyze
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP streams,
when we have more then 20 calls. The load average shown with the top command
is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with the
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all Streams),
it shows almost always 0% for packet loss. Only if we increase the load to
50 calls we can see a similar scenario where inbound calls still have < 1%
packet loss but outgoing RTP streams have up to 20 % packet loss. We also
repeated the same test with freeswitch 1.4.15 with much more powerful
hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result? Can
we more trust on the results captured with tcpdump or is there finally an
issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
***@freeswitch.org
http://www.freeswitchsolutions.com

Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-***@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Brian West
2015-04-17 20:49:58 UTC
Permalink
What i'm getting at is you should never have dropped packets on the host
itself like that, If you do then you probably have WinNic's aka RealTek
garbage.
Post by Stanislav Sinyagin
you can try playing with the buffer size (-B option), but in general,
tshark is quite CPU-intensive when it tries to analyze the UDP packet
stream in real time. Also I'm not sure if it's multithreaded -- I
guess it's not, so it tries to analyze all the streams in a single
thread in real time.
https://github.com/voxserv/voip_qos_probe
What I do is run tcpdump during the call, store the capture, and then
analyze the output with tshark in low priority.
Also I noticed that on a Xen VM on an Intel CoreDuo CPU, you need to
increase the tcpdump's buffer size with -B option. With default
settings, sometimes it's not even able to capture a single SIP session
without dropping packets.
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we
setup a
Post by mbo
test environment with 2 freeswitches, originating calls from freeswitch 1
and then answer them on freeswitch 2 and play an announcement. To analyze
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP
streams,
Post by mbo
when we have more then 20 calls. The load average shown with the top
command
Post by mbo
is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms)
Problems?
Post by mbo
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with
the
Post by mbo
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all
Streams),
Post by mbo
it shows almost always 0% for packet loss. Only if we increase the load
to
Post by mbo
50 calls we can see a similar scenario where inbound calls still have <
1%
Post by mbo
packet loss but outgoing RTP streams have up to 20 % packet loss. We also
repeated the same test with freeswitch 1.4.15 with much more powerful
hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result?
Can
Post by mbo
we more trust on the results captured with tcpdump or is there finally an
issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
*Brian West*
***@freeswitch.org


*Twitter: @FreeSWITCH , @briankwest*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com

ClueCon 2015 Call for Speakers <https://www.cluecon.com/call-for-speakers/> |
Register <https://freeswitch.com/cart.php?gid=1> TODAY! | Reddit:
/r/freeswitch <https://www.reddit.com/r/freeswitch>

*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
Stanislav Sinyagin
2015-04-17 21:13:11 UTC
Permalink
it's not dropped packets, it's the packets that didn't make it to tshark's
buffer, because it was too slow and the buffer was too short.
Post by Brian West
What i'm getting at is you should never have dropped packets on the host
itself like that, If you do then you probably have WinNic's aka RealTek
garbage.
Post by Stanislav Sinyagin
you can try playing with the buffer size (-B option), but in general,
tshark is quite CPU-intensive when it tries to analyze the UDP packet
stream in real time. Also I'm not sure if it's multithreaded -- I
guess it's not, so it tries to analyze all the streams in a single
thread in real time.
https://github.com/voxserv/voip_qos_probe
What I do is run tcpdump during the call, store the capture, and then
analyze the output with tshark in low priority.
Also I noticed that on a Xen VM on an Intel CoreDuo CPU, you need to
increase the tcpdump's buffer size with -B option. With default
settings, sometimes it's not even able to capture a single SIP session
without dropping packets.
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we
setup a
Post by mbo
test environment with 2 freeswitches, originating calls from freeswitch
1
Post by mbo
and then answer them on freeswitch 2 and play an announcement. To
analyze
Post by mbo
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP
streams,
Post by mbo
when we have more then 20 calls. The load average shown with the top
command
Post by mbo
is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms)
Problems?
Post by mbo
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with
the
Post by mbo
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all
Streams),
Post by mbo
it shows almost always 0% for packet loss. Only if we increase the load
to
Post by mbo
50 calls we can see a similar scenario where inbound calls still have <
1%
Post by mbo
packet loss but outgoing RTP streams have up to 20 % packet loss. We
also
Post by mbo
repeated the same test with freeswitch 1.4.15 with much more powerful
hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result?
Can
Post by mbo
we more trust on the results captured with tcpdump or is there finally
an
Post by mbo
issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
Post by mbo
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by mbo
http://www.freeswitch.org
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
*Brian West*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com
ClueCon 2015 Call for Speakers
<https://www.cluecon.com/call-for-speakers/> | Register
<https://freeswitch.com/cart.php?gid=1> TODAY! | Reddit: /r/freeswitch
<https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
mbo
2015-04-20 12:44:53 UTC
Permalink
I did some more test and when I use the -B 32768 option with tcpdump I can trace up to 150 calls without packets dropped by the kernel. Finally that means as soon as you have more than 15 parallel calls on the server itÂ’s very important to set this parameter to not get wrong results.

Many thanks for your support

Markus
it's not dropped packets, it's the packets that didn't make it to tshark's buffer, because it was too slow and the buffer was too short.
What i'm getting at is you should never have dropped packets on the host itself like that, If you do then you probably have WinNic's aka RealTek garbage.
you can try playing with the buffer size (-B option), but in general,
tshark is quite CPU-intensive when it tries to analyze the UDP packet
stream in real time. Also I'm not sure if it's multithreaded -- I
guess it's not, so it tries to analyze all the streams in a single
thread in real time.
https://github.com/voxserv/voip_qos_probe
What I do is run tcpdump during the call, store the capture, and then
analyze the output with tshark in low priority.
Also I noticed that on a Xen VM on an Intel CoreDuo CPU, you need to
increase the tcpdump's buffer size with -B option. With default
settings, sometimes it's not even able to capture a single SIP session
without dropping packets.
Post by mbo
Hello,
we need to monitor the voice quality on our server. To test this, we setup a
test environment with 2 freeswitches, originating calls from freeswitch 1
and then answer them on freeswitch 2 and play an announcement. To analyze
tshark -q -f 'udp' -o rtp.heuristic_rtp:TRUE -z rtp,strems -w test-pcap
The result is, that we see packet loss around 20% on outgoing RTP streams,
when we have more then 20 calls. The load average shown with the top command
is always < 0.3 with a 4 core CPU and CPU load is less then 10%.
========================= RTP Streams ========================
Src IP addr Port Dest IP addr Port SSRC Payload
Pkts Lost Max Delta(ms) Max Jitter(ms) Mean Jitter(ms) Problems?
192.168.1.134 27590 192.168.1.135 22576 0xCD7A189D ITU-T G.722
800 2 (0.2%) 59.93 0.11 0.05 X
192.168.1.135 22576 192.168.1.134 27590 0x5D4E50CD ITU-T G.722
599 202 (25.2%) 160.24 0.16 0.05 X
192.168.1.134 28172 192.168.1.135 29704 0xD53EC9AD ITU-T G.722
798 2 (0.2%) 59.99 0.07 0.04 X
192.168.1.135 29704 192.168.1.134 28172 0x5D50942D ITU-T G.722
713 86 (10.8%) 199.98 0.08 0.04 X
192.168.1.135 25794 192.168.1.134 16460 0x5D51E43D ITU-T G.722
633 165 (20.7%) 180.25 0.75 0.07 X
Instead of tshark, we also tried to capture SIP & RTP with tcpdump with the
tcpdump -i any -T rtp -vvvvv -s 0 -w test.pcap
If we analyze this file with wireshark (TELEPHONY->RTP->Show all Streams),
it shows almost always 0% for packet loss. Only if we increase the load to
50 calls we can see a similar scenario where inbound calls still have < 1%
packet loss but outgoing RTP streams have up to 20 % packet loss. We also
repeated the same test with freeswitch 1.4.15 with much more powerful
hardware with almost the same results.
The question is what do we see here. Does tshark show us wrong result? Can
we more trust on the results captured with tcpdump or is there finally an
issue in freeswitch when sending outgoing packets? Or is the method to
capture on the server in general the wrong approach?
Thanks for any comments and hints
Markus
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Brian West
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com
ClueCon 2015 Call for Speakers | Register TODAY! | Reddit: /r/freeswitch
T:+19184209001 | F:+19184209002 | M:+1918424WEST (9378)
iNUM:+883 5100 1420 9001 | ISN:410*543 | Skype:briankwest
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
http://www.freeswitchsolutions.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://confluence.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Loading...