vMX
vMX

vBNG (18.2R1.9) and PWHT issue

2 weeks ago

Hi

I only recently discovered the concept of pseudowire headend termination so find it quite exciting. I was fortunate to get a trial vMX from Juniper support a few months ago and I've been labbing on and off inside GNS3 (2.1.21) on Ubuntu 18.04.

 

Long story short, I can get the basics working (l2circuit UP, subscriber receives address via basic DHCP-INET profile) but everything seems to grind to a halt whenever COS or the dynamic IP profile comes into play. I've been using the Broadband Edge guide as an example as well as the Day One book for Subscriber Management and various KB articles on vMX / BNG but to no avail.

 

The main issue I've noticed (when RADIUS and all profiles are enabled as per the Broadband Edge guide) is that the DHCP local server constructs and appears to send the OFFER PDU with the yiaddr populated with the next available address but the DHCP client does not appear to receive it. A 'show subscribers' at this point will show the half-baked-fake subscribers that can't be reached because they don't truly exist.

 

I can't be sure but there may be a correlation between applying the hierarchical-scheduler to the anchor-point and the DHCP clients not receiving the offer? I've tried flexible-queuing-mode on the FPC, too.

 

I can post configs if that helps but it is essentially just the Broadband Edge BNG config chopped up and edited about 50 times to try and get it working Smiley Happy

 

I've been attempting to debug via:

 

general-authentication-service
smg-service
dhcp-service
interfaces

 

Is there anything else I should be looking at? I'm sure I've made a very simple mistake somewhere but I can't quite work out what. My goal was to just get IPs allocated from the dhcp-local-server and a basic 3 package (40/10, 80/20, 160/30) shaper/policer applied.

 

Any pointers much appreciated. Thanks! Smiley Happy

 

Michael

 

 

root@BNG> show version 
Hostname: BNG
Model: vmx
Junos: 18.2R1.9
JUNOS OS Kernel 64-bit  [20180614.6c3f819_builder_stable_11]
JUNOS OS libs [20180614.6c3f819_builder_stable_11]
JUNOS OS runtime [20180614.6c3f819_builder_stable_11]
JUNOS OS time zone information [20180614.6c3f819_builder_stable_11]
JUNOS network stack and utilities [20180628.003405_builder_junos_182_r1]
JUNOS libs [20180628.003405_builder_junos_182_r1]
JUNOS OS libs compat32 [20180614.6c3f819_builder_stable_11]
JUNOS OS 32-bit compatibility [20180614.6c3f819_builder_stable_11]
JUNOS libs compat32 [20180628.003405_builder_junos_182_r1]
JUNOS runtime [20180628.003405_builder_junos_182_r1]
JUNOS Packet Forwarding Engine Simulation Package [20180628.003405_builder_junos_182_r1]
JUNOS sflow mx [20180628.003405_builder_junos_182_r1]
JUNOS py extensions [20180628.003405_builder_junos_182_r1]
JUNOS py base [20180628.003405_builder_junos_182_r1]
JUNOS OS vmguest [20180614.6c3f819_builder_stable_11]
JUNOS OS crypto [20180614.6c3f819_builder_stable_11]
JUNOS mx libs compat32 [20180628.003405_builder_junos_182_r1]
JUNOS mx runtime [20180628.003405_builder_junos_182_r1]
JUNOS common platform support [20180628.003405_builder_junos_182_r1]
JUNOS mtx network modules [20180628.003405_builder_junos_182_r1]
JUNOS modules [20180628.003405_builder_junos_182_r1]
JUNOS mx modules [20180628.003405_builder_junos_182_r1]
JUNOS mx libs [20180628.003405_builder_junos_182_r1]
JUNOS mtx Data Plane Crypto Support [20180628.003405_builder_junos_182_r1]
JUNOS daemons [20180628.003405_builder_junos_182_r1]
JUNOS mx daemons [20180628.003405_builder_junos_182_r1]
JUNOS Simulation Linux Package [20180628.003405_builder_junos_182_r1]
JUNOS Simulation Package [20180628.003405_builder_junos_182_r1]
JUNOS Services URL Filter package [20180628.003405_builder_junos_182_r1]
JUNOS Services TLB Service PIC package [20180628.003405_builder_junos_182_r1]
JUNOS Services Telemetry [20180628.003405_builder_junos_182_r1]
JUNOS Services SSL [20180628.003405_builder_junos_182_r1]
JUNOS Services SOFTWIRE [20180628.003405_builder_junos_182_r1]
JUNOS Services Stateful Firewall [20180628.003405_builder_junos_182_r1]
JUNOS Services RPM [20180628.003405_builder_junos_182_r1]
JUNOS Services PCEF package [20180628.003405_builder_junos_182_r1]
JUNOS Services NAT [20180628.003405_builder_junos_182_r1]
JUNOS Services Mobile Subscriber Service Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services MobileNext Software package [20180628.003405_builder_junos_182_r1]
JUNOS Services Logging Report Framework package [20180628.003405_builder_junos_182_r1]
JUNOS Services LL-PDF Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services Jflow Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services Deep Packet Inspection package [20180628.003405_builder_junos_182_r1]
JUNOS Services IPSec [20180628.003405_builder_junos_182_r1]
JUNOS Services IDS [20180628.003405_builder_junos_182_r1]
JUNOS IDP Services [20180628.003405_builder_junos_182_r1]
JUNOS Services HTTP Content Management package [20180628.003405_builder_junos_182_r1]
JUNOS Services Flowd MS-MPC Software package [20180628.003405_builder_junos_182_r1]
JUNOS Services Crypto [20180628.003405_builder_junos_182_r1]
JUNOS Services Captive Portal and Content Delivery Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services COS [20180628.003405_builder_junos_182_r1]
JUNOS AppId Services [20180628.003405_builder_junos_182_r1]
JUNOS Services Application Level Gateways [20180628.003405_builder_junos_182_r1]
JUNOS Services AACL Container package [20180628.003405_builder_junos_182_r1]
JUNOS Extension Toolkit [20180628.003405_builder_junos_182_r1]
JUNOS J-Insight [20180628.003405_builder_junos_182_r1]
JUNOS Online Documentation [20180628.003405_builder_junos_182_r1]
JUNOS jail runtime [20180614.6c3f819_builder_stable_11]

 

 

8 REPLIES 8
vMX

Re: vBNG (18.2R1.9) and PWHT issue

2 weeks ago

Hi Michael, 

 

     Please share the configuration of your dynamic profile and the CoS conifguration and I'll take a look. 

 

     Also have you tried by just deactivating CoS and check if the subscriber comes up fine ?

 

Regards

Arpit 

vMX

Re: vBNG (18.2R1.9) and PWHT issue

2 weeks ago

Hi Arpit!

 

What's the best way of disabling CoS?

I've tried quite a few config iterations and in some of them I did notice that removing the hierarchical-scheduler maximum-hierarchy-levels 2 implicit-hierarchy from interface lt-0/0/10 seemed to allow the DHCP client (which is just a Cisco router in my lab) to receive the IPs.

 

I'll get my lab back to baseline and post some configs for you shortly, thanks.

 

vMX

Re: vBNG (18.2R1.9) and PWHT issue

[ Edited ]
2 weeks ago

Hi Michael, 

       

        Please share the complete config if you can.

        The best way to remove COS is to deactivate it under the interface config and deactivate it in the dynamic-profiles.

 

        Its a little tricky with CoS. Please make sure you have these 3 knobs configure under the chassis config :  performance-mode ,flexible-queuing-mode and loopback-device-count 1.

 

The most important is "loopback-device-count 1" if you want to use COS.

 

root# run show configuration chassis

fpc 0 {

pic 0 {

tunnel-services {

bandwidth 10g;

}

interface-type xe;

inline-services {

bandwidth 10g;

}

}

pic 1 {

inline-services;

}

performance-mode;

flexible-queuing-mode;

loopback-device-count 1;

}

network-services enhanced-ip;

 

 

Hope this helps !!

 

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !

+++++++++++++++++++++++++++++++++++++++++++++

 

Regards

Arpit

vMX

Re: vBNG (18.2R1.9) and PWHT issue

Friday
Sep 11 11:10:20 localhost user.notice root: : Number of ucode workers : 0
Sep 11 11:10:20 localhost user.notice root: : Detected Junos Device type VirtIO
Sep 11 11:10:21 localhost user.notice root: : Number of CPUs available : 7
Sep 11 11:10:21 localhost user.notice root: : Number of Ucode workers  : 0
Sep 11 11:10:21 localhost user.notice root: : Number of PORTs to service : 4, loopbacks: 0
Sep 11 11:10:21 localhost user.notice root: : vdev_str:
Sep 11 11:10:21 localhost user.notice root: : /home/pfe/riot/build/app/riot  -c 0x7f -n 2  --log-level=5   -w 00:05.0 -w 00:06.0 -w 00:07.0 -w 00:08.0 -- --rx "(0,0,0,3),(1,0,1,3),(2,0,2,3                                                 ),(3,0,3,3)," --tx "(0,3),(1,3),(2,3),(3,3)," --w "4,5,6" --f "1"  --bbe_mode "0"  --
Sep 11 11:10:21 localhost user.info kernel: traps: riot[14093] trap invalid opcode ip:41326e sp:7fff0cf38cb8 error:0 in riot[400000+1a40000]
Sep 11 11:10:22 localhost user.notice root: start_afi_server.sh : tar -xzf /usr/share/pfe/afi-server.tgz : SUCCESS
start_riot.sh: line 424: 14093 Illegal instruction     (core dumped) /home/pfe/riot/build/app/riot -c 0x7f -n 2 --log-level=5 -w 00:05.0 -w 00:06.0 -w 00:07.0 -w 00:08.0 -- --rx "(0,0,0,3)        

I've tried this on two boxes now. One within GNS3, one natively with KVM. The latter with a blank config, more or less!

 

It seems that whenever I set performance-mode, the FPC crashes? Anything more than 4 vCPUs assigned seems to trigger it. Not sure what I'm doing wrong? Smiley Happy

vMX

Re: vBNG (18.2R1.9) and PWHT issue

Friday

Hi !

 

we dont support GNS3 .

 

in performance mode more vCPU is required , 3*n + 3 , n - -number of interfaces

 

https://www.juniper.net/documentation/en_US/vmx/topics/reference/general/vmx-hw-sw-minimums.html

 

For typical configurations, we recommend the following formula to calculate the minimum vCPUs required by the VFP:

********* Without QoS—(4 * number-of-ports) + 4

********* With QoS—(5 * number-of-ports) + 4

 

For typical configurations, we recommend the following formula to calculate the minimum vCPUs required by the VFP:

********* Without QoS—(4 * number-of-ports) + 4

********* With QoS—(5 * number-of-ports) + 4

 

4x4 +4 = 20

 

 

Regards

Arpit 

Please, Mark My Solution Accepted if it Helped, Kudos are Appreciated too!!!

vMX

Re: vBNG (18.2R1.9) and PWHT issue

Monday

Hi

 

Thanks for that.

When we speak of vCPUs here do we really mean native CPU threads available?

I'm using a dual E5506 box - so 8 threads/cores available, I think:

 

root@leclerc:/media/array2/vmx# lscpu -x
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) mask:   0xff
Thread(s) per core:    1
Core(s) per socket:    4
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 26
Model name:            Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
Stepping:              5
CPU MHz:               1635.823
CPU max MHz:           2128.0000
CPU min MHz:           1596.0000
BogoMIPS:              4266.76
Virtualisation:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              4096K
NUMA node0 CPU(s):     0x55
NUMA node1 CPU(s):     0xaa
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm flush_l1d

Even if I specify 20 vCPUs for 2 interfaces in vmx.conf, RIOT crashes as soon as I enable performance mode:

 

Sep 16 12:26:15 localhost user.notice root: : Number of ucode workers : 0
Sep 16 12:26:15 localhost user.notice root: : Detected Junos Device type VirtIO
Sep 16 12:26:15 localhost user.notice root: : Number of CPUs available : 20
Sep 16 12:26:15 localhost user.notice root: : Number of Ucode workers  : 0
Sep 16 12:26:15 localhost user.notice root: : Number of PORTs to service : 2, loopbacks: 0
Sep 16 12:26:15 localhost user.notice root: : vdev_str:
Sep 16 12:26:15 localhost user.notice root: : /home/pfe/riot/build/app/riot  -c 0xfffff -n 2  --log-level=5   -w 00:05.0 -w 00:06.0 -- --rx "(0,0,0,3),(1,0,1,4)," --tx "(0,3),(1,4)," --w "5,6,7,8,9,10,11,12,13,14,15,16,17,18,19" --f "1"  --bbe_mode "0"  --rpio "local,3000,
Sep 16 12:26:15 localhost user.info kernel: traps: riot[1438] trap invalid opcode ip:41326e sp:7fff082df1a8 error:0 in riot[400000+1a40000]
Sep 16 12:26:16 localhost user.notice root: start_afi_server.sh : tar -xzf /usr/share/pfe/afi-server.tgz : SUCCESS
start_riot.sh: line 424:  1438 Illegal instruction     (core dumped) /home/pfe/riot/build/app/riot -c 0xfffff -n 2 --log-level=5 -w 00:05.0 -w 00:06.0 -- --rx "(0,0,0,3),(1,0,1,4)," --tx "(0,3),(1,4)," --w "5,6,7,8,9,10,11,12,13,14,15,16,17,18,19" --f "1" --bbe_mode "0" --rpio "local,3000,3001" --hostif "local,3002" --bsz "(32,32),(64,64),(512,512)"

Is RIOT crashing the expected behaviour?

Is it possible to overcommit KVM CPU resource for testing performance mode?

 

Thanks for your help on this!

 

Michael

vMX

Re: vBNG (18.2R1.9) and PWHT issue

Monday

Is there simply no way to test a dynamic profile with CoS without an Ivy Bridge (or later) processor?

vMX

Re: vBNG (18.2R1.9) and PWHT issue

2 hours ago

With 20 CPUs Riot shouldn't crash. 

 

Is there a riot core generated ?? 

 

I would recommend opening a JTAC case we can have a look and make this work. 

 

Regards

Arpit