vMX

Hi

I only recently discovered the concept of pseudowire headend termination so find it quite exciting. I was fortunate to get a trial vMX from Juniper support a few months ago and I've been labbing on and off inside GNS3 (2.1.21) on Ubuntu 18.04.

Long story short, I can get the basics working (l2circuit UP, subscriber receives address via basic DHCP-INET profile) but everything seems to grind to a halt whenever COS or the dynamic IP profile comes into play. I've been using the Broadband Edge guide as an example as well as the Day One book for Subscriber Management and various KB articles on vMX / BNG but to no avail.

The main issue I've noticed (when RADIUS and all profiles are enabled as per the Broadband Edge guide) is that the DHCP local server constructs and appears to send the OFFER PDU with the yiaddr populated with the next available address but the DHCP client does not appear to receive it. A 'show subscribers' at this point will show the half-baked-fake subscribers that can't be reached because they don't truly exist.

I can't be sure but there may be a correlation between applying the hierarchical-scheduler to the anchor-point and the DHCP clients not receiving the offer? I've tried flexible-queuing-mode on the FPC, too.

I can post configs if that helps but it is essentially just the Broadband Edge BNG config chopped up and edited about 50 times to try and get it working 🙂

I've been attempting to debug via:

general-authentication-service
smg-service
dhcp-service
interfaces

Is there anything else I should be looking at? I'm sure I've made a very simple mistake somewhere but I can't quite work out what. My goal was to just get IPs allocated from the dhcp-local-server and a basic 3 package (40/10, 80/20, 160/30) shaper/policer applied.

Any pointers much appreciated. Thanks! 🙂

Michael

root@BNG> show version 
Hostname: BNG
Model: vmx
Junos: 18.2R1.9
JUNOS OS Kernel 64-bit  [20180614.6c3f819_builder_stable_11]
JUNOS OS libs [20180614.6c3f819_builder_stable_11]
JUNOS OS runtime [20180614.6c3f819_builder_stable_11]
JUNOS OS time zone information [20180614.6c3f819_builder_stable_11]
JUNOS network stack and utilities [20180628.003405_builder_junos_182_r1]
JUNOS libs [20180628.003405_builder_junos_182_r1]
JUNOS OS libs compat32 [20180614.6c3f819_builder_stable_11]
JUNOS OS 32-bit compatibility [20180614.6c3f819_builder_stable_11]
JUNOS libs compat32 [20180628.003405_builder_junos_182_r1]
JUNOS runtime [20180628.003405_builder_junos_182_r1]
JUNOS Packet Forwarding Engine Simulation Package [20180628.003405_builder_junos_182_r1]
JUNOS sflow mx [20180628.003405_builder_junos_182_r1]
JUNOS py extensions [20180628.003405_builder_junos_182_r1]
JUNOS py base [20180628.003405_builder_junos_182_r1]
JUNOS OS vmguest [20180614.6c3f819_builder_stable_11]
JUNOS OS crypto [20180614.6c3f819_builder_stable_11]
JUNOS mx libs compat32 [20180628.003405_builder_junos_182_r1]
JUNOS mx runtime [20180628.003405_builder_junos_182_r1]
JUNOS common platform support [20180628.003405_builder_junos_182_r1]
JUNOS mtx network modules [20180628.003405_builder_junos_182_r1]
JUNOS modules [20180628.003405_builder_junos_182_r1]
JUNOS mx modules [20180628.003405_builder_junos_182_r1]
JUNOS mx libs [20180628.003405_builder_junos_182_r1]
JUNOS mtx Data Plane Crypto Support [20180628.003405_builder_junos_182_r1]
JUNOS daemons [20180628.003405_builder_junos_182_r1]
JUNOS mx daemons [20180628.003405_builder_junos_182_r1]
JUNOS Simulation Linux Package [20180628.003405_builder_junos_182_r1]
JUNOS Simulation Package [20180628.003405_builder_junos_182_r1]
JUNOS Services URL Filter package [20180628.003405_builder_junos_182_r1]
JUNOS Services TLB Service PIC package [20180628.003405_builder_junos_182_r1]
JUNOS Services Telemetry [20180628.003405_builder_junos_182_r1]
JUNOS Services SSL [20180628.003405_builder_junos_182_r1]
JUNOS Services SOFTWIRE [20180628.003405_builder_junos_182_r1]
JUNOS Services Stateful Firewall [20180628.003405_builder_junos_182_r1]
JUNOS Services RPM [20180628.003405_builder_junos_182_r1]
JUNOS Services PCEF package [20180628.003405_builder_junos_182_r1]
JUNOS Services NAT [20180628.003405_builder_junos_182_r1]
JUNOS Services Mobile Subscriber Service Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services MobileNext Software package [20180628.003405_builder_junos_182_r1]
JUNOS Services Logging Report Framework package [20180628.003405_builder_junos_182_r1]
JUNOS Services LL-PDF Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services Jflow Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services Deep Packet Inspection package [20180628.003405_builder_junos_182_r1]
JUNOS Services IPSec [20180628.003405_builder_junos_182_r1]
JUNOS Services IDS [20180628.003405_builder_junos_182_r1]
JUNOS IDP Services [20180628.003405_builder_junos_182_r1]
JUNOS Services HTTP Content Management package [20180628.003405_builder_junos_182_r1]
JUNOS Services Flowd MS-MPC Software package [20180628.003405_builder_junos_182_r1]
JUNOS Services Crypto [20180628.003405_builder_junos_182_r1]
JUNOS Services Captive Portal and Content Delivery Container package [20180628.003405_builder_junos_182_r1]
JUNOS Services COS [20180628.003405_builder_junos_182_r1]
JUNOS AppId Services [20180628.003405_builder_junos_182_r1]
JUNOS Services Application Level Gateways [20180628.003405_builder_junos_182_r1]
JUNOS Services AACL Container package [20180628.003405_builder_junos_182_r1]
JUNOS Extension Toolkit [20180628.003405_builder_junos_182_r1]
JUNOS J-Insight [20180628.003405_builder_junos_182_r1]
JUNOS Online Documentation [20180628.003405_builder_junos_182_r1]
JUNOS jail runtime [20180614.6c3f819_builder_stable_11]

Hi Michael, 

     Please share the configuration of your dynamic profile and the CoS conifguration and I'll take a look. 

     Also have you tried by just deactivating CoS and check if the subscriber comes up fine ?

Regards

Arpit 

Hi Arpit!

What's the best way of disabling CoS?

I've tried quite a few config iterations and in some of them I did notice that removing the hierarchical-scheduler maximum-hierarchy-levels 2 implicit-hierarchy from interface lt-0/0/10 seemed to allow the DHCP client (which is just a Cisco router in my lab) to receive the IPs.

I'll get my lab back to baseline and post some configs for you shortly, thanks.

Hi Michael, 

        Please share the complete config if you can.

        The best way to remove COS is to deactivate it under the interface config and deactivate it in the dynamic-profiles.

        Its a little tricky with CoS. Please make sure you have these 3 knobs configure under the chassis config :  performance-mode ,flexible-queuing-mode and loopback-device-count 1.

The most important is "loopback-device-count 1" if you want to use COS.

root# run show configuration chassis

fpc 0 {

pic 0 {

tunnel-services {

bandwidth 10g;

}

interface-type xe;

inline-services {

bandwidth 10g;

}

}

pic 1 {

inline-services;

}

performance-mode;

flexible-queuing-mode;

loopback-device-count 1;

}

network-services enhanced-ip;

Hope this helps !!

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !

+++++++++++++++++++++++++++++++++++++++++++++

Regards

Arpit

Sep 11 11:10:20 localhost user.notice root: : Number of ucode workers : 0
Sep 11 11:10:20 localhost user.notice root: : Detected Junos Device type VirtIO
Sep 11 11:10:21 localhost user.notice root: : Number of CPUs available : 7
Sep 11 11:10:21 localhost user.notice root: : Number of Ucode workers  : 0
Sep 11 11:10:21 localhost user.notice root: : Number of PORTs to service : 4, loopbacks: 0
Sep 11 11:10:21 localhost user.notice root: : vdev_str:
Sep 11 11:10:21 localhost user.notice root: : /home/pfe/riot/build/app/riot  -c 0x7f -n 2  --log-level=5   -w 00:05.0 -w 00:06.0 -w 00:07.0 -w 00:08.0 -- --rx "(0,0,0,3),(1,0,1,3),(2,0,2,3                                                 ),(3,0,3,3)," --tx "(0,3),(1,3),(2,3),(3,3)," --w "4,5,6" --f "1"  --bbe_mode "0"  --
Sep 11 11:10:21 localhost user.info kernel: traps: riot[14093] trap invalid opcode ip:41326e sp:7fff0cf38cb8 error:0 in riot[400000+1a40000]
Sep 11 11:10:22 localhost user.notice root: start_afi_server.sh : tar -xzf /usr/share/pfe/afi-server.tgz : SUCCESS
start_riot.sh: line 424: 14093 Illegal instruction     (core dumped) /home/pfe/riot/build/app/riot -c 0x7f -n 2 --log-level=5 -w 00:05.0 -w 00:06.0 -w 00:07.0 -w 00:08.0 -- --rx "(0,0,0,3)        

I've tried this on two boxes now. One within GNS3, one natively with KVM. The latter with a blank config, more or less!

It seems that whenever I set performance-mode, the FPC crashes? Anything more than 4 vCPUs assigned seems to trigger it. Not sure what I'm doing wrong? 🙂

Hi !

we dont support GNS3 .

in performance mode more vCPU is required , 3*n + 3 , n - -number of interfaces

https://www.juniper.net/documentation/en_US/vmx/topics/reference/general/vmx-hw-sw-minimums.html

For typical configurations, we recommend the following formula to calculate the minimum vCPUs required by the VFP:

********* Without QoS—(4 * number-of-ports) + 4

********* With QoS—(5 * number-of-ports) + 4

For typical configurations, we recommend the following formula to calculate the minimum vCPUs required by the VFP:

********* Without QoS—(4 * number-of-ports) + 4

********* With QoS—(5 * number-of-ports) + 4

4x4 +4 = 20

Regards

Arpit 

Please, Mark My Solution Accepted if it Helped, Kudos are Appreciated too!!!

Hi

Thanks for that.

When we speak of vCPUs here do we really mean native CPU threads available?

I'm using a dual E5506 box - so 8 threads/cores available, I think:

root@leclerc:/media/array2/vmx# lscpu -x
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) mask:   0xff
Thread(s) per core:    1
Core(s) per socket:    4
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 26
Model name:            Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
Stepping:              5
CPU MHz:               1635.823
CPU max MHz:           2128.0000
CPU min MHz:           1596.0000
BogoMIPS:              4266.76
Virtualisation:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              4096K
NUMA node0 CPU(s):     0x55
NUMA node1 CPU(s):     0xaa
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm flush_l1d

Even if I specify 20 vCPUs for 2 interfaces in vmx.conf, RIOT crashes as soon as I enable performance mode:

Sep 16 12:26:15 localhost user.notice root: : Number of ucode workers : 0
Sep 16 12:26:15 localhost user.notice root: : Detected Junos Device type VirtIO
Sep 16 12:26:15 localhost user.notice root: : Number of CPUs available : 20
Sep 16 12:26:15 localhost user.notice root: : Number of Ucode workers  : 0
Sep 16 12:26:15 localhost user.notice root: : Number of PORTs to service : 2, loopbacks: 0
Sep 16 12:26:15 localhost user.notice root: : vdev_str:
Sep 16 12:26:15 localhost user.notice root: : /home/pfe/riot/build/app/riot  -c 0xfffff -n 2  --log-level=5   -w 00:05.0 -w 00:06.0 -- --rx "(0,0,0,3),(1,0,1,4)," --tx "(0,3),(1,4)," --w "5,6,7,8,9,10,11,12,13,14,15,16,17,18,19" --f "1"  --bbe_mode "0"  --rpio "local,3000,
Sep 16 12:26:15 localhost user.info kernel: traps: riot[1438] trap invalid opcode ip:41326e sp:7fff082df1a8 error:0 in riot[400000+1a40000]
Sep 16 12:26:16 localhost user.notice root: start_afi_server.sh : tar -xzf /usr/share/pfe/afi-server.tgz : SUCCESS
start_riot.sh: line 424:  1438 Illegal instruction     (core dumped) /home/pfe/riot/build/app/riot -c 0xfffff -n 2 --log-level=5 -w 00:05.0 -w 00:06.0 -- --rx "(0,0,0,3),(1,0,1,4)," --tx "(0,3),(1,4)," --w "5,6,7,8,9,10,11,12,13,14,15,16,17,18,19" --f "1" --bbe_mode "0" --rpio "local,3000,3001" --hostif "local,3002" --bsz "(32,32),(64,64),(512,512)"

Is RIOT crashing the expected behaviour?

Is it possible to overcommit KVM CPU resource for testing performance mode?

Thanks for your help on this!

Michael

Is there simply no way to test a dynamic profile with CoS without an Ivy Bridge (or later) processor?

With 20 CPUs Riot shouldn't crash. 

Is there a riot core generated ?? 

I would recommend opening a JTAC case we can have a look and make this work. 

Regards

Arpit 

Hi Arpit

It looks like it does but all the files are 0 bytes, if I'm reading it correctly?

root@localhost:/var/log# ls -alF
drwxr-xr-x    2 root     root          4096 Sep 16 12:36 ./
drwxr-xr-t    6 root     root          4096 Sep 16 12:41 ../
-rw-r-----    1 root     root          3579 Sep  6 00:04 boot
-rw-r--r--    1 root     root          3119 Sep 16 12:41 fpc_core_push
-rw-r--r--    1 root     root           292 Sep 20 12:07 lastlog
-rw-r--r--    1 root     root         31644 Sep 20 12:08 messages
-rw-r--r--    1 root     root       1024020 Sep 16 12:36 messages.0
-rw-r--r--    1 root     root        288078 Sep 16 12:07 pfe.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.0.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.1.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.10.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.100.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.101.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.102.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.103.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.104.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.105.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.106.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.107.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.108.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.109.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.11.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.110.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.111.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.112.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.113.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.114.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.115.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.116.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.117.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.118.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.119.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.12.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.120.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.121.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.122.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.123.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.124.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.125.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.126.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.127.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.13.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.14.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.15.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.16.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.17.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.18.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.19.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.2.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.20.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.21.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.22.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.23.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.24.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.25.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.26.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.27.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.28.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.29.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.3.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.30.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.31.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.32.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.33.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.34.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.35.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.36.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.37.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.38.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.39.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.4.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.40.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.41.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.42.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.43.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.44.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.45.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.46.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.47.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.48.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.49.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.5.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.50.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.51.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.52.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.53.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.54.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.55.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.56.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.57.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.58.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.59.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.6.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.60.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.61.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.62.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.63.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.64.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.65.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.66.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.67.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.68.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.69.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.7.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.70.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.71.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.72.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.73.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.74.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.75.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.76.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.77.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.78.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.79.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.8.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.80.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.81.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.82.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.83.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.84.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.85.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.86.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.87.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.88.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.89.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.9.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.90.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.91.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.92.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.93.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.94.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.95.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.96.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.97.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.98.log
-rw-r--r--    1 root     root             0 Sep 16 12:06 riot.lcore.99.log
-rw-r--r--    1 root     root          1690 Sep 16 12:06 start_vmxt
-rw-r--r--    1 root     root        203904 Sep 20 12:07 wtmp

yes RIOT seems to be crashing. 

I'm not aware of any procedure to overcommit KVM CPU resource for testing performance mode.

Please file a JTAC case for RIOT cores.

Regards

Arpit 

Thanks, Arpit.

I even tried on a slighter newer server with more threads available but to no avail.

ql@vettel:~$ lscpu -x
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                24
On-line CPU(s) mask:   0xffffff
Thread(s) per core:    2
Core(s) per socket:    6
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 44
Model name:            Intel(R) Xeon(R) CPU           X5650  @ 2.67GHz
Stepping:              2
CPU MHz:               1707.183
CPU max MHz:           2666.0000
CPU min MHz:           1600.0000
BogoMIPS:              5331.81
Virtualisation:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K
NUMA node0 CPU(s):     0x555555
NUMA node1 CPU(s):     0xaaaaaa
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid dtherm ida arat flush_l1d

As soon as you increase the PFE thread count beyond 3, RIOT crashes. I'm not sure if I have the ability to raise a case as I'm basically just an enthusiast/home labber at this point but I'll give it a shot 🙂

Called Juniper support earlier.

Funnily enough, they don't allow vMX eval users to raise cases 😄

A quick update. I remembered I had another, non-server, machine I could try vMX on with a later CPU:

root@raikkonen> show chassis hardware
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                VM5D98E690B0      VMX
Midplane
Routing Engine 0                                         RE-VMX
CB 0                                                     VMX SCB
FPC 0                                                    Virtual FPC
  CPU            Rev. 1.0 RIOT-PERF    BUILTIN
  MIC 0                                                  Virtual
    PIC 0                 BUILTIN      BUILTIN           Virtual

This is the first time I've actually seen RIOT boot in PERFORMANCE mode. I can't even begin to tell you how happy I am to see those words 😄

Presumably, all the other issues I've been experiencing (with CoS) will now evaporate.

Fingers crossed 🙂

Hi Michael, 

          Apologize for a late reply. 

       I'm glad that it worked out after changing the server.

      Request you to please confirm that you were able to test COS and subscribers using the knobs below .

       If it did help, I request you to please accept this as a solution so others can help from this post. 

Please make sure you have these 3 knobs configure under the chassis config :  performance-mode ,flexible-queuing-mode and loopback-device-count 1.

The most important is "loopback-device-count 1" if you want to use COS.

root# run show configuration chassis

fpc 0 {

pic 0 {

tunnel-services {

bandwidth 10g;

}

interface-type xe;

inline-services {

bandwidth 10g;

}

}

pic 1 {

inline-services;

}

performance-mode;

flexible-queuing-mode;

loopback-device-count 1;

}

network-services enhanced-ip;

Hope this helps !!

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !

+++++++++++++++++++++++++++++++++++++++++++++

Regards

Arpit