Contrail Platform Developers
Contrail Platform Developers

Contrail Bare Metal Integration - Physical Server and VM in same VN

[ Edited ]
‎01-20-2016 02:10 AM

Hello All,

 

We are trying to integrate Bare Metal Servers with Contrail so that they can communicate with VM's in the same VN. Our setup details are as follows

 

Nodes: Config , Control NOdes *2 , Compute (Vrouter)*2 , Compute (TSN)*2 , Analytics Node

 

TOR Node: QFX5100 

 

Attached is the testbed file and configuration from QFX 5100. 

 

We are unable to establish the communication between the nodes. We have followed the below mentioned link for configuring the setup. 

 

Node Details as follows 

 

 

 

Pls. help.

 

Regards,

Lalit 

 

 

Attachments

13 REPLIES 13
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-20-2016 11:13 AM

Hi

 

I see you have configured ovsdb controller 172.16.2.75 which is your Config node. From my understanding the controller in this case should be TSN (172.16.2.80), or is it an error in config or diagram?

 

Also, are you able to see family evpn routes with server MAC on your Control nodes and vRouters? You can check this from Contrail GUI.

 

Best Regards,
PK

Juniper Ambassador, Juniper Networks Certified Instructor,
JNCIE-SEC #98, JNCIE-ENT #393, JNCIE-SP #2253
Twitter: @JuniperTrain
GitHub: https://github.com/pklimai
[Juniper Authorized Education & Support in Russia]
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-22-2016 04:15 AM

Hello PK,

 

I have configured the TSN as the controller on the QFX however I see that the QFX doesnt learn the server MAC in the ovsdb mac table. 

 

 

Captures from QFX

{master:0}
root@R4-Juniper-L2-SW> show ovsdb controller
VTEP controller information:
Controller IP address: 172.16.2.80
Controller protocol: ssl
Controller port: 9999
Controller connection: up
Controller seconds-since-connect: 6
Controller seconds-since-disconnect: 0
Controller connection status: active

{master:0}


root@R4-Juniper-L2-SW> show ovsdb mac
Logical Switch Name: Contrail-d518f1ad-b9c1-4061-8f05-1bb87f999bc3
Mac IP Encapsulation Vtep
Address Address Address
ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 172.16.2.82
02:e4:19:c5:65:19 0.0.0.0 Vxlan over Ipv4 172.16.2.78
ff:ff:ff:ff:ff:ff 0.0.0.0 Vxlan over Ipv4 172.16.2.80

 

I see some L2 (EVPN) routes on the compute nodes for the local VM's but not the BMS connected on the TOR (QFX5100). Also I was wondering if there is any hardware limitation in terms of the model used. 

 

Also I dont see any XMPP traffic getting IN or going OUT of the TSN node. Not sure if that can be a reason, if the TSN and vRouters dont communicate then how will the route and MAC tables will be exchanged.

 

Thanks in advance. 

 

Regards,

Lalit 

 

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-22-2016 12:06 PM

Hi

 

Can you tell what is 02:e4:19:c5:65:19 MAC address?

What is the MAC of BMS and can you see it on any compute/control node?

 

QFX5100 should work with Contrail using OVSDB. Probably you should check software version, are you using latest/recommended one.

 

TSN definitely must communicate to Control nodes using XMPP. Not sure why it does not do so.

 

Also, I see you configured vxlan vni 15 on your switch. Does it match vni defined in Contrail for this virtual network?

 

Best Regards,
PK

Juniper Ambassador, Juniper Networks Certified Instructor,
JNCIE-SEC #98, JNCIE-ENT #393, JNCIE-SP #2253
Twitter: @JuniperTrain
GitHub: https://github.com/pklimai
[Juniper Authorized Education & Support in Russia]
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-26-2016 11:45 PM

Hi,

 

a. The MAC Address "02:e4:19:c5:65:19" belongs on the QFX5100 VTEP interface. 


{master:0}
root@R4-Juniper-L2-SW> ...e | match 02:e4:19:c5:65:19
Contrail-d518f1ad-b9c1-4061-8f05-1bb87f999bc3 02:e4:19:c5:65:19 SO - vtep.32769

 

b. MAC address of the BMS is "3c:8a:b0:89:98:59"  and this MAC is not being learnt on neither Compute nor Control nodes. 

 

c. QFX software version we are using is the recommended relase to support the OVSDB integration (  14.1X53-D30.3)

 

d. TSN and Control nodes arent communicating over XMPP and not sure why however I will try to give the setup a restart just to check if they start communicating. 

 

e. yes, the VNI 15 matches for the VN configured on Contrail. 

 

I will give the setup a restart and will let you know of the status. 

 

Regards,

Lalit 

 

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-27-2016 03:16 PM

By the way, did you check the overall health of the system. This includes

 

- contrail-status on all nodes

- df on all nodes to make sure there is free disk space on all partitions

- ntp configuration

 

Did you try sending traffic between regular VMs, e.g. created on Compute-1 and Compute-2 nodes? This is to make sure the problem is in OVSDB and is not more general (i.e., completely broken setup)?

 

Best Regards,
PK

Juniper Ambassador, Juniper Networks Certified Instructor,
JNCIE-SEC #98, JNCIE-ENT #393, JNCIE-SP #2253
Twitter: @JuniperTrain
GitHub: https://github.com/pklimai
[Juniper Authorized Education & Support in Russia]
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-28-2016 02:59 AM

Hello PK,

 

I have cheked the health status of all the nodes in the setup and things seem to be normal. I'm attaching the same so that you can have a look at it at your convenience.  

 

 

We have 4 tenants in our setup and all of them have atleast two VM instances and a couple of tenants also have Service Instances spawned. I have verified reachability amongst themselves and also to the external gateway(MX Router) and vice versa to be working absolutely fine.

 

I believe it has got to do something with OVSDB between the QFX and TSN and maybe because of that the TSN isnt communicating with the Control nodes over XMPP as it hasnt been receiving anything from the QFX
( This is what I presume).


Regards,
Lalit

 

Attachments

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎01-28-2016 10:02 AM

Hi

 

Thanks for the info, I see health status is ok.

 

I wonder if you tried to follow this blog when you performed your setup

http://www.opencontrail.org/opencontrail-sdn-lab-testing-1-tor-switches-with-ovsdb/

 

They use tor_ovs_protocol = tcp and the comment says "always tcp, for now". I see you use pssl. Probably tcp is worth trying.

 

Also you can try to view introspect pages, http to TSN node, port 8085. Introspect pages contain a lot of information (not so easy to interpret, however).

 

Best Regards,
PK

Juniper Ambassador, Juniper Networks Certified Instructor,
JNCIE-SEC #98, JNCIE-ENT #393, JNCIE-SP #2253
Twitter: @JuniperTrain
GitHub: https://github.com/pklimai
[Juniper Authorized Education & Support in Russia]
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎02-08-2016 01:58 PM

Also, to make sure -- did you add a link between ToR switch and a BM server in the GUI (Configure -> Physical devices)?

 

Best Regards,
PK

Juniper Ambassador, Juniper Networks Certified Instructor,
JNCIE-SEC #98, JNCIE-ENT #393, JNCIE-SP #2253
Twitter: @JuniperTrain
GitHub: https://github.com/pklimai
[Juniper Authorized Education & Support in Russia]
Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎02-09-2016 03:02 AM

Hello,

 

I have changed the protocol to TCP as suggested in your previous reply , however once the same was done the connection between the TOR and the controller is always down. 

 

We have added the connection between the BM and TOR using the logical interface configuration , here I have specified the MAC address and the IP address of the BM statically.

 

Regards,

Lalit 

 

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎06-21-2016 02:50 AM

Hi Lalith K,

 

 Did you solve this issue ? I am still not able to ping from the VM and BMS

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎06-21-2016 04:24 AM

Hello,

 

Not yet. But I plan to work on the same tomorrow


Have done few changes over the last 2 days.

 

Would let all know tomorrow.

 

Regards,

Lalit

 

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

[ Edited ]
‎06-21-2016 03:19 PM

Hello,

 

 Great. But now I have an issue that you may be able to help me in. when I run the "show ovsdb mac" command, I can't see the mac of the BMS that is connected to the QFX although I configured this interface as ovsdb interface and can see this mac in "ethernet-switching table".

 

I think you can see this mac in your ovsdb mac table so you may be able to help me in this issue.

Contrail Platform Developers

Re: Contrail Bare Metal Integration - Physical Server and VM in same VN

‎11-17-2016 08:32 AM

Hi, I am also facing same situation. I have configured ovsdb connection on QFX using tcp. I can see mac address of VM running inside SDN environment on the QFX but I cannot see MAC address of BMS connected with QFX and neither I can ping from BMS to VM and vice verse.

 

Did anybody solve it????

Please mark this as accepted solution if it works for you

A kudos is a good way of appreciation



Kashif Nawaz


JNCIE-Sec #170, JNCIE-SP#2492

JNCSP Ent/ Sec
https://packet-expert.org

Attachments