Routing

We have an Juniper MX80 router  , one SRX 3600 and one Ex4500.

What if you were me, How do you design your network and where do you locate the Vlans ?

* do you set gateways on firewall
* do you set vlans on MX or EX
* do you transfer traffic with BGP / Static route
* do you use metric connection ?
....

I just really need an advice.

Thank you

The exact configuration and use of the devices will depend on what your application is and what kind of network this is supporting.  Where are the devices, how are they segmented, what is the upstream connection.

As a general rule, I would have default gateways of networks that need to be segmented for security on the SRX.

The core switch role would have RVI and the default gateways then for other subnets.  And you may be using routing instances there to group and segregate related vlans.

The use of dynamic routing protocols like BGP or OSPF depend on the size and design of the associated network.

But firstly , we have isp connection with bgp.

we do not trust srx 3600 so much because it has some bugs.  And we have 800+ servers if infected 2 server sends small pps from the inside it locks the all network what if we put all traffic to srx. Or what if it get a big attack. this is why we looking for metric solution.

Actually our all network designed with vlans. All servers has their own subnets their own subnets. And we have customer isps which we send their traffic with static route and bgp.

Seems like you want to redesign the network? Seems like this may work for you

ISP=>MX (gateway Router)=>SRX (security and firewall- apply AppSecure and Screens to protect network=>EX. SRX will protect all if one or more servers get infected.

Exactly but we do not trust the srx stability so we plan to do sth. like the given image below.

But there are mainly 3 problems.

1. Is there any possible way to policy based route the traffic if preferred route is SRX , because we want to send the UDP traffic firstly UDP Firewall 

2.  SRX require the pass the return traffic on it. If it returns directly from EX to MX this time srx start dropping incomming traffic.

3. We do not know what if our config is ok or not 😄

Upstream ISP Connection :

set protocols bgp traceoptions file bgp-trace
set protocols bgp traceoptions file world-readable
set protocols bgp traceoptions flag open
set protocols bgp traceoptions flag state detail
set protocols bgp group UPISP type external
set protocols bgp group UPISP description UPISP
set protocols bgp group UPISP local-address 10.32.35.14
set protocols bgp group UPISP import SPD-IN
set protocols bgp group UPISP export SPD-OUT
set protocols bgp group UPISP peer-as 43391
set protocols bgp group UPISP neighbor 10.32.35.13

set policy-options policy-statement SPD-OUT term F from prefix-list DDOS-PREFIX-LIST
set policy-options policy-statement SPD-OUT term F then community + Firewall
set policy-options policy-statement SPD-OUT term F then accept
set policy-options policy-statement SPD-OUT term 1 from prefix-list SPD-PREFIX-LIST
set policy-options policy-statement SPD-OUT term 1 then accept
set policy-options policy-statement SPD-OUT term 500 then reject

set policy-options policy-statement SPD-IN term 1 then accept



BGP TO SRX & Ex - MX Config



set policy-options policy-statement export_bgp term 1 from protocol static
set policy-options policy-statement export_bgp term 1 from route-filter 185.90.82.0/24 exact
set policy-options policy-statement export_bgp term 1 then accept
set policy-options policy-statement export_bgp term END then reject


set policy-options policy-statement import_bgp term RFC_1918  from route-filter 192.168.0.0/16 exact
set policy-options policy-statement import_bgp term DENY_BOGONS from prefix-list BOGON-LIST
set policy-options policy-statement import_bgp term DENY_BOGONS then reject
set policy-options policy-statement import_bgp term DENY-RFC-1918 from route-filter 10.0.0.0/8 orlonger
set policy-options policy-statement import_bgp term DENY-RFC-1918 from route-filter 172.16.0.0/12 orlonger
set policy-options policy-statement import_bgp term DENY-RFC-1918 from route-filter 192.168.0.0/16 orlonger
set policy-options policy-statement import_bgp term DENY-RFC-1918 then reject
set policy-options policy-statement import_bgp term deny-own-pool from route-filter 185.90.82.0/24 orlonger
set policy-options policy-statement import_bgp term deny-own-pool then reject
set policy-options policy-statement import_bgp term DENY-MORE-THAN-/24 from route-filter 0.0.0.0/0 prefix-length-range /25-/32
set policy-options policy-statement import_bgp term DENY-MORE-THAN-/24 then reject
set policy-options prefix-list BOGON-LIST 0.0.0.0/8
set policy-options prefix-list BOGON-LIST 127.0.0.0/8
set policy-options prefix-list BOGON-LIST 169.254.0.0/16
set policy-options prefix-list BOGON-LIST 192.0.0.0/24
set policy-options prefix-list BOGON-LIST 192.0.2.0/24
set policy-options prefix-list BOGON-LIST 198.18.0.0/15
set policy-options prefix-list BOGON-LIST 198.51.100.0/24
set policy-options prefix-list BOGON-LIST 203.0.113.0/24
set policy-options prefix-list BOGON-LIST 224.0.0.0/4




set interfaces xe-0/0/1 unit 0 family inet address 10.10.10.1/30
set interfaces xe-0/0/2 unit 0 family inet address 10.10.10.13/30

set protocols bgp group internal type internal
set protocols bgp group internal local-address 10.10.10.1
set protocols bgp group internal export export_bgp
set protocols bgp group internal neighbor 10.10.10.2
set protocols bgp group internal neighbor 10.10.10.14

set protocols ospf area 0.0.0.0 interface xe-0/0/1.0
set protocols ospf area 0.0.0.0 interface xe-0/0/2.0

set routing-options autonomous-system 57844
set routing-options router-id 10.10.10.1


SRX Config :

set interfaces xe-1/0/0 unit 0 family inet address 10.10.10.2/30
set interfaces xe-4/0/1 unit 0 family inet address 10.10.10.17/30

set protocols bgp group MXE type internal
set protocols bgp group MXE local-address 10.10.10.2
set protocols bgp group MXE local-preference 100
set protocols bgp group MXE export send-direct
set protocols bgp group MXE neighbor 10.10.10.1
set protocols bgp group MXE neighbor 10.10.10.14

set protocols bgp group Exe type external
set protocols bgp group Exe export send-direct
set protocols bgp group Exe neighbor 10.10.10.18


set policy-options policy-statement send-direct term 1 from protocol direct
set policy-options policy-statement send-direct term 1 then accept


set routing-options autonomous-system 65001
set routing-options router-id 192.168.3.1



Ex Config :

set interfaces xe-0/0/34 unit 7 family inet address 10.10.10.14/30
set interfaces xe-0/0/35 unit 8 family inet address 10.10.10.18/30

set protocols bgp group internal type internal
set protocols bgp group internal export send-direct
set protocols bgp group internal local-preference 300
set protocols bgp group internal neighbor 10.10.10.13


set protocols bgp group SRX type external
set protocols bgp group SRX export send-direct
set protocols bgp group SRX peer-as 65001
set protocols bgp group SRX neighbor 10.10.10.17



set policy-options policy-statement send-direct term 1 from protocol direct
set policy-options policy-statement send-direct term 1 then accept


set routing-options router-id 10.10.10.14

set routing-options autonomous-system 65002
set routing-options static route 185.90.82.0/24
set interface xe-0/0/0 unit 0 family inet address 185.90.82.1/24;

Unfortunately I do not have the time to look at your config, but generally your requirements can be met.

This wold definitely require spending a lot of time on all the requirements, but generally lets look at one case:
1- 1. Is there any possible way to policy based route the traffic if preferred route is SRX , because we want to send the UDP traffic firstly UDP Firewall
You would probably need to use virtual routers. Create multiple virtual routers as needed on the MX80. Lets say you create a (routing-instance) vr named "udp_traffic" (chose a name that suites you best maybe you already have a naming convention) You could use instance-type virtual-router. Place the interface that connects to the UDP FW in that routing instance.
Create a firewall filter to match UDP traffic then routing-instace "udp_traffic" and a second term to accept all other traffic that would handled by the master routing instace. And this could be place on all the ingress interface on the MX.

2. SRX require the pass the return traffic on it. If it returns directly from EX to MX this time srx start dropping incomming traffic.(not too clear)
But it depends on the mode this SRX is running in. Lets say in flow mode and you have various ZONES created. Lets say MX-ZONE1 has interface at 10.10.10.2 going to MX, EX-ZONE2 with interface 10.10.10.x going towards EX4500 and UDP-FW-ZONE3 with interface at 10.10.10.10
So as far as traffic from the EX, you would have a firewall policy to match traffic from that interface connecting to EX then permit
Not only that

Of course you would have multiple other policies for all traffic. All the devices you are massively flexible and there is practically nothing that youwant to achieve tha cannot be configured.

So for a solution with routes. 

We need Metric routes 

For UDP Traffic 

if UDP fw is not accessible then it will route the SRX 

If srx is not accessible then it will route to the EX

For tcp traffic

if srx is not accessible then it will route to the EX

But the main problem is that. SRX requires to see the return of the traffic for an healty work. so 

If we create a prefix list on MX to route the traffic to the firewall with destination addresses  then the same list must work on the EX to send the same traffic with the source addresses to the SRX

how should i apply a list on both of this devices 

with a metric route this should be a better solution. 

Is there any destination traffic upstream of the MX that is NOT on the internet?

the only way I would think it is a good idea to have that direct connection from the MX to the EX is if there is TRUSTED traffic that can use that path.  If all the traffic upstream of the MX is Internet based and therefore UNTRUSTED you really should not be going down this path.

You are correct that the SRX (as do most firewalls) reject assymetrical traffic by default.  If necessary, you could override this behavior, but there is good reason to have the symetrical path enforced.  And in your case this is likely also a requirement for the UDP firewall.

I am not sure what you mean by metric routes.  Is this a term for policy based routing using protocol?  Or some other criteria?

Actually this diagram worked perfectly with given conf but this time we need to add fwdonus filter to all of the irb units.

we should add this on the ex return because it is in family bridge mode and does not accept the routing because of tagged vlan trunks.

Or we will build all vlans on EX4500 and clean the traffic on the up but this time we have too many vlans and ex getting difficulties

root@mx80-core# show firewall
filter FWDirect {
    term UDPFW {
        from {
            destination-prefix-list {
                Firewall;
            }
            protocol udp;
        }
        then {
            log;
            next-ip 10.10.10.6/32;
        }
    }
    term TCPFW {
        from {
            destination-prefix-list {
                Firewall;
            }
            protocol tcp;
            tcp-flags 0x02,0x10,0x12,0x18;
        }
        then {
            count TCPFWTR;
            log;
            next-ip 10.10.10.2/32;
        }
    }
    term ICMP {
        from {
            destination-prefix-list {
                Firewall;
            }
            protocol icmp;
        }
        then {
            count SAYAC;
            next-ip 10.10.10.2/32;
        }
    }
    term FWBlok {
        from {
            destination-prefix-list {
                Firewall;
            }
        }
        then {
            count Bloklanan;
            discard;
        }
    }
    term Default {
        then accept;
    }
}
filter rpf-filter {
    term default {
        then {
            count rpf-failed-count;
            reject;
        }
    }
}
filter CikisTrafik {
    term icNetwork {
        from {
            source-prefix-list {
                SPD-PREFIX-LIST;
            }
        }
        then accept;
    }
    term ISP {
        from {
            source-address {
                10.32.35.12/30;
            }
        }
        then accept;
    }
    term 500 {
        then {
            count DroppedSpoof;
            discard;
        }
    }
}
filter FWDonus {
    term TrafikYonlendir {
        from {
            source-prefix-list {
                Firewall;
            }
        }
        then {
            routing-instance TCP-Routes-Donus;
        }
    }
    term 500 {
        then accept;
    }
}




root@mx80-core# show interfaces xe-0/0/1
unit 0 {
    family inet {
        address 10.10.10.1/30;
    }
}

[edit]
root@mx80-core# show interfaces xe-0/0/2
unit 0 {
    family inet {
        address 10.10.10.5/30;
    }
}
[edit]
root@mx80-core# show interfaces ae0
description Netdirekt;
aggregated-ether-options {
    minimum-links 1;
    lacp {
        active;
        periodic fast;
    }
}
unit 0 {
    family inet {
        filter {
            input FWDirect;
            output CikisTrafik;
        }
        sampling {
            input;
            output;
        }
        address 10.32.35.14/30;
    }
}

[edit]


root@mx80-core# show interfaces irb unit 100
family inet {
    filter {
        input FWDonus;
    }

I'm still not sure I understand the goal, so forgive me if this is the wrong direction.

I think you want internal gateways and routing on the MX and the MX to also handle and sort the internet traffic by protocol incoming to the SRX and UDP firewall.

If that is the case, I think current configuration solves the inbound internet portion.

For the internal gateways, you might consider creating a new virtual router routing-instance on the MX. 

Place the interface leaving the SRX to the internal network into this routing-instance

Place the uplink from the EX to the MX into this routing-instance

Create all your internal routed vlan interfaces in this routing-instance

 I am sorry maybe i've described wrong let me tell you what i need and what we have

we have 1x MX80   2xSRX3600 1xEx4500  30xEx4200 (10G connected to Ex 4500)

Conditions : 

SRX needs to see return traffic what pass on it

We need to use vlans

We do not trust the srx totally so if srx fails for some reason traffic will go from the normal route 

Last thing we just keep the traffic for some ip addresses on srx not for total of network

So we put a rule on ISP interface with inbound filter. It forward the traffic to the SRX / udp firewalls and we put an other filter on IRB so it pass the return traffic when it arrives to gateway to the SRX as return traffic .

That worked perfectly and do what we need.

Problems : 

But this time we need to add this inbound rule to all irb interfaces and also we have static routed ip subnets to our downstream isps which is not in any irb so we can not pass their traffic on both directon to SRX . 

We can not put any routing instance filter on EX connection port because it works on family bridge mode to send trunks to EX4500. Other isps also get ip from our natvie vlan and we send their subnets to their layer 3 devices from this road.

So we need to generate an alternate method and what we decide is

1. Ex connection weill be normal family inet connection and we will build all Vlans on EX4500 this time we should put a rule on this interface to send the return of traffic to SRX but the problem is we have too many servers so EX having difficulties

Solution is to buy one more Router but that costs so much. this is why we are looking for an alternative solution.

ISP   --------  VR1 A   ----- SRX/UDP FW -------- VR1 B return -------- VR2  (Vlans / Static routes ....etc) ------ EX4500

Yes that should be the solution :

For the internal gateways, you might consider creating a new virtual router routing-instance on the MX. 

Place the interface leaving the SRX to the internal network into this routing-instance

Place the uplink from the EX to the MX into this routing-instance

Create all your internal routed vlan interfaces in this routing-instance

If we should add family inet firewall filters for input on VR1A and VR1B 

but i do not know how to do it 🙂 do you have any sample for this 🙂

BGP  Needs to be on VR1 with upstream ISP

VR1 -- SRX

VR1 -- VR2 connections must be metric or OSPF

You can build your desired setup with the two VR but you will need to have a pair of interfaces put in the two VR and loop those two tegether in this configuration.  There are some internal connection methods but these are generally for leaking routes not treating an interface with a filter.

Basically assign interfaces to the VR and then treat those interfaces as if you had completly separate routers.

Overview

http://www.juniper.net/documentation/en_US/junos12.3/topics/concept/routing-instances-overview.html

Examples

http://www.juniper.net/techpubs/en_US/junos15.1/topics/topic-map/logical-systems-virtual-datacenter.html

http://www.juniper.net/documentation/en_US/junos12.3/topics/example/virtual-router-srx-use-case-edu-configuring.html

I've checked them  and start building config but just need a little help 🙂

how should i put a firewall filter on R1's input traffic that comes from R2 to R1 

chassis {
    aggregated-devices {
        ethernet {
            device-count 4;
        }
    }
    network-services all-ethernet;
}
routing-options {
    interface-routes {
        rib-group inet SPD-Route;
    }
    rib-groups {
        SPD-Route {
            import-rib [ inet.0 TCP-Routes-Donus.inet.0 ]
        }
    }
    static {
    autonomous-system 57844;
    forwarding-table {
        unicast-reverse-path feasible-paths;
    }
}
interfaces {
        xe-0/0/0 {
            gigether-options {
                802.3ad ae0;
            }
        }
        ae0 {
            description ISP;
            aggregated-ether-options {
                minimum-links 1;
                lacp {
                    active;
                    periodic fast;
                }
            }
            unit 0 {
                family inet {
                    address 10.32.35.14/30;
                }
            }
        }
        xe-0/0/1 {
            unit 0 {
                family inet {
                    filter {
                        input FWDirect;
                        output CikisTrafik;
                    }
                    address 10.10.10.1/30;
                }
            }
        }
        xe-0/0/2 {
            unit 0 {
                family inet {
                    address 10.10.10.5/30;
                }
            }
        }
        xe-0/0/3 {
            flexible-vlan-tagging;
            unit 0 {
                family bridge {
                    interface-mode trunk;
                    vlan-id-list 1-4094;
                }
            }
        }
        policy-options {
            prefix-list Firewall {
                185.9.157.15/32;
                185.90.83.0/24;
            }
            policy-statement import-from-r1 {
                term 1 {
                    from instance r1;
                    then accept;
                }
            }
            policy-statement import-from-r2 {
                term 1 {
                    from instance r2;
                    then accept;
                }
            }
            policy-statement SPD-OUT {
                term F {
                    from {
                        prefix-list DDOS-PREFIX-LIST;
                    }
                    then {
                        community + Firewall;
                        accept;
                    }
                }
                term 1 {
                    from {
                        prefix-list SPD-PREFIX-LIST;
                    }
                    then accept;
                }
                term 500 {
                    then reject;
                }
            }
            policy-statement SPD-IN {
                term 1 {
                    then accept;
                }
            }
            community Firewall members [ 9121:444 43391:111 ];
        }
        routing-instances {
            TCP-Routes-Donus {
                instance-type forwarding;
                routing-options {
                    static {
                        route 0.0.0.0/0 {
                            qualified-next-hop 10.10.10.21 {
                                metric 21;
                            }
                            qualified-next-hop 10.32.35.13 {
                                metric 22;
                            }
                        }
                    }
                }
            }
            r1 {
                instance-type virtual-router;
                protocols {
                    bgp {
                        traceoptions {
                            file bgp-trace world-readable;
                            flag open;
                            flag state detail;
                        }
                        group ISP {
                            type external;
                            description ISP;
                            local-address 10.32.35.14;
                            import SPD-IN;
                            export SPD-OUT;
                            peer-as 43391;
                            neighbor 10.32.35.13;
                        }
                    }
                }
                interface ae0.0;
                interface xe-0/0/1.0;
                interface xe-0/0/2.0;
                routing-options {
                    instance-import import-from-r2;
                }
            }
            r2 {
                instance-type virtual-router;
                interface xe-0/0/3.0;
                routing-options {
                    instance-import import-from-r1;
                }
            }
        }
    }
}
firewall {
    filter FWDirect {
        term UDPFW {
            from {
                destination-prefix-list {
                    Firewall;
                }
                protocol udp;
            }
            then {
                log;
                next-ip 10.10.10.6/32;
            }
        }
        term TCPFW {
            from {
                destination-prefix-list {
                    Firewall;
                }
                protocol tcp;
                tcp-flags 0x02,0x10,0x12,0x18;
            }
            then {
                count TCPFWTR;
                log;
                next-ip 10.10.10.2/32;
            }
        }
        term ICMP {
            from {
                destination-prefix-list {
                    Firewall;
                }
                protocol icmp;
            }
            then {
                count SAYAC;
                next-ip 10.10.10.2/32;
            }
        }
        term FWBlok {
            from {
                destination-prefix-list {
                    Firewall;
                }
            }
            then {
                count Bloklanan;
                discard;
            }
        }
        term Default {
            then accept;
        }
    }
    filter rpf-filter {
        term default {
            then {
                count rpf-failed-count;
                reject;
            }
        }
    }
    filter CikisTrafik {
        term icNetwork {
            from {
                source-prefix-list {
                    SPD-PREFIX-LIST;
                }
            }
            then accept;
        }
        term ISP {
            from {
                source-address {
                    10.32.35.12/30;
                }
            }
            then accept;
        }
        term 500 {
            then {
                count DroppedSpoof;
                discard;
            }
        }
    }
    filter FWDonus {
        term TrafikYonlendir {
            from {
                source-prefix-list {
                    Firewall;
                }
            }
            then {
                routing-instance TCP-Routes-Donus;
            }
        }
        term 500 {
            then accept;
        }
    }
}

Your issue is that you really can't apply a filter when you make the connection between R1 and R2 via instance import.

To use the filtering methods you will need to assign one interface to R1 and another to R2 then add a cable to connect those two interfaces.  Now you can configure the interfaces and assign the desired filters for traffic between the two VR.

We do not have empty interface but should we create virtual interface on MX80 ?

You can use logical tunnel interfaces to make the internal connection between the VR.  But I don't believe you will be able to apply the filters on these interfaces.  You can give it a shot.  And it may give you enough flexibility with routing via fitter based forwarding to work in your scenario.

http://www.juniper.net/techpubs/en_US/junos15.1/topics/example/logical-systems-connecting-ls-interface.html

Wonderfull idea !!!!!!

O think it should work let me complete the config , so i have 2 last questions ,

do you think it is better to complete BGP under R1 or general config ? 

I am working with virtual router first time. Where will i create the vlans ? 🙂 do you know any sample for it i have checked on google but could not find how to put bridge domains in virtual routers.

root@mx80-core# commit


commit complete

[edit]
root@mx80-core# show logical-systems
LS {
    interfaces {
        lt-0/0/1 {
            unit 1 {
                encapsulation ethernet;
                peer-unit 2;
                family inet {
                    filter {
                        input FWDonus;
                    }
                    address 10.10.11.1/30;
                }
            }
            unit 2 {
                encapsulation ethernet;
                peer-unit 1;
                family inet {
                    address 10.10.11.2/30;
                }
            }
        }
    }
}

[edit]
root@mx80-core#

I assume you will be creating your VLAN gateway addresses on the MX VR based on this.  On the EX we use RVI for this on the MX you will be creating IRB interfaces.

You will assign the IRB interface to the desired virtual router (R2 in this case).  If you don't assign them they would belong to the root routing instance.

Thanks for all your help 

we have build and tested everything and faced with one last issue and send you an email 

I hope you will check and help on it to us too 

Thank you