Routing

There is an ISP I work with who, for the purposes of this post will be called "Awesome-Net".
They have two upstream links. One to "Provider1" and one to "Provider2".
Their downstream customers have different agreements with them regarding how they can use Awesome-Net's network for Internet Transit. There are the following categories:
CUST1 - Use Awesome-Net's Provider1 uplink as primary both inbound and outbound, and are allowed to have their traffic failover to Awesome-Net's Provider2 uplink if there is an issue on Provider1's network.
CUST2 - Use Awesome-Net's Provider1 uplink only. If Provider1 has issues, their traffic does not failover to Awesome-Net's Provider2 uplink.
CUST3 - Use Awesome-Net's Provider2 uplink as primary both inbound and outbound, and are allowed to have their traffic failover to Awesome-Net's Provider1 uplink if there is an issue on Provider2's network.
CUST4 - Use Awesome-Net's Provider2 uplink only. If Provider2 has issues, their traffic does not failover to Awesome-Net's Provider1 uplink.

Awesome-Net is currently ensuring these agreements are met for traffic inbound to the customers through Awesome-Net by using BGP communities for each customer type and advertising the customer routes to Provider2 and Provider1 based on those communities, using AS-Path Prepending to make the customers' routes look more desirable on the appropriate links.

However, the hard part is ensuring these agreements are met for customer traffic outbound to the Internet through Awesome-Net. I am looking to the Juniper community to as what would be the best solution to implement to accomplish this. Should we use MPLS? Can we control this via BGP somehow? Is there another technology or solution altogether that can accomplish the proper outbound traffic control? I would rather not have to configure filter-based forwarding for every single customer.

Right now what is happening is that all customer traffic outbound to the Internet through Awesome-Net is going out Awesome-Net's Provider1 uplink.

It's also worth noting that with the current setup, Provider1 sends Awesome-Net a default route, while Provider2 sends Awesome-Net full Internet routes. This can be changed if needed.

Hi

To be able to answer the Questions I need to know which prefixes the customers are using for their own network.

Are those PI ( provider independent) or PA ( subnet from our or from upstream provider network).

The same question you need to answer regarding your own used routes in case the customer uses PA routes

regards

alexander

Many of the customers have their own public subnets and peer with Awesome-Net via BGP.They fit into categories CUST3 and CUST4.

Awesome-Net provides IPs within their own BGP AS to it's other customers. Those all fit into the category CUST1.

I just realized my answer might not have been detailed enough.

I am uploading a PDF network map at its most basic level.

In this PDF example, customers 1 thru 3 are all PI. They have their own BGP AS and assigned subnets that are independent of Awesome-Net and its upstream providers.

Any customers that would be off of R1 in the PDF, all use IPs assigned and provided by Awesome-Net. Those IPs are indepentent of Provider1 and Provider2 (ISP1 and ISP2 in the PDF). They are owned by Awesome-Net and are a parts of its BGP AS.

Does this tell you what you need to know?

Attachment(s)

Awesome-Net-Infrastructure-Basic-Example.pdf   128 KB 1 version

Hello there,

One way to accomplish this is to use MPLS L3VPN (a.k.a. VRFs) with targeted route leaking.

Rough algorithm:

0/ it would help a lot if Your Provider1 sends 0/1 & 128/1 instead of 0/0

1/ create 4 VRFs as below:

1a/ CUST1-VRF is the "primary" for CUST1 and holds all its routes. CUST2,3,4 routes are leaked into CUST1-VRF as well.

CUST1-VRF is also primary for ISP1 and receives 0/1 & 128/1 from ISP1. CUST1-VRF has a static 0/0 route with "next-table CUST2-VRF.inet.0" configured.

1b/  CUST2-VRF is the "primary" for CUST2 and holds all its routes. CUST1,3,4 routes are leaked into CUST2-VRF as well.

CUST2-VRF is also primary for ISP2 and receives full table.

1c/ CUST3-VRF is the "primary" for CUST3 and holds all its routes. CUST1,2,4 routes are leaked into CUST3-VRF as well.

ISP1 and ISP2 routes are leaked into CUST3-VRF from CUST1-VRF & CUST2-VRF. 

1d/ CUST4-VRF is the "primary" for CUST4 and holds all its routes. CUST1,2,3 routes are leaked into CUST4-VRF as well.

ISP2 routes are leaked into CUST4-VRF.

2/ the leaking between VRFs is done on a local PE by means of RIB-groups(if BGP) +auto-export (if direct/static) and on remote PE by means of crafted VRF import policies.

The advantages:

a/ You can enforce almost any business rule this way

b/ with MPLS as transport, You can have multiservice network from Day1 and gradually add E-LINE, E-LAN (VPLS), etc on the fly

Disadvantages:

i) in certain cases, when CUST2,3,4-VRFs are on same PE, that PE has to hold up to 3 copies of full table, one copy in each VRF. This cuonsumes memory but with modern Routing Engines, it is not that critical.

ii) If You are not familiar with MPLS and L3VPNs, then Your learning curve would look like Delta-4 rocket launch 

HTH

Thx

Alex

Is there any chance you would be able to give me any examples fo what this configuration might look like? I am following the idea of the solution you are suggesting, but configuring it is another matter. 

Also, what do you mean by Provider1 sending 0/1 instead of 0/0?

Hello,

Here You go.

1. I modeled full table with folowing 7 routes:

0.0.0.0/3 
32.0.0.0/3 
64.0.0.0/3 
96.0.0.0/3 
128.0.0.0/3 
160.0.0.0/3 
192.0.0.0/3 

2. Please see below the example configuration for 2 customer VRF (aptly named "CUST1" and "CUST2"), I'll let You extend this to 4 Yourself. I also let you do the transport part (IGP, MPLS and MP-BGP).

interfaces {
    ge-0/0/6 {
        description "inside CUST1 VRF";
        unit 0 {
            family inet {
                address 198.15.6.1/30;
            }
        }
    }
    ge-0/0/7 {
        description "inside CUST2 VRF";
        unit 0 {
            family inet {
                address 198.15.7.1/30;
            }
        }
    }
}
policy-options {
    policy-statement CUST1-EXP {
        term 1 {
            from protocol [ static direct bgp ];
            then {
                community add CUST1-RT;
                accept;
            }
        }
    }
    policy-statement CUST1-IMP {
        term cust1 {
            from community CUST1-RT;
            then accept;
        }
        term cust2 {
            from community CUST2-RT;
            then accept;
        }
        term prov1 {
            from community PROV1-RT;
            then accept;
        }
    }
    policy-statement CUST2-EXP {
        term 1 {
            from protocol [ static direct bgp ];
            then {
                community add CUST2-RT;
                accept;
            }
        }
    }
    policy-statement CUST2-IMP {
        term cust1 {
            from community CUST1-RT;
            then accept;
        }
        term cust2 {
            from community CUST2-RT;
            then accept;
        }
        term prov1 {
            from community PROV2-RT;
            then accept;
        }
    }
    policy-statement LOAD-BALANCE {
        term 1 {
            then {
                load-balance per-packet;
            }
        }
        then {
            load-balance per-packet;
        }
    }
    community CUST1-RT members target:203:1;
    community CUST2-RT members target:203:2;
    community PROV1-RT members target:203:100;
    community PROV2-RT members target:203:200;
}
routing-instances {
    CUST1 {
        instance-type vrf;
        interface ge-0/0/6.0;
        route-distinguisher 203.0.113.3:1;
        vrf-import CUST1-IMP;
        vrf-export CUST1-EXP;
        vrf-table-label;
        routing-options {
            static {
                route 0.0.0.0/0 next-table CUST2.inet.0;
            }
            auto-export;
        }
    }
    CUST2 {
        instance-type vrf;
        interface ge-0/0/7.0;
        route-distinguisher 203.0.113.3:2;
        vrf-import CUST2-IMP;
        vrf-export CUST2-EXP;
        vrf-table-label;
        routing-options {
            auto-export;
        }
    }
}

3. Verification printouts are given below:

regress@R3> show route table CUST1 

CUST1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 00:06:20
                      to table CUST2.inet.0 <======== NEXT BEST ROUTE IN CASE PROV1 0.0.0.0/1 AND 128.0.0.0/1 ARE NOT IN TABLLE
0.0.0.0/1          *[BGP/170] 00:14:00, localpref 600, from 203.0.113.6 
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 17
128.0.0.0/1        *[BGP/170] 00:14:00, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 17
169.254.8.0/31     *[BGP/170] 00:14:00, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 17
198.15.6.0/30      *[Direct/0] 00:04:20
                    > via ge-0/0/6.0
198.15.6.1/32      *[Local/0] 00:04:20
                      Local via ge-0/0/6.0
198.15.7.0/30      *[Direct/0] 00:04:20  <========= LEAKED FROM CUST2 VRF
                    > via ge-0/0/7.0

regress@R3> show route table CUST2    

CUST2.inet.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 00:04:23
                      to table CUST2.inet.0
0.0.0.0/3          *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
32.0.0.0/3         *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
64.0.0.0/3         *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
96.0.0.0/3         *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
128.0.0.0/3        *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
160.0.0.0/3        *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
169.254.9.0/31     *[BGP/170] 00:11:55, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
192.0.0.0/3        *[BGP/170] 00:07:10, localpref 600, from 203.0.113.6
                      AS path: I, validation-state: unverified
                    > to 169.254.13.0 via ae0.0, Push 18
198.15.6.0/30      *[Direct/0] 00:04:23  <========= LEAKED FROM CUST1 VRF
                    > via ge-0/0/6.0
198.15.7.0/30      *[Direct/0] 00:04:23 
                    > via ge-0/0/7.0
198.15.7.1/32      *[Local/0] 00:04:23
                      Local via ge-0/0/7.0

HTH

Thx

Alex