Screen OS

Hi everyone.
This concerns a Netscreen NS25 (ScreenOS 5.4.0r10).
ethernet1 = Trust (10.1.0.0/16)  Trust-VR
ethernet3 = Untrust (static WAN fixed IP) Trust-VR
ethernet4 = EDPNet (PPoE with fixed IP) EDPNet-VR
ethernet4.1 = EDPNet (extra /29 IP-range on the PPoE connection) EDPNet-VR

The ethernet3 interface is the 'default internet connection'. The 0.0.0.0/0 route is set statically in trust-vr.
My original plan was to put the 2nd internet connection on ethernet4 also in the Trust-VR router. But because it's PPoE it gets a default GW automatically and would put this default gateway in my routing table on Trust-VR as directly connected and this gets higher priority then the statically set 0.0.0.0/0 gateway for my ethernet3.

To overcome that i created a new zone called EDPNet and created a new virtual router called it EDPNet-VR.
Now the problem is that for a group of computers i want to redirect surfing the web to the EDPNet connection.
So i created a PBR with access-list ports 80 & 443:

Relevant config parts:

set vrouter trust-vr sharableset vrouter "untrust-vr"
exit
set vrouter "trust-vr"
unset auto-route-export
exit
set vrouter name "EDPNet-VR" id 1025 sharableunset vrouter "EDPNet-VR" nsrp-config-syncset vrouter "EDPNet-VR"
unset auto-route-exportexit
set zone "Trust" vrouter "trust-vr"
set zone "Untrust" vrouter "trust-vr"
set zone "DMZ" vrouter "trust-vr"
set zone "VLAN" vrouter "trust-vr"
set zone id 100 "edpnet"
set zone "edpnet" vrouter "EDPNet-VR"
set zone "Untrust-Tun" vrouter "trust-vr"
set zone "Trust" tcp-rst
set zone "Untrust" block
unset zone "Untrust" tcp-rst
set zone "MGT" block
set zone "DMZ" tcp-rst
set zone "VLAN" block
set zone "VLAN" tcp-rst
set interface "ethernet1" zone "Trust"
set interface "ethernet2" zone "Null"
set interface "ethernet3" zone "Untrust"
set interface "ethernet4" zone "edpnet"
set interface "ethernet4.1" encap "pppoe" zone "edpnet"
set interface ethernet1 ip 10.1.5.254/16
set interface ethernet1 nat
set interface ethernet3 ip 81.246.22.x
set interface ethernet3 route
set interface ethernet4 ip 77.109.83.x
set interface ethernet4 route
set interface ethernet4.1 ip 77.109.120.x
set interface ethernet4.1 route
set vrouter "trust-vr"
set source-routing enableset sibr-routing enable
unset add-default-route
set route 0.0.0.0/0 interface ethernet3 gateway 81.246.22.x
set access-list extended 50 src-ip 10.1.1.15/32 dst-port 80-80 protocol tcp entry 1
set access-list extended 50 src-ip 10.1.1.15/32 dst-port 443-443 protocol tcp entry 2
set match-group name be01c008set match-group be01c008 ext-acl 50 match-entry 50
set action-group name toEDPNetset action-group toEDPNet next-hop 212.71.0.45 action-entry 2
set pbr policy name SurfenEDPNetset pbr policy SurfenEDPNet match-group be01c008 action-group toEDPNet 1
exit
set interface ethernet1 pbr SurfenEDPNet

the route table looks like this trough webGui:

But my PBR is not working. The computer that i used in the ACL-ext can't surf anymore.
I have created the PBR in the trust-vr because i think thats needed when i apply it to clients in the trust-vr.
I have also tried to set the action to interface instead of next-hop, but this doesn't work either.

Can someone help me with this ?
If you need more info please ask me.

ps: As you can see i used to ISP2 gateway as next-hop action. And in trust-vr i added a route:
212.71.0.45/32 212.71.0.45 ethernet4 SP 20 1

Hi Stan-Gobien  ,

Configuring PBR  across virual routers needs  manythings to be taken into consideration

1. The "action-group" needs to contain a "next-hop" value only.  Do NOT enter a "next-interface" value.

I can see that you already did that

2. A self-referenced host (/32) route needs to be added for the next-hop value.

- regardless that the next-hop is reachable via the routing table or even directly connected, PBR still needs the /32 host route.

This requirement seems strange as it works okay without it in a single-VR setup; however, for cross-VR traffic PBR will fail without the host route to the next-hop address - even though a valid, less-specific route exists.

eg. set vr <egress-vr> route <next-hop>/32 interface <interface name> gateway <next-hop>

Note: You must specify the same "/32" next-hop value along with the "gateway" value. Without both, PBR will also fail.

You can refer to the following KB :

http://kb.juniper.net/index?page=content&id=KB9404&actp=search&searchid=1271701753741

So ,

you will craete the PBr on the ingress virtual router ( virtual router having the interface receiving the traffic )

&

you will put the / 32 route  at the egrees virtual router ( virtual router having the outgoing interface )

I have created the PBR on the ingress-vr, in my case called trust-vr. This is the vr where my trust interface is located where traffic is originating. I have created a route (as you suggested) in the egress-vr, in my case called EDPNet-vr. This is the vr where my outgoing interface ethernet4 is located. But this is not working !

Copy/paste of the routing tables:

trust-vr 
  IP/Netmask Gateway Interface Protocol Preference Metric Vsys Configure 
* 10.1.0.0/16   ethernet1 C     Root  - 
* 10.1.5.254/32   ethernet1 H     Root  - 
* 81.246.22.xx/30   ethernet3 C     Root  - 
* 81.246.22.xy/32   ethernet3 H     Root  - 
* 0.0.0.0/0 81.246.22.xz ethernet3 S 20 1 Root  Remove 
EDPNet-VR 
  IP/Netmask Gateway Interface Protocol Preference Metric Vsys Configure 
* 77.109.83.yx/32   ethernet4 C     Root  - 
* 77.109.83.yx/32   ethernet4 H     Root  - 
* 0.0.0.0/0 212.71.0.45 ethernet4 C   1 Root  - 
* 212.71.0.45/32 212.71.0.45 ethernet4 SP 20 1 Root  Remove 
* 77.109.120.2zx/29   ethernet4.1 C     Root  - 
* 77.109.120.2zy/32   ethernet4.1 H     Root  - 

 On the client PC i use in my ext.ACL (10.1.1.50)  i can surf the web but it's going out the normal standard interface.  So something is still wrong. Could it be because of my ISP2 gateway not being inside the range defined on my interface4 ?

And again here is my config (the relevant part):

set vrouter "untrust-vr"
exit
set vrouter "trust-vr"
set source-routing enable
set sibr-routing enable
unset add-default-route
set route 0.0.0.0/0 interface ethernet3 gateway 81.246.22.81
set access-list extended 20 src-ip 10.1.1.50/32 dst-port 80-80 protocol tcp entry 1
set match-group name 50pc
set match-group 50pc ext-acl 20 match-entry 10
set action-group name toEDPNet
set action-group toEDPNet next-hop 212.71.0.45 action-entry 2
set pbr policy name SurfenEDPNet
set pbr policy SurfenEDPNet match-group 50pc action-group toEDPNet 1
exit
set vrouter "EDPNet-VR"
set source-routing enable
set sibr-routing enable
set route 212.71.0.45/32 interface ethernet4 gateway 212.71.0.45 permanent
exit
set interface ethernet1 pbr SurfenEDPNet
set vrouter "untrust-vr"
exit
set vrouter "trust-vr"
exit
set vrouter "EDPNet-VR"
exit

PS: another thing to mention, is that i have NAT enabled on the trust interface not in my policies.

 " Could it be because of my ISP2 gateway not being inside the range defined on my interface4 ? "

Yes , how come that your next hop is not at the same subnet as your interface

change that next hop to your correct next hop for e4 & it should work

That next-hop is the default gateway for my ISP2. It is the correct default gateway. This is a PPoE ADSL connection directly on the netscreen.

How can i resolve this ?

ns25-> unset ff
unset ff
filter 0 removed
ns25-> get ff
get ff
ns25-> set ff src-ip 10.1.1.50 dst-port 80
set ff src-ip 10.1.1.50 dst-port 80
filter added
ns25-> get ff
get ff
Flow filter based on:
id:0 src ip 10.1.1.50 dst port 80
ns25-> debug flow basic
debug flow basic
ns25-> undebug all
undebug all
ns25-> get db str

****** 31691577.0: <Trust/ethernet1> packet received [48]******
  ipid = 25943(6557), @c7d01110
  packet passed sanity check.
  ethernet1:10.1.1.50/42459->72.233.89.198/80,6<Root>
  no session found
  flow_first_sanity_check: in <ethernet1>, out <N/A>
  chose interface ethernet1 as incoming nat if.
  flow_first_routing: in <ethernet1>, out <N/A>
  search route to (ethernet1, 10.1.1.50->72.233.89.198) in vr trust-vr for vsd-0/flag-0/ifp-null
PBR lookup params: dst-ip: 72.233.89.198, src-ip: 10.1.1.50, dst-port: 80, src-port: 42459, protocol: 6, dscp: 0
  [PBR route] 5.route 72.233.89.198->81.246.22.xy, to ethernet3
  routed (x_dst_ip 72.233.89.198) from ethernet1 (ethernet1 in 0) to ethernet3
  policy search from zone 2-> zone 1
 policy_flow_search  policy search nat_crt from zone 2-> zone 1
  RPC Mapping Table search returned 0 matched service(s) for (vsys Root, ip 72.233.89.198, port 80, proto 6)
  No SW RPC rule match, search HW rule
  log this session (pid=160)
  packet dropped, denied by policy

So PBR still send the packet to interface ethernet3, that is wrong. It should go to ethernet4.
This is my PBR config:

set access-list extended 20 src-ip 10.1.1.50/32 dst-port 80-80 protocol tcp entry 1
set match-group name 50pc
set match-group 50pc ext-acl 20 match-entry 10
set action-group name toEDPNet
set action-group toEDPNet next-hop 212.71.0.45 action-entry 2
set pbr policy name SurfenEDPNet
set pbr policy SurfenEDPNet match-group 50pc action-group toEDPNet 1
exit
set vrouter "EDPNet-VR"
set source-routing enable
set sibr-routing enable
set route 212.71.0.45/32 interface ethernet4 gateway 212.71.0.45 permanent
exit
set interface ethernet1 pbr SurfenEDPNet

Please advise urgently !

Remove  " permanent from the below route & run the debug again :

set route 212.71.0.45/32 interface ethernet4 gateway 212.71.0.45 permanent

I have the same issue on ScreenOS v6.2.0r1.0 (SSG5).

I have added a second ISP and because this new link is over DHCP I have bound eth0/1 to untrust-vr but the ISP gw is on the same subnet as eth0/1 IP address.

PBR is configured as in the following tutorial:

http://www.corelan.be:8800/index.php/2008/10/19/using-2-internet-links-with-juniper-screenos-firewalls-to-separate-traffic-and-apply-traffic-shaping/

and also implemented the setup according KB:

http://kb.juniper.net/index?page=content&id=KB9404&actp=search&searchid=1231989420431

the idea is to forward all traffic from few subnets (Trust) over the second ISP by default but the traffic is still forwarded over ethernet0/0 (ISP 1, Untrust in trust-vr)

Please advice!

Thanks

Frankly , i donot know what to do when your nexthop is not at the same subnet of the interface

I think that will cause the PBR not to  work because it will not be able to resolve the next hop

Can't i just put a route in trust-vr for 212.71.0.45/32 to edpnet-vr (or perhaps interface4) ?

PS: I removed the "permanent" from the route. Still no joy.

Yes , you can try it to put a route in trust-vr for 212.71.0.45/32 to edpnet-vr   ( donot put it for e4 as the trust-vr donot know about e4  )

Hi both,

That solved my issue, I have added a /32 route to the next hop in trust-vr (to untrust-vr) and now I see traffic going thru the PBR policy.

Thank you SSHSSH !

George

my case doesn't seem to work.

A quick question to try to circumvent this:

How can i give a statically set 0.0.0.0/0 route higher preference then a connected 0.0.0.0/0 route (PPoE/DHCP) in 1 and the same vritual router ?

If i can do this, then perhaps i can put interface4 in trust-vr and my problem is gone.

I'm not sure if you can setup a metric higher than a connected route, at least I don't think so. Another workaround is to add the trust interface on the same vr as eth4 but you won't have the same flexibility.

But it should work in your case also, I have exactly the same setup but is now solved with the second next-hop/32 route from trust-vr Next-hop: untrust-vr.

Also the /32 route from Juniper KB was kept, it doesn't work without.

You can also try to upgrade if you don't have ScreenOS 6.x, this might be important too.

I don't have access to ScreenOS 6 for NS25.

My routing table looks like this now, can you confirm this is how you fixed it ?

trust-vr
  	IP/Netmask 	Gateway 	Interface 	Protocol 	Preference 	Metric 	Vsys 	Configure
*	10.1.0.0/16	 	ethernet1	C	 	 	Root 	-
*	10.1.5.254/32	 	ethernet1	H	 	 	Root 	-
*	81.246.22.xx/30	 	ethernet3	C	 	 	Root 	-
*	81.246.22.xy/32	 	ethernet3	H	 	 	Root 	-
*	0.0.0.0/0	81.246.22.xz	ethernet3	S	20	1	Root 	Remove
*	212.71.0.45/32	EDPNet-VR	-	S	20	1	Root 	Remove


EDPNet-VR
  	IP/Netmask 	Gateway 	Interface 	Protocol 	Preference 	Metric 	Vsys 	Configure
*	77.109.83.yy/32	 	ethernet4	C	 	 	Root 	-
*	77.109.83.yy/32	 	ethernet4	H	 	 	Root 	-
*	212.71.0.45/32	212.71.0.45	ethernet4	S	20	1	Root 	Remove
*	0.0.0.0/0	212.71.0.45	ethernet4	C	 	1	Root 	-
*	77.109.120.xyz/29	 	ethernet4.1	C	 	 	Root 	-
*	77.109.120.xyx/32	 	ethernet4.1	H	 	 	Root 	-

Ok i have been doing some debugging:

****** 32043836.0: <Trust/ethernet1> packet received [52]******
  ipid = 31145(79a9), @c7d0b110
  packet passed sanity check.
  ethernet1:10.1.1.3/61558->72.233.89.199/80,6<Root>
  no session found
  flow_first_sanity_check: in <ethernet1>, out <N/A>
  chose interface ethernet1 as incoming nat if.
  flow_first_routing: in <ethernet1>, out <N/A>
  search route to (ethernet1, 10.1.1.3->72.233.89.199) in vr trust-vr for vsd-0/flag-0/ifp-null
PBR lookup params: dst-ip: 72.233.89.199, src-ip: 10.1.1.3, dst-port: 80, src-port: 61558, protocol: 6, dscp: 0
  [PBR route] 18.route 72.233.89.199->212.71.0.45, to ethernet4
  routed (x_dst_ip 72.233.89.199) from ethernet1 (ethernet1 in 0) to ethernet4
  policy search from zone 2-> zone 100
 policy_flow_search  policy search nat_crt from zone 2-> zone 100
  RPC Mapping Table search returned 0 matched service(s) for (vsys Root, ip 72.233.89.199, port 80, proto 6)
  No SW RPC rule match, search HW rule
Permitted by policy 199
  No src xlate   choose interface ethernet4 as outgoing phy if
  no loop on ifp ethernet4.
  session application type 6, name HTTP, nas_id 0, timeout 300sec
  service lookup identified service 0.
  flow_first_final_check: in <ethernet1>, out <ethernet4>
  existing vector list 3-31faac0.
  Session (id:19677) created for first pak 3
  flow_first_install_session======>
  route to 212.71.0.45
  serial or adsl or ml if, nsp ready.
  nsp2 wing prepared, ready
  cache mac in the session
  make_nsp_ready_no_resolve()
  search route to (ethernet4, 72.233.89.199->10.1.1.3) in vr trust-vr for vsd-0/flag-3000/ifp-ethernet1
  [ Dest] 1.route 10.1.1.3->10.1.1.3, to ethernet1
  route to 10.1.1.3
  flow got session.
  flow session id 19677
  adjust tcp mss.
  Got syn, 10.1.1.3(61558)->72.233.89.199(80), nspflag 0x801801, 0x2800
  send out through normal path.
  flow_ip_send: 79a9:10.1.1.3->72.233.89.199,6 => ethernet4(52) flag 0x0, vlan 0
 send packet to traffic shaping queue.
****** 32043839.0: <Trust/ethernet1> packet received [52]******
  ipid = 31193(79d9), @c7d06910
  packet passed sanity check.
  ethernet1:10.1.1.3/61558->72.233.89.199/80,6<Root>
  existing session found. sess token 4
  flow got session.
  flow session id 19677
  adjust tcp mss.
  Got syn, 10.1.1.3(61558)->72.233.89.199(80), nspflag 0x801801, 0x2800
  send out through normal path.
  flow_ip_send: 79d9:10.1.1.3->72.233.89.199,6 => ethernet4(52) flag 0x0, vlan 0
  send packet to traffic shaping queue.
  flow_ip_send: 79d9:10.1.1.3->72.233.89.199,6 => ethernet4(52) flag 0x20000, vlan 0
 pak has mac
  Send to ethernet4 (74)

Debugging and logging saved the day.

Logging of policy 199 (from trust to edpnet any) showed that SOURCE NAT was not being applied.

I think this is strange because i have NAT enabled on my trust interface. Must be a PBR thing.

I have now enabled SOURCE NAT on my policy and it's working !

Thanks for all the help !

This is a perfect example of why best practices always say NAT in policies not on interfaces.

Thanks for providing all the details along the way.  I've not used multiple VRs yet and this whole exercise here was very helpful in seeing how they work.

I understand, but honestly i have nearly 100 policies on this firewall.

Enabling NAT on all of them that need it, and the others not (VPN policies) is just a burden.

Interface NAT is so much easier.

Coming back on this subject...

Everything works fine except that the subnets which are routed thru the ISP2 (untrust-vr) need to access resources located on the local DMZ and Trust (trust-vr) but now all traffic that originate from those subnets is forwarded to untrust-vr.

Is there a way to solve this? I'm thingink on new ACL but I tried some scenarioos and it didn't worked.

Please advice,

Thanks!

In your ACL you could add destination address range instead of just port.

You add the destination addresses as such that you exclude your DMZ and trust ranges.

 example:

ACL-ID1: dst-ip 0.0.0.0-192.167.255.255

ACL-ID2 dst-ip 192.169.0.0-255.255.255.255

I think you get the picture.

EDIT: Above is wrong, i meant 2 sequences. See my message 2 posts below.

I know, but I need to forward all traffic except my local addresses from DMZ, Trust, etc.

Normally I should create another policy with another ACL, or at least another Action Group where to define the next hop as local DMZ interface or something like that.

But I can bind only one PBR at the time, at least this is what I saw by now.

You can have 2 sequences in 1 ACL. In my previous message i said 2 ACL-ID's, that was wrong, i meant 2 sequences for 1 ACL-ID.

So if you exclude your ranges by configuring multiple sequences with the addresses, those excluded addresses will not be effected by the PBR and will use the normal routing table & policies.

So something like:

set access-list extended 50 src-ip 0.0.0.0-192.167.255.255 dst-port 80-80 protocol tcp entry 1
set access-list extended 50 src-ip 192.168.255.255-255.255.255.255 dst-port 80-80 protocol tcp entry 2
set match-group name MATCH1
set match-group MATCH1 ext-acl 50 match-entry 50
set action-group name ACTION1
set action-group ACTION1 next-hop x.x.x.x action-entry 2
set pbr policy name POLICY1
set pbr policy POLICY1 match-group MATCH1 action-group ACTION1 1

Syntax may be wrong, edited just out of my head.

Yes but since I have to forward all traffic coming from xx.xx.xx.xx/20 (internal subnets) to 0.0.0.0/0 (internet) except my local DMZ, this is quite difficult.

Basically what I already did I have forwarded most common ports like 80, 443, 25, 110, etc but I have to check all the time if something causing huge traffic like bittorrent, DC++, IKE, etc must be added in PBR since on the second ISP I have far more bandwidth..

Ideally was to be able to specify in the ACL src IP, dst IP, EXCEPT dst IP...

But then it will be too nice...