SRX

Hi all,

I have a reth with multiple subnets configured on it and I'm having trouble getting *some* things to work between subnets.  I have an intra-zone policy with a single term configured to permit all traffic:

policy intra_vl6 {
    match {
        source-address any;
        destination-address any;
        application any;
    }
    then {
        permit;
        count;
        log {
            session-close;
        }
    }
}

The odd thing is, only some traffic (tcp/udp) is failing.  I have two servers, A and B, each on a different subnet on the same reth interface (not VLAN, but subnet).  I can ping both ways (A to B and B to A) without fail.  I can SSH from A to B without fail, but cannot SSH from B to A.  A 'show security flow session' output doesn't show any information whatsoever when an SSH attempt is made, but it will for ICMP.  I tried configuring a policy to specifically allow A to B and vice-versa, log that traffic, and count it.  Not a single log shows up, either permit or deny, and no traffic is counted.

A tcpdump on server A when the SSH attempt is made from B will show the SYN traffic coming in and a SYN-ACK response sent back to server B, but the tcpdump on server B doesn't show the SYN-ACK coming back.  A port mirror on each server's switchport shows the same behavior - SYN inbound from B, SYN-ACK inbound from A, but no SYN-ACK outbound to B (on B's port).

Has anyone else had strangeness like this on intra-zone policies before?  I don't understand why I'd be able to get from A to B, but not B to A.  Logging on intra-zone traffic doesn't seem to produce any results at all and the session flow stats are inconsistent, at best.

    reth2 {
        enable;
        redundant-ether-options {
            redundancy-group 1;
        }
        unit 0 {
            family inet {
                address 10.66.220.65/26; <<< Server B on this subnet
                address 10.66.220.129/26;
                address 10.66.221.1/28;   <<< Server A on this subnet
                address 10.66.221.17/28;
            }
        }
    }

 I'm running 11.4R4.4 on an SRX220 cluster.

I could be wrong here (not nearly enough coffee today) but at a glance it looks like this is probably a routing/switching issue not a security policy issue.

All 4 of your subnets are defined on a single interface -- reth2.0.  Traffic isn't actually traversing zones (because it's intrazone), nor is it traversing routed interfaces so it's not going to hit the intrazone policy either.

I would separate these subnets into routed interfaces (reth2.0, reth2.1, etc.).  Routing will be "automatic" because they'll all be connected routes, so you won't have to do anything fancy there.  You'll just need to configure some VLAN tags off of your switches and define them in your subinterfaces.

Unfortunately, I can't do VLANs due to limitations with the virtual servers/hardware nodes contained on these networks. 

Routing is most definitely occurring, because traffic from one subnet has to traverse the default gateway because individual servers in this network do not span multiple subnets.  I've also verified this by doing a port mirror on the switch that connects all the servers and the SRX physical ports.  Traffic comes in from server A's port, exits the SRX port, and return traffic (for the traffic that works) enters again on the SRX port and out the port to server B.

All that aside, even if what you said were true, then that does not explain why ICMP works fine between the two both ways, and tcp/udp traffic sourced from A to B works fine, but not in reverse.

I understand that it's "working", but I wouldn't consider it the ideal way to separate that traffic and have clean traffic flows.  The SRX operates at L3 primarily, and yes, technically the traffic is "routing" ("forwarding," more correctly...) but traffic is not traversing interfaces or zones, which is how the SRX is designed to operate.

A few things to try before we proceed further:

- add "log session-init" to your policy.  Denied flows are not logged by session-close, because if they're denied, the session is never created, and therefore never closed.  Logging session init will log when the session is attempted.

- can you post the "show security flow session" of a working ICMP in both directions, a working SSH from A to B, and the non-working (still blank?) from B to A, as well as the traffic logs for all of those sessions (which you should see more of with session-init).

The logging of anything inside the intra-zone policy doesn't work.  No hits for 'session-init' and no hits for 'session-close', as I mentioned in my first post.  I see some counters incrementing, depending on the way the traffic is going, but not a single entry in the logfile.

Working SSH from A to B:

{primary:node1}
srx-a> show security flow session source-prefix 10.66.221.7 destination-port 22    
node0:
--------------------------------------------------------------------------

Session ID: 47142, Policy name: intra_vl6_ipv4/4, State: Active, Timeout: 16, Valid
  In: 10.66.221.7/35185 --> 10.66.220.82/22;tcp, If: reth2.0, Pkts: 12, Bytes: 2028
  Out: 10.66.220.82/22 --> 10.66.221.7/35185;tcp, If: reth2.0, Pkts: 0, Bytes: 0
Total sessions: 1

node1:
--------------------------------------------------------------------------

Session ID: 9014, Policy name: intra_vl6_ipv4/4, State: Backup, Timeout: 14412, Valid
  In: 10.66.221.7/35185 --> 10.66.220.82/22;tcp, If: reth2.0, Pkts: 0, Bytes: 0
  Out: 10.66.220.82/22 --> 10.66.221.7/35185;tcp, If: reth2.0, Pkts: 0, Bytes: 0
Total sessions: 1

{primary:node1}
srx-a>

 Working ping from A to B:

{primary:node1}
srx-a> show security flow session source-prefix 10.66.221.7 destination-prefix 10.66.220.82 
node0:
--------------------------------------------------------------------------

Session ID: 25021, Policy name: intra_vl6_ipv4/4, State: Active, Timeout: 42, Valid
  In: 10.66.221.7/2 --> 10.66.220.82/52081;icmp, If: reth2.0, Pkts: 1, Bytes: 84
  Out: 10.66.220.82/52081 --> 10.66.221.7/2;icmp, If: reth2.0, Pkts: 0, Bytes: 0

Session ID: 25420, Policy name: intra_vl6_ipv4/4, State: Active, Timeout: 42, Valid
  In: 10.66.221.7/1 --> 10.66.220.82/52081;icmp, If: reth2.0, Pkts: 1, Bytes: 84
  Out: 10.66.220.82/52081 --> 10.66.221.7/1;icmp, If: reth2.0, Pkts: 0, Bytes: 0

Session ID: 56737, Policy name: intra_vl6_ipv4/4, State: Active, Timeout: 44, Valid
  In: 10.66.221.7/3 --> 10.66.220.82/52081;icmp, If: reth2.0, Pkts: 1, Bytes: 84
  Out: 10.66.220.82/52081 --> 10.66.221.7/3;icmp, If: reth2.0, Pkts: 0, Bytes: 0
Total sessions: 3

node1:
--------------------------------------------------------------------------
Total sessions: 0

{primary:node1}
srx-a> 

 Absolutely nothing shows up for either a working ping from B to A or a non-working SSH from B to A in the session flow output.

Hi

I'm not sure what's happening, but if a SYN packets passes through from B to A and you see no session on the SRX, the packet doesn't come through the SRX. Is there a way the servers communicate directly through the switch? Maybe  ICMP redirect or some extra routes on servers make this possible. Try to make sure that all packets travel through SRX.

Hi pk,

I agree.  It seems weird to me that the SRX wouldn't even show the B to A traffic.  I think you are on to something and I've been wondering over the past couple of days if the fact that server B is a virtual server on a hardware node that's in the same subnet as server A has something to do with it.  I think I'll have to examine my Wireshark captures a little more closely to ensure the packet flow is matching up to what I think it's supposed to be.

Wireshark on the port connected to B shows:

1. TCP SYN inbound from B destined to A.

2. No SYN-ACK.

3. No ICMP redirects.

Wireshark on the port connected to A shows:

1. TCP SYN outbound from B destined to A.

2. SYN-ACK inbound from A to B.

3. No ICMP redirects.

Wireshark on the port connected to the SRX reth interface shows:

1. No TCP SYN inbound from B to A.

2. SYN-ACK inbound from A to B.

3. No ICMP redirects.

I think it's somewhat more clear what is happening here.  Seems like the initial SYN is, in fact, getting sent directly to server A without hitting the firewall, as both of you said.  This appears to be confirmed by the source/destination MAC addresses in the capture, neither of which are the SRX.  The return traffic is actually being sent to the SRX, which is probably dropping it because it's a SYN-ACK with no matching connection.

I'm still perplexed as to how ping works, but I can sort that out later.  I may end up having to move these subnets onto an RVI on the switch instead of the firewall for the time being.