TIPS: SRX + NSM 2011.1 SYSLOG Streaming [ Edited ]

[SomeITGuy]

I have had some problems with this feature and wanted to share some important details that just do not seem to be in any of the KB or release notes.

show configuration security log


mode stream; <- Note that traffic logs will no longer APPEAR in the main logs and will only stream to NSM
format sd-syslog;
source-address [device ip]; <- IP of the device as it appears in NSM, clusters have no IP use the nodes 
stream NSM_Syslog {
    category all; <- Category MUST be set or NSM ignors the logs
    host {
        [NSM server IP];
        port 5140;
    }
}

 The BIG advantage to streaming the logs is a CPU use reduction, on my 210H units it reduced CPU use by about 40%. Also the stream logs will also include IDP logs

Howver if you are like me and use UTM features like Webfilter you will find that these logs STILL go to the main log and it logs all traffic allowed and blocked as a warning, here is my example that greatly reduces the main log buy filtering out the unnessary stuff. Note that you still need to have the default-log-messages sent to nsm to log critical warnings etc, in my example I use MATCH to filter out the two log types that are basicly spam.. If you are already logging traffic via the STREAM knowing that the webfilter allowed something is redundant.

show configuration system syslog

file default-log-messages {
    any info;
    match "!(WEBFILTER_URL_PERMITTED|WEBFILTER_SERVER_CONNECTED)";
    archive size 200k files 2;
    structured-data;
}

 Please note this should apply to JUNOS 10.2R3 and higher.

[keithr]

Good notes, thanks for posting that.

Could you post the complete syslog and security log configurations for reference?

[SomeITGuy]

Not much more to the config, other then some basic logging to the messages file and user console as per some recomendations found in other forum posts and KBs.

security

log {
    mode stream;
    format sd-syslog;
    source-address [Device IP];
    stream NSM_Syslog {
        category all;
        host {
            [NSM IP];
            port 5140;
        }
    }
}

 system

syslog {
    archive size 200k files 3;
    user * {
        any emergency;
    }
    file default-log-messages {
        any info;
        match "!(WEBFILTER_URL_PERMITTED|WEBFILTER_SERVER_CONNECTED)";
        structured-data;
    }
    file messages {
        any critical;
        authorization info;
    }
    file interactive-commands {
        interactive-commands error;
    }
}

[SomeITGuy]

Attached is an image of the CPU use on an SRX210 that was doing traffic logging locally then was switched to streaming to NSM directly.

Note that the graph is almost flat after the commit (spike) the CPU load used to climb in step with the number of open sessions every day, now it is less than the idle time!

[JNPRdhanks]

I like this feature, but just wanted to point out that the SRX branch runs on CPU anyway.  There are no ASICs.  You have to move into the SRX1400 and above to get hardware forwarding.  But I think the biggest advantage is now the syslog stream is moved to the fwdd/flowd process instead of going to the RE.

[SomeITGuy]

Regardless if the branch series uses one CPU (I didn't suggest anything else), logging in event mode uses the internal syslog server and writes to internal flash, which is VERY slow, as per my above post there is  a huge CPU load penalty to logging to the internal storage.

Thus stream mode still reduces load by a large factor and something you should use if you are doing ANY traffic/IDP logging. Take a look at my CPU graph for a 210 it is a 40% drop, but then again I am logging most traffic.

[motd]

Yes, its the flash that is the bottleneck and will slow down the RE cpu. Especially on the more powerful 650s this can become a major issue, I've seen them fail because of this. I always recommend stream logging, but I'd like to see SRX implement some sort of small in-memory buffer like netscreen had so we can look at the last X logs entries on the device itself. For the branch, that would be ideal and event mode could be removed from the default config.

Interesting you had to specify "category all" though, in my experience thats usually the one setting that causes logs not to be sent. So if anyone has troubles, try removing that statement.

Next up, getting NSM to stop logging notices to deviceDeamon.0 every time it receives a syslog message.

[pk]

Hi

By the way, why not to write logs to the RAM disk?

You can create a link to it from shell like

ln -s /jail/mfs/ramlog /var/log/ramlog

And direct logging to it

lab@srx# show system syslog file ramlog
any any;

lab@srx# run file list /var/log/ramlog detail
lrwxr-xr-x  1 lab   wheel         16 Nov 13 12:32 /var/log/ramlog@ -> /jail/mfs/ramlog
total 1

lab@srx# run file list /jail/mfs/ramlog detail  
-rw-rw----  1 root  wheel       7622 Nov 13 12:55 /jail/mfs/ramlog

It kind of works in lab, but I don't recommend this for production - just
an idea...

[great gu]

Hi Pk,

I have also faced some issues on IDP logs showing o nNSM 2011.1   I configured message any any on SRXs,but under predefined DI/IDP category, we cannot see the IDP logs. If we clear filters, we can see the logs, so I don't know if they are real IDP logs or we can directly see logs under there.. Some ideas about it?? 

[ggatten@waddeell.com]

Good morning,

I have SRX-240's - trying to implement stream based logging to HOPEFULLY address a problem with jflow and syslog F'ing each other in high load environments.

When logging in "event" mode I set the facility to 4.  My syslogd then takes all "facility 4" msgs and writes them to my common firewall log file.

When I implement "stream" mode, the msgs are written to the syslog default (/var/log/messages) and not where I want them.

How can I set the facility in stream mode?  I realize there's a stream name / tag, but what does that mean really?  I didn't notice a stream name embedded in the syslog messages.

I run rsyslogd so I can tweak this using attributes other than "facility" - but I'd prefer to stick with facility if I can.

TIA for any help!

Gary