Application Acceleration
Application Acceleration

Poor Exchange SMTP compression rate

‎08-14-2009 01:22 AM
Hello,
 

We are using a lot of WXC on our WAN links.
We are now migrating mail system from Lotus to Exchange 2007 1 server at least per site connected to a central site in HQ.

Exchange servers are communicating throught smtp traffic. That smtp traffic is very badly compressed by our wxc: just 2% compression rate !
(we use wxos V5.7.1.1 - we were reaching 50% average with Lotus Domino traffic ).

I found Exchange smtp traffic is encrypted by default by Exchange and I suppose bad compression rate comes from that - correct my if you believe there is another reason.

I am told by Exchange team that encryption is a normal default behaviour we can not change.

I am then looking for WXC / Exchange experiences :

- can the compression rate be enhanced on that traffic ?
- is it a normal behaviour that smtp encryption is activated and can not be removed on Exchange 2007 server communication ?
- how do you manage efficiently Exchange traffic over your wan links ?


Thanks for your help
19 REPLIES 19
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-14-2009 04:34 AM
Encryption if enabled will not compress traffic much.You can alternatively decide to exclude exchange traffic from compression ,can reduce latency.
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-14-2009 07:43 AM

Hi

 

Just to clarify: Are you talking about Exchange <-> Exchange traffic or Outlook <-> Exchange traffic?

 

 

Regards,
Johan
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-14-2009 07:52 AM

Hello,

 

I am speaking about Exchange_Server <=> Exchange_Server traffic. It is using now encrypted smtp traffic.

Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-14-2009 03:23 PM

Under Exchange 2007, if the server is communicating with another Exchange 2007 server then they will try to encrypt this data. If the transfer is an smtp session they will try to use TLS by default. This can be disabled.

 

To be clear, data that is encrypted before its 'seen' by the WXC will receive minimal to zero compression.

 

 

Danny Jump
Technical Marketing Manager - Access and Acceleration Business Unit
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-20-2009 03:10 PM

Hi DannyJ,

 

How is it possible to disable TLS and encryption between exchange 2007 servers? According to various Microsoft posts its not... Another thing would be to use the SSL compression feature of the WXC. Does it support secure SMTP, or only HTTPS?

 

 

thor

Thor Milde
Soulution Architect - Core Infrastructure
Firstpoint AS
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-20-2009 04:02 PM

Hi Thor,

 

The EX2007 <-> EX2007 whether Hub to Hub or Edge to Hub now use's a combination of SSL/TLS for SMTP communications. I'm not aware that this can be disabled Smiley Sad

 

 

Danny Jump
Technical Marketing Manager - Access and Acceleration Business Unit
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-21-2009 01:31 AM

Hello,

 

thanks for all your answers.From what I know now, :

- encryption on Exchange  <> Exchange smtp communication can not be deactivated on Exchange 2007 whithin a same organisation. That feature could be implemented by Microsoft in Exchange 2010.

- WXC are badly compressing  that smtp traffic since it is encrypted (I get between 2 and 10% compression rate over the time)

 

Hence, my migration from Lotus to Exchange is multiplying my traffic over our WAN links. I also realize that Exchange is consuming more volume than Lotus Notes.

 

I believe we are not alone to have that problem and it could have a big impact : what is Juniper advice to optimize Exchange 2007 traffic ? Did Juniper have a workaround with Microsoft on that point ?

 

Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-21-2009 01:48 AM

Hi Danny,

 

Let me re-phrase the question. Is it possible to use the IPSec license on the WXC to work with secure SMTP as you can with HTTPS? This way the box would be part of the SSL chain and be able to compress the SMTP traffic.

 

thor

Thor Milde
Soulution Architect - Core Infrastructure
Firstpoint AS
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-21-2009 02:19 AM

Hi

 

In theory you can use the SSL optimization feature to optimize any SSL encrypted application as it is a generic SSL optimization feature, not specific for HTTPS.

 

Now, what really matters when it comes to what SSL encrypted application would really work with the current SSL optimization feature is whether they are implicit or explicit SSL based applications.

 

An implicit SSL application will kick-off the SSL handshake directly after the TCP handshake as it "knows" the receiving end is SSL enabled, like HTTPS, IMAPS, POP3S, etc.

 

An explicit SSL application is not 100% sure if the receiving end does support SSL when it establish the TCP session. Instead of kicking of the SSL handshake directly after the TCP handshake it asks the server for it's capabilities. If the server is capable of running SSL the client will initiate the SSL handshake at a later stage in the sessions setup. If the server is not capable of running SSL the communication could continue in clear.

 

Many SMTPS implementations today is an explicit SSL based application as they keep using TCP port 25 and looking at the STARTTLS options/capability to establish a secure channel. There are also implicit SMTPS implementations out there that normally uses TCP port 465.

 

Now, I'm not 100% sure if Exchange 2007 uses an implicit or explicit SMTPS implementation. If it is an explicit implementation or can be configured as an explicit implementation (not using the STARTTLS command) you should be OK to use the WX SSL optimization feature.

 

I did look at Exchange 2003 SMTPS support a couple of years ago and then I know it used the STARTTLS option, but I did not investigate if I could turn this off.

 

 

Regards,
Johan
Application Acceleration

Re: Poor Exchange SMTP compression rate

‎08-21-2009 02:58 AM

Hi Johan,

 

According to Microsoft, Exchange does use something they call Opportunistic TLS as described below. I have problems seeing if this runs without STARTTLS, but I will kick-off wireshark later today to see if it does. Will Juniper address this Exchange2007 "issue" in future realeses? The world seems to move in the direction of more and more SSL based connections with STARTTLS..

 

Cut from Microsoft Technet:

  • Opportunistic TLS   In earlier versions of Exchange Server, you had to configure TLS manually. In addition, you had to install a valid certificate, suitable for TLS usage, on the server running Exchange Server. In Exchange 2007, Setup creates a self-signed certificate. By default, TLS is enabled. This enables any sending system to encrypt the inbound Simple Mail Transfer Protocol (SMTP) session to Microsoft Exchange. By default, Exchange 2007 also tries TLS for all remote connections.
  • Thor Milde

     

    Thor Milde
    Soulution Architect - Core Infrastructure
    Firstpoint AS
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎08-21-2009 05:04 AM

    Hi

     

    There are currently no committed enhancement to support explicit SSL sessions using the STARTTLS option at this point. I did raise this enhancement request before, but I can add another one now to include the fact Exchange 2007 is using this heavily, should put more focus on this.

     

    Thanks for the feedback

     

     

    Regards,
    Johan
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎08-21-2009 05:14 AM

    Sorry forgot

     

    I don't think "Opportunistic TLS" is related directly to the use of the STARTTLS option. I think is has more to do with the fact that the Exchange server automatically enabled SSL/TLS and adds a certificate (self signed) + that the Exchange server when acting like a SSL client don't care about who signed these certificates, they just except any certificate to encrypt the data. Not sure if this is really what you like to do if you are in fact concerned about security.

     

     

    Regards,
    Johan
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎08-21-2009 05:22 AM

    Hello Johan,

     

    in my case, I am more concerned about compression than authentication. 

    Plus, if Exchange accepts self signed certs, I believe there is no point WXC to take this as a prerequise for authentication? We just use it to exchange keys right? 


    So I am looking for a way WXC to read & decrypt SMTPS traffic to be able to compress it over the WAN.
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎08-21-2009 05:54 AM

    Hi

     

    I do understand you. The certs is not the issue, the WXC can use these self signed certificates (they are regular certificates) in the SSL optimization feature. The problem is that since we don't support SSL session that uses the STARTTLS option like Exchange 2007 seems to do we have no way to tap into the flow and compress it.

     

     

    I did a quick google on this and found this:

     

    http://www.tech-archive.net/Archive/Exchange/microsoft.public.exchange.admin/2008-11/msg00025.html

     

    This seem to disable the use of the STARTTLS option. I'm not sure if this means the communication goes in clear as regular SMTP or if this means it does run SMTPS, but in implicit mode.

     

    Let me know how this works. Unforunatly I have not any access to a Exchange 2007 server to test.

     

     

    Regards,
    Johan
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎08-21-2009 06:08 AM

    Hello Johan,

     

    I already checked with our Microsoft support : that smtp encryption can not be disabled between Exchange 2007 servers whithin a same organisation. That is enabled by default and can not be deactivated.

    I guess there is another solution somewhere as we are not the only ones to migrate from Lotus to Exchange 2007. I am wondering how others companies proceded  ? 

    Right now we migrated just a few mailboxes and WAN traffic is already multiplied by 3 !!! If it continues like this, our WAN traffic could by multiplied by 20 at the end of the migration.

     

    What did other companies ? I am sure there is a workaround. What is Juniper / Microsoft advice ?

     

    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎09-04-2009 12:17 AM

    Hi

     

    I have since the last post be reading up in more detail on this and also setup a test environment to verify a few things and here are my findings.

     

    The Exchange Server - to - Exchange Server traffic (or in fact the Hub Transport traffic) is using encrypted SMTP (SMTPS). Initially I was under the impression MS was using more standards based SMTPS implementation for the hub transport. This is not really the case.

     

    The most common approach to do SMTPS is by using a server capability called STARTTLS. This basically means that when the SMTP client connects to the SMTP server this happens in clear on port 25. During the initial SMTP negotiation the server would tell the client it is capable of switching to an encrypted session (still on port 25) by offering the STARTTLS option to the client. Next the client (if capable) would initiate the TLS session. In this case the SSL/TLS certificate is used for both authentication AND public key encryption. Allowing the server not to advertise STARTTLS or the client to not use STARTTLS you could continue to run in clear.

     

    The implementation MS uses for the hub transport is using a proprietary option called Anonymous-TLS. Basically what they do is use a certificate (either a real one or a self signed one) for public key encryption, but the authentication uses KERBEROS into the Active Directory. In the Active Directory they also store the certificates so as long as the Exchange server is part of the Active Directory they can access the public key for any other Exchange server. I did try a few ideas, like removing the certificate, making sure the Receive Connector did not offer Anonymous-TLS, etc., but the sending server will terminate the session unless there is a valid certificate available AND the two servers authenticate using KERBEROS.

     

    I have been speaking to a Exchange integrator and they tell me in fact that one of the main issues found at Exchange 2007 customers is certificate problems (expired, broken, missing, etc) which basically breaks to communication between the Hub Transport servers.

     

    Juniper have had meetings with MS and discussing this issue. MS was telling us that this specific issue has in fact made a lot of customers stay away from migrating to Exchange 2007. Many customers have grown custom to the ability of WAN optimization devices to compress the traffic on the WAN links, but this becomes impossible with the use of Anonymous TLS between the servers. MS told us they where thinking about an option to allow a customer to run hub transport without Anonymous TLS, but nothing confirmed yet.

    Regards,
    Johan
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎09-07-2009 02:48 AM

    Hello Johan,

     

    thanks for that clear answer: now we do understand well that problem.
    We do not anymore wait for a solution from Microsoft. We may take a look at others third parties solutions(like the possibility to compress attachements.)

     

    However such tool, once installed, would be difficult to remove later...

     

    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎11-16-2009 12:53 AM

    Hi,

     

    Just back from Microsoft TechEd in Berlin and the launch of Exchange 2010. The new version has the option to turn off the use of Secure SMTP between exchange servers. The setting is per connection, allowing you to turn it off only on the connections where you have WXCs or similar.

     

    Good luck with your upgradesSmiley Happy

     

    Best regards,

     

    Thor Milde

    Solution Architect - Core Infrastructure

    Firstpoint AS

    www.firstpoint.no

     

    Thor Milde
    Soulution Architect - Core Infrastructure
    Firstpoint AS
    Application Acceleration

    Re: Poor Exchange SMTP compression rate

    ‎11-17-2009 10:49 AM

    Hey Thor,

     

    Many Thanks for that update.

    Danny Jump
    Technical Marketing Manager - Access and Acceleration Business Unit