Removing Old Exchange 2013 Log Files

Exchange 2013 creates log files for everything, and this is way more than in 2010. Everything is logged, and even when the server is doing nothing, it is recording the health of the server and writing that down to the logs.
The following PowerShell script removes files over 30 days old in the IIS logs folder and the Exchange 2013 logging folder. Neither of these folders are cleaned up automatically in Exchange 2013 RTM or CU1. The transport logs in a different folder are cleaned up after 30 days, so this script uses the same duration.

Set-Executionpolicy RemoteSigned

$days=30 #You can change the number of days here 

$IISLogPath="C:\inetpub\logs\LogFiles\"

$ExchangeLoggingPath="C:\Program Files\Microsoft\Exchange Server\V15\Logging\"

Write-Host "Removing IIS and Exchange logs; keeping last" $days "days"

Function CleanLogfiles($TargetFolder)

{

    if (Test-Path $TargetFolder) {

        $Now = Get-Date

        $LastWrite = $Now.AddDays(-$days)

        $Files = Get-ChildItem $TargetFolder -Include *.* -Recurse | Where {$_.LastWriteTime -le "$LastWrite"} 

        foreach ($File in $Files)

            {Write-Host "Deleting file $File" -ForegroundColor "Red"; Remove-Item $File -ErrorAction SilentlyContinue | out-null}

       }

Else {

    Write-Host "The folder $TargetFolder doesn't exist! Check the folder path!" -ForegroundColor "red"

    }

}

CleanLogfiles($IISLogPath)

CleanLogfiles($ExchangeLoggingPath)

The above script runs in PowerShell and will delete logs on the server that you are running the script on.

The following version of the same script will get all the Exchange 2013 servers in the organization and then delete the logs (older than 30 days) across all the servers for you from one machine. You run this second script from Exchange Management Shell (run as administrator) and need remote file access to C$ (or whichever folder you set in the script) to all the servers Exchange 2013 servers.

Set-Executionpolicy RemoteSigned

$days=30 #You can change the number of days here 

$ExchangeInstallRoot = "C"

$IISLogPath="inetpub\logs\LogFiles\"

$ExchangeLoggingPath="Program Files\Microsoft\Exchange Server\V15\Logging\"

Write-Host "Removing IIS and Exchange logs; keeping last" $days "days"

Function CleanLogfiles($TargetFolder)

{

    $TargetServerFolder = "\\$E15Server\$ExchangeInstallRoot$\$TargetFolder"

    Write-Host $TargetServerFolder

    if (Test-Path $TargetServerFolder) {

        $Now = Get-Date

        $LastWrite = $Now.AddDays(-$days)

        $Files = Get-ChildItem $TargetServerFolder -Include *.* -Recurse | Where {$_.LastWriteTime -le "$LastWrite"} 

        foreach ($File in $Files)

            {

               # Write-Host "Deleting file $File" -ForegroundColor "Red" 

                Remove-Item $File -ErrorAction SilentlyContinue | out-null}

        }

Else {

    Write-Host "The folder $TargetServerFolder doesn't exist! Check the folder path!" -ForegroundColor "red"

    }

}

$Ex2013 = Get-ExchangeServer | Where {$_.IsE15OrLater -eq $true}

foreach ($E15Server In $Ex2013) {

    CleanLogfiles($IISLogPath)

    CleanLogfiles($ExchangeLoggingPath)

    }

How To Speed Up Transport Logging (Exchange 2013)

In http://blog.c7solutions.com/2012/08/how-speed-up-transport-logging.html I discuss how to speed up the interval between events happening and the log file being written to the disk for lab and testing scenarios. This is a quick update to that article to mention that in Exchange 2013 the following files can be edited to speed up log commit to disk for each of the Exchange 2013 transport services:

EdgeTransport.exe.config (for Transport service logs)

MSExchangeDelivery.exe.config (for Mailbox Transport Delivery service logs)

MSExchangeSubmission.exe.config (for Mailbox Transport Submission service logs)

MSExchangeFrontEndTransport.exe.config (for Frontend Transport service logs)

To reduce the memory cache time to 30 seconds set the following two entries in the above listed files (found in \Program Files\Microsoft\Exchange Server\v15\bin) within the AppSettings area:

<add key="SmtpSendLogFlushInterval" value="0:00:30" />
<add key="SmtpRecvLogFlushInterval" value="0:00:30" />

The two values above control different log files. Each transport log file has a different setting – so its possible to set Receive Connector protocol logging to a different value from Send Connector protocol logging if you wanted to. Once you make your changes to Edge.Transport.exe.config you need to restart the relevant service for the changes to be picked up.

Here is a list of the properties that I know about that can be changed:

• SmtpSendLogFlushInterval – Timespan value on how often to write the Send Connector protocol logging log to disk
• SmtpRecvLogFlushInterval – Timespan value on how often to write the Receive Connector protocol logging log to disk
• ConnectivityLogFlushInterval – Timespan value on how often the Connectivity log is written to disk.

In addition to the above, which are all timespan values for how often to write to disk, if the memory buffer that contains the log entries fills up then it will be written to disk as well. The default memory buffers are 1MB. So on a very busy server you might find that the log writing is not every five minutes exactly but of a more “random” nature as the buffer is filled. The following settings control the size of the buffer for the above timespans:

• SmtpSendLogBufferSize
• SmtpRecvLogBufferSize
• ConnectivityLogBufferSize

IPv6 Routed LAN with Windows

This blog is written to note down the steps needed to configure IPv6 on the whole of your LAN using Windows Server 2008 R2 as the router, but without installing RRAS.

It also uses Hurricane Electric’s IPv6 tunnel broker service to provide the IPv6 connectivity via an IPv4 tunnel as my internet provider (Virgin Media in the UK) does not provide direct IPv6 connectivity at the time of writing (Dec 2012).

Originally the plan was to do all this with the Draytek 2920 router on my network, but after days of trying I gave up as it was unable to connect to SixXS over AICCU or Freenet6/gogo via TSPC even though I had made accounts and entered the information as shown on various websites and forum. Draytek do not provide a 6in4 tunnel mode, so I needed to move to using Windows or Linux, as I have both on my LAN – though I am way more familiar with Windows!

Configuring Your Internet Router

You will need control over your internet connection as you will need to enable inbound PING responses before you can create an IPv6 tunnel. On a Draytek router this is System Maintenance > Management > untick Disable PING from the internet.

Also to allow a tunnel to traverse a NATed router, you need to allow Protocol 41 to pass the firewall. On a Draytek router this involves creating a new rule in the Default Call Filter rule set and the same under the Default Data Filter set. The settings are Direction: WAN –> LAN/RT/VPN; Source IP: Any; Destination IP: Any; Service Type: Protocol: 41; Filter: Pass Immediately.

Getting a Hurricane Electric Tunnel

Visit http://tunnelbroker.net and create an account and request a tunnel. Once you have requested a tunnel you will get the following information on the IPv6 Tunnel tab (of which only the important information is shown, and where I have changed the values to be generic):

• IPv6 Tunnel Endpoints
  • Server IPv4 Address: a.b.c.d (the endpoint of the tunnel at Hurricane Electric)
  • Server IPv6 Address: 2001:xxxx:wwww:65b::1/64 (this has wwww shown in bold and is the Hurricane Electric end of the tunnel they have created for you, and it will end in a 1.)
  • Client IPv4 Address: w.x.y.z (this is your external IP address of your internet connection)
  • Client IPv6 Address: 2001:xxxx:wwww:65b::2/64 (this has wwww shown in bold and is your end of the tunnel they have created for you, and it will end in a 2.)
• Routed IPv6 Prefixes
  • Routed /64: 2001:xxxx:yyyy:65b::/64 (this has yyyy in bold and yyyy is one number higher than wwww in the IPv6 tunnel endpoints above).

On the Example Configurations tab you will get the choice of operating system to use, and you need to select Windows Vista/2008/7 from the dropdown list. This will present you with some netsh commands as shown (where the values will be your specific values rather than the generic values I show here):

netsh interface teredo set state disabled
netsh interface ipv6 add v6v4tunnel IP6Tunnel w.x.y.z a.b.c.d
netsh interface ipv6 add address IP6Tunnel 2001:xxxx:wwww:65b::2
netsh interface ipv6 add route ::/0 IP6Tunnel 2001:xxxx:wwww:65b::1

If you are behind a NATed router then you need to change the w.x.y.z value which will show your public IP address for the private IP address of the Windows Server you are going to run this set of commands on.

Run these commands from an elevated command prompt. Once complete you should be able to reach the IPv6 internet from that machine. Try ping www.facebook.com and you should get back the IPv6 address for Facebook (showing your DNS server is IPv6 aware – Windows DNS will return AAAA, the IPv6 version of the A record, responses if your client has a valid global IPv6 address). Another destination you can attempt to ping is ipv6.google.com.

You now have working IPv6 from a single server on your LAN.

Configuring The Windows Router

The next step is to enable this single server as a router. This will allow the forwarding of packets between the LAN and the IPv6 Tunnel that exists on this server.

NOTE: This series of steps does not use RRAS, and therefore there is no firewall on this router. Therefore these steps should be for lab environments only, as you need to ensure that Windows Firewall on all your endpoints is secure (remote admin [DCOM], RPC Endpoing and 445 have default rules for open to anyone) – these will need securing to a suitably valid range, or implment IPSec on the servers so connections cannot be made from non domain members. A good IPv6 port scanner is available at ipv6.chappell-family.com

Continuing in your elevated command prompt on the tunnel Windows machine enter the following command:

netsh interface ipv6 set route ::/0 IP6Tunnel publish=yes

This adds a route for the entire IPv6 address space to go via this machine, and publishes it so that it can be see by other machines on the LAN. The publish=yes command is the only bit of this that is different from the commands provided by Hurricane Electric.

The next command to enter is:

netsh interface ipv6 add address interface="Local Area Connection" address=2001:xxxx:yyyy:65b::1

This command adds an IP address from the Routed /64 range to the network card on the machine (called “Local Area Connection” here. If your network card has a different name then change the name, and use the correct address that you want to use rather than the generic one I show here). I have chosen to end my routers IPv6 address with ::1. This means that the full address in my example is 2001:xxxx:yyyy:065b:0000:0000:0000:0001 and therefore I could choose anything for the 0000:0000:0000:0001 bit, remembering that one long list of zero’s can be collapsed to :: and leading zero’s can be removed.

Continue with:

netsh interface ipv6 set interface "Local Area Connection" forwarding=enabled advertise=enabled routerdiscovery=enabled advertisedefaultroute=enabled privacy=disabled

The command (which is long and probably wrapped on your web browser) enables forwarding on the Local Area Connection interface (forwards packets arriving on this interface to others, i.e. makes this box a router) and it will also advertise it’s routes and that it is a router. Router advertisement (both advertise=enabled routerdiscovery=enabled) allow clients on your network to find the router and generate their own IPv6 address. In this example this will therefore turn on IPv6 for your entire LAN. If you wish to do this test on just a few servers then add a valid IPv6 address using DHCPv6 with reservations or add the addresses manually on the machines you want to test IPv6 from (valid addresses are 2001:xxxx:yyyy:065b:z:z:z:z, where z:z:z:z is up to four blocks of four hex digits each). Privacy (see later) is disabled for this NIC as well.

NOTE: For any website that is IPv6 enabled, any computer that gets an IPv6 address will now use the tunnel to get to the internet. If the tunnel is down or slow then internet connectivity on all your machines will suffer. Your tunnel will be slower than your WAN speed and latency is likely to be higher. Consider carefully the advertise and routerdiscovery settings. You can always change them to disabled later if you wish (and reset your client network card to pick up the changes with netsh int ipv6 reset). I managed two days with IPv6 for every client before I changed back to IPv4. There are steps on line to change the prefix policy (netsh int ipv6 show prefix) to put IPv4 above IPv6 as an alternative to turning advertising and router discovery off.

The next command to enter is:

netsh interface ipv6 set route 2001:xxxx:yyyy:65b::/64 "Local Area Connection" publish=yes

This command publishes the route to your LAN so that the IP6Tunnel network that you created earlier can route packets to the correct interface. This is the opposite command the the first publish command you ran previously, as that one published the outbound route, this publishes the inbound route.

Finally you need to run this last command:

netsh int ipv6 set interface "IP6Tunnel" forwarding=enabled

This allows packets arriving on the IP6Tunnel from the internet to be forwarded to other networks on the machine. Again, this is the opposite of the earlier forwarding=enabled command and allows forwarding of packets arriving on the IP6Tunnel adapter to be forwarded into the LAN.

Connecting to the IPv6 Internet

Finally you are ready to go. If you open a command prompt on a Windows Vista or later client on the LAN and run ipconfig you should see an IPv6 address (and maybe a temporary IPv6 address) as well as a default gateway listing your newly configured router (reached via the Link Local address rather than the global IP address of the router if routerdiscovery is enabled on the router).

The IPv6 address you have is calculated from your Routed /64 subnet (the network portion of the address) and your MAC address. This local portion will therefore always be the same for you. This means that you are therefore trackable on the internet, as your local portion does not change. Therefore Windows 7 generates a temporary address which changes every 7 days (netsh int ipv6 show addresses and the Pref. Life column for Preferred Lifetime). After seven days the temporary address is recreated.

Open your web browser and visit http://test-ipv6.com/ to see if you have IPv6 connectivity.

You should now be able to ping www.facebook.com or ping ipv6.google.com and get a response back from the IPv6 internet.

Note that if you reboot your router or your client they will take a short while to pick up a valid IPv6 configuration from the Router Advertisements (RADV) that are running on the router (advertising the Routed /64 range you have – no requirement for DHCPv6 in this example).

Having the IPv6 Internet Connect To You (i.e. Publishing IPv6 Services)

On any machine with a valid global IPv6 address you should be able to enable the File and Printer Sharing (Echo Request - ICMPv6-In) rule in Windows Firewall and then visit http://centralops.net/co/Ping.aspx (or another IPv6 online ping test tool) and be able to ping your server or client.

Disable the ping firewall rule if needed and enable or create a firewall rule to allow a port of your choice to be published over IPv6. Configure the server to support listening on IPv6 if needed and then attempt to browse that service from another IPv6 enabled client.

Got this far – have a go at the IPv6 certification at Hurricane Electric

Domain Secure and Edge Servers

I was asked a question recently on the Microsoft Certified Master course for Exchange 2010 and was told that the answer was not clearly written up on the internet. So I thought I would write this blog post. The question was based on the idea that Domain Secure worked from a Hub Transport server in the classroom lab but not when mail flow went via an Edge server.

Domain Secure is end to end security, it cannot have anything in the middle – i.e. it cannot go via an Exchange Edge server, an Exchange 2013 Frontend Server or a third party SMTP relay.

The SMTP client in the connection (the send connector host) needs to connect to the SMTP server (the receive connector host) and swap certificates and prove the other side is who the other side say they are – i.e. mutual authentication. Also the domains must match the TLS list in TransportConfig (TLSSendDomainSecureList and TLSReceiveDomainSecureList). Therefore anything in the middle will offer a different certificate and so Domain Secure fails.

If there is a middle party and you want to do mutual authentication (i.e. swap certs to prove who you are), with one party offering their cert and not the cert of the final recipient domain (i.e. mail.messaging.microsoft.com or postini.com etc.) then use TLSAuthLevel and the DomainValidation option on the send connector (an SP1 addition to Exchange 2010). No green ticky ticky though.

Edge can do Domain Secure though. But Edge needs to be the starting point, i.e. the host of the send connector. So configure Domain Secure on the Edge (i.e. set the certificates and correct firewall settings) and ensure that the send connector for Domain Secure has the Edge server as the source. Ensure Edge Domain Secure receive connector is the target for inbound as well if you want it to work both ways. And of course you need working EdgeSync so hubs can deliver to Edge so that Edge can deliver emails for you.

Forefront Online Protection for Exchange Spam Filtering to Outlook’s Junk E-Mail Folder

Forefront Online Protection for Exchange (FOPE) is a cloud hosted email anti-spam and antivirus filtering system. Amongst the options to filter away your spam, one of the options to to allow the email to be flagged and sent on into your on-premises email system, and then managing it there.

If you have Exchange 2007 or later it is possible to write Exchange Transport Rules to process this flagged email and move it directly to the Junk E-Mail folder in your mailbox. This allows users to have their probable spam in a different location from their inbox email, but not in a different system accessed external to their email client, for which they might need a second login account or a delay before receiving the notification email. This works for probable spam as much obvious spam is filtered out at the edge of FOPE and so cannot make it to a place where users can see it.

An additional benefit of this filtering inside Outlook or OWA to the Junk E-Mail folder is that users can mark messages as safe or blocked in the client and this is picked up by Exchange and can be sent automatically to FOPE, which means FOPE flags it as spam before it reaches the Exchange organization.

To configure this you need to set FOPE to flag spam with a X-Header. This is documented at various places online, but misses out one vital piece of information which I wrote this blog to document. The missing info is what the value of the X-Header is so that you can actually write a transport rule to process it.

In FOPE, select your email domain (under Domains) and on the domain page click Edit next to Spam Action under Service Settings. Change the Spam Action to Add X-Header and type the header name that you want to use:

On your Exchange organization create a transport rule (these pictures are from Exchange 2007, but 2010 or 2013 are technically the same though visibly different). The transport rule is set to apply to messages when a message header contains specific words and the name of the header is the value set in FOPE previously (X-MoveTo-JunkEmail in this example). The value of the header will always read “This message appears to be spam.”.

It is possible to use the when the message header contains text patterns and use the RegEx expression \w* to find emails with your header in it and containing a value (\w* means any letter or digit repeated), but as we know the value for the header is always “This message appears to be spam.” then using regular expression filtering is adding un-needed CPU cycles to the Exchange Server – only use RegEx when the value can vary.

Office 365 use this process to place your probable spam in Junk E-Mail. In their case the header is X-FOSE-spam.

The transport rule continues to set the spam confidence level to a high value of your choosing, and higher than the value we will set in OrganizationConfig below.

So this rule will take all emails the FOPE marks as spam and changes the spam confidence level (SCL) value to 9 (in this example). Finally we need to set the SCLJunkThreshold property of OrganizationConfig to a value below the value in the transport rule. Exchange will place all email that exceeds this threshold into the Junk E-Mail folder in Outlook:

Set-OrganizationConfig -SCLJunkThreshold 4

If you are running the Content Filter hygiene agent then you will also want to check the Get-ContentFilterConfig values for SCLRejectEnabled, SCLDeleteEnabled and SCLQuarantineEnabled are all set to false. This ensures that SCL values that are are high are not rejected or deleted, or sent to quarantine. As all your email should be filtered by FOPE if you are using it, and the firewalls at your company or receive connectors on Exchange should be blocking email not sent from the FOPE datacenters (in FOPE admin pages click Information tab and then the Configuration link to get the list of IP addresses). The content filtering agent can be used as a second filter on-premises but if you don’t want to throw away or reject spam at this second level (recommended in this scenario) then ensure that the filter rejection, delete and quarantine settings are disabled. If you want to delete probable spam then set the transport rule to 5 and the SCLDeleteEnabled to $true and the SCLDeleteThreshold value to 9. Don’t reject or use the on-premises quarantine features when using FOPE (the transport rule cannot process the quarantined messages for a start).

Finally for administrators, consider a Message Retention Management or Retention Policy to delete without recovery email in Junk E-Mail folder after 21 days. Also consider the FOPE Directory Sync tool to push the user lists to FOPE as this upload also includes the pushing of the safe senders information as well.

Now for your users, all probable spam is managed in their email client, integrated with safe sender lists and without resorting to another application to view and deliver false positives and spam they want to read!