Sunday, August 03, 2008

GMail Log4J Appender

Application logging is vital for diagnostics on a server product, but there can be so much, how can you tell what to watch or follow? Through tools like Log4J, you can have separate logs for different levels (typically debug, info, error and fatal). In Log4J, these are called log appenders, they can be anything from console or file based, or custom log appenders. One of the custom appenders I've been playing with recently is email logging. Our team has some excellent scripts written by Senor DB which scan log files for patterns and send email reports based on it's findings. I wanted to see if I could add application logging directly into the application. I wanted an email based appender and here's how I got that.

Now, there is an SMTPAppender in the Log4J package, but I wanted to use GMail, and the Log4J appender doesn't quite set up properly for GMail. I used Spring implementation of JavaMail for sending emails and extended the SMTPAppender in the log4J package. The method you'll be most interested in overriding is 'append', this gets called when your log is called into action depending on the settings in your log4j configuration.

public GmailAppender() {
super();
notifier = new JavaMailSenderImpl();
notifier.setHost("smtp.gmail.com");
notifier.setPort(587);
notifier.setUsername(yourGmailEmailAddress);
notifier.setPassword(yourGmailEmailPassword);
Properties props = new Properties();
props.setProperty("mail.smtp.auth", "true");
props.setProperty("mail.smtp.starttls.enable", "true");
notifier.setJavaMailProperties(props);

try {
InetAddress addr = InetAddress.getLocalHost();
hostname = addr.getHostName();
} catch (UnknownHostException e) {
}

}

For brevity, I've used the constructor to build the notifier (JavaMailSenderImpl from Spring) with desired properties, you can just as easily use properties in the log4j config, and I'll do just that for the host name (which for me means which IP address the application is running on). You can see above we've set the STMP host for Gmail with the port, email address and password for the email to be sent from. The little extra bit of magic is the properties 'mail.smtp.auth' and 'mail.smpt.starttls.enable'. The last is to get the default machine name.
 
@Override
public void append(LoggingEvent event) {

super.append(event);
SimpleMailMessage message = new SimpleMailMessage();
StringBuilder builder = new StringBuilder();
builder.append(getLayout().format(event));
String[] stackTrace = event.getThrowableInformation().getThrowableStrRep();
for(int i = 0; i < stackTrace.length; i++)
builder.append(stackTrace[i] + "\n");
message.setText(builder.toString());
message.setTo(emailAddress);
message.setSubject(event.getLevel().toString() + " on host " + hostname);
try{
notifier.send(message);
} catch (MailException ex){
}
}

Using the SimpleMailMessage (again from Spring), we add the text, the address and subject. The parts to grasp here are the layout and the stack trace. The layout is derived from your configuration, this is where you'd typically set the log message/heading including timestamp, thread name and other supporting information. Passing the event to the 'getLayout().format(event)' method returns this nicely formatted string. Then to get the stack trace we grab the array (each element being a line in the stack trace) from the event throwable information method. Someone thought that a method name indicating that your getting a string should pass back that array instead of adding new line characters, so we have to fix that before setting the text as the message body. Finally, we construct a useful subject containing the log level and host/IP address.

log4j.appender.mail=com.iclutton.GmailAppender
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.layout=org.apache.log4j.PatternLayout
log4j.appender.mail.layout.ConversionPattern=%d{ISO8601} [%t] %-5p [%C{1}.%M() %L] - %m%n
log4j.appender.mail.hostname=important-live-server

I've mentioned the configuration a few times, and here is the basic one comprising the full path to the class, the threshold (meaning the lowest level to log at, I don't want an email for every debug log entry after all). There's the pattern I mentioned before, this pattern includes the date, threadname, level name, class, method and line number, then the message and a new line. Finally there's the host name. Each instance of my application has a different name to identify it, typically something development, staging, production. To enable that property in the class, simply add a Spring like named setter:

public void setHostname(String hostname){
this.hostname = hostname;
}

Properties like this can be set if you want to have the username/password of your GMail account configurable or anything else for that matter, just don't forget to set it on the JavaMailSenderImpl object before sending the email in the append method.

4 comments:

Unknown said...

Very interesting post.

You mention that you use Spring, how do i configure the spring-related XML?

Robbie said...

The appender itself is not a Spring bean, it's not a singleton, but you can use the Spring classes as I have to make sending mail easier.

Unknown said...

1. Log into your Gmail account with your username and password.
2. Open the mail.
3. To display the email headers,
* Click on the inverted triangle beside Reply. Select Show Orginal.
4. You may copy the headers and use my IP address detection script to ease the process. Or if you want to manually find the IP address, proceed to 5.
5. Look for Received: from followed by the IP address between square brackets [ ].

Received: from [69.138.30.1] by web31804.mail.mud.yahoo.com

6. If you find more than one Received: from patterns, select the last one.
7. Track the IP address of the sender
you can chk the ip address from ip-details.com

Unknown said...

1. Log into your Gmail account with your username and password.
2. Open the mail.
3. To display the email headers,
* Click on the inverted triangle beside Reply. Select Show Orginal.
4. You may copy the headers and use my IP address detection script to ease the process. Or if you want to manually find the IP address, proceed to 5.
5. Look for Received: from followed by the IP address between square brackets [ ].

Received: from [69.138.30.1] by web31804.mail.mud.yahoo.com

6. If you find more than one Received: from patterns, select the last one.
7. Track the IP address of the sender
you can chk the ip address from ip-details.com