eXtensible Logfile Format (XLF) 1.9.1

Jennifer Palonus
Graphical Dynamics, Inc.
jpalonus @ graphicaldynamics.com

(For a discussion of why we need a standard XML-based logfile format and how you can participate in its development, as well as the latest news about the project, check the XLF homepage at http://www.graphicaldynamics.com/xlf.)

Table of Contents

1  Introduction
2  Reference
    2.1  <?xml>
    2.2  <xlf>
    2.3  <session>
    2.4  <logevent>
    2.5  <debugevent>
Appendix A  The Full Schema

1 Introduction

This document describes a proposed standard format for logfiles, XLF 1.9.1. This format is currently in use in the author's software products, and should be viewed as a springboard towards a true, public "XLF 2.0" standard.

2 XLF 1.9.1 Reference

In this section we'll walk through XLF element by element. The full XML Schema is defined in Appendix A.

2.1 The <?xml> element

As an XML application, an XLF file SHOULD begin with the <?xml> element.

The most important attribute of the <?xml> tag for our purposes is the encoding attribute. The most common values for the encoding is either "UTC-8" (8-bit ASCII) or "UTC-16" (16-bit Unicode), although there are others as well.

The <?xml ...> processing instruction is written out when the logfile is first created.


<?xml version="1.0" encoding="UTF-8"?>

2.1 The <xlf> element

The <xlf> element comes after the <?xml ...> tag, and is written out when the logfile is first created.

The <xlf> tag has these attributes:

Req'dName TypeDefaultDescription
Xversionxs:string"1.9.1"The version of the XLF standard this file was written in.
 closetagsxs:boolean"1"Set this to "0" if the logfile does not end with a </xlf> tag, but instead simply has <session>, <logevent>, and <debugevent> entries appended to the end of it.
 productxs:string""The overall product that the programs that are writing to this logfile belong to, if they're all part of the same product. (Otherwise the products should be specified in each <session> element.)

Inside the <xlf> element, there are zero or more instances of <session>, <logevent>, and <debugevent>, in any order.

Unresolved issue: The full XML Schema shows the <xlf> element as a complexType. But the XML Schema language doesn't seem to provide a way to specify that a complexType element can have any number of specific children in any sequence. "xs:sequence" means that any number of the specified child elements can occur, but only in a specific sequence. "xs:all" means that they can occur in any sequence, but there can only be one of each. Neither is true in the case of XLF. So I don't know exactly how the <xlf> element's structure should be expressed in the schema.

Example with closing tags:

<?xml version="1.0" encoding="UTF-8"?> <xlf version="1.9.1"> <session ... <logevent ... <logevent ... <logevent ... <debugevent ... <logevent ... <session ... <logevent ... <logevent ... </xlf>

Example without closing tags:

<?xml version="1.0" encoding="UTF-8"?> <xlf version="1.9.1" closetags="0"> <session ... <logevent ... <logevent ... <logevent ... <debugevent ... <logevent ... <session ... <logevent ... <logevent ...

2.3 The <session> element

When an individual program starts up, it SHOULD write a <session> element. The element's body is a unique identifier of some kind—a random integer, the current system time, or even a Windows-style GUID. The purpose of the <session> element is to allow the logfile analyzer to group logfile entries together. It also collects relevant information for this logging session, such as the program that generated these log entries, its process ID, the computer this program is running on, etc.

If the <session> element is used, all subsequent <logevent> and <debugevent> entries that are written from this instance of the program SHOULD include this ID in their session attributes.

Note that the <logevent> and <debugevent> entries are not children of this session. This is because there could be mutliple programs writing to the same logfile concurrently, so their log entries would be interspersed with each other. So the log/debug entries simply refer back to their respective sessions instead.

The <session> tag has these attributes:

Req'dName TypeDefaultDescription
 dtfmt dtfmt-type
(see below)
"xml"The format that the <logevent> and <debugevent> entries will use to express the date/time. (see below)
 ipaddr xs:string""IP address of this computer.
 computerxs:string""Name of this computer on the network.
 user xs:string""Name of the user account this program is running under.
 product xs:string""Name of the product that this program is part of, if not specified in <xlf>.
 pgm xs:string""Name or pathname of this program.
 pgmver xs:string""This program's version string.
 procid xs:integer"0"This program's process ID.
 tz xs:string""The timezone this computer is in, as ±hh:mi. This is useful if you can't or don't want to include it in each <logevent>'s timestamp.
 helpuri xs:anyURI""The base URL to the file that contains information on specific <logevent> codes. (see below)

The dtfmt attribute specifies how this program will be expressing its date/times. You can specify these formats:

dtfmt Meaning
xml The default format for XLF (and XML) files: yyyy-mm-ddThh:mi:ss±hh:mi. a.k.a. the dateTime datatype.
rfc-822 The standard format for email, HTTP headers, etc.
sql SQL's DATETIME format: yyyy-mm-dd hh:mi:ss, possibly with milliseconds attached as a decimal part.
unix Number of seconds since midnight on 1/1/1970. This is generated by the time function in C, PHP, etc.
VT_DATE The default date/time format for Windows Automation objects. This is a floating-point number; the integer part is the number of days since 12/30/1899, and the decimal part is the fraction of the day since midnight.
strftime: formatspec This value should allow for any custom format a program could reasonably come up with. formatspec is the format string you would pass to the strftime function in C, PHP, Python, etc.

The helpuri attribute specifies where the log analyzer program can find the descriptions for the various codes from the <logevent> & <debugevent> entries.

Unresolved issue: The helpuri attribute is meant to enable a log analyzer program to produce a description of the various codes that the logevents & debugevents produce. The user would click on the code value for a specific logevent, and the analysis program would either popup a textbox for that code, or bring up the browser with that webpage loaded, or...? I'm assuming that there would be one html file with the descriptions for all the possible codes, with each code's description as a named anchor inside it. But it's possible that an organization might want to organize them as a separate html file for each code. It's not clear what the best way is to specify these possibilities. (Then again, few organizations document these things today, so maybe we can force a "best" way here?) It needs to be flexible, but not too difficult to implement. Then there's the question of supporting multiple languages. Maybe there should be one or more <help> elements below <session>, one for each language, with pointers to the respective help files.


<?xml version="1.0" encoding="UTF-8"?> <xlf version="1.9.1" xmlns="http://www.graphicaldynamics.com/xlf1.9.1/"> <session computer="JENNY" ipaddr="" user="Admin" procid="1668" program="D:\SRC\AutoIntern\AIEngine.exe">77057457675269</session> <logevent dt="2007-04-05T13:04:52-08:00" session="77057457675269" code="0">Scheduler engine starting.</logevent> <logevent dt="2007-04-05T19:15:00-08:00" session="77057457675269" code="5024768">Event started.</logevent> <logevent dt="2007-04-05T19:15:03-08:00" session="77057457675269" code="5024768">Event ended.</logevent> <logevent dt="2007-04-05T20:35:49-08:00" session="77057457675269" code="0">Scheduler engine ending.</logevent> </xlf>

2.4 The <logevent> element

The <logevent> element represents a message that the program writes to the logfile.

The <logevent> tag has these attributes:

Req'dName TypeDefaultDescription
XdtDepends on this session's dtfmtThe current date & time.
(normally an int)
""The session ID from this run's <session>.
(normally an int)
""A unique ID for this log event, if the program needs to generate one.
(normally an int)
""A non-unique code explaining the log event. This can be an error code, or any other non-unique data associated with this log event.
 severitystring or int"notice"Severity code. (see below)

Inside the <logevent> element is the text that the program wants to send to the log. The program could also write structured logging data here as an XML fragment.

The severity attribute comes from the syslog specification (RFC 3164). It can be an integer from 0-7 or one of these names:

0 or emergency
1 or alert
2 or critical
3 or error
4 or warning
5 or notice
6 or info
7 or debug


<logevent dt="2007-04-06T07:00:00-08:00" session="77059856805631" code="512">Event started.</logevent> <logevent dt="2007-04-06T07:00:17-08:00" session="77059856805631" code="512">Event ended.</logevent> <logevent dt="2007-04-06T07:30:00-08:00" session="77059856805631" code="10992557">Event started.</logevent> <logevent dt="2007-04-06T07:30:00-08:00" session="77059856805631" code="423">Event started.</logevent> <logevent dt="2007-04-06T07:30:04-08:00" session="77059856805631" code="10992557">1 file downloaded.</logevent> <logevent dt="2007-04-06T08:30:05-08:00" session="77059856805631" code="10992557">Event ended.</logevent> <logevent dt="2007-04-06T08:30:36-08:00" session="77059856805631" code="423">Event ended.</logevent> <logevent dt="2007-04-06T11:20:00-08:00" session="77059856805631" code="512">Event started.</logevent> <logevent dt="2007-04-06T11:20:14-08:00" session="77059856805631" code="512">Event ended.

2.5 The <debugevent> element

The <debugevent> element represents a debugging message that the program writes out to the logfile.

The <debugevent> tag has these attributes:

Req'dName TypeDefaultDescription
XdtDepends on this session's dtfmtThe current date & time.
 sessionxs:string""The session ID from this run's <session>.
 srcfilepathname""Which source file generated the debug event. In C this is generated by the __FILE__ macro.
 srclineinteger""Which line in the source file generated the debug event. In C this is generated by the __LINE__ macro.
(normally an int)
""This is normally an error code, but it can be any non-unique data associated with this debugging event.
 severitystring or int"debug"See <logevent>, above.

Inside the <debugevent> element is the debugging text that the program wants to send to the log.


<debugevent ...

Appendix A - The Schema

<?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" version="1.0" elementFormDefault="unqualified" attributeFormDefault="unqualified"> <xs:element name="logevents"> <xs:complexType> <xs:all> <xs:element ref="session" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="logevent" minOccurs="0" maxOccurs="unbounded"/> <xs:element ref="debugevent" minOccurs="0" maxOccurs="unbounded"/> </xs:sequence> <xs:attribute name="closetags" type="xs:boolean" default="1"/> <xs:attribute name="product" type="xs:string"/> </xs:complexType> </xs:element> <xs:element name="session"> <xs:complexType> <xs:attribute name="dtfmt" type="dtfmt-type" use="optional" default="iso-8601"/> <xs:attribute name="tz" type="xs:string"/> <xs:attribute name="ipaddr" type="xs:string"/> <xs:attribute name="computer" type="xs:string"/> <xs:attribute name="user" type="xs:string"/> <xs:attribute name="product" type="xs:string"/> <xs:attribute name="pgm" use="required" type="xs:string"/> <xs:attribute name="pgmver" type="xs:string"/> <xs:attribute name="procid" type="xs:integer"/> <xs:attribute name="helpuri" type="xs:anyURI"/> <xs:simpleContent><xs:extension base="sessionid-type"/></xs:simpleContent> </xs:complexType> </xs:element> <xs:element name="logevent"> <xs:complexType mixed="true"> <xs:attribute name="dt" use="required" type="xs:string"/> <xs:attribute name="session" type="sessionid-type"/> <xs:attribute name="id" type="xs:string"/> <xs:attribute name="code" type="xs:string"/> <xs:attribute name="severity" type="severity-type"/> <xs:simpleContent><xs:extension base="xs:string"/></xs:simpleContent> </xs:complexType> </xs:element> <xs:element name="debugevent"> <xs:complexType> <xs:attribute name="dt" use="required" type="xs:string"/> <xs:attribute name="session" type="sessionid-type"/> <xs:attribute name="srcfile" type="xs:string"/> <xs:attribute name="srcline" type="xs:positiveInteger"/> <xs:attribute name="code" type="xs:string"/> <xs:attribute name="severity" type="severity-type"/> <xs:simpleContent><xs:extension base="xs:string"/></xs:simpleContent> </xs:complexType> </xs:element> <xs:simpleType name="dtfmt-type"> <xs:restriction base="xs:string"> <xs:enumeration value="xml"/> <xs:enumeration value="sql"/> <xs:enumeration value="rfc-822"/> <xs:enumeration value="apache"/> <xs:enumeration value="unix"/> <xs:enumeration value="VT_DATE"/> <xs:enumeration value="strftime:*"/> </xs:restriction> </xs:simpleType> <xs:simpleType name="sessionid-type"> <xs:restriction base="xs:string"> <xs:maxLength value="36"/> </xs:restriction> </xs:simpleType> <xs:simpleType name="severity-type"> <xs:restriction base="xs:string"> <xs:enumeration value="0"/> <xs:enumeration value="1"/> <xs:enumeration value="2"/> <xs:enumeration value="3"/> <xs:enumeration value="4"/> <xs:enumeration value="5"/> <xs:enumeration value="6"/> <xs:enumeration value="7"/> <xs:enumeration value="emergency"/> <xs:enumeration value="alert"/> <xs:enumeration value="critical"/> <xs:enumeration value="error"/> <xs:enumeration value="warning"/> <xs:enumeration value="notice"/> <xs:enumeration value="info"/> <xs:enumeration value="debug"/> </xs:restriction> </xs:simpleType> </xs:schema>