[lttv.git] / doc / developer / format.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

<head>
  <title>The LTTng trace format</title>
</head>

<body>

<h1>The LTTng trace format</h1>

<p>
<em>Last update: 2008/06/02</em>
</p>

<p>
This document describes the LTTng trace format. It should be useful mainly to
developers who code the LTTng tracer or the traceread LTTV library, as this
library offers all the necessary abstractions on top of the raw trace data.
</p>

<p>
A trace is contained in a directory tree. To send a trace remotely, the
directory tree may be tar-gzipped. The trace <tt>foo</tt>, placed in the home
directory of user john, /home/john, would have the following contents:
</p>

<pre><tt>
$ cd /home/john
$ tree foo
foo/
|-- control
|   |-- facilities_0
|   |-- facilities_1
|   |-- facilities_...
|   |-- interrupts_0
|   |-- interrupts_1
|   |-- interrupts_...
|   |-- modules_0
|   |-- modules_1
|   |-- modules_...
|   |-- network_0
|   |-- network_1
|   |-- network_...
|   |-- processes_0
|   |-- processes_1
|   `-- processes_...
|-- cpu_0
|-- cpu_1
`-- cpu_...

</tt></pre>

<p>
The root directory contains a tracefile for each cpu, numbered from 0, 
in .trace format. A uniprocessor thus only contains the file cpu_0. 
A multi-processor with some unused (possibly hotplug) CPU slots may have some
unused CPU numbers. For instance an 8 way SMP board with 6 CPUs randomly 
installed may produce tracefiles named 0, 1, 2, 4, 6, 7.
</p>

<p>
The files in the control directory also follow the .trace format and are
also per cpu.  The "facilities" files only contain "core" marker_id,
marker_format and time_heartbeat events. The first two are used to describe the
events that are in the trace. The other control files contain the initial
system state and various subsequent important events, for example process
creations and exit. The interest of placing such subsequent events in control
trace files instead of (or in addition to) in the per cpu trace files is that
they may be accessed more quickly/conveniently and that they may be kept even
when the per cpu files are overwritten in "flight recorder mode".
</p>

<h2>Trace format</h2>

<p>
Each tracefile is divided into equal size blocks with a header at the beginning
of the block. Events are packed sequentially in the block starting right after
the block header.
</p>

<p>
Each block consists of :
</p>

<pre><tt>
block start/end header
trace header
event 1 header
event 1 variable length data
event 2 header
event 2 variable length data
....
padding
</tt></pre>

<h3>The block start/end header</h3>

<pre><tt>
begin
	* the beginning of buffer information
	uint64 cycle_count
		* TSC at the beginning of the buffer
	uint64 freq
		* frequency of the CPUs at the beginning of the buffer.
end
	* the end of buffer information
	uint64 cycle_count
		* TSC at the end of the buffer
	uint64 freq
		* frequency of the CPUs at the end of the buffer.
uint32 lost_size
	* number of bytes of padding at the end of the buffer.
uint32 buf_size
	* size of the sub-buffer.
</tt></pre>


<h3>The trace header</h3>

<pre><tt>
uint32 magic_number
	* 0x00D6B7ED, used to check the trace byte order vs host byte order.
uint32 arch_type
	* Architecture type of the traced machine.
uint32 arch_variant
	* Architecture variant of the traced machine. May be unused on some arch.
uint32 float_word_order
	* Byte order of floats and doubles, sometimes different from integer byte
	  order. Useful only for user space traces.
uint8 arch_size
	* Size (in bytes) of the void * on the traced machine.
uint8 major_version
	* major version of the trace.
uint8 minor_version
	* minor version of the trace.
uint8 flight_recorder
	* Is flight recorder mode activated ? If yes, data might be missing
	  (overwritten) in the trace.
uint8	has_heartbeat
	* Does this trace have heartbeat timer event activated ?
		Yes (1) -> Event header has 32 bits TSC
		No (0) -> Event header has 64 bits TSC
uint8 alignment
	* Are event headers in this trace aligned ?
		Yes -> the value indicates the alignment
		No (0) -> data is packed.
uint8 tsc_lsb_truncate
	* Used for compact channels
uint8 tscbits
	* Used for compact channels
uint8 compact_data_shift
	* Used for compact channels
uint32 freq_scale
		event time is always calculated from :
			trace_start_time + ((event_tsc - trace_start_tsc) * (freq / freq_scale))
uint64 start_freq
	* CPUs clock frequency at the beginnig of the trace.
uint64 start_tsc
	* TSC at the beginning of the trace.
uint64 start_monotonic
	* monotonically increasing time at the beginning of the trace.
		(currently not supported)
start_time
	* Real time at the beginning of the trace (as given by date, adjusted by NTP)
		This is the only time reference with the real world : the rest of the trace
		has monotonically increasing time from this point (with TSC difference and
		clock frequency).
	uint32 seconds
	uint32 nanoseconds
</tt></pre>


<h3>Event header</h3>

<p>
Event headers differ according to the following conditions : does the
traced system have a heartbeat timer? Is tracing alignment activated?
</p>

<p>
Event header :
</p>
<pre><tt>
{ uint32 timestamp
	or
	uint64 timestamp }
	* if has_heartbeat : 32 LSB of the cycle counter at the event record time.
	* else : 64 bits complete cycle counter.
uint8 facility_id
	* Numerical ID of the facility corresponding to the event. See the facility
	  tracefile to know which facility ID matches which facility name and
		description.
uint8 event_id
	* Numerical ID of the event inside the facility.
uint16 event_size
	* Size of the variable length data that follows this header.
</tt></pre>

<p>
Event header alignment
</p>

<p>
If trace alignment is activated (<tt>alignment</tt>), the event header is
aligned. In addition, padding is automatically added after the event header so
the variable length data is automatically aligned on the architecture size.
</p>

<!--
<h2>System description</h2>

<p>
The system type description, in system.xml, looks like:
</p>

<pre><tt>
&lt;system 
 node_name="vaucluse"
 domainname="polymtl.ca" 
 cpu=4
 arch_size="ILP32" 
 endian="little" 
 kernel_name="Linux" 
 kernel_release="2.4.18-686-smp" 
 kernel_version="#1 SMP Sun Apr 14 12:07:19 EST 2002"
 machine="i686" 
 processor="unknown" 
 hardware_platform="unknown"
 operating_system="Linux" 
 ltt_major_version="2"
 ltt_minor_version="0"
 ltt_block_size="100000"
&gt;
Some comments about the system
&lt;/system&gt;
</tt></pre>

<p>
The system attributes kernel_name, node_name, kernel_release,
 kernel_version, machine, processor, hardware_platform and operating_system
come from the uname(1) program. The domainname attribute is obtained from
the "hostname &#045;&#045;domain" command. The arch_size attribute is one of
LP32, ILP32, LP64 or ILP64 and specifies the length in bits of integers (I),
long (L) and pointers (P). The endian attribute is "little" or "big".
While the arch_size and endian attributes could be deduced from the platform
type, having these explicit allows analysing traces from yet unknown
platforms. The cpu attribute specifies the maximum number of processors in
the system; only tracefiles 0 to this maximum - 1 may exist in the cpu
directory.
</p>

<p>
Within the system element, the text enclosed may describe further the
system traced.
</p>


<h2>Bookmarks</h2>

<p>
Bookmarks are user supplied information added to a trace. They contain user
annotations attached to a time interval.
</p>


<pre><tt>
&lt;bookmarks&gt;
  &lt;location name=name cpu=n start_time=t end_time=t&gt;Some text&lt;/location&gt;
  ...
&lt;/bookmarks&gt;
</tt></pre>

<p>
The interval is defined using either "time=" or "start_time=" and 
"end_time=", or "cycle=" or "start_cycle=" and "end_cycle=". 
The time is in seconds with decimals up to nanoseconds and cycle counts 
are unsigned integers with a 64 bits range. The cpu attribute is optional.
</p>

-->
</body>
</html>
Commit	Line	Data
1f39128c	1	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/xhtml1-strict.dtd">
	2	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	3
584db146	4	<head>
a25fb9c4	5	<title>The LTTng trace format</title>
584db146	6	</head>
1f39128c	7
1f39128c	8	<body>
584db146	9
a25fb9c4	10	<h1>The LTTng trace format</h1>
a25fb9c4	11
1f39128c	12	<p>
	13	<em>Last update: 2008/06/02</em>
	14	</p>
147c6f7f	15
1f39128c	16	<p>
147c6f7f	17	This document describes the LTTng trace format. It should be useful mainly to
a25fb9c4	18	developers who code the LTTng tracer or the traceread LTTV library, as this
a25fb9c4	19	library offers all the necessary abstractions on top of the raw trace data.
1f39128c	20	</p>
584db146	21
1f39128c	22	<p>
147c6f7f	23	A trace is contained in a directory tree. To send a trace remotely, the
	24	directory tree may be tar-gzipped. The trace <tt>foo</tt>, placed in the home
	25	directory of user john, /home/john, would have the following contents:
1f39128c	26	</p>
584db146	27
1f39128c	28	<pre><tt>
584db146	29	$ cd /home/john
	30	$ tree foo
	31	foo/
584db146	32	\|-- control
a25fb9c4	33	\| \|-- facilities_0
	34	\| \|-- facilities_1
	35	\| \|-- facilities_...
	36	\| \|-- interrupts_0
	37	\| \|-- interrupts_1
	38	\| \|-- interrupts_...
	39	\| \|-- modules_0
	40	\| \|-- modules_1
	41	\| \|-- modules_...
c81f8c7f	42	\| \|-- network_0
	43	\| \|-- network_1
	44	\| \|-- network_...
	45	\| \|-- processes_0
	46	\| \|-- processes_1
a25fb9c4	47	\| `-- processes_...
	48	\|-- cpu_0
	49	\|-- cpu_1
	50	`-- cpu_...
	51
1f39128c	52	</tt></pre>
584db146	53
1f39128c	54	<p>
a25fb9c4	55	The root directory contains a tracefile for each cpu, numbered from 0,
a25fb9c4	56	in .trace format. A uniprocessor thus only contains the file cpu_0.
584db146	57	A multi-processor with some unused (possibly hotplug) CPU slots may have some
147c6f7f	58	unused CPU numbers. For instance an 8 way SMP board with 6 CPUs randomly
584db146	59	installed may produce tracefiles named 0, 1, 2, 4, 6, 7.
1f39128c	60	</p>
584db146	61
1f39128c	62	<p>
147c6f7f	63	The files in the control directory also follow the .trace format and are
	64	also per cpu. The "facilities" files only contain "core" marker_id,
	65	marker_format and time_heartbeat events. The first two are used to describe the
	66	events that are in the trace. The other control files contain the initial
	67	system state and various subsequent important events, for example process
	68	creations and exit. The interest of placing such subsequent events in control
	69	trace files instead of (or in addition to) in the per cpu trace files is that
	70	they may be accessed more quickly/conveniently and that they may be kept even
	71	when the per cpu files are overwritten in "flight recorder mode".
1f39128c	72	</p>
584db146	73
1f39128c	74	<h2>Trace format</h2>
584db146	75
1f39128c	76	<p>
a25fb9c4	77	Each tracefile is divided into equal size blocks with a header at the beginning
	78	of the block. Events are packed sequentially in the block starting right after
	79	the block header.
1f39128c	80	</p>
	81
	82	<p>
a25fb9c4	83	Each block consists of :
1f39128c	84	</p>
	85
	86	<pre><tt>
a25fb9c4	87	block start/end header
	88	trace header
	89	event 1 header
	90	event 1 variable length data
	91	event 2 header
	92	event 2 variable length data
	93	....
	94	padding
1f39128c	95	</tt></pre>
a25fb9c4	96
1f39128c	97	<h3>The block start/end header</h3>
a25fb9c4	98
1f39128c	99	<pre><tt>
a25fb9c4	100	begin
a25fb9c4	101	* the beginning of buffer information
a25fb9c4	102	uint64 cycle_count
	103	* TSC at the beginning of the buffer
	104	uint64 freq
	105	* frequency of the CPUs at the beginning of the buffer.
	106	end
	107	* the end of buffer information
a25fb9c4	108	uint64 cycle_count
147c6f7f	109	* TSC at the end of the buffer
a25fb9c4	110	uint64 freq
d88e4d7c	111	* frequency of the CPUs at the end of the buffer.
a25fb9c4	112	uint32 lost_size
	113	* number of bytes of padding at the end of the buffer.
	114	uint32 buf_size
	115	* size of the sub-buffer.
1f39128c	116	</tt></pre>
a25fb9c4	117
	118
	119
1f39128c	120	<h3>The trace header</h3>
a25fb9c4	121
1f39128c	122	<pre><tt>
a25fb9c4	123	uint32 magic_number
	124	* 0x00D6B7ED, used to check the trace byte order vs host byte order.
	125	uint32 arch_type
	126	* Architecture type of the traced machine.
	127	uint32 arch_variant
	128	* Architecture variant of the traced machine. May be unused on some arch.
	129	uint32 float_word_order
	130	* Byte order of floats and doubles, sometimes different from integer byte
	131	order. Useful only for user space traces.
	132	uint8 arch_size
	133	* Size (in bytes) of the void * on the traced machine.
	134	uint8 major_version
	135	* major version of the trace.
	136	uint8 minor_version
	137	* minor version of the trace.
	138	uint8 flight_recorder
	139	* Is flight recorder mode activated ? If yes, data might be missing
	140	(overwritten) in the trace.
	141	uint8 has_heartbeat
	142	* Does this trace have heartbeat timer event activated ?
	143	Yes (1) -> Event header has 32 bits TSC
	144	No (0) -> Event header has 64 bits TSC
147c6f7f	145	uint8 alignment
	146	* Are event headers in this trace aligned ?
	147	Yes -> the value indicates the alignment
a25fb9c4	148	No (0) -> data is packed.
147c6f7f	149	uint8 tsc_lsb_truncate
c81f8c7f	150	* Used for compact channels
147c6f7f	151	uint8 tscbits
c81f8c7f	152	* Used for compact channels
147c6f7f	153	uint8 compact_data_shift
c81f8c7f	154	* Used for compact channels
99f2111d	155	uint32 freq_scale
cb310b57	156	event time is always calculated from :
cb310b57	157	trace_start_time + ((event_tsc - trace_start_tsc) * (freq / freq_scale))
a25fb9c4	158	uint64 start_freq
	159	* CPUs clock frequency at the beginnig of the trace.
	160	uint64 start_tsc
	161	* TSC at the beginning of the trace.
	162	uint64 start_monotonic
	163	* monotonically increasing time at the beginning of the trace.
	164	(currently not supported)
	165	start_time
	166	* Real time at the beginning of the trace (as given by date, adjusted by NTP)
	167	This is the only time reference with the real world : the rest of the trace
	168	has monotonically increasing time from this point (with TSC difference and
	169	clock frequency).
	170	uint32 seconds
	171	uint32 nanoseconds
1f39128c	172	</tt></pre>
a25fb9c4	173
584db146	174
1f39128c	175	<h3>Event header</h3>
584db146	176
1f39128c	177	<p>
147c6f7f	178	Event headers differ according to the following conditions : does the
147c6f7f	179	traced system have a heartbeat timer? Is tracing alignment activated?
1f39128c	180	</p>
a25fb9c4	181
1f39128c	182	<p>
a25fb9c4	183	Event header :
1f39128c	184	</p>
1f39128c	185	<pre><tt>
a25fb9c4	186	{ uint32 timestamp
	187	or
	188	uint64 timestamp }
	189	* if has_heartbeat : 32 LSB of the cycle counter at the event record time.
	190	* else : 64 bits complete cycle counter.
a25fb9c4	191	uint8 facility_id
	192	* Numerical ID of the facility corresponding to the event. See the facility
	193	tracefile to know which facility ID matches which facility name and
	194	description.
	195	uint8 event_id
	196	* Numerical ID of the event inside the facility.
	197	uint16 event_size
	198	* Size of the variable length data that follows this header.
1f39128c	199	</tt></pre>
a25fb9c4	200
1f39128c	201	<p>
a25fb9c4	202	Event header alignment
1f39128c	203	</p>
a25fb9c4	204
1f39128c	205	<p>
147c6f7f	206	If trace alignment is activated (<tt>alignment</tt>), the event header is
	207	aligned. In addition, padding is automatically added after the event header so
	208	the variable length data is automatically aligned on the architecture size.
1f39128c	209	</p>
a25fb9c4	210
147c6f7f	211	<!--
1f39128c	212	<h2>System description</h2>
584db146	213
1f39128c	214	<p>
584db146	215	The system type description, in system.xml, looks like:
1f39128c	216	</p>
584db146	217
1f39128c	218	<pre><tt>
584db146	219	<system
	220	node_name="vaucluse"
	221	domainname="polymtl.ca"
	222	cpu=4
	223	arch_size="ILP32"
	224	endian="little"
	225	kernel_name="Linux"
	226	kernel_release="2.4.18-686-smp"
	227	kernel_version="#1 SMP Sun Apr 14 12:07:19 EST 2002"
	228	machine="i686"
	229	processor="unknown"
	230	hardware_platform="unknown"
	231	operating_system="Linux"
	232	ltt_major_version="2"
	233	ltt_minor_version="0"
	234	ltt_block_size="100000"
	235	>
	236	Some comments about the system
	237	</system>
1f39128c	238	</tt></pre>
584db146	239
1f39128c	240	<p>
584db146	241	The system attributes kernel_name, node_name, kernel_release,
	242	kernel_version, machine, processor, hardware_platform and operating_system
	243	come from the uname(1) program. The domainname attribute is obtained from
147c6f7f	244	the "hostname --domain" command. The arch_size attribute is one of
584db146	245	LP32, ILP32, LP64 or ILP64 and specifies the length in bits of integers (I),
	246	long (L) and pointers (P). The endian attribute is "little" or "big".
	247	While the arch_size and endian attributes could be deduced from the platform
	248	type, having these explicit allows analysing traces from yet unknown
	249	platforms. The cpu attribute specifies the maximum number of processors in
	250	the system; only tracefiles 0 to this maximum - 1 may exist in the cpu
	251	directory.
1f39128c	252	</p>
584db146	253
1f39128c	254	<p>
584db146	255	Within the system element, the text enclosed may describe further the
584db146	256	system traced.
1f39128c	257	</p>
584db146	258
584db146	259
1f39128c	260	<h2>Bookmarks</h2>
	261
	262	<p>
584db146	263	Bookmarks are user supplied information added to a trace. They contain user
584db146	264	annotations attached to a time interval.
1f39128c	265	</p>
1f39128c	266
584db146	267
1f39128c	268	<pre><tt>
584db146	269	<bookmarks>
	270	<location name=name cpu=n start_time=t end_time=t>Some text</location>
	271	...
	272	</bookmarks>
1f39128c	273	</tt></pre>
584db146	274
1f39128c	275	<p>
584db146	276	The interval is defined using either "time=" or "start_time=" and
	277	"end_time=", or "cycle=" or "start_cycle=" and "end_cycle=".
	278	The time is in seconds with decimals up to nanoseconds and cycle counts
	279	are unsigned integers with a 64 bits range. The cpu attribute is optional.
1f39128c	280	</p>
1f39128c	281
147c6f7f	282	-->
1f39128c	283	</body>
1f39128c	284	</html>