1 #LyX 1.5.5 created this file. For more info see http://www.lyx.org/
10 \font_typewriter default
11 \font_default_family default
17 \paperfontsize default
25 \paperorientation portrait
28 \paragraph_separation skip
30 \quotes_language english
33 \paperpagestyle default
34 \tracking_changes false
43 A Set of APIs for a Third-Generation Trace Viewer
48 Pierre-Marc Fournier, Michel Dagenais, Mathieu Desnoyers
53 <pierre-marc.fournier _at_ polymtl.ca>
62 École Polytechnique de Montréal
69 \begin_layout Standard
70 This document proposes some APIs for a trace viewing and analysis infrastructure.
73 \begin_layout Standard
74 Design choices were made based on the experience acquired by the design,
75 development and use of two previous generations of trace viewers, respectively
76 the Trace Visualizer from the Linux Trace Toolkit and its successor, the
77 LTTV from the LTTng tracing toolkit.
85 Provide an infrastructure for fast, interactive visualization and analysis
86 of huge traces (>10 GB) on standard workstations
90 \begin_layout Standard
91 Efficiency is of great importance.
92 It is tempting to think that as computers get faster, it will be possible
93 to run more complex analyses on traces in a shorter time.
94 In fact, the size of the traces grows as computers get more powerful, because
95 they do more operations than before in the same amount of time and therefore
97 The current tendency to parallelize systems is another factor that results
106 \begin_layout Itemize
107 Allow efficient manipulation of traces that are larger than the system memory
110 \begin_layout Itemize
111 Support processing of live traces
114 \begin_layout Itemize
115 Allow merging of traces of different formats
118 \begin_layout Itemize
119 Design with both command line utilities (C language) and graphical interfaces
120 (Java and others) in mind, sharing code and standardizing interfaces as
125 \begin_layout Standard
126 Both command-line interfaces and graphical interfaces have an important
127 role to play in trace analysis.
128 While graphical interfaces allow for complex views, command-line utilities
129 can be used quickly in more environments; they can also be more convenient
130 in some circumstances.
131 A good design should be such that they can share code and APIs.
135 \begin_layout Section
139 \begin_layout Itemize
140 All data structures must be accessed through API methods, in order to allow
141 for internal change without breaking compatibility.
144 \begin_layout Itemize
145 The APIs in this document are described as C functions and datatypes.
146 When an operation applies to an abstraction that could be an object in
147 an object-oriented language, it is specified as the first argument of the
151 \begin_layout Subsection
155 \begin_layout Standard
156 The framework needs to represent points in the time line and time ranges.
157 This is done by two data structures, respectively:
160 \begin_layout Itemize
164 \begin_layout Itemize
165 struct trace_time_range
168 \begin_layout Standard
169 The preferred representation for times is seconds.
172 \begin_layout Standard
173 Ranges are a span between two points in time.
174 They are represented as these two points.
175 One or both of them can have the special value
182 \begin_layout Standard
183 Each time is accompanied by an uncertainty value.
186 \begin_layout Subsection
190 \begin_layout Standard
191 The following modules should be part of the framework.
192 This list does not include modules related to the GUI, the controlling
193 of traces and the transfer of traces.
196 \begin_layout Description
197 Trace Has one instance per open trace.
198 Allows the access to a trace by exporting the Low-level Trace Reading API.
199 The underlying implementation may change from one trace to another to adapt
200 to the specific format of each trace.
203 \begin_layout Description
204 Traceset Receives data from several Traces using their Low-Level Trace Reading
205 API and merge them in a single stream.
206 May also receive data from other instances of Traceset.
207 Exports the High-level Trace Reading API and the Low-level Trace Reading
209 Handles time offsetting if necessary to align traces.
213 \begin_layout Description
215 providers They receive the stream of events from the Traceset, using
216 the High-level Trace Reading API.
217 They detect state changes.
218 Each one is designed to detect states of a particular level (kernel, applicatio
219 n-level, UML states).
220 It pushes the detected states to the State module using the State Setting
224 \begin_layout Description
225 State Records changes in the system state received from State providers
226 by the State Setting API.
227 It implements an efficient state searching engine.
228 Exports the State Accessing API to allow plugins to access the information.
231 \begin_layout Description
233 description Receives the stream of events from the Traceset using the
234 High-level Trace Reading API.
235 It reads the events that describe other events and stores the information
237 Other components can access this information through an API it exports.
240 \begin_layout Description
242 These are the graphical views, analyses and other components that need
243 access to the events.
244 They receive data from a traceset using the High-level Trace Reading API.
245 They use the services they need (State, Event Description, and/or others).
246 They may export a special API if other components need to access their
250 \begin_layout Section
251 Low level reading of traces
254 \begin_layout Subsection
258 \begin_layout Standard
259 This API presents a uniform way of accessing traces at a low level, that
260 is to access randomly the events of the trace in an physical-encoding-independa
264 \begin_layout Standard
265 \begin_inset Note Note
268 \begin_layout Subsubsection
272 \begin_layout Standard
273 Timestamps, at the framework level, should be in an architecture-independent
275 One possibility would be to have them stored in seconds (with decimals).
276 Offsetting, if necessary, could be handled by the traceset (trace merging)
278 An uncertainty value should accompany the timestamp.
279 Timestamps in cycles should be available if they are provided by the tracer.
287 \begin_layout Standard
288 \begin_inset Note Note
291 \begin_layout Subsubsection
295 \begin_layout Itemize
296 void get_supported_formats(void)
300 \begin_layout Standard
304 \begin_layout Standard
308 \begin_layout Itemize
309 a list of supported formats
313 \begin_layout Itemize
318 \begin_layout Standard
322 \begin_layout Itemize
327 \begin_layout Itemize
331 \begin_layout Itemize
332 other special url to establish a network connection
335 \begin_layout Itemize
340 \begin_layout Itemize
341 trace format (optional, for cases where auto detection would not work or
345 \begin_layout Standard
349 \begin_layout Itemize
358 \begin_layout Subsection
359 Methods of the Low Level Trace Reading API
362 \begin_layout Itemize
363 void close(struct trace *tr)
367 \begin_layout Standard
368 Close the trace and unallocate all resources associated to this struct trace
369 including the handle.
370 After this call, tr is invalid and its memory is freed.
373 \begin_layout Standard
377 \begin_layout Itemize
381 \begin_layout Standard
386 \begin_layout Itemize
387 struct event *current_event(struct trace *tr, struct cursor *cur)
391 \begin_layout Standard
392 Return the event pointed by the cursor cur.
395 \begin_layout Standard
399 \begin_layout Itemize
403 \begin_layout Itemize
404 cur: the cursor indicating the position of the wanted event
407 \begin_layout Standard
411 \begin_layout Itemize
412 the event currently pointed by the position pointer
415 \begin_layout Itemize
416 NULL, if current_event is called on a new cursor that is positioned before
417 the beginning of the trace
421 \begin_layout Itemize
422 int advance(struct trace *tr, struct trace_cursor *cur)
426 \begin_layout Standard
427 Advance the cursor to the next event in the trace.
430 \begin_layout Standard
434 \begin_layout Itemize
438 \begin_layout Itemize
439 cur: the cursor that should be advanced
442 \begin_layout Standard
446 \begin_layout Itemize
451 \begin_layout Itemize
452 Success (TRACE_ADVANCE_OK)
455 \begin_layout Itemize
456 The cursor was not advanced because the end of trace is reached (end of
457 a non-live trace or of a completed live trace) (TRACE_ADVANCE_END)
460 \begin_layout Itemize
461 The cursor was not advanced because no new events are available at this
462 time (with a live trace that is still running) (TRACE_ADVANCE_TRY_AGAIN)
467 \begin_layout Itemize
468 int seek_time(struct trace *tr, struct trace_cursor *cur, struct trace_time
473 \begin_layout Standard
474 Place a cursor at a particular time index in a trace.
477 \begin_layout Standard
481 \begin_layout Itemize
482 tr: the trace that should be seeked
485 \begin_layout Itemize
486 cur: the cursor to seek
489 \begin_layout Itemize
490 time: the time to seek to
493 \begin_layout Standard
497 \begin_layout Itemize
501 \begin_layout Itemize
502 != 0: time out of range
506 \begin_layout Itemize
507 struct trace_cursor *trace_new_cursor(struct trace *tr)
511 \begin_layout Standard
512 Return a new cursor associated to the trace.
513 The position of this new cursor is just before the first event, therefore
514 advance() must be called before reading the first event.
517 \begin_layout Standard
521 \begin_layout Itemize
525 \begin_layout Standard
529 \begin_layout Itemize
534 \begin_layout Itemize
535 struct trace_cursor *trace_cursor_copy(struct trace_cursor *cur)
539 \begin_layout Standard
543 \begin_layout Standard
547 \begin_layout Itemize
548 cur: the cursor to copy
551 \begin_layout Standard
555 \begin_layout Itemize
556 a new cursor that is at the same location as cur
560 \begin_layout Itemize
561 void trace_cursor_destroy(struct trace_cursor *cur)
565 \begin_layout Standard
566 Free all resources associated to a cursor.
567 After this call, cur is invalid.
570 \begin_layout Standard
574 \begin_layout Itemize
575 cur: the cursor to destroy
578 \begin_layout Standard
583 \begin_layout Itemize
584 struct trace_time get_time_start(struct trace *tr)
588 \begin_layout Standard
592 \begin_layout Itemize
596 \begin_layout Standard
600 \begin_layout Itemize
601 the timestamp of the first event in the trace
605 \begin_layout Itemize
606 struct trace_time get_time_end(struct trace *tr)
610 \begin_layout Standard
614 \begin_layout Standard
618 \begin_layout Itemize
619 the timestamp of the last event in the trace
623 \begin_layout Itemize
624 register_callback_new_event(struct trace *tr, struct trace_time after, void
625 (*cb)(struct trace *, void *priv), void *private)
629 \begin_layout Standard
630 Register a callback that is called when a new event becomes available in
632 It is also called when the live trace ends.
635 \begin_layout Standard
639 \begin_layout Itemize
640 after (timestamp) : call only if the timestamp of the event is later than
646 \begin_layout Itemize
647 cb: the callback function
648 \begin_inset Note Note
651 \begin_layout Standard
652 specify its args and return val
660 \begin_layout Standard
664 \begin_layout Itemize
665 Result (success or failure)
669 \begin_layout Section
670 High level reading of traces
673 \begin_layout Standard
674 When reading/analyzing/viewing several traces of heterogenous types, these
675 traces are read by translator modules, which export the Low Level Trace
677 The traceset service then uses this API to read each of these traces individual
678 ly, merging them along the way.
679 It may apply timestamp offsetting or other synchronization techniques.
680 To allow views and analyses to access events, it in turn exports the High
681 Level Trace Reading API.
684 \begin_layout Standard
685 The goal of this API is to provide a uniform way for analyses and views
686 to obtain large sets of trace events from a traceset (merge of many traces
690 \begin_layout Subsection
691 Methods of the high-level trace reading API
694 \begin_layout Itemize
695 struct request_handle *traceset_new_event_request(struct traceset *tr, struct
696 trace_time t1, struct trace_time t2, struct event_filter *filter, void
697 (*cb)(void *priv, ), void *private)
701 \begin_layout Standard
702 Request a range of events from a traceset
705 \begin_layout Standard
709 \begin_layout Itemize
713 \begin_layout Itemize
717 \begin_layout Itemize
718 t2: stop timestamp (special value for infinity, for live traces)
721 \begin_layout Itemize
722 filter: filter with complex expressions
725 \begin_layout Itemize
726 private: private pointer to be passed to the callback
729 \begin_layout Standard
733 \begin_layout Itemize
734 handle to the request for cancelling it
738 \begin_layout Itemize
739 void event_request_cancel(struct request_handle *req)
743 \begin_layout Standard
747 \begin_layout Standard
751 \begin_layout Itemize
752 req: the handle to the request
756 \begin_layout Subsection
760 \begin_layout Standard
761 States are key/value pairs associated with a time range.
762 Keys can be (and generally are) duplicated as long as they do not apply
763 to overlapping ranges.
766 \begin_layout Standard
767 Keys are character strings.
770 \begin_layout Standard
771 Values may be of various types:
774 \begin_layout Itemize
778 \begin_layout Itemize
782 \begin_layout Itemize
786 \begin_layout Itemize
787 blob (binary block of arbitrary length)
790 \begin_layout Itemize
794 \begin_layout Itemize
798 \begin_layout Standard
799 The state information must be persistent between executions of the framework.
802 \begin_layout Standard
803 It is possible to assign a state to the range -infinity..infinity to indicate
804 that it is global to the trace.
807 \begin_layout Standard
808 The key names should be hierarchical.
811 \begin_layout Subsubsection
812 Methods of the State Accessing API
815 \begin_layout Itemize
816 struct state_value *state_get_value_at_time(char *key, struct trace_time
821 \begin_layout Standard
822 Request the value of a given key at a point in time
825 \begin_layout Standard
829 \begin_layout Itemize
830 var: the state variables (string)
833 \begin_layout Itemize
837 \begin_layout Standard
841 \begin_layout Itemize
846 \begin_layout Itemize
847 struct state_value_range **state_get_values_in_range(char *key, struct state_val
848 ue *val, struct trace_time_range range)
852 \begin_layout Standard
853 Request all the states changes of a given set of state variables between
857 \begin_layout Standard
861 \begin_layout Itemize
865 \begin_layout Itemize
866 range: the time range
870 \begin_layout Itemize
871 Other functions for getting values for a set of keys at once?
874 \begin_layout Subsubsection
875 Methods of the State Setting API
878 \begin_layout Itemize
879 set a particular state
882 \begin_layout Itemize
886 \begin_layout Section
887 Describing event types
890 \begin_layout Subsection
894 \begin_layout Standard
895 Because tracepoints may be created dynamically, information about the descriptio
896 n of events is just as dynamic.
897 In this context, one simple way to communicate the event description informatio
898 n to upper layers would be to send them as events, as it is done in recent
900 The core events used to describe other events are the only ones whose descripti
901 on is hardcoded in the framework.
904 \begin_layout Standard
905 These event-type-describing events could then be received and interpreted
906 by the Event Description Service, which would be a client to the high-level
907 tracing API at the same level as normal views and analyses.
908 It would store the information and allow the other views and analyses to
909 access it via this API.
912 \begin_layout Standard
913 Each event has a timestamp, a name and arguments of various types.
914 The framework should support the following types:
917 \begin_layout Itemize
921 \begin_layout Itemize
925 \begin_layout Itemize
929 \begin_layout Itemize
933 \begin_layout Itemize
937 \begin_layout Subsection
938 Events-describing events
941 \begin_layout Itemize
942 Event type declaration event
946 \begin_layout Standard
947 Announce the existence of an event type
950 \begin_layout Itemize
955 \begin_layout Itemize
956 Argument declaration event
960 \begin_layout Standard
961 Announce the existence of an event argument
964 \begin_layout Itemize
968 \begin_layout Itemize
972 \begin_layout Itemize
977 \begin_layout Itemize
982 \begin_layout Standard
983 Announce that an event type ceased to exist
986 \begin_layout Itemize
991 \begin_layout Subsection
992 Methods of the Event Type Description API
995 \begin_layout Standard
996 The event type description service provides the following functions.
999 \begin_layout Itemize
1000 GArray<struct event_type *> *traceset_get_all_event_types(struct traceset
1005 \begin_layout Standard
1006 Get the list of all the event types
1009 \begin_layout Standard
1013 \begin_layout Itemize
1014 ts: the traceset of which we want the event types
1017 \begin_layout Standard
1021 \begin_layout Itemize
1022 A GArray of of struct event_type.
1023 The GArray must be gfree()'d by the caller when it is done reading it.
1027 \begin_layout Itemize
1028 struct event_type *traceset_get_event_type_by_name(struct traceset *ts,
1033 \begin_layout Standard
1034 Find an event type by name
1037 \begin_layout Standard
1041 \begin_layout Itemize
1042 ts: the traceset of which we want the event type
1045 \begin_layout Itemize
1046 name: the name of the of the event type we are looking for
1049 \begin_layout Standard
1053 \begin_layout Itemize
1054 A pointer to the event type (must not be free'd) or NULL if not found
1058 \begin_layout Itemize
1059 GArray<struct event arg *> *event_type_get_all_args(struct event_type *evtype)
1063 \begin_layout Standard
1064 Get the list of arguments of an event
1067 \begin_layout Standard
1071 \begin_layout Itemize
1072 eventype: the event type of which we want the arguments
1075 \begin_layout Standard
1079 \begin_layout Itemize
1080 A GArray of struct event_args.
1081 The GArray must be gfree()'d by the caller when it is done reading it.
1085 \begin_layout Itemize
1086 struct event_arg *event_type_get_arg_by_name(struct event_type *evtype)
1090 \begin_layout Standard
1091 Find an argument by name
1095 \begin_layout Itemize
1096 Functions for accessing struct event_arg fields
1099 \begin_layout Section
1103 \begin_layout Subsection
1107 \begin_layout Standard
1108 Events contain the following information.
1111 \begin_layout Itemize
1115 \begin_layout Itemize
1116 Event type identifier - an event id (integer) - hidden to the API users,
1117 manipulated as pointers/references to struct event_type
1120 \begin_layout Itemize
1121 A reference to the trace it was in
1124 \begin_layout Subsection
1125 Methods of the Event inspecting API
1128 \begin_layout Itemize
1129 struct event_type *event_get_type(struct traceset *ts, struct event *ev)
1133 \begin_layout Standard
1134 get the event type corresponding to an event
1137 \begin_layout Standard
1141 \begin_layout Itemize
1145 \begin_layout Itemize
1149 \begin_layout Standard
1153 \begin_layout Itemize
1154 The event type or NULL if no information
1158 \begin_layout Itemize
1159 struct trace_time event_get_time(struct event *ev)
1163 \begin_layout Standard
1168 \begin_layout Itemize
1169 struct trace *event_get_trace(struct event *ev)
1172 \begin_layout Itemize
1173 get the name of the machine on which the event occured or other location
1177 \begin_layout Itemize
1178 get information on the type of tracing technology that was used
1181 \begin_layout Itemize
1182 get the corresponding tracepoint (machine/tracing technology/name/location
1183 in code(if available))
1186 \begin_layout Itemize
1187 uint32 event_read_arg_uint32(struct event *ev, struct event_arg *arg)
1190 \begin_layout Itemize
1191 int32 event_read_arg_int32(struct event *ev, struct event_arg *arg)
1194 \begin_layout Itemize
1195 uint64 event_read_arg_uint64(struct event *ev, struct event_arg *arg)
1198 \begin_layout Itemize
1199 int64 event_read_arg_int64(struct event *ev, struct event_arg *arg)
1202 \begin_layout Itemize
1203 float32 event_read_arg_float32(struct event *ev, struct event_arg *arg)
1206 \begin_layout Itemize
1207 float64 event_read_arg_float64(struct event *ev, struct event_arg *arg)
1210 \begin_layout Section
1214 \begin_layout Standard
1215 A filtering API is proposed.
1218 \begin_layout Section
1219 Controlling the tracing of a system
1222 \begin_layout Subsection
1226 \begin_layout Standard
1227 The goal of this API is to provide a portable way of controlling tracing
1228 of heterogenous systems.
1231 \begin_layout Standard
1232 It should enable to do the following actions.
1235 \begin_layout Itemize
1236 Set the parameters of a trace (channels, buffering, destination of data
1237 (file, network, process)...)
1240 \begin_layout Itemize
1241 Control the recording of events (start, stop, pause the trace)
1244 \begin_layout Itemize
1245 Create tracepoints (on dynamic tracers) and control which tracepoints are
1246 activated (some may be at trace level, others might only permit system
1250 \begin_layout Subsection
1251 Methods of the Trace Control API
1254 \begin_layout Itemize
1255 List the static tracepoints available on a system
1259 \begin_layout Standard
1260 These may be static tracepoints (active or inactive) or dynamic tracepoints
1261 (active or proposed).
1265 \begin_layout Itemize
1266 Add a new dynamic tracepoint
1269 \begin_layout Itemize
1270 Activate a tracepoint
1273 \begin_layout Itemize
1274 Deactivate a tracepoint
1277 \begin_layout Itemize
1278 List available probes
1281 \begin_layout Itemize
1282 Connect a probe to a tracepoint
1285 \begin_layout Itemize
1289 \begin_layout Itemize
1293 \begin_layout Itemize
1294 \begin_inset Note Greyedout
1297 \begin_layout Standard