1 #LyX 1.5.5 created this file. For more info see http://www.lyx.org/
10 \font_typewriter default
11 \font_default_family default
17 \paperfontsize default
25 \paperorientation portrait
28 \paragraph_separation skip
30 \quotes_language english
33 \paperpagestyle default
34 \tracking_changes false
43 A Set of APIs for a Third-Generation Trace Viewer
48 Pierre-Marc Fournier, Michel Dagenais, Mathieu Desnoyers
53 <pierre-marc.fournier _at_ polymtl.ca>
62 École Polytechnique de Montréal
69 \begin_layout Standard
70 This document proposes some APIs for a trace viewing and analysis infrastructure.
73 \begin_layout Standard
74 Design choices were made based on the experience acquired by the design,
75 development and use of two previous generations of trace viewers, respectively
76 the Trace Visualizer from the Linux Trace Toolkit and its successor, the
77 LTTV from the LTTng tracing toolkit.
85 Provide an infrastructure for fast, interactive visualization and analysis
86 of huge traces (>10 GB) on standard workstations
90 \begin_layout Standard
91 Efficiency is of great importance.
92 It is tempting to think that as computers get faster, it will be possible
93 to run more complex analyses on traces in a shorter time.
94 In fact, the size of the traces grows as computers get more powerful, because
95 they do more operations than before in the same amount of time and therefore
97 The current tendency to parallelize systems is another factor that results
106 \begin_layout Itemize
107 Allow efficient manipulation of traces that are larger than the system memory
110 \begin_layout Itemize
111 Support processing of live traces
114 \begin_layout Itemize
115 Allow merging of traces of different formats
118 \begin_layout Itemize
119 Design with both command line utilities (C language) and graphical interfaces
120 (Java and others) in mind, sharing code and standardizing interfaces as
125 \begin_layout Standard
126 Both command-line interfaces and graphical interfaces have an important
127 role to play in trace analysis.
128 While graphical interfaces allow for complex views, command-line utilities
129 can be used quickly in more environments; they can also be more convenient
130 in some circumstances.
131 A good design should be such that they can share code and APIs.
135 \begin_layout Section
139 \begin_layout Itemize
140 All data structures must be accessed through API methods, in order to allow
141 for internal change without breaking compatibility.
144 \begin_layout Itemize
145 The APIs in this document are described as C functions and datatypes.
146 When an operation applies to an abstraction that could be an object in
147 an object-oriented language, it is specified as the first argument of the
151 \begin_layout Subsection
155 \begin_layout Standard
156 The framework needs to represent points in the time line and time ranges.
157 This is done by two data structures, respectively:
160 \begin_layout Itemize
164 \begin_layout Itemize
165 struct trace_time_range
168 \begin_layout Standard
169 The preferred representation for times is seconds.
172 \begin_layout Standard
173 Ranges are a span between two points in time.
174 They are represented as these two points.
175 One or both of them can have the special value
182 \begin_layout Standard
183 Each time is accompanied by an uncertainty value.
186 \begin_layout Subsection
190 \begin_layout Standard
191 The following modules should be part of the framework.
192 This list does not include modules related to the GUI, the controlling
193 of traces and the transfer of traces.
196 \begin_layout Description
197 Trace Has one instance per open trace.
198 Allows the access to a trace by exporting the Low-level Trace Reading API.
199 The underlying implementation may change from one trace to another to adapt
200 to the specific format of each trace.
203 \begin_layout Description
204 Traceset Receives data from several Traces using their Low-Level Trace Reading
205 API and merge them in a single stream.
206 May also receive data from other instances of Traceset.
207 Exports the High-level Trace Reading API and the Low-level Trace Reading
209 Handles time offsetting if necessary to align traces.
213 \begin_layout Description
215 providers They receive the stream of events from the Traceset, using
216 the High-level Trace Reading API.
217 They detect state changes.
218 Each one is designed to detect states of a particular level (kernel, applicatio
219 n-level, UML states).
220 It pushes the detected states to the State module using the State Setting
224 \begin_layout Description
225 State Records changes in the system state received from State providers
226 by the State Setting API.
227 It implements an efficient state searching engine.
228 Exports the State Accessing API to allow plugins to access the information.
231 \begin_layout Description
233 description Receives the stream of events from the Traceset using the
234 High-level Trace Reading API.
235 It reads the events that describe other events and stores the information
236 they contain, possibly using the State module.
237 Other components can access this information through an exported API.
240 \begin_layout Description
242 These are the graphical views, analyses and other components that need
243 access to the events.
244 They receive data from a traceset using the High-level Trace Reading API.
245 They use the services they need (State, Event Description, and/or others).
246 They may export a special API if other components need to access their
250 \begin_layout Section
251 Low level reading of traces
254 \begin_layout Subsection
258 \begin_layout Standard
259 This API presents a uniform way of accessing traces at a low level, that
260 is to access randomly the events of the trace in an physical-encoding-independa
264 \begin_layout Standard
265 \begin_inset Note Note
268 \begin_layout Subsubsection
272 \begin_layout Standard
273 Timestamps, at the framework level, should be in an architecture-independent
275 One possibility would be to have them stored in seconds (with decimals).
276 Offsetting, if necessary, could be handled by the traceset (trace merging)
278 An uncertainty value should accompany the timestamp.
279 Timestamps in cycles should be available if they are provided by the tracer.
287 \begin_layout Standard
288 \begin_inset Note Note
291 \begin_layout Subsubsection
295 \begin_layout Itemize
296 void get_supported_formats(void)
300 \begin_layout Standard
304 \begin_layout Standard
308 \begin_layout Itemize
309 a list of supported formats
313 \begin_layout Itemize
318 \begin_layout Standard
322 \begin_layout Itemize
327 \begin_layout Itemize
331 \begin_layout Itemize
332 other special url to establish a network connection
335 \begin_layout Itemize
340 \begin_layout Itemize
341 trace format (optional, for cases where auto detection would not work or
345 \begin_layout Standard
349 \begin_layout Itemize
358 \begin_layout Subsection
359 Methods of the Low Level Trace Reading API
362 \begin_layout Itemize
363 void close(struct trace *tr)
367 \begin_layout Standard
368 Close the trace and unallocate all resources associated to this struct trace
369 including the handle.
370 After this call, tr is invalid and its memory is freed.
373 \begin_layout Standard
377 \begin_layout Itemize
381 \begin_layout Standard
386 \begin_layout Itemize
387 struct event *current_event(struct trace *tr, struct cursor *cur)
391 \begin_layout Standard
392 Return the event pointed by the cursor cur.
395 \begin_layout Standard
399 \begin_layout Itemize
403 \begin_layout Itemize
404 cur: the cursor indicating the position of the wanted event
407 \begin_layout Standard
411 \begin_layout Itemize
412 the event currently pointed by the position pointer
415 \begin_layout Itemize
416 NULL, if current_event is called on a new cursor that is positioned before
417 the beginning of the trace
421 \begin_layout Itemize
422 int advance(struct trace *tr, struct trace_cursor *cur)
426 \begin_layout Standard
427 Advance the cursor to the next event in the trace.
430 \begin_layout Standard
434 \begin_layout Itemize
438 \begin_layout Itemize
439 cur: the cursor that should be advanced
442 \begin_layout Standard
446 \begin_layout Itemize
451 \begin_layout Itemize
452 Success (TRACE_ADVANCE_OK)
455 \begin_layout Itemize
456 The cursor was not advanced because the end of trace is reached (end of
457 a non-live trace or of a completed live trace) (TRACE_ADVANCE_END)
460 \begin_layout Itemize
461 The cursor was not advanced because no new events are available at this
462 time (with a live trace that is still running) (TRACE_ADVANCE_TRY_AGAIN)
467 \begin_layout Itemize
468 int seek_time(struct trace *tr, struct trace_cursor *cur, struct trace_time
473 \begin_layout Standard
474 Place a cursor at a particular time index in a trace.
477 \begin_layout Standard
481 \begin_layout Itemize
482 tr: the trace that should be seeked
485 \begin_layout Itemize
486 cur: the cursor to seek
489 \begin_layout Itemize
490 time: the time to seek to
493 \begin_layout Standard
497 \begin_layout Itemize
501 \begin_layout Itemize
502 != 0: time out of range
506 \begin_layout Itemize
507 struct trace_cursor *trace_new_cursor(struct trace *tr)
511 \begin_layout Standard
512 Return a new cursor associated to the trace.
513 The position of this new cursor is just before the first event, therefore
514 advance() must be called before reading the first event.
517 \begin_layout Standard
521 \begin_layout Itemize
525 \begin_layout Standard
529 \begin_layout Itemize
534 \begin_layout Itemize
535 struct trace_cursor *trace_cursor_copy(struct trace_cursor *cur)
539 \begin_layout Standard
543 \begin_layout Standard
547 \begin_layout Itemize
548 cur: the cursor to copy
551 \begin_layout Standard
555 \begin_layout Itemize
556 a new cursor that is at the same location as cur
560 \begin_layout Itemize
561 void trace_cursor_destroy(struct trace_cursor *cur)
565 \begin_layout Standard
566 Free all resources associated to a cursor.
567 After this call, cur is invalid.
570 \begin_layout Standard
574 \begin_layout Itemize
575 cur: the cursor to destroy
578 \begin_layout Standard
583 \begin_layout Itemize
584 struct trace_time get_time_start(struct trace *tr)
588 \begin_layout Standard
592 \begin_layout Itemize
596 \begin_layout Standard
600 \begin_layout Itemize
601 the timestamp of the first event in the trace
605 \begin_layout Itemize
606 struct trace_time get_time_end(struct trace *tr)
610 \begin_layout Standard
614 \begin_layout Standard
618 \begin_layout Itemize
619 the timestamp of the last event in the trace
623 \begin_layout Itemize
624 register_callback_new_event(struct trace *tr, struct trace_time after, void
625 (*cb)(struct trace *, void *priv), void *private)
629 \begin_layout Standard
630 Register a callback that is called when a new event becomes available in
632 It is also called when the live trace ends.
635 \begin_layout Standard
639 \begin_layout Itemize
640 after (timestamp) : call only if the timestamp of the event is later than
646 \begin_layout Itemize
647 cb: the callback function
648 \begin_inset Note Note
651 \begin_layout Standard
652 specify its args and return val
660 \begin_layout Standard
664 \begin_layout Itemize
665 Result (success or failure)
669 \begin_layout Section
670 High level reading of traces
673 \begin_layout Subsection
677 \begin_layout Standard
678 When reading/analyzing/viewing several traces of heterogenous types, these
679 traces are read by translator modules, which export the Low Level Trace
681 The traceset service then uses this API to read each of these traces individual
682 ly, merging them along the way.
683 It may apply timestamp offsetting or other synchronization techniques.
684 To allow views and analyses to access events, it in turn exports the High
685 Level Trace Reading API.
688 \begin_layout Standard
689 The goal of this API is to provide a uniform way for analyses and views
690 to obtain large sets of trace events from a traceset (merge of many traces
694 \begin_layout Subsection
695 Methods of the high-level trace reading API
698 \begin_layout Itemize
699 struct request_handle *traceset_new_event_request(struct traceset *tr, struct
700 trace_time t1, struct trace_time t2, struct event_filter *filter, void
701 (*cb)(void *priv, ), void *private)
705 \begin_layout Standard
706 Request a range of events from a traceset
709 \begin_layout Standard
713 \begin_layout Itemize
717 \begin_layout Itemize
721 \begin_layout Itemize
722 t2: stop timestamp (special value for infinity, for live traces)
725 \begin_layout Itemize
726 filter: filter with complex expressions
729 \begin_layout Itemize
730 private: private pointer to be passed to the callback
733 \begin_layout Standard
737 \begin_layout Itemize
738 handle to the request for cancelling it
742 \begin_layout Itemize
743 void event_request_cancel(struct request_handle *req)
747 \begin_layout Standard
751 \begin_layout Standard
755 \begin_layout Itemize
756 req: the handle to the request
760 \begin_layout Section
764 \begin_layout Standard
765 States are key/value pairs associated with a time range.
766 Keys can be (and generally are) duplicated as long as they do not apply
767 to overlapping ranges.
770 \begin_layout Standard
771 Keys are character strings.
772 They are organized in a filesystem-like hierarchy.
775 \begin_layout Standard
776 Each key/value pair is associated either to a specific trace or to the traceset.
779 \begin_layout Standard
780 State can persist between runs of the program.
781 This is useful both to reduce pre-calculation times when re-opening a trace,
782 and also to keep general parameters like bookmarks.
785 \begin_layout Standard
786 It is possible to assign a state to the time range -infinity..infinity to
787 indicate that it is global to the trace.
790 \begin_layout Standard
791 Values may be of various types:
794 \begin_layout Itemize
798 \begin_layout Itemize
802 \begin_layout Itemize
806 \begin_layout Itemize
810 \begin_layout Itemize
814 \begin_layout Itemize
815 blob (binary block of arbitrary length)
818 \begin_layout Itemize
822 \begin_layout Itemize
826 \begin_layout Itemize
830 \begin_layout Subsection
831 Methods of the State Accessing API
834 \begin_layout Itemize
835 struct state_value *state_get_value_at_time(char *key, struct trace_time
840 \begin_layout Standard
841 Request the value of a given key at a point in time
844 \begin_layout Standard
848 \begin_layout Itemize
849 var: the state variables (string)
852 \begin_layout Itemize
856 \begin_layout Standard
860 \begin_layout Itemize
862 A struct state_value contains the value and the time interval that applies
867 \begin_layout Itemize
868 struct state_value_range **state_get_values_in_range(char *key, struct state_val
869 ue *val, struct trace_time_range range)
873 \begin_layout Standard
874 Request all the state changes of a given set of state variables between
878 \begin_layout Standard
882 \begin_layout Itemize
886 \begin_layout Itemize
887 range: the time range
890 \begin_layout Standard
894 \begin_layout Itemize
899 \begin_layout Itemize
900 Other functions for getting values for a set of keys at once?
903 \begin_layout Subsection
904 Methods of the State Setting API
907 \begin_layout Itemize
908 set a particular state
911 \begin_layout Itemize
915 \begin_layout Section
916 Describing event types
919 \begin_layout Subsection
923 \begin_layout Standard
924 Because tracepoints may be created dynamically, information about the descriptio
925 n of events is just as dynamic.
926 In this context, one simple way to communicate the event description informatio
927 n to upper layers would be to send them as events, as it is done in recent
929 The core events used to describe other events are the only ones whose descripti
930 on is hardcoded in the framework.
933 \begin_layout Standard
934 These event-type-describing events could then be received and interpreted
935 by the Event Description Service, which would be a client to the high-level
936 tracing API at the same level as normal views and analyses.
937 It would store the information and allow the other views and analyses to
938 access it via this API.
941 \begin_layout Standard
942 Each event has a timestamp, a name and arguments of various types.
943 The framework should support the following types:
946 \begin_layout Itemize
950 \begin_layout Itemize
954 \begin_layout Itemize
958 \begin_layout Itemize
962 \begin_layout Itemize
966 \begin_layout Subsection
967 Events-describing events
970 \begin_layout Itemize
971 Event type declaration event
975 \begin_layout Standard
976 Announce the existence of an event type
979 \begin_layout Itemize
984 \begin_layout Itemize
985 Argument declaration event
989 \begin_layout Standard
990 Announce the existence of an event argument
993 \begin_layout Itemize
997 \begin_layout Itemize
1001 \begin_layout Itemize
1006 \begin_layout Itemize
1011 \begin_layout Standard
1012 Announce that an event type ceased to exist
1015 \begin_layout Itemize
1020 \begin_layout Subsection
1021 Methods of the Event Type Description API
1024 \begin_layout Standard
1025 The event type description service provides the following functions.
1028 \begin_layout Itemize
1029 GArray<struct event_type *> *traceset_get_all_event_types(struct traceset
1034 \begin_layout Standard
1035 Get the list of all the event types
1038 \begin_layout Standard
1042 \begin_layout Itemize
1043 ts: the traceset of which we want the event types
1046 \begin_layout Standard
1050 \begin_layout Itemize
1051 A GArray of of struct event_type.
1052 The GArray must be gfree()'d by the caller when it is done reading it.
1056 \begin_layout Itemize
1057 struct event_type *traceset_get_event_type_by_name(struct traceset *ts,
1062 \begin_layout Standard
1063 Find an event type by name
1066 \begin_layout Standard
1070 \begin_layout Itemize
1071 ts: the traceset of which we want the event type
1074 \begin_layout Itemize
1075 name: the name of the of the event type we are looking for
1078 \begin_layout Standard
1082 \begin_layout Itemize
1083 A pointer to the event type (must not be free'd) or NULL if not found
1087 \begin_layout Itemize
1088 GArray<struct event arg *> *event_type_get_all_args(struct event_type *evtype)
1092 \begin_layout Standard
1093 Get the list of arguments of an event
1096 \begin_layout Standard
1100 \begin_layout Itemize
1101 eventype: the event type of which we want the arguments
1104 \begin_layout Standard
1108 \begin_layout Itemize
1109 A GArray of struct event_args.
1110 The GArray must be gfree()'d by the caller when it is done reading it.
1114 \begin_layout Itemize
1115 struct event_arg *event_type_get_arg_by_name(struct event_type *evtype)
1119 \begin_layout Standard
1120 Find an argument by name
1124 \begin_layout Itemize
1125 Functions for accessing struct event_arg fields
1128 \begin_layout Section
1132 \begin_layout Subsection
1136 \begin_layout Standard
1137 Events contain the following information.
1140 \begin_layout Itemize
1144 \begin_layout Itemize
1145 Event type identifier - an event id (integer) - hidden to the API users,
1146 manipulated as pointers/references to struct event_type
1149 \begin_layout Itemize
1150 A reference to the trace it was in
1153 \begin_layout Subsection
1154 Methods of the Event inspecting API
1157 \begin_layout Itemize
1158 struct event_type *event_get_type(struct traceset *ts, struct event *ev)
1162 \begin_layout Standard
1163 get the event type corresponding to an event
1166 \begin_layout Standard
1170 \begin_layout Itemize
1174 \begin_layout Itemize
1178 \begin_layout Standard
1182 \begin_layout Itemize
1183 The event type or NULL if no information
1187 \begin_layout Itemize
1188 struct trace_time event_get_time(struct event *ev)
1192 \begin_layout Standard
1197 \begin_layout Itemize
1198 struct trace *event_get_trace(struct event *ev)
1201 \begin_layout Itemize
1202 get the name of the machine on which the event occured or other location
1206 \begin_layout Itemize
1207 get information on the type of tracing technology that was used
1210 \begin_layout Itemize
1211 get the corresponding tracepoint (machine/tracing technology/name/location
1212 in code(if available))
1215 \begin_layout Itemize
1216 uint32 event_read_arg_uint32(struct event *ev, struct event_arg *arg)
1219 \begin_layout Itemize
1220 int32 event_read_arg_int32(struct event *ev, struct event_arg *arg)
1223 \begin_layout Itemize
1224 uint64 event_read_arg_uint64(struct event *ev, struct event_arg *arg)
1227 \begin_layout Itemize
1228 int64 event_read_arg_int64(struct event *ev, struct event_arg *arg)
1231 \begin_layout Itemize
1232 float32 event_read_arg_float32(struct event *ev, struct event_arg *arg)
1235 \begin_layout Itemize
1236 float64 event_read_arg_float64(struct event *ev, struct event_arg *arg)
1239 \begin_layout Section
1243 \begin_layout Standard
1244 A filtering API is proposed.
1247 \begin_layout Section
1248 Controlling the tracing of a system
1251 \begin_layout Subsection
1255 \begin_layout Standard
1256 The goal of this API is to provide a portable way of controlling tracing
1257 of heterogenous systems.
1260 \begin_layout Standard
1261 It should enable to do the following actions.
1264 \begin_layout Itemize
1265 Set the parameters of a trace (channels, buffering, destination of data
1266 (file, network, process)...)
1269 \begin_layout Itemize
1270 Control the recording of events (start, stop, pause the trace)
1273 \begin_layout Itemize
1274 Create tracepoints (on dynamic tracers) and control which tracepoints are
1275 activated (some may be at trace level, others might only permit system
1279 \begin_layout Subsection
1280 Methods of the Trace Control API
1283 \begin_layout Itemize
1284 List the static tracepoints available on a system
1288 \begin_layout Standard
1289 These may be static tracepoints (active or inactive) or dynamic tracepoints
1290 (active or proposed).
1294 \begin_layout Itemize
1295 Add a new dynamic tracepoint
1298 \begin_layout Itemize
1299 Activate a tracepoint
1302 \begin_layout Itemize
1303 Deactivate a tracepoint
1306 \begin_layout Itemize
1307 List available probes
1310 \begin_layout Itemize
1311 Connect a probe to a tracepoint
1314 \begin_layout Itemize
1318 \begin_layout Itemize
1322 \begin_layout Itemize
1323 \begin_inset Note Greyedout
1326 \begin_layout Standard