Documentation of live usage and protocol
[lttng-tools.git] / doc / live-reading-protocol.txt
1 LTTng Live trace reading protocol
2
3 Julien Desfossez
4 September 18th, 2013
5
6 This document describes the protocol of live trace reading. It is mainly
7 intended for trace-viewer developers.
8
9 Live trace reading allows a viewer to read safely a LTTng trace while it is
10 being recorded. The protocol ensures that the viewer is never in a position
11 where it can read for which it does not have the metadata, and also allows to
12 viewer to know about inactive streams and skip those up to a certain point in
13 time.
14
15 This feature is implemented starting at lttng-tools 2.4
16
17 * General informations
18 All the data structures required to implement this protocole are provided in
19 the lttng-viewer.h header. All integer are encoded in big endian during the
20 transfer.
21
22 Just like the streaming protocol, this protocol works always in only one
23 direction : from the viewer to the relay. After each command, the relay sends a
24 reply to the viewer (or closes the connection on fatal error).
25
26 When the viewer sends a command to the relay, it sends a struct
27 lttng_viewer_cmd which contains the command (enum lttcomm_relayd_command) and
28 the size of the actual command if the command requires a payload.
29 For example, if the viewer wants to list the sessions, it sends a struct
30 lttng_viewer_cmd with :
31 cmd = htobe32(VIEWER_CONNECT);
32 data_size = htobe64(sizeof(struct lttng_viewer_connect));
33
34 The cmd_version is currently unused, but might get useful when we extend the
35 protocol later.
36
37 * Protocol sequence
38 In this section, we describe the normal sequence of event during a live-reading
39 session. The viewer is abbreviated V and the relay R for conciseness.
40
41 Establishing a connection :
42 - V connects to R on port TCP/5344
43 - V establishes a session by sending a VIEWER_CONNECT command, payload in
44 struct lttng_viewer_connect. In this struct, it sets its major and minor
45 version of the protocol it implements, and the type of connection it wants,
46 for now only VIEWER_CLIENT_COMMAND is supported.
47 - If the protocol implemented by V and R are compatible, R sends back the same
48 struct with its own version and the newly assigned viewer_session_id.
49 Protocols are compatible if they have the same major number. At this point,
50 if the communication continues and the minor are not the same, it is implied
51 that the two processes will use the min(minor) of the protocol during this
52 connection. Protocol versions follow lttng-tools version, so if R implements
53 the 2.5 protocol and V implements the 2.4 protocol, R will use the 2.4
54 protocol for this connection.
55
56 List the sessions :
57 Once V and R agree on a protocol, V can start interacting with R. The first
58 thing to do is list the sessions currently active on R.
59 - V sends the command VIEWER_LIST_SESSIONS with no payload (data_size ignored)
60 - R sends back a struct lttng_viewer_list_sessions which contains the number of
61 sessions to receive, and then for each session (sessions_count), it sends a
62 struct lttng_viewer_session
63 - V must first receive the struct lttng_viewer_list_sessions and then receive
64 all the lttng_viewer_session structs. The live_timer corresponds to the value
65 set by the user who created the tracing session, if it is 0, the session
66 cannot be read in live. This timer in microseconds is the minimum rate at
67 which R receives information about the running session. The "clients" field
68 contains the number of connected clients to this session, for now only one
69 client at a time can attach to session.
70
71 Attach to a session :
72 Now V can select a session and attach to it. In order to do so, it sends the
73 command VIEWER_ATTACH_SESSION with the session_id it wants. The "seek"
74 parameter allows the viewer to attach to a session from its beginning (it will
75 receive all trace data still on the relayd) or from now (data will be available
76 to read starting at the next packet received on the relay). Only one session
77 can be established by connection. Lots of clock synchronisation issues can
78 happen when connecting to multiple sessions from multiple hosts at the same
79 time, we let the viewers have fun with it ;-)
80 R replies with a struct lttng_viewer_attach_session_response with a status and
81 the number of streams currently active in this session. Then, for each stream,
82 it sends a struct lttng_viewer_stream. Just like with the session list, V must
83 receive the first header and then all the stream structs. The ctf_trace_id
84 parameter in the struct lttng_viewer_stream is particularly important since it
85 allows the viewer to match all the streams belonging to the same CTF trace (so
86 one metadata file and multiple stream files). If the stream is a metadata
87 stream, metadata_flag will be set to 1.
88
89 #### below needs to be well written, but the essential is here ###
90
91 Get metadata :
92 A CTF trace cannot be read without the complete metadata.
93 Send the command VIEWER_GET_METADATA and the struct lttng_viewer_get_metadata.
94
95 Once we have all the metadata, we can start processing the trace. In order to
96 do that, we work with the indexes. Whenever we need to read a new packet from a
97 stream, we first ask for the next index for this stream and then ask for a
98 trace packet at a certain offset and length.
99
100 Get the next index :
101 Command VIEWER_GET_NEXT_INDEX
102 struct lttng_viewer_get_next_index
103 Receive back a struct lttng_viewer_index
104
105 Get data packet :
106 Command VIEWER_GET_PACKET
107 struct lttng_viewer_get_packet
108 Receive back a struct lttng_viewer_trace_packet
109
110 For the VIEWER_GET_NEXT_INDEX and VIEWER_GET_PACKET, the viewer must check the
111 "flags" element of the struct it receives, because it contains important
112 information such as the information that new metadata must be received before
113 being able to receive and read the next packet.
114 When new metadata is added during a session, the GET_NEXT_INDEX will succeed
115 but it will have the flag LTTNG_VIEWER_FLAG_NEW_METADATA, but the
116 GET_DATA_PACKET will fail with the same flag as long as the metadata is not
117 downloaded.
This page took 0.044344 seconds and 5 git commands to generate.