configure: add --enable-Werror
[lttng-tools.git] / doc / live-reading-protocol.txt
CommitLineData
24bc0841
JD
1LTTng Live trace reading protocol
2
3Julien Desfossez
6e92aae6 4February 6th, 2014
24bc0841
JD
5
6This document describes the protocol of live trace reading. It is mainly
7intended for trace-viewer developers.
8
9Live trace reading allows a viewer to read safely a LTTng trace while it is
10being recorded. The protocol ensures that the viewer is never in a position
11where it can read for which it does not have the metadata, and also allows to
12viewer to know about inactive streams and skip those up to a certain point in
13time.
14
15This feature is implemented starting at lttng-tools 2.4
16
17* General informations
18All the data structures required to implement this protocole are provided in
6e92aae6 19the lttng-viewer-abi.h header. All integer are encoded in big endian during the
24bc0841
JD
20transfer.
21
22Just like the streaming protocol, this protocol works always in only one
23direction : from the viewer to the relay. After each command, the relay sends a
24reply to the viewer (or closes the connection on fatal error).
25
26When the viewer sends a command to the relay, it sends a struct
27lttng_viewer_cmd which contains the command (enum lttcomm_relayd_command) and
28the size of the actual command if the command requires a payload.
29For example, if the viewer wants to list the sessions, it sends a struct
30lttng_viewer_cmd with :
31cmd = htobe32(VIEWER_CONNECT);
32data_size = htobe64(sizeof(struct lttng_viewer_connect));
33
34The cmd_version is currently unused, but might get useful when we extend the
35protocol later.
36
37* Protocol sequence
38In this section, we describe the normal sequence of event during a live-reading
39session. The viewer is abbreviated V and the relay R for conciseness.
40
41Establishing a connection :
42- V connects to R on port TCP/5344
43- V establishes a session by sending a VIEWER_CONNECT command, payload in
44 struct lttng_viewer_connect. In this struct, it sets its major and minor
45 version of the protocol it implements, and the type of connection it wants,
46 for now only VIEWER_CLIENT_COMMAND is supported.
47- If the protocol implemented by V and R are compatible, R sends back the same
48 struct with its own version and the newly assigned viewer_session_id.
49 Protocols are compatible if they have the same major number. At this point,
50 if the communication continues and the minor are not the same, it is implied
51 that the two processes will use the min(minor) of the protocol during this
52 connection. Protocol versions follow lttng-tools version, so if R implements
53 the 2.5 protocol and V implements the 2.4 protocol, R will use the 2.4
54 protocol for this connection.
55
56List the sessions :
57Once V and R agree on a protocol, V can start interacting with R. The first
58thing to do is list the sessions currently active on R.
59- V sends the command VIEWER_LIST_SESSIONS with no payload (data_size ignored)
60- R sends back a struct lttng_viewer_list_sessions which contains the number of
61 sessions to receive, and then for each session (sessions_count), it sends a
62 struct lttng_viewer_session
63- V must first receive the struct lttng_viewer_list_sessions and then receive
64 all the lttng_viewer_session structs. The live_timer corresponds to the value
65 set by the user who created the tracing session, if it is 0, the session
66 cannot be read in live. This timer in microseconds is the minimum rate at
67 which R receives information about the running session. The "clients" field
68 contains the number of connected clients to this session, for now only one
69 client at a time can attach to session.
70
71Attach to a session :
6e92aae6
JD
72Now V can select and attach one or multiple session IDs, but first, it needs to
73create a viewer_session. Creating a viewer session is done by sending the
74command LTTNG_VIEWER_CREATE_SESSION. In the future, this would be the place
75where we could specify options global to the viewer session.
76Once the session is created, the viewer can issue one or multiple
77VIEWER_ATTACH_SESSION commands with the session_id it wants. The "seek"
24bc0841
JD
78parameter allows the viewer to attach to a session from its beginning (it will
79receive all trace data still on the relayd) or from now (data will be available
6e92aae6
JD
80to read starting at the next packet received on the relay). The viewer can
81issue this command multiple times and at any moment in the process.
24bc0841
JD
82R replies with a struct lttng_viewer_attach_session_response with a status and
83the number of streams currently active in this session. Then, for each stream,
84it sends a struct lttng_viewer_stream. Just like with the session list, V must
85receive the first header and then all the stream structs. The ctf_trace_id
86parameter in the struct lttng_viewer_stream is particularly important since it
87allows the viewer to match all the streams belonging to the same CTF trace (so
88one metadata file and multiple stream files). If the stream is a metadata
89stream, metadata_flag will be set to 1.
6e92aae6
JD
90The relay ensures that it sends only ready ctf traces, so once this command is
91complete, V knows that it has all the streams ready to be processed.
92A quick note about the "sessions": from the relay perspective we see one
93session for each domain (kernel, ust32, ust64) of a session as created on the
94sessiond. For example, if the user creates a session and adds events in the
95kernel and UST and has only 64-bit applications running, the relay sees two
96sessions with the same name and hostname. So in order to display a session as
97seen by the user, the viewer has to attach to all the sessions with the same
98hostname and session name. There might be clashes if two servers on the network
99have the same hostname and create the same session name, but the relay cannot
100distinguish these cases, so it is currently a known limitation.
101During a session, new streams might get added (especially in per-pid tracing)
102so the viewer must be ready to add new streams while processing the trace. To
103inform V that new streams are available, R sets the
104LTTNG_VIEWER_FLAG_NEW_STREAM flag on LTTNG_VIEWER_GET_NEXT_INDEX and
105LTTNG_VIEWER_GET_PACKET replies. The viewer must then issue the
106LTTNG_VIEWER_GET_NEW_STREAMS command and receive all the streams, just like
107with the attach command.
24bc0841
JD
108
109#### below needs to be well written, but the essential is here ###
110
111Get metadata :
112A CTF trace cannot be read without the complete metadata.
113Send the command VIEWER_GET_METADATA and the struct lttng_viewer_get_metadata.
114
115Once we have all the metadata, we can start processing the trace. In order to
116do that, we work with the indexes. Whenever we need to read a new packet from a
117stream, we first ask for the next index for this stream and then ask for a
118trace packet at a certain offset and length.
119
120Get the next index :
121Command VIEWER_GET_NEXT_INDEX
122struct lttng_viewer_get_next_index
123Receive back a struct lttng_viewer_index
6e92aae6
JD
124We might also receive flags :
125- LTTNG_VIEWER_FLAG_NEW_METADATA the viewer must ask new metadata
126 (LTTNG_VIEWER_GET_METADATA)
127- LTTNG_VIEWER_FLAG_NEW_STREAM the viewer must get the new streams
128 (LTTNG_VIEWER_GET_NEW_STREAMS)
24bc0841
JD
129
130Get data packet :
131Command VIEWER_GET_PACKET
132struct lttng_viewer_get_packet
133Receive back a struct lttng_viewer_trace_packet
6e92aae6
JD
134We might also receive a LTTNG_VIEWER_GET_PACKET_ERR and some flags :
135- LTTNG_VIEWER_FLAG_NEW_METADATA the viewer must ask new metadata
136 (LTTNG_VIEWER_GET_METADATA)
137- LTTNG_VIEWER_FLAG_NEW_STREAM the viewer must get the new streams
138 (LTTNG_VIEWER_GET_NEW_STREAMS)
24bc0841
JD
139
140For the VIEWER_GET_NEXT_INDEX and VIEWER_GET_PACKET, the viewer must check the
141"flags" element of the struct it receives, because it contains important
142information such as the information that new metadata must be received before
143being able to receive and read the next packet.
144When new metadata is added during a session, the GET_NEXT_INDEX will succeed
145but it will have the flag LTTNG_VIEWER_FLAG_NEW_METADATA, but the
146GET_DATA_PACKET will fail with the same flag as long as the metadata is not
147downloaded.
2bedf848
JD
148
149Detach from a session:
150Closing the network connection detaches a client from all the sessions it is
151currently attached to. It is also possible to detach from a specific session
152without disconnecting by sending the LTTNG_VIEWER_DETACH_SESSION command. The
153payload of this command is the session ID.
This page took 0.061381 seconds and 4 git commands to generate.