Commit | Line | Data |
---|---|---|
24bc0841 JD |
1 | LTTng Live trace reading protocol |
2 | ||
3 | Julien Desfossez | |
6e92aae6 | 4 | February 6th, 2014 |
24bc0841 JD |
5 | |
6 | This document describes the protocol of live trace reading. It is mainly | |
7 | intended for trace-viewer developers. | |
8 | ||
9 | Live trace reading allows a viewer to read safely a LTTng trace while it is | |
10 | being recorded. The protocol ensures that the viewer is never in a position | |
11 | where it can read for which it does not have the metadata, and also allows to | |
12 | viewer to know about inactive streams and skip those up to a certain point in | |
13 | time. | |
14 | ||
15 | This feature is implemented starting at lttng-tools 2.4 | |
16 | ||
17 | * General informations | |
18 | All the data structures required to implement this protocole are provided in | |
6e92aae6 | 19 | the lttng-viewer-abi.h header. All integer are encoded in big endian during the |
24bc0841 JD |
20 | transfer. |
21 | ||
22 | Just like the streaming protocol, this protocol works always in only one | |
23 | direction : from the viewer to the relay. After each command, the relay sends a | |
24 | reply to the viewer (or closes the connection on fatal error). | |
25 | ||
26 | When the viewer sends a command to the relay, it sends a struct | |
27 | lttng_viewer_cmd which contains the command (enum lttcomm_relayd_command) and | |
28 | the size of the actual command if the command requires a payload. | |
29 | For example, if the viewer wants to list the sessions, it sends a struct | |
30 | lttng_viewer_cmd with : | |
31 | cmd = htobe32(VIEWER_CONNECT); | |
32 | data_size = htobe64(sizeof(struct lttng_viewer_connect)); | |
33 | ||
34 | The cmd_version is currently unused, but might get useful when we extend the | |
35 | protocol later. | |
36 | ||
37 | * Protocol sequence | |
38 | In this section, we describe the normal sequence of event during a live-reading | |
39 | session. The viewer is abbreviated V and the relay R for conciseness. | |
40 | ||
41 | Establishing a connection : | |
42 | - V connects to R on port TCP/5344 | |
43 | - V establishes a session by sending a VIEWER_CONNECT command, payload in | |
44 | struct lttng_viewer_connect. In this struct, it sets its major and minor | |
45 | version of the protocol it implements, and the type of connection it wants, | |
46 | for now only VIEWER_CLIENT_COMMAND is supported. | |
47 | - If the protocol implemented by V and R are compatible, R sends back the same | |
48 | struct with its own version and the newly assigned viewer_session_id. | |
49 | Protocols are compatible if they have the same major number. At this point, | |
50 | if the communication continues and the minor are not the same, it is implied | |
51 | that the two processes will use the min(minor) of the protocol during this | |
52 | connection. Protocol versions follow lttng-tools version, so if R implements | |
53 | the 2.5 protocol and V implements the 2.4 protocol, R will use the 2.4 | |
54 | protocol for this connection. | |
55 | ||
56 | List the sessions : | |
57 | Once V and R agree on a protocol, V can start interacting with R. The first | |
58 | thing to do is list the sessions currently active on R. | |
59 | - V sends the command VIEWER_LIST_SESSIONS with no payload (data_size ignored) | |
60 | - R sends back a struct lttng_viewer_list_sessions which contains the number of | |
61 | sessions to receive, and then for each session (sessions_count), it sends a | |
62 | struct lttng_viewer_session | |
63 | - V must first receive the struct lttng_viewer_list_sessions and then receive | |
64 | all the lttng_viewer_session structs. The live_timer corresponds to the value | |
65 | set by the user who created the tracing session, if it is 0, the session | |
66 | cannot be read in live. This timer in microseconds is the minimum rate at | |
67 | which R receives information about the running session. The "clients" field | |
68 | contains the number of connected clients to this session, for now only one | |
69 | client at a time can attach to session. | |
70 | ||
71 | Attach to a session : | |
6e92aae6 JD |
72 | Now V can select and attach one or multiple session IDs, but first, it needs to |
73 | create a viewer_session. Creating a viewer session is done by sending the | |
74 | command LTTNG_VIEWER_CREATE_SESSION. In the future, this would be the place | |
75 | where we could specify options global to the viewer session. | |
76 | Once the session is created, the viewer can issue one or multiple | |
77 | VIEWER_ATTACH_SESSION commands with the session_id it wants. The "seek" | |
24bc0841 JD |
78 | parameter allows the viewer to attach to a session from its beginning (it will |
79 | receive all trace data still on the relayd) or from now (data will be available | |
6e92aae6 JD |
80 | to read starting at the next packet received on the relay). The viewer can |
81 | issue this command multiple times and at any moment in the process. | |
24bc0841 JD |
82 | R replies with a struct lttng_viewer_attach_session_response with a status and |
83 | the number of streams currently active in this session. Then, for each stream, | |
84 | it sends a struct lttng_viewer_stream. Just like with the session list, V must | |
85 | receive the first header and then all the stream structs. The ctf_trace_id | |
86 | parameter in the struct lttng_viewer_stream is particularly important since it | |
87 | allows the viewer to match all the streams belonging to the same CTF trace (so | |
88 | one metadata file and multiple stream files). If the stream is a metadata | |
89 | stream, metadata_flag will be set to 1. | |
6e92aae6 JD |
90 | The relay ensures that it sends only ready ctf traces, so once this command is |
91 | complete, V knows that it has all the streams ready to be processed. | |
92 | A quick note about the "sessions": from the relay perspective we see one | |
93 | session for each domain (kernel, ust32, ust64) of a session as created on the | |
94 | sessiond. For example, if the user creates a session and adds events in the | |
95 | kernel and UST and has only 64-bit applications running, the relay sees two | |
96 | sessions with the same name and hostname. So in order to display a session as | |
97 | seen by the user, the viewer has to attach to all the sessions with the same | |
98 | hostname and session name. There might be clashes if two servers on the network | |
99 | have the same hostname and create the same session name, but the relay cannot | |
100 | distinguish these cases, so it is currently a known limitation. | |
101 | During a session, new streams might get added (especially in per-pid tracing) | |
102 | so the viewer must be ready to add new streams while processing the trace. To | |
103 | inform V that new streams are available, R sets the | |
104 | LTTNG_VIEWER_FLAG_NEW_STREAM flag on LTTNG_VIEWER_GET_NEXT_INDEX and | |
105 | LTTNG_VIEWER_GET_PACKET replies. The viewer must then issue the | |
106 | LTTNG_VIEWER_GET_NEW_STREAMS command and receive all the streams, just like | |
107 | with the attach command. | |
24bc0841 JD |
108 | |
109 | #### below needs to be well written, but the essential is here ### | |
110 | ||
111 | Get metadata : | |
112 | A CTF trace cannot be read without the complete metadata. | |
113 | Send the command VIEWER_GET_METADATA and the struct lttng_viewer_get_metadata. | |
114 | ||
115 | Once we have all the metadata, we can start processing the trace. In order to | |
116 | do that, we work with the indexes. Whenever we need to read a new packet from a | |
117 | stream, we first ask for the next index for this stream and then ask for a | |
118 | trace packet at a certain offset and length. | |
119 | ||
120 | Get the next index : | |
121 | Command VIEWER_GET_NEXT_INDEX | |
122 | struct lttng_viewer_get_next_index | |
123 | Receive back a struct lttng_viewer_index | |
6e92aae6 JD |
124 | We might also receive flags : |
125 | - LTTNG_VIEWER_FLAG_NEW_METADATA the viewer must ask new metadata | |
126 | (LTTNG_VIEWER_GET_METADATA) | |
127 | - LTTNG_VIEWER_FLAG_NEW_STREAM the viewer must get the new streams | |
128 | (LTTNG_VIEWER_GET_NEW_STREAMS) | |
24bc0841 JD |
129 | |
130 | Get data packet : | |
131 | Command VIEWER_GET_PACKET | |
132 | struct lttng_viewer_get_packet | |
133 | Receive back a struct lttng_viewer_trace_packet | |
6e92aae6 JD |
134 | We might also receive a LTTNG_VIEWER_GET_PACKET_ERR and some flags : |
135 | - LTTNG_VIEWER_FLAG_NEW_METADATA the viewer must ask new metadata | |
136 | (LTTNG_VIEWER_GET_METADATA) | |
137 | - LTTNG_VIEWER_FLAG_NEW_STREAM the viewer must get the new streams | |
138 | (LTTNG_VIEWER_GET_NEW_STREAMS) | |
24bc0841 JD |
139 | |
140 | For the VIEWER_GET_NEXT_INDEX and VIEWER_GET_PACKET, the viewer must check the | |
141 | "flags" element of the struct it receives, because it contains important | |
142 | information such as the information that new metadata must be received before | |
143 | being able to receive and read the next packet. | |
144 | When new metadata is added during a session, the GET_NEXT_INDEX will succeed | |
145 | but it will have the flag LTTNG_VIEWER_FLAG_NEW_METADATA, but the | |
146 | GET_DATA_PACKET will fail with the same flag as long as the metadata is not | |
147 | downloaded. |