Update version to 2.0-pre3
[lttng-tools.git] / doc / session-daemon-model.txt
1 RFC - New processes model for UST and LTTng
2
3 Author: David Goulet <david.goulet@polymtl.ca>
4
5 Contributors:
6 * Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
7 * Yannick Brosseau <yannick.brosseau@polymtl.ca>
8 * Nils Carlson <nils.carlson@ericsson.com>
9 * Michel Dagenais <michel.dagenais@polymtl.ca>
10 * Stefan Hajnoczi <stefanha@gmail.com>
11
12 Version:
13 - v0.1: 17/01/2011
14 * Initial proposal
15
16 - v0.2: 19/01/2011
17 After multiple reply from all the contributors above, here is the list
18 of what has changed:
19 * Change/Add Terminology elements from the initial model
20 * New figures for four new scenarios
21 * Add inprocess library section
22 * LTTng kernel tracer support proposition
23 * More details for the Model and Components
24 * Improve the basic model. Quite different from the last one
25
26 - v0.3: 28/01/2011
27 In response from Michel Dagenais and Nils Carlson comments:
28 * Add scaling to reasons of this re-engineering
29 * Purpose of the session ID
30 * Explain why ltt-sessiond creates the tracing buffers
31 * ust-consumerd interaction schema
32 * Clarify inprocess library behavior
33
34 - v0.4: 01/02/2011
35 After Mathieu Desnoyers and Michel Dagenais comments:
36 * Add section Introduction
37 * Define the global and per-user ltt-sessiond
38 * Add details for ltt-sessiond in the inprocess lib section
39 * Session ID are now UUID
40 * Add buffer snapshot schema for ust-consumerd
41 * ltt-sessiond validate inprocess lib version
42 * ltt-sessiond socket validation by the inprocess lib.
43 * Add lttng definition
44 * Add consumer to the Model section
45
46 Terminology
47 -----------------
48
49 ltt-sessiond - Main daemon for trace session registry for UST and LTTng
50
51 ust-consumerd - Daemon that consume UST buffers for a specific application
52
53 ltt-consumerd - Daemon that consume LTTng buffers
54
55 tracing session - A trace linked to a set of specific tracepoints and to a set
56 of tracing buffers
57
58 tracing buffers - Buffers containing tracing data
59
60 tracing data - Data created by tracing an application
61
62 inprocess library - UST library linked with the application
63
64 shared memory - system V shared memory
65
66 application common named pipe - Global named pipe that triggers application
67 registration, on pipe event, to ltt-sessiond
68
69 lttng - New command line tool for LTTng and UST tracing control
70
71 Introduction
72 -----------------
73
74 This RFC propose a brand new UST and LTTng daemon model. This re-engineering
75 was mostly driven by the need of:
76
77 * Better security in terms of access rights on tracing data
78 * Manage tracing session
79 * Scaling in terms of thread/processes needed to perform tracing
80 * LTTng and UST integration in terms of merging traces and session control
81 * Networking such as streaming and remote control over different traces
82
83 The new model follows the basic principles of having a session registry
84 (ltt-sessiond) and consumers for each tracing session (ust-consumerd and
85 ltt-consumerd).
86
87 With this proposal, LTTng and UST will share the same tracing session, be
88 managed by the same tool and bring a complete integration between these two
89 powerful tools.
90
91 NOTE: This proposal does NOT makes UST dependent on LTTng and vice versa.
92
93 Model
94 -----------------
95
96 A global and/or per-user registry keeps track of all tracing sessions. Any user
97 that wants to manage either a kernel trace using LTTng or an application trace
98 with UST must interact with that registry for any possible actions.
99
100 The model address multiple tracing use cases based on the fact that we
101 introduce a tracing Unix group (tracing group). Only users in that group or
102 root can use the global registry. Other users will create a local registry
103 (per-user registry) that will be completely independent from the global one.
104
105 Two cases:
106
107 1) Users in the tracing group, it's tracing session can consume all tracing
108 buffers from all applications and the kernel.
109
110 2) Users NOT in the tracing group, it's tracing session can only consume
111 data from its own applications' buffers hence tracing his applications.
112
113 A session stored by the registry consist of:
114
115 * Session name (given by the user or automatically assigned)
116 * List of traces (LTTng or UST)
117 * Tracepoints/markers associated to a trace of that session
118 * UUID
119 * Associated user (UID)
120
121 Then, consumers are used to extract data from tracing buffers. These consumers
122 are daemon consuming either UST or/and LTTng buffers. For a single session,
123 only one UST consumer and one LTTng consumer is necessary. The daemon CAN
124 handle multiple tracing buffers for network streaming by example or for quick
125 snapshot. These consumers are told by the inprocess library or the kernel to
126 start getting out data on disk or network.
127
128 For the next subsections, every components of this new proposal is explained
129 from the global and per-user registry perspective.
130
131 LTT-SESSIOND:
132
133 The ltt-sessiond daemon acts as a session registry i.e. by keeping reference to
134 all active session and, by active, it means a session in any state other than
135 destroyed. Each entity we are keeping track of, here session, will have a
136 universal unique identifier (UUID) assigned to it. The purpose of this UUID is
137 to track a session in order to apply any kind of actions (Ex: Attach, Destroy).
138 A human readable version SHOULD be consider in order to facilitate the session
139 identification when listed by lttng.
140
141 The daemon creates two local Unix sockets (AF_UNIX). The first one is for what
142 we call client communication i.e. interaction with lttng (or any other
143 compatible tools). That socket is set with the ltt-sessiond credentials with
144 read-write mode for both user and group. The second one is a global socket for
145 application registration for the UST case (see inprocess lib subsection below).
146
147 This daemon is also responsible for tracing buffers creation. Two main reasons
148 motivate this design:
149
150 * The ltt-sessiond needs to keep track of all the shared memory segments in
151 order to be able to give reference to any other possible consumer.
152
153 * For the case of sharing tracing buffers between all userspace
154 applications, having the registry allocating them will allow that but, if
155 the inprocess library was allocating them, we will need to redesign the
156 whole model.
157
158 For all tracing actions either to interact with a session or a specific trace,
159 the lttng client MUST go through ltt-sessiond. The daemon will take care of
160 routing the command to the write inprocess library or the kernel.
161
162 Global registry:
163
164 A global registry SHOULD be started, idealy at boot, with credentials UID root
165 and GID of the tracing group. Only user within the tracing group will be able
166 to interact with that registry. All applications will try to register to that
167 registry using the global socket (second one discuss above).
168
169 Per-user registry:
170
171 This type of registry address two use cases. The first one is when a session
172 creation is requested from lttng but no global ltt-sessiond exist. So, a
173 ltt-sessiond will be spawned in order to manage the tracing of that user. The
174 second use case is when a user is not in the tracing group thus he cannot
175 communication with the global registry.
176
177 However, care MUST be put in order to manage the socket's daemon. They are not
178 global anymore so they should be created in the home directory of the user
179 requesting tracing.
180
181 In both cases, for global and per-user registry, all applications MUST try to
182 register to both ltt-sessiond. (see inprocess library subsection for details)
183
184 The trace roles of ltt-sessiond:
185
186 Trace interaction - Create, Destroy, Pause, Stop, Start, Set options
187
188 Registry - keep track of trace's information:
189 * shared memory location (only the keyid)
190 * application PID (UST)
191 * type (kernel or UST)
192 * session name
193 * UID
194
195 Buffers creation - creates shared memory for the tracing buffers.
196
197 UST-CONSUMERD:
198
199 The purpose of this daemon is to consume the UST trace buffers for only a
200 specific session. The session MAY have several traces for example two different
201 applications. The client tool, lttng has to create the ust-consumerd if NONE
202 is available for that session. It is very important to understand that for a
203 tracing session, there is only one ust-consumerd for all the traced
204 applications.
205
206 This daemon basically empty the tracing buffers when asked for and writes that
207 data to disk for future analysis using LTTv or/and TMF (Tracing Monitoring
208 Frameworks). The inprocess library is the one that tells the ust-consumerd
209 daemon that the buffers are ready for consumption.
210
211 Here is a flow of action to illustrate the ust-consumerd life span:
212
213 1)
214 +-----------+ ops +--------------+
215 | lttng A |<---------->| ltt-sessiond |
216 +-----------+ +--------------+
217
218 lttng ask for tracing an application using the PID and the session UUID. The
219 shared memory reference is given to lttng and the ust-consumerd communication
220 socket if ust-consumerd already exist.
221
222 2a) If ust-consumerd EXIST
223
224 +-----------+
225 | lttng A |
226 +-----------+
227 | mem ref.
228 | +---------------+ read +------------+
229 +-->| ust-consumerd |--------->| shared mem |
230 +---------------+ +------------+
231
232 In that case, lttng only ask ust-consumerd to consume the buffers using
233 the reference it previously got from ltt-sessiond.
234
235 2b) If ust-consumerd DOES NOT EXIST
236
237 +-----------+ +--------------+
238 | lttng A | +---->| ltt-sessiond |
239 +-----------+ | +--------------+
240 | ID |
241 | mem ref. | register
242 | +---------------+
243 +-->| ust-consumerd |
244 +---------------+
245
246 lttng spawns the ust-consumerd for the session using the session UUID in
247 order for the daemon to register as a consumer to ltt-sessiond for that
248 session.
249
250 Quick buffer snapshot:
251
252 1) Here, lttng will request a buffer snapshot for an already running session.
253
254 +-----------+ +--------------+
255 | lttng A |-------- ops ------->| ltt-sessiond |
256 +-----------+ +--------------+
257 | | command
258 | +-----------------+ +-------+<--+
259 | | ust-consumerd 1 |<----| app_1 |-+
260 | +-----------------+ +-------+ | write
261 | 1 | v
262 | | +-------------+
263 | +--- read ----->| shared mem. |
264 | +-------------+
265 | ^
266 | +-----------------+ |
267 +->| ust-consumerd 2 |----------+
268 +-----------------+ snapshot
269 | write
270 |
271 +---> disk/network
272
273 The first ust-consumerd (1) was already consuming buffers for the current
274 session. So, lttng ask for a live snapshot. A new ust-consumerd (2) is
275 spawned, snapshot the buffers using the shared memory reference from
276 ltt-sessiond, writes date to disk and die after all.
277
278 On the security side, the ust-consumerd gets UID/GID from the lttng
279 credentials since it was spawned by lttng and so the files containing the
280 tracing data will also be set to UID/GID of the lttng client. No setuid or
281 setgid is used, we only use the credentials of the user.
282
283 The roles of ust-consumerd:
284
285 Register to ltt-sessiond - Using a session UUID and credentials (UID/GID)
286
287 Consume buffers - Write data to a file descriptor (on disk, network, ...)
288
289 Buffer consumption is triggered by the inprocess library which tells
290 ust-consumerd when to consume.
291
292 LTT-CONSUMERD:
293
294 The purpose of this daemon is to consume the LTTng trace buffers for only a
295 specific session.
296
297 For that kernel consumer, ltt-sessiond will pass different anonymous file
298 descriptors to the ltt-consumerd using a Unix socket. From these file
299 desriptors, it will be able to get the data from a special function export by
300 the LTTng kernel.
301
302 ltt-consumerd will be managed by the exact same way as ust-consumerd. However,
303 in order to trace the kernel, you are either root (UID=0) or in the tracing
304 group.
305
306 The roles of ltt-consumerd:
307
308 Register to ltt-sessiond - Using a session UUID and credentials (UID/GID)
309
310 Consume buffers - Write data to a file descriptor (on disk, network, ...)
311
312 Kernel triggers ltt-consumerd for buffer consumption.
313
314 UST INPROCESS LIBRARY:
315
316 When the application starts, this library will check for the global named pipe
317 of ltt-sessiond. If present, it MUST validate that root is the owner. This
318 check is very important to prevent ltt-sessiond spoofing. If the pipe is root,
319 we are certain that it's the privileged user that operates tracing. Then, using
320 it's UID, the application will try to register to the per-user ltt-sessiond
321 again verifying before the owner ship of the named pipe that should match the
322 UID.
323
324 Before registration, the inprocess library MUST validate with the ltt-sessiond
325 the library version for compatibility reason. This is mechanism is useful for
326 library compatibility but also to see if ltt-sessiond socket is valid (means
327 that an actual ltt-sessiond is listening on the other side). Having no response
328 for over 10 seconds, the application will cut communication on that socket and
329 fallback to the application common named pipe (explain below).
330
331 If the socket is valid, it will register as a traceable application using the
332 apps credentials and will open a local Unix socket, passed to ltt-sessiond, in
333 order to receive an eventual shared memory reference. It will then wait on it
334 if any other command are given by the lttng client. This socket becomes the
335 only channel of communication between the registry and the application.
336
337 If no ltt-sessiond is present at registration, the application tries to open
338 the application common named pipe or create it if it does not exist and wait on
339 it (using poll or epoll Linux API). Having any type of event on that pipe, the
340 inprocess library will then try to register to the global and per-user
341 ltt-sessiond. If it fails again, it goes back again to wait on that pipe.
342
343 SHARED MEMORY
344
345 For UST, this is the memory area where the tracing buffers will be held and
346 given access in read-write mode for the inprocess library of the application.
347
348 On the LTTng side (for ltt-consumerd), these buffers are in the kernel space
349 and given access by opening a file in the debugfs file system. With an
350 anonymous file desriptor, this consumer will be able to extract the data.
351
352 This memory is ONLY used for the tracing data. No communication between
353 components is done using that memory.
354
355 A shared memory segment for tracing MUST be set with the tracing group GID for
356 the UST buffers. This is the job of ltt-sessiond.
357
358 PREREQUISITES:
359
360 The global ltt-sessiond daemon MUST always be running as "root" or an
361 equivalent user having the same privilege as root (UID = 0).
362
363 The ltt-sessiond daemon SHOULD be up and running at all time in order to trace
364 a tracable application.
365
366 The new lttng library API MUST be used to interact with the
367 ltt-sessiond registry daemon for every trace action needed by the user.
368
369 A tracing group MUST be created. Whoever is in that group is able to access the
370 tracing data of any buffers and is able to trace any application or the kernel.
371
372 WARNING: The tracing group name COULD interfere with other already existing
373 groups. Care should be put at install time for that (from source and packages)
374
375 The next section illustrates different use cases using that new model.
376
377 Use Cases
378 -----------------
379
380 Each case considers these :
381
382 * user A - UID: A; GID: A, tracing
383 * user B - UID: B; GID: B, tracing
384
385 Scenario 1 - Single user tracing app_1
386 ------
387
388 This first scenario shows how user A will start a trace for application app_1
389 that is not running.
390
391 1) lttng ask ltt-sessiond for a new session through a Unix socket. If
392 allowed, ltt-sessiond returns a session UUID to the client.
393 (Ex: ops --> new session)
394
395 +-----------+ ops +--------------+
396 | lttng A |<---------->| ltt-sessiond |
397 +-----------+ +--------------+
398
399 2) The app_1 is spawned by lttng having the user A credentials. Then, app_1
400 automatically register to ltt-sessiond has a "tracable apps" through the global
401 named pipe of ltt-sessiond using the UID/GID and session UUID.
402
403 The shared memory is created with the app_1 UID (rw-) and tracing group GID
404 (r--) and a reference is given back to app_1
405
406 +-----------+ +--------------+
407 | lttng A | | ltt-sessiond |
408 +-----------+ +--------------+
409 | ^ |
410 | +-------+ | | +-------------+
411 +-->| app_1 |<--------+ +-->| shared mem. |
412 +-------+ +-------------+
413
414 3) app_1 connect to the shared memory and ust-consumerd is spawned with the
415 session UUID and lttng credentials (user A). It then register to ltt-sessiond
416 for a valid session to consume using the previous session UUID and credentials.
417
418 +-----------+ +--------------+
419 | lttng A | +-->| ltt-sessiond |----------+
420 +-----------+ | +--------------+ |
421 | | |
422 | +---------------+ read | commands
423 +-->| ust-consumerd |---------+ | and
424 +---------------+ v | options
425 ^ | +-------------+ |
426 | v +------>| shared mem. | |
427 +-------+ | +-------------+ |
428 | app_1 |-------- |
429 +-------+ write |
430 ^ |
431 +---------------------------------------
432
433 Scenario 2 - Single user tracing already running app_1
434 ------
435
436 1) lttng ask ltt-sessiond for a new session through a Unix socket. If allowed
437 (able to write on socket), ltt-sessiond returns a session UUID to the client.
438
439 +-----------+ ops +--------------+
440 | lttng A |<---------->| ltt-sessiond |
441 +-----------+ +--------------+
442 ^
443 +-------+ read |
444 | app_1 |----------+
445 +-------+
446
447 NOTE: At this stage, since app_1 is already running, the registration of app_1
448 to ltt-sessiond has already been done. However, the shared memory segment is
449 not allocated yet until a trace session is initiated. Having no shared memory,
450 the inprocess library of app_1 will wait on the local Unix socket connected to
451 ltt-sessiond for the reference.
452
453 +-----------+ +--------------+
454 | lttng A | | ltt-sessiond |
455 +-----------+ +--------------+
456 ^ |
457 +-------+ | | +-------------+
458 | app_1 |<--------+ +-->| shared mem. |
459 +-------+ commands +-------------+
460 | ^
461 +---------- write ----------+
462
463 2) lttng spawns a ust-consumerd for the session. We get the same figure as
464 step 3 in the first scenario.
465
466 There is a small difference though. The application MAY NOT be using the same
467 credentials as user A (lttng). However, the shared memory is always GID of
468 the tracing group. So, in order for user A to trace app_1, is MUST be in the
469 tracing group otherwise, if the application is not set with the user
470 credentials, user A will not be able to trace app_1
471
472 Scenario 3 - Multiple users tracing the same running application
473 ------
474
475 1) Session are created for the two users. Using the same exact mechanism as
476 before, the shared memory and consumers are created. Two users, two sessions,
477 two consumers and two shared memories for the same application.
478
479 +-----------+ +--------------+
480 | lttng A |-------- ops ------->| ltt-sessiond |
481 +-----------+ ^ +--------------+
482 | ^ commands
483 +-----------+ | +-------+<--+
484 | lttng B |------+ +--->| app_1 |------- write -----+
485 +-----------+ | +-------+ |
486 | |
487 +-----------------+ | +-------------+ |
488 | ust-consumerd A |--O--- read ----->| shared mem. |<-+
489 +-----------------+ | +-------------+ |
490 | |
491 +-----------------+ v +-------------+ |
492 | ust-consumerd B |--+--- read ----->| shared mem. |<-+
493 +-----------------+ +-------------+
494
495 ust-consumerd A - UID: user A (rw-), GID: tracing (r--)
496 ust-consumerd B - UID: user B (rw-), GID: tracing (r--)
497
498 Scenario 4 - User not in the tracing group
499 ------
500
501 For this particular case, it's all goes back to the first scenario. The user
502 MUST start the application using his credentials. The session will be created
503 by the per-user ltt-sessiond but he will not be able to trace anything that the
504 user does not owned.
This page took 0.061424 seconds and 4 git commands to generate.