Julien Desfossez [Fri, 26 Jan 2018 21:56:52 +0000 (16:56 -0500)]
Dedicated error message when relay does not support rotations
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 19 Jan 2018 16:14:35 +0000 (11:14 -0500)]
Fix: add missing includes for embedded help
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Philippe Proulx [Thu, 18 Jan 2018 22:12:02 +0000 (17:12 -0500)]
Document tracing session rotation features
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 10 Jan 2018 21:23:45 +0000 (16:23 -0500)]
Check for pending notification on notification channel activity
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 14:16:16 +0000 (10:16 -0400)]
Clarify error logging statement of rotation thread
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 14:14:17 +0000 (10:14 -0400)]
Fix: rotation state marked as completed before relayd has completed
The session rotation state is updated to COMPLETED before the
relay daemon has signaled that its rotation has been completed.
This causes users using the "rotation get_info" API to receive
this status before the session archive is readable on the
relay daemon's end.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 14:13:26 +0000 (10:13 -0400)]
Fix: cmd_rotate_set_schedule returns positive error codes
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 14:11:34 +0000 (10:11 -0400)]
Fix: unchecked return value of domain_mkdir()
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 14:07:32 +0000 (10:07 -0400)]
Add initial "no rotation" state to session rotation states
ltt_session structures are initialized in the
LTTNG_ROTATION_sTATE_COMPLETED state which is unexpected for
users of the get_info API.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 29 Mar 2018 20:43:14 +0000 (16:43 -0400)]
Fix: erroneous use of kernel consumer error codes
Errors related to the kernel consumer are returned in a code path
that is only used by the userspace tracer, probably as a result
of copy-pasting code.
This patch changes the codes to the corresponding CONSUMER32/64
ones and makes them negative to honor the convention indicated
in the function's header.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 29 Mar 2018 19:13:34 +0000 (15:13 -0400)]
Fix: unhandled prev_seq initial value
The previous sequence number of a stream is initialized to -1ULL
and comparing the current sequence number against it to perform
a rotation will yield unexepected results.
The assumption that the previous sequence number is less than
the current one is assert()'ed on elsewhere.
This case triggers whenever a rotation is performed before the
relay daemon has received a packet for a given stream.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 9 Feb 2018 19:53:32 +0000 (14:53 -0500)]
Size-based rotation
The user can now configure the desired size of each chunk, every time a
chunk is bigger than the specified size, a rotation is automatically
started. The size of a chunk is measured by polling from the monitoring
thread, so the accuracy depends on the monitoring sampling rate.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 10 Jan 2018 19:12:35 +0000 (14:12 -0500)]
Add lttng_notification_channel_has_pending_notification()
This new API allows notification channel users to check for
pending notifications without necessarily blocking until
a new notification is ready. Moreoever, the pending notification
is not consumed by this new API.
lttng_notification_channel_get_next_notification() must still
be called to consume the new notification.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 9 Jan 2018 22:00:41 +0000 (17:00 -0500)]
Fix: channel lock must be taken to check for pending notifications
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 9 Jan 2018 22:00:27 +0000 (17:00 -0500)]
Docs: typo in notification channel header
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 20 Dec 2017 19:52:33 +0000 (14:52 -0500)]
Fix: circular inclusion of lttng.h results in warning
The circular inclusion of lttng.h, which includes all
public headers, from condition.h results in the following
warning for users of the API:
warning: ‘struct lttng_evaluation’ declared inside
parameter list will not be visible outside of this definition
or declaration
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Fri, 15 Dec 2017 21:03:52 +0000 (16:03 -0500)]
Remove unneeded domain.h include
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 29 Nov 2017 16:21:53 +0000 (17:21 +0100)]
Docs: wrong enum value used in evaluation API description
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Fri, 9 Feb 2018 20:07:56 +0000 (15:07 -0500)]
Remove unneeded forward declaration in condition headers
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 11 Jan 2018 18:44:56 +0000 (13:44 -0500)]
Add the GMT offset in the rotated chunk path
The path of a rotated chunk is composed of the start and end timestamp
of the trace inside that chunk. In order to support distributed
environments, we now specify the GMT offset in this path as well. The
date is now formatted in ISO 8601. Here is an example:
~/lttng-traces/<session-name>/20180118T144610-0500-20180118T144611-0500-1
Chunk start: 2018-01-18 14:46:10
Chunk end: 2018-01-18 14:46:11
GMT offset: GMT-5 on both timestamps
Chunk ID: 1 (number of rotations that occured in this session so far)
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 21 Dec 2017 20:32:16 +0000 (15:32 -0500)]
Tests for the session rotation feature
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 21 Dec 2017 20:28:56 +0000 (15:28 -0500)]
Fix validate_trace_empty test check
Since the output of babeltrace was directly piped into wc, the return
code was never an error even if the trace was invalid. We now split the
commands in two parts: process the trace with babeltrace and check the
error code, and then count the number of lines.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 21 Dec 2017 19:57:49 +0000 (14:57 -0500)]
Example client to use the session rotation API
This client creates a session with all system calls enabled, offers
the option to rotate periodically the session (unlimited or only a
number of times), and call a script on each rotated chunk. The script
provided compresses the chunk and deletes the original.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 21 Dec 2017 19:32:32 +0000 (14:32 -0500)]
Save, restore and list the rotation parameters
Add the support to save, restore and list the automatic rotation
parameters.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 9 Feb 2018 19:50:28 +0000 (14:50 -0500)]
Session consumed size notification
Add the support for notifications about the total amount of trace data
consumed for a session. The user can register itself to be notified when
a session has consumed more than a threshold. This sums the data for all
channels in a session.
For the review: part of this code was written by Jérémie, but it was on
top of my development branch with major updates on my early work with
notifications, so I had to squash it because it made no sense to keep
Jérémie's code separate.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 21 Dec 2017 16:06:11 +0000 (11:06 -0500)]
Fix: previous channel total is not updated
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 21 Dec 2017 16:03:33 +0000 (11:03 -0500)]
Add likely/unlikely annotations on channel sample handling path
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 20 Dec 2017 21:19:57 +0000 (16:19 -0500)]
Separate session info from channel info in notification thread
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Wed, 20 Dec 2017 20:43:22 +0000 (15:43 -0500)]
Rotate timer
Allow the user to configure a timer to rotate a session periodically.
The user can configure this setting with the API or the new
enable-rotation/disable-rotation commands:
lttng enable-rotation --timer 10s
lttng disable-rotation --timer
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 3 Apr 2018 16:11:24 +0000 (12:11 -0400)]
Simplify lock handling in enqueue_timer_rotate_job()
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Simon Marchi [Wed, 17 Jun 2015 18:39:51 +0000 (14:39 -0400)]
Use utils_parse_time_suffix in create and enable-channel command
Signed-off-by: Simon Marchi <simon.marchi@polymtl.ca>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Simon Marchi [Wed, 17 Jun 2015 18:07:58 +0000 (14:07 -0400)]
Introduce utils_parse_time_suffix
This function is based on utils_parse_size_suffix, but is to parse
(relatively short) time suffixes. It returns the time in microseconds.
So far, it supports:
- u/us: microseconds, x1, same as no suffix
- m/ms: milliseconds, x1 000
- s: seconds, x1 000 000
For example:
- 32u becomes 32
- 32us becomes 32
- 32m becomes 32 000
- 32ms becomes 32 000
- 32s becomes 32 000 000
Signed-off-by: Simon Marchi <simon.marchi@polymtl.ca>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Tue, 27 Mar 2018 18:37:32 +0000 (14:37 -0400)]
Fix: use metadata key instead of fd for consumer rotation command
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Wed, 14 Mar 2018 21:35:05 +0000 (17:35 -0400)]
Fix: double similar condition
Based on the pattern of the function, threshold_bytes should be used
inside the "else if" condition.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Fri, 23 Mar 2018 20:08:53 +0000 (16:08 -0400)]
Fix: missing type definitions in mi-lttng-3.0.xsd
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Fri, 23 Mar 2018 19:58:13 +0000 (15:58 -0400)]
Fix: out of tree build fails on missing header
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 18 Dec 2017 21:51:41 +0000 (16:51 -0500)]
lttng rotate command
The command line and API interface to the lttng rotate command.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 18 Dec 2017 21:04:44 +0000 (16:04 -0500)]
Relay rotate pending command
When a session rotation completes and the session is configured to send
its traces to a relay, we have to poll the relay to know when all the
chunk's data are written on its disk. To do that, we define a timer in
the sessiond and arm it when the rotation is complete. When the rotation
is complete on the relay, we clear the "rotate_pending" flag in the
session and the client can access the chunk safely.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 18 Dec 2017 19:45:56 +0000 (14:45 -0500)]
Rotate command
This is the core of the session rotation command in the session daemon,
no client interface for now. For each channel in the session, we send a
request to the related consumer to rotate the channel and add that
channel key and domain in the channel_pending_rotate_ht HT. When the
consumer has finished the rotation of all the streams in the channel, it
sends back a notification. The rotation thread in the session daemon
looks up the channel information in the HT and finds the corresponding
session. When all channels of a session have finished, the rotation
thread asks the consumer to rename the chunk folder to append the
timestamp of the end of the rotation.
On the first rotation, we have an extra step to change the session
directory layout from "<session-name>/<domain>" to
"<session-name>/<session-start-time>-<rotate-end-time>-1/<domain>".
When the rotation starts, the new chunk folder is created immediately
in: "<session-name>/<previous-rotate-start-time>-2/<domain>" so we won't
have to move the domain folder(s) after the next rotate has finished,
just rename the chunk folder.
The "mkdir" and "rename" commands are all propagated to the relay if
needed, only the rotate_pending check on the relay is not part of this
patch.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 15 Dec 2017 21:49:42 +0000 (16:49 -0500)]
Sessiond timer thread
This new thread allows to handle timers in the session daemon. The
structure is the same as the consumer-timer thread. For now it does not
perform any action, but it will be used by the session rotation feature.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Thu, 15 Mar 2018 20:16:58 +0000 (16:16 -0400)]
Fix: ret is uninitialized on standard path
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 21:27:49 +0000 (16:27 -0500)]
Sessiond rotation thread
This thread is responsible to receive the notifications from the
consumers that a channel has finished its rotation and perform the
rename of the chunk ready to be processed by the client when all the
channels of a session have completed their rotation.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 19:29:52 +0000 (14:29 -0500)]
Consumer rotate a channel
This command is sent from the session daemon for each channel in a
session. When the consumer receives the command, it stores the position
at which we need to rotate the stream. If a stream is ready to be
rotated, we perform the rotation immediately.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 19:15:57 +0000 (14:15 -0500)]
Consumer perform the rotation when extracting a packet
When the consumer reads a subbuffer, it checks if the stream needs to be
rotated before or after writing the data. The post-rotation action must
take place after we have released the stream lock, so we need to add a
flag to the read_subbuffer functions to know if a rotation occurred
while the stream lock was held.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 18:38:20 +0000 (13:38 -0500)]
Consumer rotate stream
Perform the action of rotating a stream locally or send the command to
do it on the relayd. Rotating a stream file consists of:
- closing the current tracefile and index,
- opening a new tracefile and index in the new chunk folder,
- resetting the stream rotation flags,
- updating the counter of streams waiting for a rotation in a channel,
If the stream is a metadata stream, we also need to trigger the action
to re-dump the content of the metadata cache after the rotation has been
performed.
The caller of lttng_consumer_rotate_stream() always calls
consumer_post_rotation() after having released the stream lock to update
the counter of streams waiting for a rotation in a channel and notifying
the session daemon if this counter reaches 0.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 16:00:30 +0000 (11:00 -0500)]
Implement the RELAYD_ROTATE_PENDING relay daemon command
This command allows the sessiond to check if a rotation is complete
from the relayd point of view. There can be a significant delay
between the time the consumer has finished extracting the data from
the buffers and the time the relay has finished writing them on disk,
and we can only inform the user that the rotation is complete when all
the data is on disk. So the RELAYD_ROTATE_PENDING command is used to
poll the relayd after the consumer has finished extracting the data
until everything is on the relayd disk.
This command also takes care of streams that did not exist on the
consumer when the rotation started, or streams that appeared after the
last rotation started. The chunk_id field is used to distinguish those
cases.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 14 Dec 2017 15:05:19 +0000 (10:05 -0500)]
Implement the RELAYD_ROTATE_STREAM relay daemon command
Support for the RELAYD_ROTATE_STREAM command on the relay. This
command informs the relay that the current stream must rotate after it
has written the data and index for the net_seq_num passed. After each
data and index written on disk we check if it is time to rotate (in
case it was in flight when the rotate command was received). On the
other hand, if too much data has been written when we receive the
rotate command, we move the excess data to a new tracefile and
truncate the current one.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Wed, 13 Dec 2017 18:24:34 +0000 (13:24 -0500)]
Channel rotate pipe between sessiond and the consumers
This new pipe is used by the consumers to inform the session daemon (in
the rotation_thread) that it has finished the rotation of a channel. In
this patch, we only setup the pipe between the daemons, it is not yet in
use.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:58:51 +0000 (16:58 -0500)]
Support to dump the kernel metadata cache from the beginning
On demand, the consumer can ask for the kernel tracer to dump the
content of the metadata cache (depends on lttng-modules 2.11). This
allows to extract the exact same metadata (compared to regenerating it
which could change the epoch offset).
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:56:17 +0000 (16:56 -0500)]
Add ustctl_flush_buffer to the consumer API
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:50:23 +0000 (16:50 -0500)]
Common consumer functions to read current positions
Introduce lttng_consumer_sample_snapshot_positions and
lttng_consumer_get_consumed_snapshotconsumer_flush_buffer to take a
snapshot of the positions and read the consumed position.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:42:20 +0000 (16:42 -0500)]
Dedicated function to wakeup the consumer metadata pipe
Extract the logic to wakeup the consumer metadata pipe to a dedicated
function, it will be used after a rotation in UST.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:17:48 +0000 (16:17 -0500)]
Keep read-only copies of fields from the channel to the stream
In the consumer, we sometimes need to read the channel pathname or
tracefile_size from a stream, but we cannot always access those values
safely. So we now keep a copy of those values when we add or allocate
streams.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:13:19 +0000 (16:13 -0500)]
Cleanup: keep the number of pipes used by poll in a variable
The "2" hardcoded at multiple places in the consumer is prone to error
when adding new FDs. Keep it all in one place to make it easier next
time we modify it.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 21:11:09 +0000 (16:11 -0500)]
Fix: kernel snapshot handling of EAGAIN
kernctl_snapshot can return EAGAIN, this is not an error, it only means
there is no data to read.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 20:07:23 +0000 (15:07 -0500)]
Command to rename a folder
This new command allows the sessiond to ask the consumer or relay to
rename a folder. This will be useful for the session rotation to
rename a completed chunk.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 6 Mar 2018 22:07:46 +0000 (17:07 -0500)]
Fix: create_output_path() relayd util is not const-correct
Code using this utility assumes that the path name passed to
this function is not modified. Using 'const' enforces this
assumption.
Moreover, this change makes it easier to write const-correct
code in the relayd.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 6 Mar 2018 16:35:23 +0000 (11:35 -0500)]
Fix: relayd send_command() util not logging on failure
send_command() only logs if it succeeds in sending a command to
the relay daemon.
This commit makes the helper log _before_ sending the command
so that errors can be associated back to the command being sent.
Moreover, PERROR() is used to log errors returned by sendmsg().
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 6 Mar 2018 16:33:19 +0000 (11:33 -0500)]
Clean-up: relayd send_command() helper is not const-correct
The payload sent to the relayd as part of a command is not
expected to be modified when it is sent. This commit makes
the data parameter 'const' to enforce this assumption.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 6 Mar 2018 15:58:58 +0000 (10:58 -0500)]
Clean-up: remove unneeded cast
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 19:41:45 +0000 (14:41 -0500)]
Create the session and domain directories on start
When the session starts, we now create the <session-name> folder and the
domain folder(s) regardless of the activity or registered apps.
This will be useful for the session rotation feature to avoid dealing
with empty rotations. Also, it is more consistent for the user to have
an empty trace folder instead of no folder at all if a session is
completely inactive.
This also removes the directory creation made by the session daemon
which did not belong there.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 19:03:52 +0000 (14:03 -0500)]
Command to make a directory on the consumer or relay
This new command allows the session daemon to create a directory in the
session folder (local or remote), this gives more control over the
directory creation which is currently lazy (when the first stream is
created).
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Fri, 2 Mar 2018 15:41:34 +0000 (10:41 -0500)]
Use free running metadata channel key between sessiond and kernel consumer
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Thu, 1 Mar 2018 22:03:52 +0000 (17:03 -0500)]
Fix: leftover use of channel fd as identifier
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Mon, 5 Mar 2018 16:09:42 +0000 (11:09 -0500)]
Clean-up: use LTTNG_PATH_MAX rather than PATH_MAX
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 11 Dec 2017 19:16:06 +0000 (14:16 -0500)]
Keep the base directory of a relay session separate
For the session rotation feature, we need a way to modify the path of a
channel, so we need to keep the "<hostname>/<session-name>" information
separate in the consumer_net structure for future use.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Mon, 24 Jul 2017 20:01:30 +0000 (16:01 -0400)]
Change trace_path to session_root_path and chunk_path
Prepare for the trace rotation feature where we need to store the root
path of the session and create a subdirectory for each chunk of trace.
For now, the chunk_path is \0, so the behaviour does not change.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 1 Dec 2017 19:46:23 +0000 (14:46 -0500)]
Make kernel tracer version global to the session daemon
A version check is already performed during the initialization of
the kernel tracer. This patch makes the result available as a
global variable in main.c in order to check for support of
the kernel tracer commands required to support the rotation of
sessions.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 7 Nov 2017 23:01:42 +0000 (18:01 -0500)]
Clean-up: typo fixes in notification thread comments
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Tue, 7 Nov 2017 22:57:15 +0000 (17:57 -0500)]
Bump minor notification protocol version
LTTng 2.11 introduces a number of new condition types related
to the session rotation feature.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 2 Nov 2017 17:19:43 +0000 (13:19 -0400)]
Clean-up: hash table utils are unnecessarily non-const
Non-const internal utilities make it hard/impossible to write
const-correct code in the rest of the project.
This clean-up allows the notification subsystem to improve on
this code-quality front (see follow-up patches).
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 10 Nov 2017 21:22:13 +0000 (16:22 -0500)]
Fix: stream_per_chan_id_ht should allow duplicates
Fixes: #1134
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Fri, 10 Nov 2017 21:21:46 +0000 (16:21 -0500)]
Fix: lttng logs nanoseconds
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Thu, 30 Nov 2017 20:31:27 +0000 (15:31 -0500)]
Fix: use a free running channel key between sessiond and kernel consumer
We currently use the channel FD number opened by the session daemon to
reference a channel in the consumer. This can lead to races where the
session daemon destroys a channel and recreates one with the same FD
number before the consumer has time to cleanup everything on its side,
so all the commands in between that use that FD number has a key may end
up working on the wrong objects.
This fix introduces a free running counter as the channel key, so this
decouples the channel key in the consumer from the channel FD in the
session daemon. This fixes the race observed in stress tests.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Tue, 27 Feb 2018 20:32:00 +0000 (15:32 -0500)]
Tests: Fix: arm64 use sys_openat instead of sys_open
gen-syscall-events is failing to build on arm64 because of the following
error:
```
gen-syscall-events.c: In function ‘main’:
gen-syscall-events.c:35:15: error: ‘SYS_open’ undeclared (first use in
this function)
fd = syscall(SYS_open, "/proc/cpuinfo", O_RDONLY);
```
SYS_open is not available using the syscall(2) arm64 glibc function.
SYS_openat should be used instead.
Change test app and test cases to use SYS_openat.
Other projects have encountered the same issue:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=758521
https://bugs.launchpad.net/linaro-aarch64/+bug/
1100782
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Jonathan Rajotte [Wed, 21 Feb 2018 03:06:31 +0000 (22:06 -0500)]
Fix: set errno with value from SO_ERROR on error.
Debugging output at the same time in case of asynchronous handling.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Mon, 26 Feb 2018 21:03:02 +0000 (16:03 -0500)]
Fix: missing value handling for lttng_event_context_type
Handling of the following enum are added:
LTTNG_EVENT_CONTEXT_INTERRUPTIBLE
LTTNG_EVENT_CONTEXT_PREEMPTIBLE
LTTNG_EVENT_CONTEXT_NEED_RESCHEDULE
LTTNG_EVENT_CONTEXT_MIGRATABLE
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Wed, 21 Feb 2018 21:59:54 +0000 (16:59 -0500)]
Fix: Use SOL_SOCKET level for SO_KEEPALIVE on all platform
On Linux, COMPAT_SOCKET_LEVEL was set to SOL_TCP instead
of SOL_SOCKET, this resulted in execution of:
setsockopt(..., SOL_TCP, TCP_DEFER_ACCEPT, ...)
Instead of:
setsockopt(..., SOL_SOCKET, SO_KEEPALIVE, ...)
Hence, TCP keep alive was not enabled.
COMPAT_SOCKET_LEVEL is not needed. All platforms supported use the
SOL_SOCKET level for SO_KEEPALIVE.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Wed, 21 Feb 2018 21:57:36 +0000 (16:57 -0500)]
Fix: error out on leftover arguments
All the commands currently ignore leftover arguments, this can lead to
wrong usage of the commands and waste of time debugging. For example,
this command enables the vpid context on all channels instead of only on
the "mychan" channel:
$ lttng add-context -u mychan -t vpid
The correct usage is:
$ lttng add-context -u -c mychan -t vpid
We now output an error on leftover arguments:
$ lttng add-context -u mychan -t vpid
Error: Unknown argument: mychan
Error: Command error
Some commands accept one leftover argument (create, start, stop,
destroy), so we check if there are other leftovers:
$ lttng create mysess allo
Error: Unknown argument: allo
Error: Command error
Only the snapshot command is not handled since it has a second level of
command and does not consume the popt arguments.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Michael Jeanson [Tue, 20 Feb 2018 17:40:49 +0000 (12:40 -0500)]
Load preemptirq lttng-modules probe
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Julien Desfossez [Wed, 14 Feb 2018 21:32:45 +0000 (16:32 -0500)]
Fix: reply to version check even on protocol mismatch
In the relay, we currently put() the connection when we detect that
the major version from the session daemon is not compatible. We don't
reply to the version check message. The relay still holds a reference
on the connection so it is not closed and the session daemon is left
blocking in recvmsg.
The relay now replies to the version check so the session daemon knows
it is not compatible, and the relay completely closes the connection on
its side and removes the FD from the poll set.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Julien Desfossez [Wed, 14 Feb 2018 21:32:44 +0000 (16:32 -0500)]
Fix: error handling on relay version check
If a network error occurs while performing the version check between
the session daemon and the relay, we should not report to the user that
there is a version mismatch. LTTNG_ERR_RELAYD_VERSION_FAIL is now
returned by relayd_version_check() when the daemons are not compatible
while a negative value is returned if sendmsg()/recvmsg() fail on
network errors.
Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Mon, 5 Feb 2018 23:19:01 +0000 (18:19 -0500)]
Document add-context limitation for started session
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jonathan Rajotte [Mon, 5 Feb 2018 23:19:00 +0000 (18:19 -0500)]
Fix: add-context cannot be performed after a session has been started
The following scenario lead to a corrupted trace/metadata layout problem:
- lttng create test
- lttng enable-channel -u test
- lttng enable-event -u -a -c test
- lttng start
- ./instrumented-application
- lttng stop
- lttng add-context -u -t procname -c test
- lttng start
- ./instrumented-application
- lttng stop
- lttng view
Babeltrace 1.5.x will fail with:
[error] Unexpected end of packet. Either the trace data stream is corrupted or metadata description does not match data layout.
[error] Reading event failed.
Error printing trace.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Fri, 16 Feb 2018 19:48:49 +0000 (14:48 -0500)]
Fix: duplicated kernel consumer socket locking
Commit
9d1103e introduced a bug causing a deadlock on snapshot record.
Function consumer_snapshot_channel is called with the lock held causing
the pthread_mutex_lock call inside to hang forever.
Because consumer_snapshot_channel now acquires the lock before using the
socket. No need to acquire the lock before calling the function.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Wed, 19 Apr 2017 16:19:56 +0000 (12:19 -0400)]
Tests: Change syscall tests to use `gen-syscall-events` testapp
Use `gen-syscall-events` testapp in conjuction with the LTTng PID
tracker to improve the reliability of the syscall tracing tests by only
tracing the test app's activity.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Wed, 5 Apr 2017 15:53:51 +0000 (11:53 -0400)]
Tests: Add test app to generate syscalls
This app launches and waits for the creation of a file specified in the
arguments before executing syscalls. This can be used with the PID
tracker feature of LTTng to only trace the syscalls made by the test
process and thus makes the tests more reliable.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Thu, 8 Jun 2017 21:14:46 +0000 (17:14 -0400)]
Tests: Move script synchronization functions to utils library
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Wed, 13 Dec 2017 17:08:34 +0000 (12:08 -0500)]
Fix: remove unused event types in MI XML schema
KPROBE and KRETPROBE event types are never produced by the MI output,
PROBE and FUNCTION are rightfully used. Using KPROBE and KRETPROBE would
be exposing the inner workings of the kernel tracer that should be
abstracted to the user.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Francis Deslauriers [Wed, 17 Jan 2018 21:58:13 +0000 (16:58 -0500)]
Updating lttng-ust-ctl header file
This file is a near-complete duplicate of the include/lttng/ust-ctl.h
file in lttng-ust repo.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 15 Feb 2018 16:53:17 +0000 (11:53 -0500)]
Tests: cleanly exit from test apps on reception of SIGTERM
There is a known lttng-ust limitation that can cause a buffer
to become unreadable if an application is killed or preempted
indefinitely between the reserve and commit operations in
while trying to record to a subbuffer.
A buffer being unreadable will cause some tests to fail since
events that are expected to be visible in a given stream
may not be shown by the trace viewers as the consumer was
unable to "get" that subbuffer.
It was fairly easy to reproduce this failure scenario using
the test_ust_fast snapshot test, in the "post_mortem" case.
This test case performs the following sequence of operations:
* setup a tracing session in snapshot mode
* launch an app
* kill(1) it after one event is known to have been produced
* record a snapshot
* try to read the resulting snapshot
Adding logging allowed the confirmation that the "get"
operation was indeed failing on the subbuffer to which the
application had run. This resulted in an empty stream
(file size == 0) being produced by the snapshot record operation.
The test was then failing because babeltrace reported that no
events were contained in the resulting trace.
Since there are no concrete solution to this limitation yet,
the test suite must ensure that the applications exit cleanly
on reception of a signal.
This patch introduces a SIGTERM signal handler in the test
applications which sets a "should_quit" flag to 1 and is
tested between every iteration of their event production loop.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 22:44:05 +0000 (17:44 -0500)]
Document consumer socket locking assumptions
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 21:13:51 +0000 (16:13 -0500)]
Fix: consumer socket lock not held during snapshot record
This missing lock was identified while stress-testing the
snapshot tracing mode.
The "post_mortem" test case would sometimes hang on a
push_metadata() call waiting for a status reply from the
consumer daemon.
This test demonstrated a race that consists in killing an
application and taking a snapshot near-simultaneously.
This causes the app management thread to issue a "push metadata"
command to the consumerd while the lttng client is issuing
a snapshot record command.
Since the snapshot record does not acquire the consumer socket lock,
the "push metadata" and "snapshot" commands end-up mixed-up on
the socket which ultimately causes the "apps management" thread
to wait for a reply forever while holding the socket's lock.
This prevents the client, invoked by the test script, from
completing the "stop" operation on the session.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 21:05:18 +0000 (16:05 -0500)]
Fix: set_relayd_for_snapshot does not acquire the consumer socket lock
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 20:24:40 +0000 (15:24 -0500)]
Fix: send_channel_monitor_pipe does not take the consumer socket lock
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 21:04:33 +0000 (16:04 -0500)]
Document the locking assumptions of consumerd-relayd socket passing
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 21:14:21 +0000 (16:14 -0500)]
Assert that the consumer lock is held while sending FDs to consumerd
The consumer_data lock must be held during the communications
between the consumerd and sessiond.
The consumer_data lock is refered-to by each consumer_socket
instance; they point to their consumer's global data lock.
The lock can't be taken in consumer_send_msg() or consumer_send_fds()
since we want to protect a complete "transaction". Some commands
require both functions to be called and we want to hold the lock
over the duration of both calls to protect against other
threads initiating a communication between the two calls.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Wed, 14 Feb 2018 19:59:35 +0000 (14:59 -0500)]
Assert that the consumer socket lock is taken during communication
The consumer_data lock must be acquired during any communication
between the session and consumer daemons.
Stress tests have shown a number of deadlocks that have been
traced down to this type of errors.
Individual fixes follow this commit.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Fri, 9 Feb 2018 21:40:39 +0000 (16:40 -0500)]
Tests: refuse to run test suite if lttng processes are present
The test suite often fails because of unclean environments where
stale LTTng processes are left running. Since the test suite
assumes that no LTTng process (daemons and test applications) are
running, it makes sense to force the user to kill all those
processes before running the test suite.
The warn_processes.sh script now prints an error and returns 1
to indicate an early failure to the test harness.
It is possible to circumvent this check by invoking the tests
manually or by removing the "exit 1" from the warn_processes.sh
script if there is a need to have persistent processes across
the execution of the test suite.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Jérémie Galarneau [Thu, 8 Feb 2018 23:25:55 +0000 (18:25 -0500)]
Fix: metadata channel leak when using the snapshot tracing mode
While running stress tests involving the snapshot mode, it
becomes apparent that the lttng-consumerd leaks a number of file
descriptors.
To isolate the problem, the test was narrowed down to
* Create a session in snapshot mode
* Enable a userspace channel
* Enable all userspace events
* Start tracing
* Run a traced application
* Stop tracing
* Destroy session
This has shown that 5 file descriptors were leaked on each
iteration of the above.
As the comments in this change indicate, the ownership and
lifetime of metadata channels varies depending on the tracing
mode being used.
In non-snapshot tracing modes, metadata channels are owned by
their respective streams. On destruction of a metadata stream,
consumer_del_channel() is invoked since the stream releases its
ownership of the metadata channel.
However, this relationship between metadata streams and channels
does not exist in snapshot mode; streams are created and
destroyed on every snapshot record. Hence, the
LTTNG_CONSUMER_CLOSE_METADATA command must immediately clean the
metadata channel.
The channel's "monitor" flag is used to determine whether or not
the metadata channel is in "snapshot" mode or not.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
This page took 0.045011 seconds and 4 git commands to generate.