trunk/lttv/QUICKSTART

   1 Linux Trace Toolkit Quickstart
   2 ------------------------------
   3 Author : Mathieu Desnoyers, September 2005
   4 Last update : January 9th, 2009
   5
   6
   7 This document is made of four parts : the first one explains how to install
   8 LTTng and LTTV from Debian and RPM binary packages, the second one explains how
   9 to install LTTng and LTTV from sources and the third one describes the steps
  10 to follow to trace a system and view it. The fourth and last part explains
  11 briefly how to add a new trace point to the kernel and to user space
  12 applications.
  13
  14 What you will typically want is to read sections 2 and 3 : install LTTng from
  15 sources and use it.
  16
  17 These operations are made for installing the LTTng 0.67 tracer on a linux 2.6.X
  18 kernel. You will also find instructions for installation of LTTV 0.12.x : the
  19 Linux Trace Toolkit Viewer.
  20
  21 To see the list of compatibilities between LTTng, ltt-control, LTTV and
  22 markers-userspace, please refer to :
  23 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
  24
  25
  26
  27 The following lttng patch is necessary to have the tracing hooks in the kernel.
  28 The following ltt-control module controls the tracing.
  29
  30 Required programs and libraries are assumed to be automatically installed in an
  31 installation with Debian or RPM packages. In the case of an installation from
  32 sources, the dependencies are listed.
  33
  34
  35 ** Current development status **
  36
  37 LTTng :
  38 supported architectures :
  39 Intel Pentium (UP/SMP) with TSC
  40 PowerPC 32 and 64 bits
  41 ARM
  42 x86_64
  43 C2 Microsystems (variant of MIPS)
  44
  45 LTTV :
  46 supported architectures :
  47 Intel i386 and better
  48 Intel 64 bits
  49 PowerPC 32 and 64 bits
  50
  51
  52 ***********************************************************
  53 ** Section 1 * Installation from Debian or RPM packages  **
  54 ***********************************************************
  55
  56 * Create custom LTTV Debian packages
  57
  58 Use : dpkg-buildpackage -rfakeroot
  59
  60 You should then have your LTTV .deb files created for your architecture.
  61
  62 * Create custom LTTng packages
  63
  64 For building LTTng Debian packages :
  65
  66 Get the build tree with patches applies as explained in section 2.
  67
  68 make menuconfig (or xconfig or config) (customize your configuration)
  69 make-kpkg kernel_image
  70
  71 You will then see your freshly created .deb in /usr/src. Install it with
  72 dpkg -i /usr/src/(image-name).deb
  73
  74 Then, follow the section "Editing the system wide configuration" in section 2.
  75
  76
  77 ***********************************************************
  78 ** Section 2 * Installation from sources                 **
  79 ***********************************************************
  80
  81 * Prerequisites
  82
  83 Tools needed to follow the package download steps :
  84
  85 o wget
  86 o bzip2
  87 o gzip
  88 o tar
  89
  90 You have to install the standard development libraries and programs necessary
  91 to compile a kernel :
  92
  93 (from Documentation/Changes in the Linux kernel tree)
  94 o  Gnu C                  2.95.3                  # gcc --version
  95 o  Gnu make               3.79.1                  # make --version
  96 o  binutils               2.12                    # ld -v
  97 o  util-linux             2.10o                   # fdformat --version
  98 o  module-init-tools      0.9.10                  # depmod -V
  99
 100 You might also want to have libncurses5 to have the text mode kernel
 101 configuration menu, but there are alternatives.
 102
 103 Prerequisites for LTTV 0.x.x installation are :
 104
 105 gcc 3.2 or better
 106 gtk 2.4 or better development libraries
 107   (Debian : libgtk2.0, libgtk2.0-dev)
 108   (Fedora : gtk2, gtk2-devel)
 109   note : For Fedora users : this might require at least core 3 from Fedora,
 110   or you might have to compile your own GTK2 library.
 111 glib 2.4 or better development libraries
 112   (Debian : libglib2.0-0, libglib2.0-dev)
 113   (Fedora : glib2, glib2-devel)
 114 libpopt development libraries
 115   (Debian : libpopt0, libpopt-dev)
 116   (Fedora : popt)
 117 libpango development libraries
 118   (Debian : libpango1.0, libpango1.0-dev)
 119   (Fedora : pango, pango-devel)
 120 libc6 development librairies
 121   (Debian : libc6, libc6-dev)
 122   (Fedora : glibc, glibc)
 123
 124 * Reminder
 125
 126 See the list of compatibilities between LTTng, ltt-control, LTTV and
 127 markers-userspace at :
 128 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
 129
 130
 131
 132 * Getting the LTTng packages
 133
 134 su -
 135 mkdir /usr/src/lttng
 136 cd /usr/src/lttng
 137 (see http://ltt.polymtl.ca/lttng for package listing)
 138 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
 139 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
 140
 141
 142 * Getting LTTng kernel sources
 143
 144 su -
 145 cd /usr/src
 146 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
 147 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
 148 cd linux-2.6.X
 149 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
 150 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
 151   or use quilt
 152 cd ..
 153 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
 154
 155
 156 * Installing a LTTng kernel
 157
 158 su -
 159 cd /usr/src/linux-2.6.X-lttng-0.x.xx
 160 make menuconfig (or make xconfig or make config)
 161     Select the < Help > button if you are not familiar with kernel
 162     configuration.
 163     Items preceded by [*] means they has to be built into the kernel.
 164     Items preceded by [M] means they has to be built as modules.
 165     Items preceded by [ ] means they should be removed.
 166   go to the "General setup" section
 167     Select the following options :
 168     [*] Prompt for development and/or incomplete code/drivers
 169     [*] Activate markers
 170     [*] Activate userspace markers ABI (experimental, optional)
 171     [*] Immediate value optimization (optional)
 172     [*] Linux Trace Toolkit Next Generation (LTTng) --->
 173     <M> or <*> Compile lttng tracing probes
 174     <M> or <*> Linux Trace Toolkit High-speed Lockless Data Relay
 175     <M> or <*> Linux Trace Toolkit Lock-Protected Data Relay
 176     <M> or <*> Linux Trace Toolkit Serializer
 177     <M> or <*>   Linux Trace Toolkit Marker Control
 178     <M> or <*>     Linux Trace Toolkit Tracer
 179     [*] Align Linux Trace Toolkit Traces
 180     <M> or <*> Support logging events from userspace
 181     [*] Support trace extraction from crash dump
 182     <M> or <*> Linux Trace Toolkit Trace Controller
 183     <M> or <*> Linux Trace Toolkit State Dump
 184   Select <Exit>
 185   Select <Exit>
 186   Select <Yes>
 187 make
 188 make modules_install
 189 (if necessary, create a initrd with mkinitrd or your preferate alternative)
 190 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
 191
 192 -- on X86, X86_64
 193 make install
 194 reboot
 195 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 196
 197 -- on PowerPC
 198 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
 199 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
 200 cp .config /boot/config-2.6.X-lttng-0.x.xx
 201 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 202 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 203 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
 204 that comes first is the default kernel)
 205 ybin
 206 select the right entry at the yaboot prompt (see choices : tab, select : type
 207 the kernel name followed by enter)
 208 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 209 --
 210
 211
 212
 213 * Editing the system wide configuration
 214
 215 You must activate debugfs and specify a mount point. This is typically done in
 216 fstab such that it happens at boot time.
 217
 218 If you have never used DebugFS before, these operation would do this for you :
 219
 220 mkdir /mnt/debugfs
 221 cp /etc/fstab /etc/fstab.lttng.bkp
 222 echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
 223
 224 then, rebooting or issuing the following command will activate debugfs :
 225
 226 mount /mnt/debugfs
 227
 228 You need to load the LTT modules to be able to control tracing from user
 229 space. This is done by issuing the following commands. Note however
 230 these commands load all LTT modules. Depending on what options you chose to
 231 compile statically, you may not need to issue all these commands.
 232
 233 modprobe ltt-trace-control
 234 modprobe ltt-marker-control
 235 modprobe ltt-tracer
 236 modprobe ltt-serialize
 237 modprobe ltt-relay
 238 modprobe ipc-trace
 239 modprobe kernel-trace
 240 modprobe mm-trace
 241 modprobe net-trace
 242 modprobe fs-trace
 243 modprobe syscall-trace
 244 modprobe trap-trace
 245 #if locking tracing is wanted, uncomment the following
 246 #modprobe lockdep-trace
 247
 248 If you want to have complete information about the kernel state (including all
 249 the process names), you need to load the ltt-statedump module. This is done by
 250 issuing the command :
 251
 252 modprobe ltt-statedump
 253
 254 You can automate at boot time loading the ltt-control module by :
 255
 256 cp /etc/modules /etc/modules.bkp
 257 echo ltt-trace-control >> /etc/modules
 258 echo ltt-marker-control >> /etc/modules
 259 echo ltt-tracer >> /etc/modules
 260 echo ltt-serialize >> /etc/modules
 261 echo ltt-relay >> /etc/modules
 262 echo ipc-trace >> /etc/modules
 263 echo kernel-trace >> /etc/modules
 264 echo mm-trace >> /etc/modules
 265 echo net-trace >> /etc/modules
 266 echo fs-trace >> /etc/modules
 267 #if locking tracing is wanted, uncomment the following
 268 #echo lockdep-trace >> /etc/modules
 269
 270
 271 * Getting and installing the ltt-control package (on the traced machine)
 272 (note : the ltt-control package contains lttd and lttctl. Although it has the
 273 same name as the ltt-control kernel module, they are *not* the same thing.)
 274 su -
 275 cd /usr/src
 276 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
 277 gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
 278 cd ltt-control-0.x-xxxx2006
 279 (refer to README to see the development libraries that must be installed on you
 280 system)
 281 ./configure
 282 make
 283 make install
 284
 285 * Userspace tracing
 286
 287 Simple userspace tracing is available through
 288 echo "some text to record" > /mnt/debugfs/ltt/write_event
 289
 290 It will appear in the trace under event :
 291 channel : userspace
 292 event name : event
 293
 294 * Getting and installing the LTTV package (on the visualisation machine, same or
 295   different from the visualisation machine)
 296
 297 su -
 298 cd /usr/src
 299 wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
 300 gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
 301 cd lttv-0.x.xx-xxxx2008
 302 (refer to README to see the development libraries that must be installed on your
 303 system)
 304 ./configure
 305 make
 306 make install
 307
 308
 309 * Getting and installing the markers-userspace package for user space tracing
 310 (experimental)
 311 See http://ltt.polymtl.ca/packages/markers-userspace-0.5.tar.bz2 or more recent.
 312
 313
 314
 315 ***********************************************************
 316 ** Section 3 * Using LTTng and LTTV                      **
 317 ***********************************************************
 318
 319 * IMPORTANT : Arm Linux Kernel Markers after each boot
 320
 321 ltt-armall
 322
 323 * Use graphical LTTV to control tracing and analyse traces
 324
 325 lttv-gui (or /usr/local/bin/lttv-gui)
 326   - Spot the "Tracing Control" icon : click on it
 327       (it's a traffic light icon)
 328     - enter the root password
 329     - click "start"
 330     - click "stop"
 331     - Yes
 332       * You should now see a trace
 333
 334 * Use text mode LTTng to control tracing
 335
 336 The tracing can be controlled from a terminal by using the lttctl command (as
 337 root).
 338
 339 Start tracing :
 340
 341 lttctl -C -w /tmp/trace1 trace1
 342
 343 Stop tracing and destroy trace channels :
 344
 345 lttctl -D trace1
 346
 347 see lttctl --help for details.
 348
 349 (note : to see if the buffers has been filled, look at the dmesg output after
 350 lttctl -R or after stopping tracing from the GUI, it will show an event lost
 351 count. If it is the case, try using larger buffers. See lttctl --help to learn
 352 how. lttv now also shows event lost messages in the console when loading a trace
 353 with missing events or lost subbuffers.)
 354
 355 * Use text mode LTTV
 356
 357 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
 358 graphical plugins available.
 359
 360 For example, a simple trace dump in text format is available with :
 361
 362 lttv -m textDump -t /tmp/trace
 363
 364 see lttv -m textDump --help for detailed command line options of textDump.
 365
 366 It is, in the current state of the project, very useful to use "grep" on the
 367 text output to filter by specific event fields. You can later copy the timestamp
 368 of the events to the clipboard and paste them in the GUI by clicking on the
 369 bottom right label "Current time". Support for this type of filtering should
 370 be added to the filter module soon.
 371
 372 * Hybrid mode
 373
 374 Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used :
 375 hybrid. It can be especially useful when studying big workloads on a long period
 376 of time.
 377
 378 When using this mode, the most important, low rate control information will be
 379 recorded during all the trace by lttd (i.e. process creation/exit). The high
 380 rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a
 381 flight recorder buffer (now named flight-channelname_X).
 382
 383 The following lttctl commands take an hybrid trace :
 384
 385 Create trace channel, start lttd on normal channels, start tracing:
 386 lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
 387
 388 Stop tracing, start lttd on flight recorder channels, destroy trace channels :
 389 lttctl -D -w /tmp/trace2 trace2
 390
 391 Each "overwrite" channel is flight recorder channel.
 392
 393 * Flight recorder mode
 394
 395 The flight recorder mode writes data into overwritten buffers for all channels,
 396 including control channels, except for the facilities tracefiles. It consists of
 397 setting all channels to "overwrite".
 398
 399 The following lttctl commands take a flight recorder trace :
 400
 401 lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
 402 ...
 403 lttctl -D -w /tmp/trace3 trace3
 404
 405
 406 **************************************************************
 407 ** Section 4 * Adding new instrumentations with the markers **
 408 **************************************************************
 409
 410 See Documentation/markers.txt and Documentation/tracepoints.txt in your kernel
 411 tree.
 412
 413 * Add new events to userspace programs with userspace markers
 414 http://ltt.polymtl.ca/packages/
 415
 416 Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It
 417 allows inserting markers in executables and libraries, currently only on x86_32
 418 and x86_64.
 419