tags/lttv-0.10.0-pre15-12082008/QUICKSTART

   1 Linux Trace Toolkit Quickstart
   2 ------------------------------
   3 Author : Mathieu Desnoyers, September 2005
   4 Last update : July 31, 2008
   5
   6
   7 This document is made of four parts : the first one explains how to install
   8 LTTng and LTTV from Debian and RPM binary packages, the second one explains how
   9 to install LTTng and LTTV from sources and the third one describes the steps
  10 to follow to trace a system and view it. The fourth and last part explains
  11 briefly how to add a new trace point to the kernel and to user space
  12 applications.
  13
  14 What you will typically want is to read sections 2 and 3 : install LTTng from
  15 sources and use it.
  16
  17 These operations are made for installing the LTTng 0.12 tracer on a linux 2.6.X
  18 kernel. You will also find instructions for installation of LTTV 0.8.x : the
  19 Linux Trace Toolkit Viewer.
  20
  21 To see the list of compatibilities between LTTng, ltt-control, LTTV and
  22 markers-userspace, please refer to :
  23 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
  24
  25
  26
  27 The following lttng patch is necessary to have the tracing hooks in the kernel.
  28 The following ltt-control module controls the tracing.
  29
  30 Required programs and libraries are assumed to be automatically installed in an
  31 installation with Debian or RPM packages. In the case of an installation from
  32 sources, the dependencies are listed.
  33
  34
  35 ** Current development status **
  36
  37 LTTng :
  38 supported architectures :
  39 Intel Pentium (UP/SMP) with TSC
  40 PowerPC 32 and 64 bits
  41 ARM
  42 x86_64
  43 C2 Microsystems (variant of MIPS)
  44
  45 LTTV :
  46 supported architectures :
  47 Intel i386 and better
  48 Intel 64 bits
  49 PowerPC 32 and 64 bits
  50
  51
  52 ***********************************************************
  53 ** Section 1 * Installation from Debian or RPM packages  **
  54 ***********************************************************
  55
  56 * Create custom LTTV Debian packages
  57
  58 Use : dpkg-buildpackage -rfakeroot
  59
  60 You should then have your LTTV .deb files created for your architecture.
  61
  62 * Create custom LTTng packages
  63
  64 For building LTTng Debian packages :
  65
  66 Get the build tree with patches applies as explained in section 2.
  67
  68 make menuconfig (or xconfig or config) (customize your configuration)
  69 make-kpkg kernel_image
  70
  71 You will then see your freshly created .deb in /usr/src. Install it with
  72 dpkg -i /usr/src/(image-name).deb
  73
  74 Then, follow the section "Editing the system wide configuration" in section 2.
  75
  76
  77 ***********************************************************
  78 ** Section 2 * Installation from sources                 **
  79 ***********************************************************
  80
  81 * Prerequisites
  82
  83 Tools needed to follow the package download steps :
  84
  85 o wget
  86 o bzip2
  87 o gzip
  88 o tar
  89
  90 You have to install the standard development libraries and programs necessary
  91 to compile a kernel :
  92
  93 (from Documentation/Changes in the Linux kernel tree)
  94 o  Gnu C                  2.95.3                  # gcc --version
  95 o  Gnu make               3.79.1                  # make --version
  96 o  binutils               2.12                    # ld -v
  97 o  util-linux             2.10o                   # fdformat --version
  98 o  module-init-tools      0.9.10                  # depmod -V
  99
 100 You might also want to have libncurses5 to have the text mode kernel
 101 configuration menu, but there are alternatives.
 102
 103 Prerequisites for LTTV 0.x.x installation are :
 104
 105 gcc 3.2 or better
 106 gtk 2.4 or better development libraries
 107   (Debian : libgtk2.0, libgtk2.0-dev)
 108   (Fedora : gtk2, gtk2-devel)
 109   note : For Fedora users : this might require at least core 3 from Fedora,
 110   or you might have to compile your own GTK2 library.
 111 glib 2.4 or better development libraries
 112   (Debian : libglib2.0-0, libglib2.0-dev)
 113   (Fedora : glib2, glib2-devel)
 114 libpopt development libraries
 115   (Debian : libpopt0, libpopt-dev)
 116   (Fedora : popt)
 117 libpango development libraries
 118   (Debian : libpango1.0, libpango1.0-dev)
 119   (Fedora : pango, pango-devel)
 120 libc6 development librairies
 121   (Debian : libc6, libc6-dev)
 122   (Fedora : glibc, glibc)
 123
 124
 125 * Getting the LTTng packages
 126
 127 su -
 128 mkdir /usr/src/lttng
 129 cd /usr/src/lttng
 130 (see http://ltt.polymtl.ca/lttng for package listing)
 131 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
 132 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
 133
 134
 135 * Getting LTTng kernel sources
 136
 137 su -
 138 cd /usr/src
 139 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
 140 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
 141 cd linux-2.6.X
 142 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
 143 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
 144   or use quilt
 145 cd ..
 146 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
 147
 148
 149 * Installing a LTTng kernel
 150
 151 su -
 152 cd /usr/src/linux-2.6.X-lttng-0.x.xx
 153 make menuconfig (or make xconfig or make config)
 154     Select the < Help > button if you are not familiar with kernel
 155     configuration.
 156     Items preceded by [*] means they has to be built into the kernel.
 157     Items preceded by [M] means they has to be built as modules.
 158     Items preceded by [ ] means they should be removed.
 159   go to the "General setup" section
 160     Select the following options :
 161     [*] Activate tracepoints
 162     [*] Activate markers
 163     [*] Activate userspace markers ABI
 164     <*> Compile generic tracing probes
 165     Linux Trace Toolkit  --->
 166     [LTTng fine-grained-timestamping]
 167     [*] Linux Trace Toolkit Instrumentation Support
 168     <M> or <*> Linux Trace Toolkit Relay+DebugFS Support
 169     <M> or <*> Linux Trace Toolkit Serializer
 170     <M> or <*> Linux Trace Toolkit Marker Control
 171     <M> or <*> Linux Trace Toolkit Tracer
 172       It makes no difference for the rest of the procedure whether the Tracer
 173       is compiled built-in or as a module.
 174     activate :
 175        [*] Align Linux Trace Toolkit Traces
 176        <M> Linux Trace Toolkit Netlink Controller
 177        <M> Linux Trace Toolkit State Dump
 178     your choice (see < Help >) :
 179        [ ] Write heartbeat event to shrink traces
 180        [ ] Support trace extraction from crash dump
 181   Select <Exit>
 182   Select <Exit>
 183   Select <Yes>
 184 make
 185 make modules_install
 186 (if necessary, create a initrd with mkinitrd or your preferate alternative)
 187 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
 188
 189 -- on X86, X86_64
 190 make install
 191 reboot
 192 Select the Linux 2.6.17-lttng-0.x.xx kernel in your boot loader.
 193
 194 -- on PowerPC
 195 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
 196 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
 197 cp .config /boot/config-2.6.X-lttng-0.x.xx
 198 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 199 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 200 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
 201 that comes first is the default kernel)
 202 ybin
 203 select the right entry at the yaboot prompt (see choices : tab, select : type
 204 the kernel name followed by enter)
 205 Select the Linux 2.6.17-lttng-0.x.xx kernel in your boot loader.
 206 --
 207
 208
 209
 210 * Editing the system wide configuration
 211
 212 You must activate debugfs and specify a mount point. This is typically done in
 213 fstab such that it happens at boot time.
 214
 215 If you have never used DebugFS before, these operation would do this for you :
 216
 217 mkdir /mnt/debugfs
 218 cp /etc/fstab /etc/fstab.lttng.bkp
 219 echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
 220
 221 then, rebooting or issuing the following command will activate debugfs :
 222
 223 mount /mnt/debugfs
 224
 225 You need to load the LTT modules to be able to control tracing from user
 226 space. This is done by issuing the following commands. Note however
 227 these commands load all LTT modules. Depending on what options you chose to
 228 compile statically, you may not need to issue all these commands.
 229
 230 modprobe ltt-control
 231 modprobe ltt-marker-control
 232 modprobe ltt-tracer
 233 modprobe ltt-serialize
 234 modprobe ltt-relay
 235 modprobe ipc-trace
 236 modprobe kernel-trace
 237 modprobe mm-trace
 238 modprobe net-trace
 239 modprobe fs-trace
 240 #if locking tracing is wanted, uncomment the following
 241 #modprobe lockdep-trace
 242
 243 If you want to have complete information about the kernel state (including all
 244 the process names), you need to load the ltt-statedump module. This is done by
 245 issuing the command :
 246
 247 modprobe ltt-statedump
 248
 249 You can automate at boot time loading the ltt-control module by :
 250
 251 cp /etc/modules /etc/modules.bkp
 252 echo ltt-control >> /etc/modules
 253 echo ltt-marker-control >> /etc/modules
 254 echo ltt-tracer >> /etc/modules
 255 echo ltt-serialize >> /etc/modules
 256 echo ltt-relay >> /etc/modules
 257 echo ipc-trace >> /etc/modules
 258 echo kernel-trace >> /etc/modules
 259 echo mm-trace >> /etc/modules
 260 echo net-trace >> /etc/modules
 261 echo fs-trace >> /etc/modules
 262 #if locking tracing is wanted, uncomment the following
 263 #echo lockdep-trace >> /etc/modules
 264
 265
 266 * Getting and installing the ltt-control package (on the traced machine)
 267 (note : the ltt-control package contains lttd and lttctl. Although it has the
 268 same name as the ltt-control kernel module, they are *not* the same thing.)
 269 su -
 270 cd /usr/src
 271 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
 272 gzip -cd ltt-control-0.x-xxxx2006.tar.gz | tar xvof -
 273 cd ltt-control-0.x-xxxx2006
 274 (refer to README to see the development libraries that must be installed on you
 275 system)
 276 ./configure
 277 make
 278 make install
 279
 280 * Getting and installing the markers-userspace package for user space tracing
 281 See http://ltt.polymtl.ca/packages/markers-userspace-0.5.tar.bz2 or more recent.
 282
 283
 284 * Getting and installing the LTTV package (on the visualisation machine, same or
 285   different from the visualisation machine)
 286
 287 su -
 288 cd /usr/src
 289 wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
 290 gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
 291 cd lttv-0.x.xx-xxxx2008
 292 (refer to README to see the development libraries that must be installed on your
 293 system)
 294 ./configure
 295 make
 296 make install
 297
 298
 299
 300
 301 ***********************************************************
 302 ** Section 3 * Using LTTng and LTTV                      **
 303 ***********************************************************
 304
 305 * IMPORTANT : Arm Linux Kernel Markers after each boot
 306
 307 ltt-armall
 308
 309 * Use graphical LTTV to control tracing and analyse traces
 310
 311 lttv-gui (or /usr/local/bin/lttv-gui)
 312   - Spot the "Tracing Control" icon : click on it
 313       (it's a traffic light icon)
 314     - enter the root password
 315     - click "start"
 316     - click "stop"
 317     - Yes
 318       * You should now see a trace
 319
 320 * Use text mode LTTng to control tracing
 321
 322 The tracing can be controlled from a terminal by using the lttctl command (as
 323 root).
 324
 325 Start tracing :
 326
 327 lttctl -n trace -d -l /mnt/debugfs/ltt -t /tmp/trace
 328
 329 Stop tracing and destroy trace channels :
 330
 331 lttctl -n trace -R
 332
 333 see lttctl --help for details.
 334
 335 (note : to see if the buffers has been filled, look at the dmesg output after
 336 lttctl -R or after stopping tracing from the GUI, it will show an event lost
 337 count. If it is the case, try using larger buffers. See lttctl --help to learn
 338 how.)
 339
 340 * Use text mode LTTV
 341
 342 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
 343 graphical plugins available.
 344
 345 For example, a simple trace dump in text format is available with :
 346
 347 lttv -m textDump -t /tmp/trace
 348
 349 see lttv -m textDump --help for detailed command line options of textDump.
 350
 351 It is, in the current state of the project, very useful to use "grep" on the
 352 text output to filter by specific event fields. You can later copy the timestamp
 353 of the events to the clipboard and paste them in the GUI by clicking on the
 354 bottom right label "Current time". Support for this type of filtering should
 355 be added to the filter module soon.
 356
 357 * Hybrid mode
 358
 359 Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used :
 360 hybrid. It can be especially useful when studying big workloads on a long period
 361 of time.
 362
 363 When using this mode, the most important, low rate control information will be
 364 recorded during all the trace by lttd (i.e. process creation/exit). The high
 365 rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a
 366 flight recorder buffer (now named flight-channelname_X).
 367
 368 The following lttctl commands take an hybrid trace :
 369
 370 Create trace channel, start lttd on normal channels, start tracing:
 371 lttctl -n tracename -d -l /mnt/debugfs/ltt -t /tmp/trace1 -m hybrid
 372
 373 Stop tracing, start lttd on flight recorder channels, destroy trace channels :
 374 lttctl -n tracename -f -l /mnt/debugfs/ltt -t /tmp/trace1 -m hybrid
 375
 376
 377 We will need to tweak what we consider "important" medium rate events. For
 378 instance, thread branding events are actually considered a "high rate" event
 379 when it should be considered "medium rate". The same should apply for the
 380 state dump process enumeration.
 381
 382 * Flight recorder mode
 383
 384 The flight recorder mode writes data into overwritten buffers for all channels,
 385 including control channels, except for the facilities tracefiles.
 386
 387 The following lttctl commands take a flight recorder trace :
 388
 389 lttctl -n trace -c -m flight
 390 lttd -n -d -t /tmp/trace -c /mnt/debugfs/ltt/trace
 391 lttctl -n trace -s
 392 .. do stuff
 393 lttctl -n trace -q
 394 lttd -f -d -t /tmp/trace -c /mnt/debugfs/ltt/trace
 395 lttctl -m trace -r
 396
 397
 398 **************************************************************
 399 ** Section 4 * Adding new instrumentations with the markers **
 400 **************************************************************
 401
 402 See Documentation/markers.txt and Documentation/tracepoints.txt in your kernel
 403 tree.
 404
 405 * Add new events to userspace programs with userspace markers
 406 http://ltt.polymtl.ca/packages/
 407
 408 Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It
 409 allows inserting markers in executables and libraries, currently only on x86_32
 410 and x86_64.
 411