NAME

ttrace - Trace-based interpreter initialization

SYNOPSIS

package require Tcl 8.4

package require Thread ?2.6?

ttrace::eval arg ?arg ...?

ttrace::enable

ttrace::disable

ttrace::cleanup

ttrace::update ?epoch?

ttrace::getscript

ttrace::atenable cmd arglist body

ttrace::atdisable cmd arglist body

ttrace::addtrace cmd arglist body

ttrace::addscript name body

ttrace::addresolver cmd arglist body

ttrace::addcleanup body

ttrace::addentry cmd var val

ttrace::getentry cmd var

ttrace::getentries cmd ?pattern?

ttrace::delentry cmd

ttrace::preload cmd

DESCRIPTION

This package creates a framework for on-demand replication of the interpreter state accross threads in an multithreading application. It relies on the mechanics of Tcl command tracing and the Tcl unknown command and mechanism.

The package requires Tcl threading extension but can be alternatively used stand-alone within the AOLserver, a scalable webserver from America Online.

In a nutshell, a short sample illustrating the usage of the ttrace with the Tcl threading extension:

    % package require Ttrace
    2.6.5
    % set t1 [thread::create {package require Ttrace; thread::wait}]
    tid0x1802800
    % ttrace::eval {proc test args {return test-[thread::id]}}
    % thread::send $t1 test
    test-tid0x1802800
    % set t2 [thread::create {package require Ttrace; thread::wait}]
    tid0x1804000
    % thread::send $t2 test
    test-tid0x1804000

As seen from above, the ttrace::eval and ttrace::update commands are used to create a thread-wide definition of a simple Tcl procedure and replicate that definition to all, already existing or later created, threads.

USER COMMANDS

This section describes user-level commands. Those commands can be used by script writers to control the execution of the tracing framework.

ttrace::eval arg ?arg ...?: This command concatenates given arguments and evaluates the resulting Tcl command with trace framework enabled. If the command execution was ok, it takes necessary steps to automatically propagate the trace epoch change to all threads in the application. For AOLserver, only newly created threads actually receive the epoch change. For the Tcl threading extension, all threads created by the extension are automatically updated. If the command execution resulted in Tcl error, no state propagation takes place.
This is the most important user-level command of the package as it wraps most of the commands described below. This greatly simplifies things, because user need to learn just this (one) command in order to effectively use the package. Other commands, as desribed below, are included mostly for the sake of completeness.
ttrace::enable: Activates all registered callbacks in the framework and starts a new trace epoch. The trace epoch encapsulates all changes done to the interpreter during the time traces are activated.
ttrace::disable: Deactivates all registered callbacks in the framework and closes the current trace epoch.
ttrace::cleanup: Used to clean-up all on-demand loaded resources in the interpreter. It effectively brings Tcl interpreter to its pristine state.
ttrace::update ?epoch?: Used to refresh the state of the interpreter to match the optional trace ?epoch?. If the optional ?epoch? is not given, it takes the most recent trace epoch.
ttrace::getscript: Returns a synthetized Tcl script which may be sourced in any interpreter. This script sets the stage for the Tcl unknown command so it can load traced resources from the in-memory database. Normally, this command is automatically invoked by other higher-level commands like ttrace::eval and ttrace::update.

CALLBACK COMMANDS

A word upfront: the package already includes callbacks for tracing following Tcl commands: proc, namespace, variable, load, and rename. Additionaly, a set of callbacks for tracing resources (object, clasess) for the XOTcl v1.3.8+, an OO-extension to Tcl, is also provided. This gives a solid base for solving most of the real-life needs and serves as an example for people wanting to customize the package to cover their specific needs.

Below, you can find commands for registering callbacks in the framework and for writing callback scripts. These callbacks are invoked by the framework in order to gather interpreter state changes, build in-memory database, perform custom-cleanups and various other tasks.

ttrace::atenable cmd arglist body: Registers Tcl callback to be activated at ttrace::enable. Registered callbacks are activated on FIFO basis. The callback definition includes the name of the callback, cmd, a list of callback arguments, arglist and the body of the callback. Effectively, this actually resembles the call interface of the standard Tcl proc command.
ttrace::atdisable cmd arglist body: Registers Tcl callback to be activated at ttrace::disable. Registered callbacks are activated on FIFO basis. The callback definition includes the name of the callback, cmd, a list of callback arguments, arglist and the body of the callback. Effectively, this actually resembles the call interface of the standard Tcl proc command.
ttrace::addtrace cmd arglist body: Registers Tcl callback to be activated for tracing the Tcl cmd command. The callback definition includes the name of the Tcl command to trace, cmd, a list of callback arguments, arglist and the body of the callback. Effectively, this actually resembles the call interface of the standard Tcl proc command.
ttrace::addscript name body: Registers Tcl callback to be activated for building a Tcl script to be passed to other interpreters. This script is used to set the stage for the Tcl unknown command. Registered callbacks are activated on FIFO basis. The callback definition includes the name of the callback, name and the body of the callback.
ttrace::addresolver cmd arglist body: Registers Tcl callback to be activated by the overloaded Tcl unknown command. Registered callbacks are activated on FIFO basis. This callback is used to resolve the resource and load the resource in the current interpreter.
ttrace::addcleanup body: Registers Tcl callback to be activated by the trace::cleanup. Registered callbacks are activated on FIFO basis.
ttrace::addentry cmd var val: Adds one entry to the named in-memory database.
ttrace::getentry cmd var: Returns the value of the entry from the named in-memory database.
ttrace::getentries cmd ?pattern?: Returns names of all entries from the named in-memory database.
ttrace::delentry cmd: Deletes an entry from the named in-memory database.
ttrace::preload cmd: Registers the Tcl command to be loaded in the interpreter. Commands registered this way will always be the part of the interpreter and not be on-demand loaded by the Tcl unknown command.

DISCUSSION

Common introspective state-replication approaches use a custom Tcl script to introspect the running interpreter and synthesize another Tcl script to replicate this state in some other interpreter. This package, on the contrary, uses Tcl command traces. Command traces are registered on selected Tcl commands, like proc, namespace, load and other standard (and/or user-defined) Tcl commands. When activated, those traces build an in-memory database of created resources. This database is used as a resource repository for the (overloaded) Tcl unknown command which creates the requested resource in the interpreter on demand. This way, users can update just one interpreter (master) in one thread and replicate that interpreter state (or part of it) to other threads/interpreters in the process.

Immediate benefit of such approach is the much smaller memory footprint of the application and much faster thread creation. By not actually loading all necessary procedures (and other resources) in every thread at the thread initialization time, but by deffering this to the time the resource is actually referenced, significant improvements in both memory consumption and thread initialization time can be achieved. Some tests have shown that memory footprint of an multithreading Tcl application went down more than three times and thread startup time was reduced for about 50 times. Note that your mileage may vary. Other benefits include much finer control about what (and when) gets replicated from the master to other Tcl thread/interpreters.

KEYWORDS

command tracing, introspection