Feature-Specific Profiling

This package provides experimental support for profiling the costs of specific language and library features.

Unlike Racket’s regular profiler, which reports time spent in each function, the feature-specific profiler reports time spent in feature instances: a particular pattern matching form, a specific contract, etc. This pinpoints which uses of expensive features are the sources of performance issues and should be replaced by less expensive equivalents. Conversely, the feature-specific profiler also reports which feature instances are not problematic and should be left alone.

Libraries may supply additional plug-ins for the features they introduce. For example, the Marketplace library includes a process accounting plug-in.

1 Quick Start

To use the profiler, wrap the code you wish to profile with the feature-profile form. Then, execute the program with the feature-profile-compile-handler active.

Some features, such as contracts, can be profiled without using the compile handler. To get information about all features, the compile handler is required.

Here is an example feature profile:

"fizzbuzz.rkt"

#lang racket

(require feature-profile)

(define (divisible x n)

(= 0 (modulo x n)))

(define (fizzbuzz n)

(for ([i (range n)])

(cond [(divisible i 15) (printf "FizzBuzz\n")]

[(divisible i 5)  (printf "Buzz\n")]

[(divisible i 3)  (printf "Fizz\n")]

[else             (printf "~a\n" i)])))

(feature-profile

(parameterize ([current-output-port (open-output-nowhere)])

(fizzbuzz 10000000)))

2 Using the Profiler

For each profiled feature, reports how much time was spent in the feature overall, then breaks down that time by feature instance. Features are sorted in decreasing order of time, and only features for which time was observed are displayed.

The optional features argument contains the lists of features that should be observed by the feature profiler. It defaults to default-features.

2.1 Default features

3 Adding Profiling Support to Libraries

Not all expensive features come from the standard library; some are provided by third-party libraries. For this reason, feature-specific profiling support for third-party library features can be useful.

Adding feature-specific profiling for a library requires implementing a feature-specific plug-in and adding feature marks to the library’s implementation (see Feature Instrumentation).

3.1 Feature Instrumentation

Implementing feature-specific profiling support for a library requires instrumenting its implementation. Specifically, library code must arrange to have feature marks on the stack whenever feature code is running. This is usually a non-intrusive change, and does not change program behavior. Feature marks allow the profiler’s sampling thread to observe when programs are executing feature code.

Conceptually, feature marks are conditionally enabled continuation marks that map feature keys to payloads. Feature keys can be any value, but must uniquely identify features and must be consistent with with the feature’s plug-in (see Implementing Feature Plug-Ins).

Payloads can also be any value (except #f), but they should uniquely identify feature instances (e.g. a specific pattern matching form). The source location of feature instances is an example of a good payload. Payloads with additional information can be useful when implementing more sophisticated feature-specific analyses (see Implementing Feature-Specific Analyses).

To avoid attributing time to a feature when feature code transfers control to user code (e.g. when a feature calls a function that was provided by the user), you can install antimarks which are feature marks with the 'antimark symbol as payload. Antimarks are recognized specially by the profiling library, and any sample taken while an antimark is the most recent feature mark will not contribute time toward that feature.

Feature marks come in three flavors, each with different tradeoffs and appropriate for different use cases.

• Active marks correspond directly to continuation marks. Feature code can be instrumented with active marks by wrapping it in

  (with-continuation-mark feature-key payload body)

  Active marks are the simplest flavor of feature marks, and do not require recompiling programs (with feature-profile-compile-handler) before profiling. On the other hand, they impose a performance overhead even when not profiling, and must be used with caution.

• Syntactic latent marks are syntax properties attached to the syntax of feature code, that are turned into actual continuation marks by feature-profile-compile-handler or a similar compile handler that supports the key (see make-latent-mark-compile-handler).

  To instrument feature code with syntactic latent marks, syntax objects corresponding to feature code must be wrapped with

  (syntax-property stx feature-key payload)

  If the payload is #f, the compile handler will automatically use the syntax object’s source location as payload.

  Syntactic latent marks, unlike active marks, require recompilation but do not impose overhead unless activated by the compile handler (which is typically only done when profiling). Because of their reliance on syntax properties, they are only appropriate for features implemented as syntactic extensions.

• Functional latent marks are functions that are recognized specially and whose calls are wrapped in continuation marks by a latent mark compile handler. Functional latent marks make it possible to instrument features that are implemented as functions, while avoiding the overhead of active marks when not profiling.

  For more details on how to use functional latent marks, see make-latent-mark-compile-handler.

  Compile handlers automatically use source locations as payloads for functional latent marks.

The first argument is the list of feature keys used by the latent marks that should be activated. It should usually be an extension of default-syntactic-latent-mark-keys.

The second argument is a dictionary mapping the names of functions whose calls should be profiled to the key of the feature that the function corresponds to. It should usually be an extension of default-functional-latent-marks.

3.2 Implementing Feature Plug-Ins

At its core, a plug-in is a feature struct.

key is the continuation mark key used by the feature’s feature marks. It can be any value, but must be consistent with the key used by the feature’s instrumentation.

grouper is a function that should be used to group mark payloads that correspond to a single feature instance. The function takes a payload as argument, and returns a value that identifies the payload’s equivalence class. That is, all payloads that should be grouped together must return the same value. The grouping function only considers the payload of the most recent mark for a given feature.

A value of #f will result in the plug-in using the default grouping functions, which is usually what you want.

analysis is a function that performs feature-specific analysis to present profile results in a custom format. It is expected to produce its results as a side-effect. Writing analysis functions is covered in Implementing Feature-Specific Analyses.

A value of #f will result in the plug-in using the default analysis provided by the profiling library. This analysis groups costs by feature instance and prints them to standard output. It is usually best to use this option to start with, and eventually migrate to a more sophisticated analysis.

The most basic feature plug-in is a feature struct with a name and a key, and uses the default grouping and analysis. To include the new feature when profiling, pass the feature struct to feature-profile’s or feature-profile-thunk’s #:extra-features argument.

3.3 Implementing Feature-Specific Analyses

While the basic analysis provided by default by the profiling library (grouping costs by feature instance) is useful, instances of some features (such as contracts) carry enough information to make further analysis worthwhile. Further analysis can be used to produce precise reports tailored specifically to a given feature.

This section describes the API provided by the profiling library for building feature-specific analyses. This API is still experimental, and is likely to change in the future.

A feature-specific analysis is a function that takes a feature-report structure as an argument and emits profiling reports as a side effect. This function should be used as the analysis field of the relevant feature object.

The feature field contains the feature structure that corresponds to the feature being profiled and analyzed.

The core-samples field contains feature core samples, that is each sample is a list of all marks related to the feature of interest that are on the stack at the time the sample is taken. The most recent mark is at the front of the list. These core samples include antimarks.

The raw-samples field contains the samples collected by the regular Racket profiler during program execution, in the (intentionally) undocumented format used by the profiler. These samples can be passed to the regular profiler’s analyzer, and the result correlated with feature data.

The grouped-samples field contains a list of groups (lists) of samples, grouped using the feature’s grouping function. Each sample has the most recent payload as its car and a sample in the regular profiler’s format as its cdr.

The total-time field contains the total time (in milliseconds) observed by the sampler.

The feature-time field contains the time (in milliseconds) for which a feature mark was observed, as estimated by interpolating sample timestamps.

This library also provides helper functions for common analysis tasks.