Home > Solutions > Survey Builder Pro 2008 Tool Suite > Survey Builder Pro 2008: Metric Builder

Survey Builder Pro 2008: Metric Builder

Lock Media's Survey Builder Pro Metric Builder allows the behavioral researcher to fulfill the fundamental activities of defining, measuring, adapting, and validating measures to address specific research questions. With Metric Builder you can create and carry out measurements coupled with the analytic tools for interpreting survey assessment results in the light of previous or comparable survey assessments. This includes the capability to differentiate between different survey instruments and methods while maintaining a central focus on your core metrics.

Metrics are at the Heart of the Survey Builder Pro 2008 Tool Suite

Metrics are usually specialized by the subject area, in which case they are valid within a certain industry or functional domain and can be benchmarked over periodic intervals. Survey questions are created as the instrument that researches use to apply a value to the metric and create a baseline for further benchmarking. Metric Builders ability to tie a survey question to the underling metric is what truly makes this a powerfully integrated set of software tools.

Defining Metrics - Overview:

A metric is a verifiable measure, stated in either quantitative or qualitative terms and defined with respect to a reference point. Performance metrics quantify the units performance. Project metrics tell you whether the project is meeting its goals. Business metrics define the business' progress in measurable terms. Metrics, in the context of a behavioral survey instrument, represent a system of parameters or ways of quantitative and periodic assessment of a thought process or belief that is to be measured.

Most quantitative comparisons of interest in terms of behavioral research are based on numeric values obtained from multi-item surveys, also known as multivariate research. In order for these comparisons to be of value to the researcher, a meaningful survey instrument must be utilized that can be tied to specific metrics, measured, and benchmarked over multiple data sets.

Metrics provide researchers with three basic functions:

1. Control: Metrics enable researchers to evaluate and control the measurement of behavioral outcomes in a quantifiable method.
   
   
2. Communication: Metrics communicate behavioral outcomes not only to internal researchers for purposes of control, but to external stakeholders for other purposes as well (e.g., Wall Street, the EPA or to a bank). Many times stakeholders and users of metrics do not understand the workings and processes of a firm or operation, nor do they need to. Well-designed and communicated metrics provide the user a sense of knowing what needs to be done without necessarily requiring him/her to understand the intricacies of related processes. Poorly developed or implemented metrics can lead to frustration, conflict, and confusion.
   
   
3. Improvement: Metrics identify gaps (between performance and expectation) that ideally point the way for intervention and improvement. The size of the gap and the direction of the gap (positive or negative) provide information and feedback that can be used to identify productive process adjustments or other actions.

The Typology of Metrics

Metrics are the core component of the Survey Builder Pro solution because they allow the researcher to make comparisons between survey data sets that have asked questions differently. The following table shows the elements of a metric that can be defined within the Metric Builder Tool:

Metrics can be readily classified according to two primary attributes: metrics focus and metrics tense. Metrics focus pertains to the resource that is the focus of the metric. Metrics tense, refers the how the metrics are intended to be used. Metrics can be used to both to judge outcome performance and to predict future performance. An outcome-oriented use of a metric implicitly assumes that the problems and lessons uncovered from a study of past outcomes can be applied to current situations.

Metrics used in behavioral research and deployed within various survey instruments are for performance or behavior value modeling and are typically tracked under a classification hierarchy. For example, metrics can be used to better understand how a company is performing compared to other companies within its own industry utilizing specific metric Benchmarks.

Metric Catagory - Classification Levels

Specifically, Metric Builder addresses the functions of metrics; their focus and tense; their operational and strategic contexts; as well as setting up the characteristic distinctions between metrics, metric sets and metric systems. Survey Builder Pro Metric Builder allows the user to define these hierarchical classifications and categorized metric sets. An example might look like:

The intention is to identify future-state objectives, relate them to specific goals that can be achieved through critical success factors (CSF) which are then monitored and measured by a specific metric. Through a hierarchy, organizations can define and communicate relationships between metrics and how they contribute to the achievement of organizational goals and objectives.

The term, "metrics" is usually used to define one of three types of constructs:

1. Individual Metric
2. Metric Sets
3. Performance or outcome measurement system (Survey Instrument)
   

The recognition of the different levels of metrics and their interactions is important for the research and design of metrics.

At the highest level, the performance measurement system level integrates. That is, it is responsible for coordinating metrics across the various functions and for aligning the metrics from the strategic to the operational levels. For every activity, product, function, or relationship, multiple metrics can be developed, implemented, and measured. What is important is to design a structure to the metrics (i.e., grouping them together) and extracting an overall sense of outcomes from them.

Metric Sets - True Multi-Variant Research Analysis

The metric set consists of the metrics assigned by a higher level of authority to direct, and evaluate a single behavior or specific activity, process, or function. The metrics set is critical because it is often the relevant unit of analysis, and because the scope and complexity of the metrics set can be viewed as its multivariate outcomes.

At the core is the individual metric, the foundation in behavioral research. Individual metrics are aggregated to form various metrics sets. Each metric set provides the researcher with the ability to look at patterns of outcomes. Coordinating the development of the various individual metrics and the metrics sets allows the researcher to create effective survey instruments that provide a measurement across the metrics.