Criteria for measurements

A crucial aspect of any analysis is the nature and quality of the data that are involved, which we call measurements, using that term in a very general sense. Criteria for satisfactory measurements typically include:

• relevance;

• adequate precision;

• economy; and

• absence of distortion of the features studied.

In fields with a solid history of investigation, the measurement techniques necessary to capture the relevant features may well be firmly established. In other contexts, defining a measurement procedure that will yield insightful information may be crucial to successful work. Usually we require of a procedure some mixture of face validity, that is, apparent relevance, and construct validity. The latter implies that a procedure has proven success in establishing meaningful and stable relationships with other variables. These further variables should include those for which there is strong prior reason to expect a connection.

The costs, however measured, of obtaining the data should be commensurate with the objectives. In particular, while individual data quality is of great importance, the collection of unnecessarily large amounts of data is to be avoided; in any case the intrinsic quality of data, for example the response rates of surveys, may be degraded if too much is collected.