Hypothesis-driven experiments

Null and alternative hypotheses

The basic outlines of hypothesis-driven research were provided in Section 1.3.1, but when setting out to create an experiment to test a hypothesis, there is much more to consider.

Your experiment should have at least one control or independent variable, and at least one response or dependent variable. The independent variable is something you are sure will change during the course of experimental measurements. The dependent variable is what you will measure, although you may not be sure if it will change. Indeed the point of the experiment may be to determine if it does or does not change. In laboratory experiments it is common to call the independent variable a control variable because you decide upon its value and control it. For example, in an experiment concerning the effect of fertilizer doses on size of tomatoes, you control the concentration of fertilizer given to each plant. In observational research, the independent variable may be something you cannot control. For example, you might conduct a study of tomatoes grown outside and investigate whether they grow faster on hot days than on cool days. In this case, nature will bring you the hot and cold days; you cannot control when they will happen, but you can still study their effects. Experiments are usually better when the investigator controls the independent variable, but this is not always possible.

Introduction

The most annoying problems that are encountered during the use of electronic systems are often intermittent in nature. Electromagnetic interference, corona and arcing in high-voltage circuits, and some other causes of potentially intermittent faults, are covered in this chapter. Other reliability problems in electronic hardware, such the failure of high-power equipment, are also discussed below.

Difficulties involving electrical contacts, connectors and cables are very common in electronic work, and frequently intermittent. These are dealt with in Chapter 12. Problems with mains power disturbances and overheating of equipment are considered in Sections 3.6 and 3.4.1, respectively. Vibrations can be a cause of noise in electronic systems in the form of microphonics, and these are discussed in Section 3.5.3. A general survey of some of the causes of intermittent failures in experimental work is presented in Section 3.3.

With some exceptions, details of the design, construction, and troubleshooting of electronic circuits and equipment items (e.g. self-contained electronic instruments) are not discussed in this chapter. Such information is provided in references listed in the “Further reading” section on page 403.

Electromagnetic interference

Grounding and ground loops

General points

Importance of grounding arrangements

Proper grounding is generally necessary for the reliability of electronic systems. This includes both the topology of ground networks, and the quality of the electrical contacts made between ground conductors. (The latter is discussed in Section 12.2.4.) Failure to provide an adequate ground arrangement often leads to erratic noise problems and system malfunctions.

Introduction

Among the most common problems encountered when dealing with electronic items are those caused by faulty contacts, connectors, wires and cables. For example, bad solder joints are among the most frequent causes of failures in electronic devices in general. In any given electronic system, the most unreliable components will be its connectors and cables. (Perhaps somewhat surprisingly, the number of connector problems that occurred during the Apollo space program was greater than that of all other problems combined.) More specifically, faulty connectors and cables are an important source of trouble in experimental work, in the form of unreliable measurements and wasted time.

Information on this subject is widely dispersed. The purpose of the following chapter is to bring together knowledge about these mundane but troublesome items from some of the various scattered sources.

Some contact, connector, wire, and cable problems are:

1. (a) excessively high or low resistances, or complete open or short circuits,

2. (b) noisy contacts (perhaps in the form of 1/f noise),

3. (c) production and reception of electromagnetic interference (including crosstalk between nearby wires and cables),

4. (d) leakage currents over insulators in high-impedance circuits,

5. (e) noise produced as a result of cable vibrations in high-impedance circuits (“microphonics”),

6. (f) nonlinear I–V characteristics of poor-quality contacts (i.e. they can become non-ohmic – this may lead to interference with sensitive measurements due to rectification of electromagnetic noise in the environment, and also signal distortion),

7. (g) reflections on cables at radio (and particularly microwave) frequencies, owing to impedance mismatches,

8. (h) arcing and corona discharge at contacts, and in cables and connectors, at high voltages,

9. […]

Ingredients of mathematical modeling

Applied mathematics refers to the portion of mathematics most often useful in representing the physical world. Many scientists spend most of their time working on the theories of physics, chemistry, geology, astronomy, or biology; most of their time is spent with mathematics as well. Statistical analysis is a portion of that analysis, but there is much more as well. Another name for this sort of activity is mathematical modeling.

Numbers

The most useful idea in mathematical modeling is also the oldest and most basic. It is the idea of using numbers to represent the world. There are many different sorts of numbers, and it may be useful to review them before proceeding to more complicated operations.

Positive integers These represent increasing quantities of essentially identical objects. The basic idea is simple yet abstract, that one object can be essentially the same as another, while not literally being identical. Even two symbols, X, X, are different if only because they are in different places, but everyone seems to understand what it means to say that they are the same, and there are two of them.

Zero This is a number representing the absence of some quantity. It makes it possible to state that something is not present.

Negative integers These numbers seem to have been conceived in India during our Medieval period, along with the Arabic numerals (brought by Arabs from India to Europe).

Course goals

Science starts with curiosity about the world. It starts with questions about why the sky is blue, why dogs wag their tails, whether electric power lines cause cancer, and whether God plays dice with the universe. But while curiosity is necessary for science, it is not enough. Science also depends upon logical thought, carefully planned experiments, mathematical and computer models, specialized instruments, and many other elements. In this course, you will learn some of the research methods that turn curiosity into science. In particular, you will learn to

• Create your own experiments to answer scientific questions.

• Design experiments to reduce systematic and random errors and use statistics to interpret the results.

• Use probes and computers to gather and analyze data.

• Treat human subjects in an ethical fashion.

• Apply safe laboratory procedures.

• Find and read articles in the scientific literature.

• Create mathematical models of scientific phenomena.

• Apply scientific arguments in matters of social importance.

• Write scientific papers.

• Review scientific papers.

• Give oral presentations of scientific work.

You will not just be learning about these skills. You will be acquiring and applying them by carrying out scientific inquiries.

Kinds of questions

Testable questions Every scientific inquiry answers questions, and most scientific inquiries begin with specific questions.

Introduction

The ubiquity of computers in just about every aspect of experimental work, from their use in preliminary literature searches, through the collection and analysis of data, to the final publication of results, means that the consequences of their imperfections are unavoidable. Although hardware and software problems are both a possible cause of frustrations, the latter can be particularly insidious, and harder to solve.

The failure of hardware is sometimes heralded by unusual physical phenomena such as excessive vibrations or noises. On the other hand, failures of software take place suddenly and without warning. Also, again unlike the situation with hardware, the concept of safety margins (or “derating”) normally does not exist in software design. The enormous complexity of many types of software when compared with hardware makes the former particularly susceptible to unforeseen failure modes. (For example, operating systems and large applications often contain millions, or even tens of millions, of lines of source code.) These characteristics can all make software failures especially troublesome.

Computers and operating systems

Selection

Compared with many other electronic devices, computers (including their hardware, software, data storage, security and interconnection aspects) are generally very unreliable. Hence, and since they play such an important part in so many facets of research, it pays to use high-quality computer equipment and software.

The quality of computer hardware varies over a wide range – from systems of marginal dependability, to ones that can be counted on to give years of reliable service.

Introduction

Generally, nearly all of the equipment, devices, substances, and software that are employed in scientific research are of commercial origin. It is clear from everyday experience that the reliability and usability of such products can range from superb to abysmal. (Usability has an impact on overall reliability through its effect on human error.) Low-quality items, coupled with poor documentation (e.g. operating instructions), and lack of after-sales technical support from the companies that made them, can have a very harmful effect on laboratory productivity. The purpose of this chapter is to provide ways of avoiding such problems.

Using established technology and designs

As was pointed out in a general context on page 4, if reliability is important, one should not normally employ devices or software that use a technology which has not previously been in commercial production. Most innovative things have unforeseen problems (sometimes in very significant numbers), which can be found and dealt with only after such things have been tested by many users over an extended period. This is a particularly serious issue with computer software (see page 504).

Introduction

Mechanisms in general are a notorious source of reliability problems. This chapter deals with mechanical devices and systems (broadly construed) that are of particular concern in experimental work. The former include precision mechanisms, static and dynamic vacuum seals, and valves. A brief discussion of devices for preventing mechanical overtravel and overload is also included.

Mechanical devices that are used under extreme conditions (e.g. in ultrahigh-vacuum and cryogenic environments) often suffer from friction and wear problems due to a lack of lubrication. A section of the chapter is devoted to lubricants (and especially dry lubricants and self-lubricating solids) that are useful under such conditions.

Water-cooling systems are a common source of trouble in apparatus ranging from small-scale tabletop equipment (e.g. lasers) to large particle accelerators. Such problems include leaks, cooling-line blockages, corrosion, and condensation. These issues, and others, are discussed below.

Mechanical vacuum pumps are dealt with in Chapter 7, while electrical connectors and cables (which are partly mechanical devices) are discussed in Chapter 12.

Mechanical devices

Overview of conditions that reduce reliability

Some agents and environmental conditions that tend to diminish the reliability of mechanical parts and systems are as follows.

1. (a) Particulate matter such as dust and grit can be very harmful to mechanical devices. It leads to accelerated wear and damage of moving parts (especially antifriction bearings, such as ball bearings), leaks in sealing devices, a decrease in the accuracy of precision mechanisms, and instabilities in mechanical assemblies requiring accurate registration of surfaces.

2. (b) Moisture (including high humidity) causes corrosion and promotes biological growth (i.e. mold and fungi, which can release corrosive chemicals), degrades lubricants, and holds particulate matter that would otherwise escape on surfaces. […]

The following quotation is from Galileo's final book, Dialogues on Two New Sciences [213] (179) translated by Henry Crew and Alfonso de Salvo (Macmillan, New York, 1914). Salvatore, a character who stands in for Galileo himself, has just been challenged to justify the claim that falling objects accelerate uniformly.

These grading rubrics provide an attempt to formalize the way that scientists evaluate scientific work. Most of the rubric is subtractive. This means that the rubric describes various sorts of mistakes, and the numbers of points that can be deducted for them. The total number of points that can be deducted from each category adds to much more than 100 because scientific work can be rendered invalid by poor performance in any of these areas. Unsafe practices or plagiarism lead immediately to a failing grade.

At the very end of the rubric, there is room for additions of points. Adding points accounts for exceptional or innovative work. This part of the rubric captures the idea that the best scientific work has an element of creativity and effort that cannot be captured by the simple avoidance of doing anything wrong.

To use the rubric, the instructor will decide on the baseline value from which points will be subtracted for errors and omissions. This value may be less than 100. Getting the maximum score of 100 may require the addition of points from the final section of the rubric.

Please note that no rubric or checklist can fully capture the full range of strengths and difficulties that describe independent inquiries. This rubric provides a guide to help you identify common misconceptions and errors, but cannot fully cover all cases.

This book accompanies a one-semester undergraduate introduction to scientific research. The course was first developed at The University of Texas at Austin for students preparing to become science and mathematics teachers, and has since grown to include a broad range of undergraduates who want an introduction to research. The heart of the course is a set of scientific inquiries that each student develops independently. In years of teaching the course, the instructors have heard many questions that students naturally ask as they gather data, develop models, and interpret them. This book contains answers to those most common questions.

Because the focus is on supporting student inquiries, the text is relatively brief, and focuses on concepts such as the meaning of standard error, p-values, and deterministic modeling. If a single statistical test, such as χ2, is adequate to deal with most student experiments, the text does not introduce alternatives, such as ANOVA, even if they are standard for professional researchers to know.

The mathematical level of the book is intermediate, and in some places presumes knowledge of calculus. It could probably be used with students who don't know calculus, skipping these sections without great loss.

There is an instructor's manual that describes daily activities for a 14-week class that meets two hours per week in a classroom and two hours per week in a lab. It is available at www.cambridge.org/Marder.

Introduction

The design and construction of scientific apparatus is often accompanied by serious pitfalls. These can be a cause of much wasted time, and a source of frustration to those who are not familiar with the problems. This chapter examines a few of most common difficulties, and some useful general strategies for easing the development of reliable apparatus. Detailed design and construction considerations are discussed elsewhere in the book, and in the references at the end of the chapter.

Commercial vs. self-made items

One should generally not make equipment and parts that can be obtained commercially. An important reason for this is that it very often takes much longer to design and construct such items in a research laboratory than is originally envisaged (see Section 5.3). Most of the things that are done in a laboratory are not new. Hence, much time and effort can be saved by making full use of the multitude of products that are available on the market, and easily located on the Web.

Many commercial devices of a given type may be made by a company, and several generations may be designed and manufactured over the years. For this reason, the people who design these devices (often professional engineers) frequently have considerable experience in dealing with their potential problems. Furthermore, since large development costs can be spread out over many units, the reliability, ease-of-use, and performance of these can be much higher, given the cost, than those of equivalent devices designed and built in-house.

Introduction

If a scientist carries out a major research project, but no one knows about it, or no one can understand it, the research is of little use. So a large part of science is communication. The communication happens informally between colleagues in the hallway, at conferences through presentations, and most durably with global distribution of information through articles and books. Scientists have developed many conventional ways to explain their results, including equations, figures, and specialized vocabulary. Once you have completed a research project, you will need to practice communicating the results, to round out the set of scientific skills you have acquired. Each of the sections of this chapter touches just on a few essentials. You can find much more complete discussion of all the topics mentioned here in Valiela (2001).

Writing a proposal

Scientists spend a large fraction of their time writing proposals, whether they want to or not. Proposals are necessary to apply for grant funding, or for permission to conduct research. The research proposals needed for funding from federal agencies are usually a minimum of 15 pages in length, not including budgets, bibliography, and supporting documents. In this class you will not have to write proposals of that length. However, your instructors may ask you to write proposals a page or two long as you are beginning your inquiries.

Spreadsheet programs

Spreadsheets record and manipulate data. The purpose of this appendix is to emphasize the features of spreadsheets that make them useful for manipulating scientific data, and to perform computations based on scientific theories.

Excel for Windows

By far the most common spreadsheet is Excel, which is developed and marketed by the Microsoft Corporation as part of Microsoft Office. There are many different versions of Excel in circulation, but all of them have all the functions described here. Excel is available for all the different Windows operating systems Microsoft has written for IBM-compatible personal computers. The major releases of Excel in current use are Excel 2003, and Excel 2007. The latest release, Excel 2007, involves a completely new way of accessing menu items, but all functions of Excel 2003 are still available. Note that the native file format of Excel 2007 is a new format (.xlsx) that can cause difficulties for other programs. Although this format is becoming universal, you may still sometimes find it useful to read and write files in the old Excel 2003 format (Compatibility Mode).

Excel for Macintosh

Excel 2008 is available for computers running Apple's OSX operating system. The menu items are arranged somewhat differently from the Windows releases, but all the same functions are available in a nearly identical way. This version of Excel can read the .xlsx files of the recent Excel releases for Windows.

Introduction

A number of basic qualities or conditions are of value whenever reliability is an issue. These include: (a) simplicity, (b) redundancy (providing duplicate or backup components or systems), (c) margins of safety, (d) modularity (dividing complicated things into simple components), and (e) conservatism (using conservative technology). These factors, and others, are considered in the following chapter.

Human error is, of course, a very important cause of problems in all activities. It might be thought that little can be done to prevent such errors, but this is far from the case. For example, numerous investigations have been carried out (mostly in the aviation and nuclear industries), which show that errors are generally not completely random and unpredictable events, but usually follow regular patterns. These results, which are discussed below, suggest ways of avoiding errors, or at least mitigating their consequences.

Other sections of the chapter discuss record keeping in the laboratory (the lack of which is a common cause of problems), the maintenance and calibration of equipment, and general strategies for troubleshooting apparatus and software.

Central points

The following are very general principles of reliability that recur repeatedly in all activities in research.

Introduction

In books on mathematical methods in physics, there is a very frequent tendency to ignore the subject of errors, and what can be done to prevent them. In conversations on this topic, two common points of view are: (1) error prevention is an ability that is acquired through practice (i.e. it is not something that is taught explicitly), and (2) one just has to be careful. While both viewpoints contain elements of truth, it is also true that techniques exist for preventing and detecting errors. Furthermore, these can be passed on explicitly like other skills (i.e. they do not have to be learned through hard experience.) Such techniques are the subject of the present chapter. These are mostly concerned with the prevention of errors in symbolic (i.e. algebraic) calculations, rather than numerical ones, unless otherwise indicated.

Sources and kinds of error

Conceptual problems

The first and most subtle types of error in analysis in general arise from conceptual problems: understanding the essential physics of a problem and expressing it in mathematical form.

Transcription errors

A second very common type, frequently encountered when calculations are done by hand, is transcription errors. These occur when formulae and numbers are copied from one line in the calculation to the next, often taking place when the handwriting is untidy and cramped. They also arise very frequently if more than one mathematical operation is attempted per line in the calculation.

Introduction

The following chapter discusses a number of important issues that affect many classes of experimental hardware. For instance: vibrations can cause problems with optical, cryogenic, and other apparatus, and overheating may afflict electronic instruments, computers, and numerous other items.

Stress derating

The practice of derating (otherwise known as “providing a margin of safety”) is a means of improving the reliability of devices exposed to physical stresses (including power, temperature, electric current, pressure, etc.). If devices are operated continuously at their rated maximum stress levels, they are not unlikely to undergo failure in a relatively short time. Derating consists of operating them at reduced stress levels, at which the probability of failure is comparatively small. The derating factor of a device is usually expressed as a ratio of the stress needed to achieve good reliability to the rated maximum stress. While the term “derating” is normally used in conjunction with electrical and electronic components, the principle is a general one.

Generally, systems comprising a collection of components (e.g. a power supply or a power amplifier) are not derated as a whole, because derating has (or should have been) already been done for each of the constituent components. That is, derating is usually done at the component level, rather than the system (collection of components) level. (An exception is the temperature derating of power supplies – see below.)