What branch of statistics gathers data from the whole population and makes a conclusion for the whole population?

Inferential statistics is concerned with making inferences (decisions, estimates, predictions, or generalizations) about a population of measurements based on information contained in a sample of those measurements. The two basic types of statistical inference are estimation and hypothesis testing. The goal of estimation is to estimate a value for an unknown population characteristic (parameter) based on a sample statistic. Both point estimators and interval estimators can be used for this task. Hypothesis testing, which is more involved than estimation, consists of six steps constituting a formal procedure whose goal is to draw inferences or conclusions about the value of one or more population parameters based on sample statistics estimating those parameters. Uncertainty is inherent in both estimation and hypothesis testing since statistics can vary from one sample to another. A sampling distribution depicts the probability of occurrence of all values a statistic can assume when computed for all possible independent, random samples of size n. Sampling distributions are used to evaluate the reliability of inferences made using statistics.

4 Inductive and Transductive Inference: Sample and Population Statistics

Inferential statistics is a way of making inferences about populations based on samples. Given information about a subset of examples, how do we draw conclusions about the full set (including other specific examples in that full set)? Inferences based on principles of evidence use sample statistics. We suggest that matching and similarity-based inferences are based on population statistics. That is, if one has sample statistics then the inferential problem is to treat those statistics as evidence. If one has population statistics then the inferential problem is to assess a degree of match. Psychologists are very familiar with inferential statistics and evidence evaluation: Our studies tend to draw conclusions about populations based on samples. Rarely, if ever, do we have information about the whole population. What are inferences based on population statistics? Fortunately, there is an account of population-statistical inferences, and even a label for such inferences: Transductive.

Developmental psychologists are familiar with the term “transductive inference” from Piaget (1959). Piaget used “transductive” to describe a preoperational form of reasoning that connects specific cases with no general rule or underlying mechanism. Events that are experienced together are taken to be causally related. A car horn honks and the lights go off: The horn caused the darkness. The inference goes from one particular event (honking) to another (darkness) with no notion of a general mechanism. That is, how can honking horns cause darkness? How could this relation be generally sustained? Piaget's transductive inference seems to be a version of the “principle of association” in sympathetic magic (Frazer, 1894; events perceived together are taken to be causally related). Such a principle is also characteristic of a Humean or associative view of causation and causal inference: A causal relation just is the perception of association. Once a person associates horns and darkness, they will come to expect one given the other. Transductive inference (horn → darkness), therefore, seems to be familiar associative learning and similarity-based induction. Perhaps, Piaget was illustrating some special characteristics of young children's similarity-based inferences (e.g., “one-trial” associations) but the general process of inference is familiar. In particular, it is not clear how Piaget's transductive inference addresses the point that similarity-based inferences rely on population statistics. There is another account of transductive inference that does, however.

Apparently independent of Piaget, the term “transductive inference” was introduced into the machine learning/statistical learning theory literatures by Vladimir Vapnik (1998). Vapnik also uses “transductive” to refer to inferences from particular examples to particular examples. For Vapnik, though, transductive inferences are a kind of simplifying assumption, a way of approaching complex learning problems. Vapnik argues that (standard, evidential) statistical inference is an attempt to solve a general problem: From experience with a set of examples how can the learner construct a rule or pattern that can then be applied to new examples? The key feature of evidential inference is that the class of potential new examples is infinite. The point of transductive inference is that often the class of potential new examples is finite. For example, an inspector might encounter the problem of predicting which widgets in a batch of 100 are defective (see Fig. 1.1). Solving the problem for the particular batch will usually be much simpler than solving the problem of identifying defective widgets in general. For Vapnik, transductive inference is the strategy of limiting focus to the specific examples that the learner will actually encounter. Here is the link to Piaget's transductive inference. If the child's conclusion (horns cause darkness) is restricted just to that particular observed situation, then it seems less problematic: It is only when generalized that it falls apart. Similarly, the link to sample and population statistics becomes more apparent. In (standard, evidential) statistical inference, the 100 widgets are a sample. The learner's task is to use this sample to form representation of the class or population of widgets. In transductive inference, the 100 widgets are the population. The learner's task is to learn and use statistics to make predictions about this population.

An example will help illustrate the relation between transductive-population and evidential-sample inferences. Consider the 100 widgets. Suppose an inspector has examined all 100 and discovered the following: There are 10 black widgets and 90 white widgets moreover 5 of the black widgets are defective and 9 of the white are defective. These are descriptive statistics. The inspector's challenge is to use these descriptive statistics to make some inferences. We distinguish two kinds of inferences the inspector could be called upon to make: transductive or evidential (what we have been calling “standard statistical inference”). If the inference concerns the 100 widgets, the statistics observed are population statistics. For example, suppose the inspector is shown one of the widgets and told that it is white. The inspector recalls that only 10% of white widgets were defective and predicts that it will be fine. This is clearly a kind of inductive inference in that it is not guaranteed to be correct: The inspector's past experience makes the conclusion probable but not certain. But it is a special kind of inductive inference, a transductive inference. The inspector's problem is relatively simple. After having calculated the descriptive statistic, p(defective|white) = 0.1, there is really very little work to be done. The inspector can be confident using the descriptive statistic to guide his inferences because the statistic was calculated based on the examples he is making inferences about. In a certain respect, the inspector is not even making an inference, just reporting a description of the population. To move from “9 of these 90 white widgets are defective” to “one of these white widgets has a 10% chance of being defective” to “a white widget selected at random is probably not defective” hardly seems like much of an inductive leap at all. Put slightly differently, once the inspector has solved the descriptive problem (what is p(defective|white) among the 100 widgets?) the inferential problem of making a prediction about a randomly selected widget is easy.

The inspector faces a more difficult problem when making inferences about a widget not in the “training” set, widget 101. In this case, the descriptive statistics (e.g., p(defective|white) = 0.1) are characteristics of a sample and the inspector must calculate inferential statistics to make predictions. In this case, the inspector must consider the evidential relation between his sample (of 100 widgets) and the general population (from which the new widget was drawn). Is the sample biased? Recognizing and adjusting for sample bias is a specific problem of evidential inference. It is this inferential problem that distinguishes transductive inference from evidential inference. Sample bias does not matter when making a transductive inference to one of the 100 original widgets.

Consider the problem of predicting the color of a widget identified as defective. If the defective widget was one of the 100, the prediction is clear: It is probably white. Nine of the 14 defective widgets encountered were white. If the defective widget is new, widget 101, the prediction is less clear. The 100 original widgets were mostly white. Is that true of the general population widgets or is that a bias in the sample? Unless the inspector knows about the relation between his sample and the population he cannot use the former to make predictions about the latter. However, figuring out that relation, solving this inferential problem, is irrelevant for a transductive inference. If the 100 widgets are considered the population, there is no sampling bias. In this way, transductive inference can be used as a kind of simplifying assumption for inductive inference. Transductive inference is inductive inference where sample-population relations are ignored, where sample statistics are treated as population statistics. The challenge of transductive inference is limited to developing useful descriptions (characterizing the patterns in the available data).

3.02.3.4 Use of Statistics with Single-subject Data

For the most part, inferential statistics were designed for use in between-group comparisons. The assumptions underlying the widely accepted classical model of statistics are usually violated when statistical tests based on the model are applied to single-subject data. To begin with, presentation of conditions is not generally random in single-subject designs, and randomization is a necessary prerequisite to statistical analysis. More importantly (and more constantly), the independence of data required in classical statistics is generally not achieved when statistical analyses are applied to time-series data from a single subject (Sharpley & Alavosius, 1988). Busk and Marascuilo (1988) found, in a review of 101 baselines and 125 intervention phases from various single-subject experiments, that autocorrelations between data, in most cases, were significantly greater than zero and detectable even in cases of low statistical power. Several researchers have suggested using analyses based on a randomization task to circumvent the autocorrelation problem (Edgington, 1980; Levin, Marascuilo, & Hubert, 1978; Wampold & Furlong, 1981). For example, data from an alternating treatment design or extended complex phase change design, where the presentation of each phase is randomly determined, could be statistically analyzed by a procedure based on a randomization task. Some controversy surrounds the issue (Huitema, 1988), but the consensus seems to be that classical statistical analyses are too risky to use in individual time-series data unless at least 35–40 data points per phase are gathered (Horne, Yang, & Ware, 1982). Very few researchers have the good fortune to collect so much data.

Time-series analyses where collected data is simply used to predict subsequent behavior (Gottman, 1981; Gottman & Glass, 1978) can also be used, and is useful when such predictions are desired. However, such an analysis is not suitable for series with less than 20 points, as serial dependence and other factors will contribute to an overinflated alpha in such cases (Greenwood & Matyas, 1990). In cases where statistical analysis indicates the data is not autocorrelated, basic inferential statistical procedures such as a t-test may be used. Finally, the Box-Jenkins procedure (Box & Jenkins, 1976) can technically be used to determine the presence of a main effect based on the departure of observed data from an established pattern. However, this procedure would require a minimum of about 50 data points per phase, and thus is impractical for all but a few single-subject analyses.

In addition, most statistical procedures are of unknown utility when used with single-subject data. As most statistical procedures and interpretations of respective statistical results were derived from between-group studies, use of these procedures in single-subject designs yields ambiguous results. The meaning of a statistically significant result with a lone subject does not mean the same thing as a statistically significant result with a group, and the assumptions and evidentiary base supporting classical statistics simply dose not tell us what a significant result with a single subject means. Beyond the technical incorrectness of using nomethetic statistical approaches to ideographic data, it is apparent that such use of these statistics is of extremely limited use in guiding further research and bolstering confidence about an intervention's efficacy with an individual subject. If, for example, a statistically significant result were to be obtained in the treatment of a given client, this would tell us nothing about that treatment's efficacy with other potential clients. Moreover, data indicating a clinically significant change in a single client would be readily observable in a well-conducted and properly graphed single-subject experiment. Statistics—so necessary in detecting an overall positive effect in a group of subjects where some improved, some worsened, and some remained unchanged—would not be necessary in the case of one subject exhibiting one trend at any given time.

Examples of parametric tests from the project management website dataset

The following are examples of the use of inferential statistics on parametric data through investigations of questions 3 and 4, outlined earlier in the chapter. For each test, an example is given of the way the results of the test could be reported. Typically, the value of p is not reported: just the level that was used to determine significance. Also, the value of the statistic resulting from a test is only reported if significance is found. For all tests in these examples, the level of significance used is 0.05.

4.1 Introduction

Statistical analysis can be subdivided into two parts descriptive statistics and inferential statistics. In descriptive statistics, one summarizes and graphically represents data of a sample or a whole population. In inferential statistics, one not only collects numerical data as a sample from a population but also analyzes it and, based on this analysis, draws conclusions with estimated uncertainties (i.e., by using probability theory) about the population. It goes without saying that in order to measure aspects of scientific communication and to evaluate scientific research, scientists use statistical techniques. Although hundreds of books have been written on statistics, few deal explicitly with statistics in the framework of information and library science. A basic introductory text for library professionals is Vaughan (2001), while Egghe and Rousseau (2001) is more elementary. One quarter of Introduction to Informetrics (Egghe & Rousseau, 1990) is devoted to statistics. Ding et al. (2014) contains a practical introduction to recent developments in informetrics, including statistical methods.

The term population refers to the set of entities (physical or abstract ones) about which one seeks information. The publications of scientists forming a research group, of scientists in a country, of scientists active in a scientific domain; of articles included in Scopus and published during the year 2015, are all examples of populations.

In order to investigate a population, the investigator collects data. If it is possible, the best option is to include the whole population in this investigation. Yet, it is often impossible to collect data on the whole population, so the statistician collects a representative sample. This means that a subset is collected in such a way that it provides a miniature image of the whole population. If, moreover, the sample is large enough, then a diligent analysis of the sample will lead to conclusions that are, to a large extent, also valid for the whole population. Such conclusions must be reliable, which includes that the probability to be correct must be known.

Classical inferential statistics draws samples from a population and then tries to obtain conclusions that are valid for the whole population (with a specified level of confidence). In informetrics there often are no samples, but one tries to draw conclusions based on an observed population e.g., all journals included in Scopus. Does this make sense? We will not answer this question, but refer to Section 4.14 for some useful references related to this question.

This chapter is subdivided into two main parts. In Part A, we describe some techniques from descriptive statistics, while in Part B we discuss inferential statistics, including a short introduction to the normal distribution and a few nonparametric tests. Although we will not consider parametric tests, we nevertheless briefly introduce the normal distribution as it is near-impossible to talk about statistics without in some way involving the normal distribution e.g., when talking about z-scores. Multivariate techniques are beyond the scope of this introductory book. For these we refer the reader to specialized literature.

IBMB 200: Introduction to Research Methods and Methodology

Three hours of lectures and one two-hour lab per week. Introduces descriptive and inferential statistics and their roles in clinical and technical practice and evaluation. Topics include correlation, regression, t-test, Chi square, ANOVA, nonparametric methods, sample-size estimation, and probability. Also included are topics in experimental design and correlation analysis, including multiple correlation and multiple regression, curvilinear relationships, missing data, interactions, ANOVA and its generalization, logistical regression, selected complex factorial designs, multiple comparisons, and meta-analyses; causal models, construction estimation, and testing of causal models for correlational data. Particular attention is paid to models with unobserved variables.

(3 credits)

1.2 Inferential versus Descriptive Statistics and Data Mining

Descriptive statistics, as the name implies, is the process of categorizing and describing the information. Inferential statistics, on the other hand, includes the process of analyzing a sample of data and using it to draw inferences about the population from which it was drawn. With inferential statistics, we can test hypotheses and begin to explore causal relationships within data and information. In data mining, we are looking for useful relationships in the information or models, particularly those that can be used to anticipate or predict future events. Therefore, data mining more closely resembles descriptive statistics.

It was not that long ago that the process of exploring and describing data, descriptive statistics, was seen as the necessary though unglamorous prerequisite to the more important and exciting process of inferential statistics and hypothesis testing. In many ways, though, the creative exploration of data and information associated with descriptive statistical analysis is the essence of data mining, a process that, in skilled hands, can open new horizons in data and our understanding of the world.

Sampling Bias

Whenever statements are made about a population on the basis of a sample, there is at least some risk of error. The tools of inferential statistics cannot eliminate this risk, but they provide powerful ways of assessing it and working with it. Practically every analysis in archaeology (whether statistical or not) involves characterizing a larger set of things than are actually observed, so the perspectives of inferential statistics have implications for archaeological analyses that reach far beyond the quantitative contexts they were designed for. The risk of error in characterizing a population on the basis of a sample arises from two fundamentally different sources. One is that the process of selecting the sample from the population may systematically produce a sample with characteristics different from those of the population at large. This is known as ‘sampling bias’. It happens, for example, when lithic debitage is recovered by passing excavated deposits through screens with 6 mm mesh. Lithic debitage is known to include very small waste flakes, many of which will pass through mesh of this size, so the sample recovered from the screen will be systematically larger than the complete population. The mean weight of such a sample of waste flakes would be higher than that of the population as a whole, and any statement made on the basis of this sample about the weight of waste flakes in general would be inflated as a direct consequence of this sampling bias.

Precisely the same is true even in entirely nonquantitative analyses. An archaeologist might subjectively characterize the lithic technology of the Archaic period in some region as reflecting a broad application of a high degree of technical skill. If this characterization of the large, vaguely defined population consisting of all lithic artifacts produced in the region during the Archaic period is based on a sample recovered by artifact collectors who value well-made bifacial tools and never bother to keep anything as mundane as a utilized flake, then the sample is clearly biased toward well-made tools, and the breadth of application of high technical skill will be overvalued as a direct consequence of sampling bias.

Rigorously random procedures for sample selection are designed to avoid bias, and neither of the sampling procedures in the examples above is random. There are no statistical tools for removing bias from a sample once it has been selected, whether by screening deposits through large mesh or by collectors. Indeed, the tools of inferential statistics are often said to ‘assume’ that samples are unbiased, and thus to be inapplicable to the analysis of biased samples. This is not a useful way to approach statistical analysis in archaeology, because archaeologists are often forced to work with samples they know to be biased. The prescription offered in other disciplines (collect another sample with rigorously random procedures that avoid bias) is often impossible in archaeology. Fortunately, there are at least two common ways of working with biased samples. Archaeologists may want to say things about populations that would not be affected by the bias present in the available sample. It might, for example, be perfectly possible to make unbiased conclusions about the proportions of raw materials represented in lithic debitage on the basis of the screened sample discussed above. It would only be necessary to assume that different raw materials would not occur among the small waste flakes that fell through the screen in proportions very different from those among the larger flakes that would not escape the sample in this way. Biased samples can also often be accurately compared to each other if the same biased operated to the same degree in the selection of all samples to be compared. Thus, two samples of lithics recovered from 6 mm screen may show rather different flake weights. Such a difference cannot be attributed to sampling biases if the sampling biases were the same in both cases, and it is valid to say that the flakes in one sample are heavier than those in the other.

Precisely, the same principles apply to subjective and qualitative comparisons. To compare the application of technical flint-knapping skill to the production of lithic tools in the Archaic with that in the Formative, it may well be possible to work successfully with biased samples. (This is likely to be necessary in any event.) As long as the same sampling biases operated in the recovery of both Archaic and Formative artifacts, then they can be compared. If, however, the Formative tools come from systematic excavations and the Archaic ones are those accumulated by artifact collectors, then the sampling biases are different and likely to affect very strongly precisely the characteristics of interest. Such a difference in sampling biases affects any comparison based on the abundance of ‘nice’ well-made tools, whether the comparison is quantitative or not. To repeat, statistical tools cannot eliminate bias once a sample has been selected (and the utter elimination of all kinds of sampling bias from the process of archeological recovery is an unrealistic goal in any event). Statistics, however, does provide two very useful things: first, a reminder of the important effects that sampling bias can have, and, second, some useful concepts for thinking about sampling bias and how serious a worry it is in the specific context of particular observations of potential interest.

Robust estimation

Measures of central tendency such as trimmed or Winsorized means are referred to as robust estimators. Some researchers (e.g., Keselman et al., 2007) argue that inferential statistics based on robust estimators can be preferable, when data are not normally distributed, than either traditional parametric and nonparametric tests, or even permutation tests. They suggest that robust estimators are less likely to lead to the wrong decision over whether to reject the Null Hypothesis. For more detail on robust methods, see Wilcox (2001), Wilcox and Keselman (2003), and Maxwell and Delaney (2004).

1.2 Inferential versus descriptive statistics and data mining

Descriptive statistics, as the name implies, is the process of categorizing and describing information. Inferential statistics, on the other hand, includes the process of analyzing a sample of data and using it to draw inferences about the population from which it was drawn. With inferential statistics, we can test hypotheses and begin to explore causal relationships within data and information. In data mining, we are looking for useful trends, patterns, or relationships in the information. Therefore, data mining more closely resembles descriptive statistics. Predictive analytics, on the other hand, can be seen as a logical extension or complement to the data mining process, and can be used to create models, particularly those that can be used to anticipate or predict future events, surface and characterize hidden patterns or trends, and provide meaningful insight in support of anticipation and influence.

It was not that long ago that the process of exploring and describing data, descriptive statistics was seen as the necessary though unglamorous prerequisite to the more important and exciting process of inferential statistics and hypothesis testing. In many ways, though, the creative exploration of data and information associated with descriptive statistical analysis is the essence of data mining, a process that, in skilled hands, can open new horizons in data and our understanding of the world. For example, summative content analysis is a descriptive technique used for the analysis of unstructured narrative that includes the counting and comparison of key words or content.1 Figure 1.1 illustrates the results of this technique when applied to media content associated with the Tahrir Square protests during the Arab Spring.2 Through the additional segmentation of English language sources into the period immediately before and after the resignation of President Hosni Mubarak, the analyst can compare and contrast perspective and viewpoint, as reflected by language frequency and use.