Validity serves the main goals of any science - knowledge and power.
Today it is often argued that the concept of validity is unique. But an evaluation criterion or evaluation procedure is valid or invalid, and nothing more. Verification methods are varied and numerous, some of which are theoretical or constructive, while others are pragmatic or simply empirical.
Validation techniques are very common and their number is growing rapidly. But, there are only two fundamental types of validity, which are otherwise interdependent: theoretical and practical. They correspond to the two fundamental goals of science: knowledge and power. On the one hand, to cognize reality, explain and understand it, that is, to take into account one aspect of reality (explain it) in connection with other aspects (understand them).
On the other hand, to be able to intervene in reality in order to dominate it to some extent, it is better to adapt to it or modify it so that it better suits our purposes. These are two different goals, but not independent.
In psychometrics, validity is understood as the degree to which an instrument or procedure measures what it is intended to measure. In other words, a measurement method will be valid as long as it effectively reflects the psychological variable it is aimed at. This variable is assessed by its relationship with a certain criterion, for example, results in other variables or related tasks. In this sense, different types of validity can be considered: predictive validity, convergent validity, construct validity, etc.
Validity is a judgment or assessment of how well a test measures (assesses) what it is intended to measure in a given context. Specifically, it is making evidence-based judgments about the appropriateness of differences made on the basis of test results.
Validity ensures that what is being measured is what is being proposed and not something else. A test is considered valid if it meets our purposes.
Validation is a process that allows a measurement “instrument” to be increasingly refined, constrained and improved. Thus, a measurement theory test will be valid if it actually measures, for example, the knowledge that students have in the subject and not (perhaps unintentionally) something else.
Finally, validity term refers to the extent to which a test measures what it intends to measure. In this sense, a test will be suitable for measuring spatial reasoning, for example, if it measures this type of reasoning and not something else.
Main types of experiments in psychology
Experiment (Latin “test, experience”) is the leading method of obtaining scientific knowledge, the purpose of which is to identify cause-and-effect relationships. It is characterized by the creation of optimal conditions for the study of mental phenomena and targeted and controlled changes in these conditions.
The experiment is characterized by: the active position of the researcher himself. Can cause a mental phenomenon as many times as necessary to test a hypothesis (creation of a pre-thought-out artificial situation in which the property being studied is best assessed). In experimental studies, it is important that the subjects are equal in age, health, motivation to participate, etc.
The reliability of the hypothesis being tested is achieved either by repeating experiments many times, or by having a sufficient number of subjects followed by mathematical processing.
Laboratory - carried out in specially organized conditions, different from real ones: a- using equipment; b- using equipment.
A natural experiment is one conducted under real-life conditions in which the researcher deliberately varies some of these conditions. Alexander Fedorovich Lazursky contributed to the methodology for organizing a psychological and pedagogical natural experiment.
To eliminate the negative influence of laboratory conditions on the subject, an experiment was developed, which is carried out in the natural conditions of the group, a training workshop - 3. Field E. Depending on the nature of the researcher’s actions, a distinction is made between ascertaining and design experiments.
Statement involves determining the actual state and level of certain features of mental development, as well as the relationship of cause and effect.
Formative - active formation of the studied trait (in accordance with the hypothesis) in the process of specially organized experimental training and education. Depending on the degree of development of the problem, experiments are distinguished:
Research (exploratory) with the aim of obtaining fundamentally new results in the area under study. Such experiments are conducted when it is unknown whether a cause-and-effect relationship exists between H and ST, or when the nature of the dependent variable is uncertain;
Elucidative (confirmatory) experiments that reveal the nature of the functional quantitative relationship between independent and dependent variables. Their purpose is to determine the limits within which a particular theory or law operates. The conditions, methodology, and subjects of the study usually differ from the original experiments;
Critical, organized with the aim of refuting an existing theory or law with new facts;
Reproductive (confirming), i.e. exact repetition of the experiments of previous scientists to establish the validity, reliability and objectivity of their results. Research algorithm:
A hypothesis is formulated about a qualitative cause-and-effect relationship between A and B.
If the hypothesis is not confirmed, another qualitative hypothesis is put forward and a new exploratory experiment is conducted; If the qualitative hypothesis is confirmed, a quantitative functional hypothesis is put forward.
The hypothesis about the nature of the relationship between the variables is accepted (or rejected) and clarified.
A pilot study conducted before the main work begins helps identify missing categories, missed units of analysis, and inaccurate instructions. Pilot study (the area under study is unknown, there is no system of hypotheses).
The concept of psychodiagnostics
Psychodiagnostics is an applied discipline of psychology that studies and uses for practical purposes methods of classification, ranking and measurement of psychophysiological and psychological characteristics of people.
Its main methods are:
- experiment;
- observation;
- survey;
- testing.
Factors influencing the internal validity of an experiment
Internal validity is a type of validity, that is, the degree to which the independent variable influences the dependent variable. The greater the influence of uncontrolled conditions on the change in the dependent variable, the lower the internal validity and the greater the likelihood that the facts discovered in the experiment are artifacts.
The main factors that reduce the internal validity of an experiment are:
Change over time (dependence of the subjects and the environment on the time of day and season, changes in the subject himself - aging, fatigue and distraction during long-term studies, changes in the motivation of the subjects and the experimenter, etc.; see the section “Natural evolution”). Need to retest - the actual image may not be identical to the real one.
The sequence effect is the influence of one experiment on the next; the influence of one condition of one experiment on the next.
Rosenthal (Pygmalion) effect. Draws conclusions about the influence of experimenter bias on the outcome of the study. The influence of the researcher's biases, expectations, and previous experiences on the outcome of the study. This can happen at any stage of research and in any science: when conducting an experimental procedure, when processing results, when interpreting research results, etc. It is impossible to study groups that are biased in any way.
The Hawthorne effect is triggered when the subject tries to please the experimenter. This is a condition in which novelty, interest in the experiment, or increased attention to the subject lead to a biased, often overly favorable result. Experimental participants behave differently, more carefully than usual, simply because of the awareness that they are participating in an experiment.
Placebo effect. A placebo is a physiologically inert substance used as a medicine, the positive therapeutic effect of which is associated with the unconscious psychological expectation of the patient. A placebo has no direct effect on the condition for which it is prescribed.
The term “placebo effect” refers to the real phenomenon of non-drug effects not only of drugs, but also, for example, radiation (sometimes with various “flashing” devices, even “laser therapy”, etc.). Testing the effectiveness of some psychotherapy methods.
Example - get rid of the fear of speaking in front of an audience: Psychoanalysis 60% Behaviorism 86% Medicine 56% Cognitive 50% Placebo 50% (group not exposed to any influence - buffer group).
The audience effect is the influence of the presence of an audience member on human behavior.
The first impression effect is the influence of the opinion about a person formed by the subject in the first minutes of the first meeting on a further assessment of the activities and personality of this person. added Khudyakov.
The Barnum effect is the general observation that people place extreme importance on the accuracy of descriptions of their personality that they believe are tailored to them, but which are actually so vague and generalized that they could just as easily be applied to others. to many other people. The Barnum effect partly explains the enormous popularity of astrological charts, palmistry and various pseudosciences. The less specifics, the more faith. People tend to accept the most general descriptions as truth.
Confounding is a threat to internal validity in which the dependent variable is usually influenced by a lot of irrelevant stimuli that you basically can't get rid of. The researcher's job is to determine the extent to which these stimuli affect the dependent variable in order to be confident that changing the independent variable of interest to the experimenter will have an effect on it.
Sampling factors. Highlighted by Kepbel.
Incorrect selection (inequivalence of groups in composition, leading to systematic errors in the results).
Experimental attrition (uneven exclusion of subjects from comparison groups, resulting in groups that are not equivalent in composition). The first sample must be larger than the next
The history effect is the events that occur between the first and second meetings between the subject and the experimenter.
Content
- 1 Details
- 2 Examples of threats 2.1 Ambiguous time priority
- 2.2 Confusing
- 2.3 Selection criteria
- 2.4 History
- 2.5 Maturation
- 2.6 Retesting (also called testing effects)
- 2.7 Changing tools (instrumentation)
- 2.8 Regression to the mean
- 2.9 Mortality/differential loss
- 2.10 Interaction between selection and maturation
- 2.11 Distribution
- 2.12 Compensatory rivalry/resentful demoralization
- 2.13 Experimenter bias
Factors that violate the external validity of an experiment
Validity is the validity (or degree of reliability) of a conclusion drawn from the results of a real experiment in comparison with the results of an ideal experiment.
The validity of a study characterizes the reliability of its results. There is internal and external validity of experimental studies. Internal validity is the extent to which the fact established in the experiment reflects the true cause-effect relationship. Campbell internalB- whether the experimental effect (NP) actually led to changes in a particular experiment (EO).
External - the validity of conclusions drawn from the results of a real experiment in comparison with the results of a full agreement experiment. An increase in external variability is ensured by achieving agreement between the levels of additional variables in the experiment and the levels of these variables in the reality under study. An experiment that does not have external validity is false and does not correspond to the established hypothesis (but may be suitable for testing another hypothesis).
Factors that violate the external validity of an experiment: Campbell identifies the following main reasons for violating the external validity:
Experimental effects - a decrease or increase in the sensitivity of subjects to experimental influences as a result of the experiment. For example, pretesting students' knowledge can increase their interest in new learning material. Because the general population was not pretested, results may not be representative of the general population.
Interference of experimental influences (mutual overlap of experimental influences).
Subjects have memory and learning abilities. If an experiment consists of several series, then the first influences do not pass unnoticed for them and influence the occurrence of effects from subsequent influences - residual “traces” of previous experimental influences - “learning”.
Study conditions as a factor that caused an inadequate reaction of subjects to participation in the study. They force subjects to respond to the experiment. Therefore, his data cannot be applied to subjects who did not participate in the experiment; these subjects are the entire general population, except for the experimental sample.
Influence of interaction between selection factors and experimental content. Their effects are artifacts (in experiments with volunteers or subjects participating under duress).
Most reasons for the violation of external validity are due to the characteristics of psychological experiments with people that distinguish psychological research from experiments in other natural sciences.
An artifact (lat. arte factum - artificially produced) is a research result that is a consequence of a change in the dependent variable under the influence of secondary variables.
An artifact is a consequence of errors or insufficient control of the research conditions. The same phenomenon can be an artifact in one experimental design and a fact in another, so phenomena that cannot be explained by the accepted theory are often considered artifacts.
Thus, the main features of the experiment as a basic research method are the identification of dependent, independent, external variables; formation of control and experimental groups; control of validity, especially preliminary planning of the experiment. An element of planning is the choice of a specific research design.
Operational validity is a type of external validity for laboratory experiments; the congruence (or degree of congruence) of the methodological procedures used with the theoretical concepts contained in the experimental hypothesis.
A special form of external validity is ecological validity, which determines the ability to extrapolate the conclusions of a particular study to real-world conditions rather than to other laboratory conditions;
Recommendations
- Brewer, M. (2000). Study design and validity issues. In Reis, H. and Judd, C. (eds.) Handbook of Research Methods in Social and Personality Psychology. Cambridge: Cambridge University Press.
- ^ a b
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference Boston: Houghton Mifflin. - Levine, G., & Parkinson, S. (1994). Experimental methods in psychology. Hillsdale, NJ: Lawrence Erlbaum.
- Libert, R. M., & Libert, L. L. (1995). Science and Behavior: An Introduction to Psychological Research Methods. Englewood Cliffs, NJ: Prentice Hall.
- Wortman, P. M. (1983). “Evaluative Research—A Methodological Perspective.” Annual Review of Psychology
.
34
: 223–260. Doi:10.1146/annurev.ps.34.020183.001255.
Psychometric properties of psychodiagnostic methods
These include reliability, validity and representativeness.
The reliability of the methodology is a quantitative and statistical indicator of the stability of the results, which is measured using various correlation coefficients. It, however, does not answer the question of what mental property is being measured.
This information can be obtained by examining the validity indicator. It is also measured using correlation coefficients, but more theoretical and practical considerations are taken into account.
In psychology, validity is the compliance with the objectives of the developed methods and the results of their research.
Its important feature is the impossibility of establishing and using this evidence once as the final argument “for” the quality of a technique or experiment. By considering the compliance of the research results with the stated objectives, one can judge the degree of validity. But it is not measured in any conventional units.
The test of representativeness is often reduced to the normal distribution of test scores. But this can be considered a special case, a sufficient condition, but not necessary.
Today, psychometric properties are considered the main sign of scientific validity. In global psychodiagnostics, this is a prerequisite for obtaining a certificate from national testing commissions as a technique suitable for practical use.
Theoretical and pragmatic validation
Validation is a test of the validity of a method. Divided into theoretical and pragmatic.
Theoretical validation refers to the technique itself. The researcher is interested in the property that is being measured.
Pragmatic validation refers to the purpose of using a technique. The emphasis is on demonstrating that what the technique measures is related to specific areas of practice.
