The broad scientific definition of inbreeding is that it is the mating of individuals more closely related to each other than the average relationship within the population concerned. This statement is really only valid for large populations, since with small populations inbreeding is inevitable, even with random mating. To be more precise, an inbred person is defined as someone whose parents are related. In practice this means close direct relationship or, more usually, related through recent common ancestors, since all members of the same species are related to some extent.

The adverse effects of inbreeding in animals are well known. The incidence of metabolic disorders, structural abnormalities and inherited disease conditions, caused by harmful recessive genes, increases following inbreeding. Performance in several characters, particularly those concerned with reproduction and survival, declines following the mating of close relatives. This is known as inbreeding depression. These effects are mainly due to an increase in the frequency of homozygous genotypes (AA and aa) at the expense of heterozygotes (Aa), which is caused by inbreeding. It is only harmful, however, when the dominance is directional, which means that the undesirable member of a pair of genes is usually recessive. When a high proportion of these harmful genes are present in the heterozygous state (Aa) the animal is protected from their debilitating effects by the dominance of the normal gene; but when some of the heterozygotes are replaced by homozygous recessives (aa), following inbreeding, their harmful effects become manifest. Other types of gene action are sometimes responsible for inbreeding damage, but are thought to be less important. These are: overdominance, epistatic interaction and the overall level of heterozygosity.

Overdominance occurs when the heterozygote (A1A2) is superior in performance to either of the two homozygotes (A1A1 or A2A2). In this situation, an increase in homozygosity following inbreeding also causes inbreeding depression.

Epistatic interaction between different pairs of genes occurs when one pair affects the expression of another pair at a different locus. With one particular type, called complementary epistasis, two dominants, one from each of two separate loci, are necessary for normal development or metabolism,Thus, AABB, AaBB, AABb and AaBb will be normal, but AAbb, aaBB Aabb, aaBb and aabb will be defective. This situation arises when a metabolic pathway requires two enzymes for the essential end-product to be synthesised. Since each enzyme requires a different dominant gene for its synthesis, the absence of one or both will result in a defective individual. Inbreeding in a population with a mixture of the above genotypes will lead to a break-up of the favourable gene combinations, with more inferior genotypes, particularly aaBB, AAbb and aabb, being produced.

Finally, Lerner (1954) found evidence that some abnormal conditions in animals were not caused by single genes but by a drop in the general level of heterozygosity throughout the whole genome. His theory of developmental homeostasis suggests that for an animal to be able to cope with developmental accidents and environmental stress there is a minimum or obligate level of heterozygosity for normal development. The implication being that heterozygotes in general are more versatile because they can produce a greater variety of enzymes and other proteins. This means that the heterozygosity level per se, as well as the effects of the genes themselves, may be a contributing factor.

The opposite of inbreeding depression is known as heterosis or hybrid vigour and can result from the crossing of unrelated inbred animals or lines with different genetic backgrounds. What is lost from inbreeding is usually restored when several inbred lines are crossed randomly. Deliberate inbreeding and crossing, followed by selection between lines, is sometimes used with farm plants and animals to improve yields. Its significance in humans is that greater mobility means that people travel further to find a spouse and are less likely to marry a person from the same locality with a similar genotype. Thus, although most of the increase in height and improved survival is the result of better nutrition and disease control, a small part may also be due to heterosis following a change in the mating system.