The architectural feats of animals -- from beaver dams to birds' nests -- not only make for great nature television, but, since the plans for such constructions seem largely inherited, they also offer an opportunity for scientists to tackle the profoundly difficult question of how genes control complicated behavior in animals and humans.
A long term study of the construction of burrows by deer mice has the beginnings of an answer. Hailed as innovative and exciting by other scientists, the report, in the current issue of Nature, identifies four regions of DNA that play a major role in telling a mouse how long a burrow to dig and whether to add an escape tunnel.
The research could eventually lead to a better understanding of what kind of internal reward system motivates mice to dig, or tells them to stop. And, although humans don't dig burrows, that, said the leader of the three-person research team, Hopi E. Hoekstra of Harvard, could "tell us something about behavioral variation in humans."
Dr. Hoekstra, an evolutionary and molecular biologist, said the work, largely carried out by graduate students in her laboratory, Jesse N. Weber, now at the Howard Hughes Medical Institute at the University of Texas at Austin, and Brant K. Peterson, showed that "complex behaviors may be encoded by just a few genetic changes."
While other genes have been found in various species from worms to voles that govern various kinds of behavior, like mating and aggression, Dr. Hoekstra and her colleagues took on an unusually complicated behavior with an approach that involved nearly a decade of work on ecology and evolutionary biology as well as genetics. The result, said Cori Bargmann, who studies the genetics of behavior in roundworms at Rockefeller University, is "really exciting." She added that "it was done with great intelligence. The genetics are beautiful."
Robert Anholt, a specialist in behavior and genetics in fruit flies at North Carolina State University, said it was "courageous to undertake this particular work" for Dr. Hoekstra because of the great difficulty of dealing with complicated behavior, and that the approach was "very innovative" and pushed forward what was possible in behavioral genetics.
Dr. Hoekstra started with a species called the oldfield mouse (Peromyscus polionotus), the smallest of the deer mice. For 80 years or more, field scientists have documented its behavior, including excavating characteristically long burrows with an escape tunnel, which the mice will dig even after generations of breeding in cages in a laboratory.
Dr. Hoekstra treated tunnel length and architecture as a physical, measurable trait, much like tail length or weight, by filling burrows with foam that would produce a mold easily measured and catalogued – behavior made solid.
She and her students did this in the field and repeated it in the laboratory by putting the mice in large, sandbox-like enclosures, letting them burrow and then making molds of the burrows. They did the same with another deer mouse species, Peromyscus maniculatus, that digs short burrows without escape tunnels.
The team bred the two species together (they are close enough to interbreed) and measured the burrows of the offspring. Their tunnels varied in length. Further breeding crosses between the hybrids and the original short-burrow species were conducted and the tunnels measured again.They showed a blend of characteristics, varying in length and with and without escape tunnels.
Then the scientists matched variations in tunnel architecture to variations in DNA. What they found were three areas of DNA that contributed to determining tunnel length, and one area affecting whether or not the crossbred mice dug an escape tunnel. That was a separate behavior inherited on its own, so that the mice could produce tunnels of any length, with or without escape tunnels.
All complicated behaviors are affected by many things, Dr. Hoekstra said, so these regions of DNA do not determine tunnel architecture and length by themselves. But tunnel length is about 30 percent inherited, she said, and the three locations account for about half of that variation. The rest is determined by many tiny genetic effects. As for the one location that affected whether or not mice dug an escape tunnel, if a short-burrow mouse had the long-burrow DNA region, it was 40 percent more likely to dig a complete escape tunnel.
Both Dr. Anholt and Dr. Bargmann said that for complex behaviors, which can be affected in ways too small to measure by many other genes, the effects of these DNA locations were very significant.
These are, however, regions of DNA, not actual genes. Next comes the attempt to find the specific genes and then the pathways from genes to behavior. Dr. Anholt said "this is really only a first step," and that the next phase would be even more difficult. Dr. Bargmann said "the hardest thing about studying natural traits is that end game," getting from the region of DNA down to a particular gene.
But Dr. Hoekstra is confident and said the research that should lead to identifying the actual genes is already going on.
"We know exactly how to do it," Dr. Weber said. "We've always had the intention of finding these genes."
Correction: January 16, 2013, Wednesday
This article has been revised to reflect the following correction: An earlier version of this article misstated the characteristics of tunnels built by the first generation of crossbred mice. They all had escape tunnels; it was not the case that some did not have escape tunnels.
This article originally appeared in The New York Times.