Newborn boys weighing less than 3kg (6lb 10oz) could be at greater risk of infertility when they become adults, a Danish study suggests.
By their early 30s, 8.3% of men who were small at birth had fertility problems, compared with 5.7% of those born within normal weight range.
The researchers say poor growth in the womb could lead to poor development of sperm production and other organs.
About 12.5% of couples are affected by infertility - a third of cases are men.
Women with fertility problems make up another third while the remainder of cases are due to problems in both the man and woman, or of an unknown cause.
The study found that infertility in women was not linked to weight at birth.
Scientists from Aarhus University looked at more than 5,500 men and 5,300 women born between 1984 and 1987 and followed them into adulthood, until 2017.
Of those, 10% of men and women were born small for their age; 80% were a "normal" weight at birth and nearly 9% were born large.
Babies born at 40 weeks had a weight range of 2.5kg-4.5kg in this study. Babies in the bottom 10% were defined as being small.
Mothers of the low-weight babies had the highest level of cigarette and alcohol consumption, the study found, with smoking known to damage the foetus.
Two specific conditions, hypospadias (a malformation where the opening of the urethra is not at the tip of the penis) and cryptorchidism (undescended testis) were behind most of the male fertility problems.
Anne Thorsted, from the department of public health at Aarhus University, said it was not yet clear what lay behind the link between birth weight and infertility.
"A sub-optimal growth environment for the foetus, for whatever reason, could itself be detrimental to the development of sperm production and reproductive organs.
"It could also be speculated that the mother's health and lifestyle during pregnancy could affect both foetal growth and the development of reproductive functions. For instance, we know already that if the mother smokes, this can have an impact on the foetus."
She added: "Our results show that sometimes we must look at the very early life to find explanations of health problems that occur later in life."
Allan Pacey, professor of andrology at the University of Sheffield, said the study showed "an intriguing association".
"It's important to note that this observation does not demonstrate cause and effect, although it does fit the theory that one of the most critical parts of a man's life with regard to his fertility actually occurs before he was ever born.
"This is a bit hard to get one's head around as a concept, but there is a lot of evidence emerging to support the idea, to which this new study now forms an important part," he said.
The researchers acknowledge that the study, published in Human Reproduction, had some limitations.
The average age of people in the study was 32 so they had not yet reached the end of their reproductive years, and a follow-up study in another 10 years could change the conclusions.
Those classified as "infertile" had a diagnosis of infertility or were part of a couple seeking fertility treatment, but that included lesbian and single mothers, which may have skewed the findings slightly.
How to reduce the risk of having a small baby
- reduce or give up smoking
- don't use recreational drugs, especially cocaine
- eat healthily and lead a healthy lifestyle
Women are more likely to have a smaller baby if they are over 40 or have high blood pressure, kidney problems or diabetes complications.
Sometimes babies don't grow as well as expected for other reasons, such as:
- the placenta not working well
- an infection during pregnancy
- having a baby with a genetic or developmental problems