We know about fates of most members of House Targaryen. Basically, what you have to consider first is that children of any female Targaryen weren´t Targaryens themselves, but members of different houses. So, even if women of this house that we know nothing of (there are few) survived into adulthood, their lines basically ended with them.
Now, what happened. There were quite a lot of Targaryens before Dance of Dragons (their civil war), but after that, only few women and two boys (Aegon and Viserys) remained. Aegon´s sons died childless and Viserys had only two, one of which became member of Kingsguard (Aemon the Dragonknight). His brother Aegon had son Daeron and daughter Daenerys and many bastards, but that was it. So, all Targaryens in the world that were born close to the end of second century and aferwards were children of one man.
So, Daeron the Good. He had four sons. Oldest, Baelor, died and during Great sickness, Daeron and his sons died too, leaving his second son, Aerys, to rule. But Aerys liked books more than women and died childless. Third son, Rhaegel, was mad and died before Aerys and his son Aelor died soon afterwards by accident caused by his sister-wife, who killed herself soon after that. He also had daughter, who was married to his brother´s son, Aerion. So, crown went to Maekar, fourth son who had four sons. Oldest, Aerion, was married to Rhaegel´s daughter. This was the guy who drank wilfire and died. His son, Maegor, was just a child when he was claimant to the throne, but nobody wanted him there, because he could be mad like his dad. Seeing that he is no longer mentioned, it can be safely assumed he died either during childhood or when he was young. Second son of Maekar was Daeron, who died of pox and had only kind of stupid daughter, who was also considered as a queen after her grandfather´s death, but nobody wanted her and was either married off to some other house, but most likely died in young age. Third son was Aemon, maester of Castle Black. So, after Maekar died, only his fourth son, Aegon, was able to carry on Targaryen name. Maekar´s daughters were married to other houses, so they aren´t important.
Aegon had three sons. Duncan chose to marry lowborn woman and gave up his claim. They had no children. Jaehaerys married his sister, Shaera and had Aerys "the Mad King" with her. Daeron was homosexual, who died in battle. Rhaelle was married to Ormund Baratheon, lord of Stormlands.
And now about Summerhall. It was summer seat of Targaryens and former seat of Maekar. In year 259, Aerys and his sister-wife Rhaella had a son, Rhaegar and all Targaryens came there to celebrate (this could include living children of Aegon´s brothers). Aegon tried some ritual to hatch dragon eggs that Targaryen still had and failed spectacularly. Only Jaehaerys, his wife, children and Rhaegar escaped.
Jaehaerys didn´t reign long and while Aerys had many children with his wife, they were either stillborn or didn´t live for long. Only Viserys and later Daenerys survived. Rhaegar had son and daughter with Elia Martell.
These were all living Targaryens at the time of Robert´s rebellion (who was actually rightful king in case of death of all Targaryens, as his grandmother was Targaryen), excluding Daenerys, who was not born at that time. Rhaegar died in battle, his children were killed (SPOILER: well, unless you consider Aegon who appears in 5th book to be his son). Viserys and his mother were brought to Dragonstone and she died in childbirth of Daenerys.
So, to answer your question, while there were many Targaryens throughout the history and their house was quite secure at the time of rebellion (Rhaegar, Aegon and Viserys, that´s three heirs in line), all of them were children of Aerys, because most Targaryens died in Summerhall.