"I saw two people dead in front of the school, an adult and a child ... Inside, it was a vision of horror, the bodies of two small children," one father, searching for his son at the Ozar Hatorah school among crowds of distraught parents and children, told RTL radio.
The gunman killed a 30-year old Hebrew teacher, his two children aged three and six, and another child, Toulouse prosecutor Michel Valet said. A 17-year-old was wounded.
"The attacker was shooting people outside the school, then pursued children into the school, before fleeing on a heavy motorbike," Valet told reporters.
The soldiers, one of Caribbean and two of Muslim origin, also been killed in drive-by shootings and prosecutors opened an anti-terrorism investigation into all three attacks although it was not clear whether the motive was political or purely racist.
President Nicolas Sarkozy and Francois Hollande, the Socialist opposing him in his uphill bid for re-election in May, both rushed to the scene.
Sarkozy said he was struck by the similarities between the three attacks.
"We don't know who this killer is and what is the exact link is to the drama that has hit the military community," Sarkozy said after arriving in Toulouse. "Everything must be done so that the killer is stopped and has to pay for his crimes."
"Our schools must keep functioning, our compatriots that want to worship at synagogues, mosques and churches must be able to continue to do so. We should not give ground to terror."
The prosecutor confirmed that the same calibre firearm was used in Monday's attack as in the killing of the three soldiers in two separate attacks by a man who escaped on a scooter. He could not say if it was exactly the same weapon.
Police cordoned off the school and a spokesman for the interior ministry said security was being tightened at all other Jewish schools in the country. France's 600,000-strong Jewish community is Europe's largest.
There was mayhem around the small school in a leafy upscale neighborhood of Toulouse, a booming industrial town. "All the children at this school were my children," one tearful mother told LCI television.
The father looking for his son expressed disbelief: "How can they attack something as sacred as a school, attack children only 60 centimeters (two feet) tall?" he told RTL radio.
As messages of condolence poured in from across Europe, representatives of France's Jewish community voiced their solidarity.
"The whole Jewish community is in mourning," said Rabbi Moshe Lewin, a spokesman for France's great rabbi. "In the face of such a drama, such a horror, one cannot but go there."
Public prosecutor Valet said investigators were studying video evidence from the school shooting and the attack on Thursday in the nearby town of Montauban that killed two soldiers and left a third seriously injured.
The three men, aged between 24 and 28, were shot while in uniform as they tried to withdraw money from a cash machine close to the barracks of the 17th parachute regiment.
A third soldier, aged 30, was killed the previous weekend in Toulouse. Investigators said the same weapon had been used in both incidents.
The shootings could thrust security back to the top of the agenda in a bitter electoral campaign that has been dominated by issues of taxation and immigration.
Political analyst Stefane Rozes, head of CAP political consultancy, said the shooting was unlikely to have a decisive impact on France's election campaign as all candidates were strongly condemning the violence.