Many Sunni residents, however, stayed and took their chances with the militant group members, also Sunni.
"In the beginning, the Daesh fighters would come to us and say 'We're here to support the Muslims, and you're our brothers,'" said village resident Ahmad Iqaab, using the Arabic acronym for the extremist group.
The militants, though known for enforcing strict living conditions and conducting extreme acts of violence, at times could be more respectful than government officials, he said. But danger was always present.
As the Iraqi government offensive launched Oct. 17 to retake the key city of Mosul from Islamic State has pressed forward, residents of villages in the surrounding area have been shifting from living under the control of extremists back to government rule.
Many residents have greeted the soldiers as liberators, recounting horrific stories of brutal treatment and deprivation.
On Friday, the United Nations said Islamic State appeared to be using citizens in the Mosul area as human shields. The U.N.'s human rights office had received reports of more than 200 people being killed for refusing to comply with the extremist group's orders or for having belonged to Iraqi forces, officials said.
Nevertheless, some residents have described a life under Islamic State, at least for Sunni communities, that at times seemed preferable to what the government had offered.
Iqaab said that in the first nine months of controlling Tob Zawa, the militants disbursed a monthly allowance of 100,000 Iraqi dinars, or roughly $85, to those who couldn't work, though it was later halved and halved again. Families would receive a sack of flour.
The militants were also respectful, especially compared with the Iraqi army under former Prime Minister
"If a Daesh guy came to buy a sheep from me and I didn't accept the price, he would apologize and say 'forgive me,'" said Iyad, an 18-year old shepherd who gave only his first name.
He contrasted the jihadists' behavior with the actions of the government troops who reclaimed Tob Zawa this week from the group.
"I was on the road when army soldiers saw me and offered me 25,000 Iraqi dinars for a sheep I had. I said it cost 100,000," he said. "But they threatened me with their guns, took the sheep, paid the 25,000 and left."
A friend, Abdul Karim Taqi Din, 17, squatted beside Iyad near a water point in the new refugee camp set up near Khazir, 12 miles from Mosul. The camp had about 1,000 people who were evacuated from Tob Zawa because of military operations.
"It wasn't a bad life," Din said. "If you didn't do anything against them [Islamic State], if you didn't violate their rules, they were very respectful."
Some of those rules, the refugees said, included being forbidden from wearing jeans and cutting short the traditional Arab dishdash robe to match the garb worn by jihadi fighters in Afghanistan. Women were forced to wear a full face covering.
Men were forced to attend five daily prayers, checking their names off a list posted on the door of the village mosque.
The group forbade television and satellite receivers, said Aqeelah, Din's mother. SIM cards and mobile phones were also not allowed.
"They told us there were no mobiles in the time of the Prophet [Mohammad]," Din said.
But there was a semblance of justice.
Iyad spoke with admiration about the time a militant had killed one of his guard dogs. When he had complained, the fighter was lashed 50 times and made to pay a fine.
"Their commander, they call him an emir [prince], if he hit you, you could go to complain and if it had been done without reason and you could hit him back in court," he said.
"If it was proven that their emir had stolen money, they would cut his hand. Even [Islamic State leader Abu Bakr] Baghdadi, if they had proof he had stolen something, they would punish him as well."
Islamic State espouses a harsh interpretation of Sharia or Islamic law, one which applies the hudood, a set of punishments that include amputations, beheadings and crucifixions.
The jihadists replaced the state curricula with one more in line with their ideology, posing math questions such as "how many IEDs do you have if you have three and bring three more?" or asking children to choose which wire to cut, some residents said.
"The whole idea is to condition children to the norms of life in the caliphate," said Aymenn Jawad Tamimi, an expert on militant groups with an archive of Islamic State documents. "It doesn't necessarily mean every child is intended to become a future jihadi warrior … but there's obvious indoctrination here."
Iqaab said he took his son Alaa out of school and hid him in the house, fearing the schooling would make him an Islamic State fighter.
"In the last two years, I couldn't even walk past the door. We couldn't let Islamic State know about me," said Alaa, a slight 17-year old with a wisp of hair on his chin and a mustache.
For him, life had improved.
"It's good to be outside," he said, smiling.
Bulos is a special correspondent.