Ban was unharmed as he ducked behind a podium after a rocket or mortar round landed near the prime minister's office while the two men were holding a news conference.
VIDEO: U.N. chief ducks behind podium
Small chips of debris floated down from the ceiling above the U.N. chief after the explosion rattled the building in the Green Zone.
Ban will be in Baghdad for only one day of meetings with Prime Minister Nouri al-Maliki, whose forces are working with the U.S. military in a major operation to curb violence in the Iraqi capital and the surrounding territory.
The U.N. officials, who spoke on condition of anonymity because they were not authorized to release the information, would give no other details about the visit.
They said Ban boarded a helicopter immediately on arrival at Baghdad airport and flew to the Green Zone, a heavily fortified area in the heart of the city that houses the U.S. Embassy as well as the offices of al-Maliki and Iraq's parliament.
The top U.N. official was last in Baghdad when Ban's predecessor, Kofi Annan, visited the capital in November 2005.
Meanwhile, Saad Yousif al-Muttalibi of the Ministry of National Dialogue and Reconciliation said talks with Sunni insurgent groups were initiated at the request of the insurgents and have been taking place inside and outside Iraq over the past three months.
He refused to identify the groups, but said they did not include al-Qaeda in Iraq or Saddam Hussein loyalists. Members of the former president's outlawed Baath party took part, he added.
Speaking to The Associated Press in a telephone interview Thursday, al-Muttalibi said the negotiations were deadlocked over the insurgent groups' insistence that they would lay down their arms only when a timetable for the withdrawal of U.S.-led coalition troops in Iraq is announced.
The government's response was that such a move could only be taken when security is restored.
Future rounds of negotiations are planned, he said, but did not elaborate.
Al-Muttalibi's comments came one day after he expressed optimism in an interview with the British Broadcasting Corp. that Prime Minister Nouri al-Maliki's government was making progress in talks with insurgent groups, predicting some factions might be close to laying down their arms.
"One of the aims is to join with them in the fight against al-Qaeda (in Iraq)," he told the BBC.
Reports have periodically surfaced in the past three years of talks between Iraqi and U.S. authorities and representatives of Sunni insurgent groups, but details of the contents of these negotiations and whether they made any progress always have been sketchy.
Groups said to have taken part in such talks often denied their participation in statements posted on the Internet.
A senior al-Sadr aide, meanwhile, was released from U.S. custody Wednesday and handed over to al-Maliki, according to the military. Ahmed al-Shibani was captured in the holy city of Najaf during clashes in 2004 between U.S. forces and Mahdi Army militiamen.
The government has released a photograph showing al-Maliki receiving a smiling al-Shibani at his office, a treatment that underlines the close ties between the prime minister and radical Shiite cleric Muqtada al-Sadr, whose support won al-Maliki his job last year.
In Basra, authorities slapped an indefinite curfew in the southern city following clashes Thursday between militiamen loyal to al-Sadr and guards outside the headquarters of the rival Shiite Fadhila party.
Police said the militiamen captured eight Fadhila members before the building caught fire. Twelve Mahdi Army fighters were wounded in the gunbattle, they added.
Clashes also erupted between the two sides near the residence of Basra's Fadhila governor Mohammed al-Waeli and continued into the afternoon, police said.
In the volatile city of Baqouba, located northeast of Baghdad, the bullet-ridden body of a kidnapped local official was found dumped on a city street Thursday, one day after masked gunmen stormed her house and took her away handcuffed, police said.
Ilham Namik Shahin, 43, was a Shiite member of the Baqouba provincial council. Her brother, Najah Namik Shahin, said 10 gunmen stormed the family home Wednesday night, ordered everyone into the living room before they handcuffed his sister and left with her.
"It took just 10 minutes and we were really scared. We couldn't talk," said the brother.
Baqouba, located in Diyala province 35 miles northeast of Baghdad, has seen an upsurge in violence and sectarian killings in recent weeks, with Sunni insurgents loyal to al-Qaeda in Iraq stepping up attacks as violence appears to ebb in Baghdad since a security push began Feb. 14.
The clashes in Basra, 340 miles southeast of Baghdad, followed the arrest earlier Thursday of a Mahdi Army militiaman by policemen thought to be loyal to Fadhila.
The arrested militiaman worked for the city's electricity company. His comrades retaliated by kidnapping a colleague of his known to be a Fadhila supporter, according to police. His body was later found, police said without giving details.
Thursday's clashes came days after British forces pulled out of their main base in the heart of Basra, Iraq's second city. Capt. Katie Brown, a British military spokeswoman Basra, said she had no information on the fighting.
Clashes between rival Shiite groups in the mainly Shiite south of Iraq are not uncommon. They reflect the competition for power between major parties, including those in al-Maliki's government.
The rivalry in the oil-rich region poses the most serious threat to the unity of Iraq's majority Shiites.
In the latest episode of Shiite-Shiite violence, three mortar shells hit the headquarters of the Supreme Council for the Islamic Revolution in Iraq, or SCIRI, Thursday in Kut, 100 miles southeast of Baghdad, police said.
They reported no casualties and did not say who might have been behind the attack.
In Dujailah, a town near Kut, about 2,000 angry mourners marched with the coffins of three Mahdi Army militiamen gunned down Wednesday by unknown assailants, witnesses reported.
The Mahdi Army is loyal to al-Sadr, whose loyalists have 30 of parliament's 275 seats and six Cabinet posts. His Mahdi Army has in the past clashed with gunmen loyal to SCIRI, the senior partner in al-Maliki's governing coalition, and other Shiite groups.
Ban will be in Baghdad for only one day of meetings with Prime Minister Nouri al-Maliki, whose forces are working with the U.S. military in a major operation to curb violence in the Iraqi capital and the surrounding territory, U.N. officials said.