The swearing-in was held before Dewan Negara deputy president Doris Sophia Brodi at Parliament House.
Abu Zahar, 69, who is a former president of the Council of Former Elected Representatives (Mubarak), was reappointed senator on April 26 for a second term.
He was appointed president of the Dewan Negara on April 26, 2010, to succeed Datuk Wong Foon Meng whose term of office ended on April 12, 2010.
Speaking to reporters after the swearing-in, Abu Zahar expressed his determination to carry out his duties with responsibility, trust and dedication.
Abu Zahar started his career in 1964 as an inspector in the Royal Malaysia Police and the last post he held in the force was deputy superintendent in the Commercial Crime Investigation Department in Bukit Aman in 1979.
Between 1980 and June 1986, he was engaged in the legal profession in Kuala Lumpur prior to being elected as the state assemblyman for Pilah and appointed as a state executive councillor of Negeri Sembilan, in charge of housing and local government.
In 1991, he defended the state seat, after which he was elected as the MP for Kuala Pilah for one term, from 1995 to 1999.