Bharara said in a tweet on Saturday "I did not resign. Moments ago I was fired."
The 48-year-old was appointed by former President Barack Obama in 2009 and was one of 46 prosecutors from the previous administration who were called on to resign by new US Attorney General Jeff Sessions on Friday.
Although it is customary for the 93 federal attorneys to step down once a new president in office, Bharara held a meeting with then President-elect Donald Trump at New York's Trump Tower shortly after his election win, and was asked to stay on in the position.
US media said the abrupt nature of the resignation call came as a surprise - given that so many were asked to leave at one time.
The decision has raised questions about whether the Trump administration would be hindered in its ability to enforce the nation's laws. Career attorneys are to carry on the work until new US attorneys are put in place, the Justice Department said.
Bharara's Manhattan office handles some of the most critical business and criminal cases passing through the federal judicial system. His purview thus included Trump Tower, where the president's sons now run his businesses. It was also at the center of a baseless wiretapping claim President Trump made against his predecessor Barack Obama last week.
He has successfully prosecuted over a dozen state lawmakers, Democrats and Republicans alike.
Time magazine famously lauded Bharara on its cover as the man who is "busting Wall Street," after he handled a string of high-profile corruption and white-collar criminal cases.
Earlier, a source for Bharara had said he was taking Trump at his word and would not comply with Session's request to resign.
But Saturday's tweet suggests he was not given a choice.
He went on to say in the tweet that being the US Attorney in the Southern District of New York will "forever be the greatest honor of my professional life."
mm/jm (AFP, AP, Reuters)