Peter Stöger has left his role as Cologne coach with the side bottom of the Bundesliga and without a win in the league this season.
A club statement on Sunday said U19 coach Stefan Ruthenbeck will take over until the mid-season break at the end of the month.
"The decision to part ways with Peter Stöger was a difficult one for us to make," said club president Werner Spinner.
"Until recently, we had hoped that we could stay up in the league with this constellation including Peter, his staff, and the players."
The club said the decision to part ways was mutual.
Just minutes after Cologne battled to a 2-2 draw at Schalke on Saturday, local Cologne paper Express reported that Stöger had told his players he would be leaving.
Stöger, 51, went over to the away fans at the end of the game, removed his hat and applauded them.
It was his 168th and last game in charge of the club.
"It was clear on Friday that Schalke would be our last game. In order to concentrate fully on the game, we said we would communicate this on Sunday," Stöger said.
Worst ever start to a Bundesliga season
The Austrian joined Cologne in 2013 and guided them back into the Bundesliga as second division champions. He kept the side in the league and led them to a remarkable fifth-placed finish last season and a return to Europe for the first time in 25 years.
This campaign, though, has been a tough one. Cologne have gone 14 league games without a win — the worst ever start to a Bundesliga season — and find themselves eight points off second-last Werder Bremen and 11 points off safety.
Cologne, though, will qualify for the first knockout round of the Europa League if they win at Red Star Belgrade on Thursday.
Holstein Kiel boss Markus Anfang is favorite for the job long-term after leading the northern side from the German third tier to the top of the second division. However, the club say they will try to keep Anfang in Kiel.