"The supervisory board has come to a mutual agreement with Leo Apotheker not to extend his contract as an executive board member," the company said in a statement.
Apotheker has been with SAP for over 20 years. In April 2008, he was named co-chief executive of the German software company with Henning Kagermann. He took over as the sole CEO after Kagermann's retirement in July 2009.
SAP has once again opted to return to a split leadership, with executive board members Bill McDermott and Jim Hagemann Snabe being appointed as new co-chief executives.
Apotheker's resignation caused SAP's shares to fall by more than 1 per cent in early trading.