Palmisano will serve as a senior adviser until he retires from the company Dec. 1.
Rometty succeeded Palmisano as CEO in January, becoming the company’s ninth chief executive after holding senior leadership positions in IBM’s services, sales, strategy and marketing units.
She also joined the board of directors in January.
According to the company, she led the integration of PricewaterhouseCoopers’ consulting business and built a team of more than 100,000 business consultants and services experts.
Palmisano became CEO in 2002 and board chairman in 2003, with the company crediting him for increasing investments in analytics, cloud computing and other high-value businesses and technologies.
IBM created over $100 billion of total shareholder value during his tenure as CEO, the company said.