President Obama said the conduct involving the Internal Revenue Service's targeting of conservative groups for extra tax scrutiny was "inexcusable."
"America has a right to be angry about it and I am angry," the president said.
He said the administration will accept the resignation of the acting IRS commissioner, Steve Miller, hold other responsible parties accountable and put in place safeguards to prevent such action happening in the future.
Earlier, the speaker of the House of Representatives raised the possibility of jail time on Wednesday for law violations in the growing scandal.
"My question isn't about who is going to have to resign, my question is who is going to jail over this scandal," John Boehner told reporters.
The House Oversight and Government Reform Committee will hold a hearing on May 22, according to an aide for Rep. Darrell Issa, chairman of the Republican committee.
Deputy Treasury Secretary Neil Wolin; former IRS Commissioner Doug Shulman; Lois Lerner, chief of the IRS tax exempt unit; and Treasury Inspector General for Tax Administration J. Russell George are being asked to testify.
The Justice Department began a criminal investigation Tuesday, while a number of congressional Republicans called for the resignation of acting IRS Commissioner Steven Miller.
On Friday, a separate House committee will hold the first hearing on the matter since it erupted last week. George and Miller will testify there.
"There are laws in place to prevent this type of abuse," Boehner said, referring to the IRS's giving extra scrutiny to requests for tax-exempt status by Tea Party and other conservative groups.
"Someone made a conscious decision to harass and hold up these requests," he said. "We need to know who they are, whether they violated the law. Clearly, someone violated the law."
This piece originally appeared at CNBC
Read more at CNBC:
House Speaker Suggests Jail Time for Violations in IRS Scandal