Fr. William Casey, 76, was arrested on a fugitive from justice warrant from McDowell County, N.C. early Monday morning.
He is under investigation by the sheriff's department there for allegations by Warren Tucker, 44, who came forward publicly last week.
Tucker filed a report with the sheriff's department in September 2009 saying Casey brought him to North Carolina as a boy.
Tucker spoke with 6 News by phone from his home in Indiana Monday afternoon after hearing about the arrest.
"We feel kids are safer, and I feel they need to know that they are not alone and justice delayed is still justice, and it has really helped restore my faith in authorities and also these religious leaders," Tucker said.
Court officials say Casey agreed in court to be extradited to North Carolina to face the charges against him.
Until then he is being held in the Greene County Detention Center.
Tucker says the abuse happened in North Carolina among other places.
"It happened in the church in Kingsport, in his bedroom," said Tucker. "It happened in his car where he would pull over to some spot where he knew. It happened in Greene County in his rural cabin."
Tucker held a press conference in Knoxville on April 14 to announce that he was a victim of Casey from 1975 through 1980.
The next day, Bishop Richard Stika said Casey admitted to the abuse and was removed from the priesthood.
Casey served several places in East Tennessee, including Chattanooga, Greeneville, Rogersville, Farragut, Newport, and Kingsport, where Tucker's abuse took place.
Parishes in all those cities are now part of the Diocese of Knoxville.
The group called Survivors Network of those Abused by Priests (SNAP) encouraged Tucker to come forward in this case last week. Members of the group stood with him at the press conference.
On Monday, SNAP released a statement, saying, "We call on Knoxville's bishop to personally go to each parish where Casey worked and beg victims and witnesses to break their silence, get some help, contact law enforcement, and help protect kids."