Joe Banister traded his agent’s badge for a crusader’s mantle. His critics call him a danger to taxpayers.

As seminar crowds go, the one at the hotel in Irvine was eclectic. Longhaired men in suits and women in flowing dresses jostled with elderly people in wheelchairs and walkers as they sampled organic foods and anti-tax paraphernalia.

But they were united as they scrambled for their seats when a slim, dark-haired accountant named Joe Banister got up to speak at the Health and Freedom Rally.

Banister is their kind of hero. “I’m a big fan of his,” said Robert Schulz, chairman of We the People, a Washington-based anti-tax group that helped organize the seminar.

For five years, Banister had been an IRS special agent, with a gun and a badge, sworn to uphold the tax laws of the United States. Then, in the mid-1990s, he began to have doubts about his calling.

He became convinced that the tax laws were unconstitutional. The 16th Amendment, which established the income tax, was improperly ratified by the states, he came to believe. And because the Internal Revenue Service doesn’t collect taxes in person, a legal principle called “voluntary compliance” should, he decided, make tax payments optional.

“There are laws that have been passed by Congress that do require some classes of people and/or entities to pay the federal income tax,” Banister said in an interview. “But that group of people is not as big or large a group as the American people have been led to believe.”

Banister’s theories have been discredited. The Supreme Court rejected the argument about faulty ratification in 1916, and other claims that the income tax is illegal have gone nowhere.

But schemes to dodge income taxes are proliferating on the Internet and at anti-tax seminars like the one at the Atrium Hotel in Irvine last month. And after a lapse in enforcement, the IRS is starting to crack down on tax violators like Banister and Irwin Schiff, author of “The Great Income Tax Hoax: Why You Can Immediately Stop Paying This Illegally Enforced Tax,” according to lawyers and others who track the agency.

“Schiff and Banister have been proselytizing their anti-tax philosophy for so many years that it has reached a boiling point,” said Elliott Kajan, a partner at Beverly Hills tax law firm Kajan Mather & Barish. “It is the government’s expectation that if the charges against them are proven, it will have a profound effect in communicating to others that the IRS now means business.”

IRS officials won’t say they have ever been soft on people who won’t pay taxes. But they do acknowledge that enforcement took a hit from budget cuts in the late 1990s that forced deep reductions in staffing. Recent gains in staff productivity, partly because of Americans’ increased use of the Internet for filing tax returns, have allowed the agency to free up additional resources for enforcement, IRS Commissioner Mark W. Everson said in recent testimony to Congress.

Criminal tax prosecutions in 2003 reached 1,353, a 32% increase from the year before.

Although the number of cases referred to the Justice Department for prosecution may still seem paltry, Dale Hart, an IRS deputy commissioner, pointed out that criminal prosecutions are a last resort. Most cases are either settled in civil court or resolved when the IRS sends tax violators a letter demanding that they pay up, said Hart, the IRS official in charge of individual taxpayer compliance.

“We take these people very seriously,” she said, referring to anti-tax authors, lecturers and others. “We prosecute them; we get civil injunctions against them; we spend time searching the Internet trying to figure out who they are and stop them from selling this kind of stuff.”

At the same time, Hart and others say, the IRS continues to concentrate its enforcement efforts on big-time tax cheats. After all, people who buy how-to books and videos to duck Uncle Sam tend to have smaller tax liabilities than those who hide money in offshore accounts or squirrel it away in complex tax shelters.

Armed With Credentials

Just how many people refuse to pay taxes as a kind of tax protest is unclear. Hart said the number was probably in the thousands. Schulz of We the People Foundation contends that there are tens of thousands — and perhaps hundreds of thousands — of Americans who thumb their noses at the IRS and refuse to file tax returns.

To this group, Banister is an icon, in large part because of his former career as an IRS agent.

“This person has credentials,” Kajan said. “That makes him a little more dangerous to the government than a person off the street.”

Banister, a graduate of San Jose State, joined the IRS in 1993 after spending several years in private practice as an accountant. His job as a special agent in San Jose was to chase down the most serious tax violators. This group includes drug traffickers and other criminals, which is why Banister packed a gun.

He said his conversion came after about five years on the job, while he listened to anti-tax activist Devvy Kidd on a talk radio program.

“At the time, I thought those ideas were kooky, outrageous,” said Banister, 41. “If I had heard her say those things in a store or a street corner, I would have dismissed it and kept going.”

He set out to prove Kidd was wrong and ordered her books to become more familiar with her views. And gradually, he changed sides. He found himself agreeing with those who contend that several of the state legislatures that ratified the 16th Amendment weren’t legally able to do so, leaving it shy of the required three-quarters majority.

What’s more, Banister came to believe that filing an IRS Form 1040 violates the 5th Amendment guarantee against self-incrimination, since it requires taxpayers to admit under penalty of perjury how much they have earned.

Banister put together a 95-page memo raising these and other issues, which he presented to his bosses for a response.

“Instead of answering my questions,” he recalled, “they asked me to resign.”

It was the beginning of a new career. He had already come into contact with many like-minded individuals while researching his memo, and they welcomed him into the lecture circuit. He flew to Washington to confront the IRS with other tax fighters and began to assemble an e-mail list through which he could keep his followers apprised about what was happening in the “truth in taxation” movement.

He turned his memo into a book titled “Investigating the Federal Income Tax” and starred in two videos claiming to show how the tax laws are illegal.

Banister, who practices what he preaches and doesn’t pay federal income tax, says he makes about $100,000 a year from the sale of books and videos, speaking gigs, representing tax clients in battles against the IRS and appearing as an expert witness in tax cases. That’s more than the $80,000 annual salary he pulled down with the IRS and, he said, enough to provide a comfortable living for him, his wife and two sons in San Jose.

“He’s a very decent and honest man,” said Aaron Russo, a Hollywood producer (his credits include “The Rose” and “Trading Places”) who is attempting to win the nomination to run for president on the Libertarian Party ticket. “He asked some questions of the IRS to make sure that what he was doing was right and lawful. They refused to answer his questions, and in one week he was gone. They let him go. I admire his courage. I respect the man.”

Seen as a Danger

Banister’s critics say he is a danger to taxpayers, who may be unaware that they risk fines and jail time by taking on the IRS.

“Tax returns signed under penalties of perjury should not represent an offer to negotiate legal theories, particularly theories that have often been held frivolous,” said Beverly Hills tax attorney Charles P. Rettig.

Banister himself was sued in U.S. Tax Court in San Francisco last year by the IRS, which wants to block him from representing tax clients, on grounds that he misrepresents the law and has failed to file his own returns. Judge William B. Moran granted the order in January; Banister is appealing it.

The IRS would not comment on whether further charges against him were planned.

“Banister’s assertions have been addressed by so many federal courts that they are no longer afforded the dignity of repeating the explanations as to why the claims are meritless,” Moran said in his decision. “The very significant problem with Banister’s advice to his clients is that it is absolutely wrong.”

In another high-profile case, the IRS last year sued Schiff to stop him from selling the “Income Tax Hoax” book and other writings, claiming that erroneous information in them had harmed taxpayers. The case, which has attracted some attention because of its 1st Amendment implications, is scheduled for a hearing April 26 in Las Vegas.

Even Schiff’s attorney appears to harbor doubts about his client’s reasoning.

“Schiff’s belief system appears to be completely circular: Within that system, Schiff is right, the government and courts are wrong, and he remains impervious to rational discussion,” attorney William A. Cohan said in a court filing.

The Justice Department indicted Schiff last month on tax fraud charges. Another of Schiff’s attorneys said his client would plead not guilty.

As it seeks to counter the arguments of tax protesters like Banister and Schiff, the IRS points to a spate of recent court rulings in its favor:

  • In September, Lanis R. Metteer was sentenced by a federal judge in Oregon to 33 months in prison for failing to file personal income tax returns and obstructing the IRS. Metteer also was ordered to pay the $13,884 cost of his own prosecution after claiming at trial that he believed the federal income tax was voluntary.
  • In January, the owner of Arrow Custom Plastics, Richard Simkanin, was convicted in federal court in Texas of 29 counts of willful failure to withhold taxes and submitting false or fraudulent claims for refunds. He is scheduled to be sentenced April 30.
  • In February, Ricky Paul Brunet, a Tennessee member of the Save-a-Patriot Fellowship, was sentenced by a federal judge in Nashville to serve 27 months for tax evasion. Brunet’s contention — like Banister’s — was that he could find no legal requirement that compelled him to pay taxes or file a return.

Banister said such court decisions didn’t bother him.

“I went to Catholic schools where they talk about the Ten Commandments,” he said. “One of them is that you shall not bear false witness. I couldn’t think of a more egregious violation of that commandment than to learn what I learned and keep quiet about it.”

By Kathy M. Kristof, Los Angeles Times Staff Writer