Several Websites are now offering to help you separate the purveyors of hot tips from the blowers of hot air. Whether you tune in to the talking heads on Wall Street Week, the gurus at Money magazine, or financial chat room chatterers, you can track the performance of their tips on one of these sites.

If it sounds like a great idea, not everyone is thrilled with its execution. "I think it's a bullshit product," says James J. Cramer, cofounder of TheStreet.com and hedge fund manager at Cramer, Berkowitz, and Co. "I think it's stupid."

Cramer's peeve is with Validea.com (formerly Reesegroup.com). Validea is the first comprehensive, ongoing effort to hold investment gurus' feet to the fire. The site's founders (and company president and CEO, respectively), Keith M. Ferry and John Reese, have taken on the monster task of tracking the picks and pans of virtually anyone who acts the part of a professional stock-picker. "We've been researching for over four years and have 55,000 recommendations in our database and another 20,000 that we've discarded because they were either for mutual funds or because there was no clear recommendation--no way to say whether this author intended it to be a positive or negative investment idea," says Ferry.

To keep that database current, the 20 researchers at Validea.com search financial Websites and magazines every day to see if any of the 980 authors (on-line and print), 4,000 analysts, 275 fund managers, 12 magazines, 15 on-line sources, and 225 columns they cover have made a recommendation, pro or con. Every new stock suggestion becomes part of the source's portfolio of "Investment Ideas," with long positions in picks, short ones in pans. Cramer's "portfolio," for instance, consists of over 30 stocks he's mentioned in his column, "Wrong!," which he posts on TheStreet.com as many as seven times a day.

The mentioned stock's performance, Ferry continues, is measured for one week, one month, three months, six months, and a year, from the date the recommendation is actually in the hands of the average subscriber. In the case of Ohm=, that means Validea would begin tracking suggestions appearing in the June issue sometime in May.

Once a stock becomes part of a recommender's portfolio, it stays there unless and until an offsetting pick or pan is issued. "The one exception to that would be that if a pick loses 50 percent of its value, for example, we assume that the person that picked that stock would sell," Ferry explains. "It's almost like we're saying, Hey, the individual investor is not going to sit there and wait for the person to tell them to sell."

The site offers two channels. If you wanted to evaluate an expert's advice, you'd surf the "Top Ranked Stock-Pickers" channel. If you wanted to see what the pros thought about one of your investment ideas, you'd simply change to "Most Talked-About Stocks."

For example, to see what James J. Cramer is complaining about, you'd click on "Personalities" in the stock-picker's channel. On the day I checked him out, I learned that Validea rated him the 15th-best picker over a three-month period among personalities (apparently, people with star power on Wall Street), with an average return of 11.04 percent on 274 picks. I also discovered that he had a 21.52 percent return over a one-year period on 122 picks, good enough for a ranking of four lightbulbs out of a possible five (the site used to use stars but switched when it changed its name). And I watched his name scroll by on the right side of the page under a banner declaring him "today's top-ranked stock-picker," ahead of runners-up Morgan Stanley Dean Witter analyst Mary Meeker and David Kugler, president of Monument Funds Group. Not bad company. Finally, I noticed that "Wrong!" was ranked fifth among all on-line sources, behind two other TheStreet.com columns and two MSN MoneyCentral columns.

To validate your own picking prowess, you'd try the other channel. Just enter your stock pick--PMC-Sierra (PMCS), say--in the Quick Search box and select Validea Report from the dropdown box. Up comes an array of five tabs that will give you at a glance what the professionals are saying about PMCS. From recent recommendations to current buzz, from pros to investment hobbyists (the "community"), you can see whether anyone agrees with you. In the case of PMC-Sierra, eight top-ranked analysts back your judgment.

If that's not enough for you, click on the guru-analysis tab to see how PMCS would fare under the microscope of investment greats (both living and departed) Peter Lynch, Benjamin Graham, William O'Neil, and six others. Validea has reduced the philosophies of each of these giants to a set of formulas into which it pours the subject company's fundamentals to determine whether the financial legend in question would invest in it. In PMCS's case, a momentum investor, as defined by William O'Neil, would have "strong interest"; the others, no interest at all.

There's no doubt these are very cool and potentially very useful tools. What some people question is how well they really work. One issue is how the site determines what constitutes a pick or a pan. This is easy, says Ferry, when a recommender "actually says something like 'I like IBM and would recommend buying it as part of my portfolio.' " The task is harder with implied recommendations. In such cases, Validea researchers weigh the positive and negative comments in a magazine article or analyst's report, trying to decide if a reader would interpret a mention as an investment idea. "We're not going to claim 100 percent accuracy," Ferry admits. "But if a researcher misconstrues a recommender's comments, we have a review committee that constantly goes back and audits the review."

Still, mistakes do slip through. In September 1999, for instance, Matthew Stichnoth, now a Bloomberg Personal contributor, wrote a "Screening Room" column for Individual Investor in which he called Premark International (PMI) a "screaming bargain." A Validea researcher labeled Premark a pick and placed it in Stichnoth's portfolio. Fair enough. However, the same researcher also added to the same portfolio eight other picks based on a chart at the bottom of the page that was actually associated with (and directly below) a column by David Lindorf, who got credit only for Crane (CR), the stock mentioned in his column. Stichnoth earned five lightbulbs for his performance, Lindorf just three.

Acknowledging the possibility of human error, Ferry still maintains that the site's methodology is "absolutely solid, bulletproof," in large part because Validea is open to input from recommenders who feel their ratings have fallen unfairly. "We want to work with authors, editors, et cetera, as much as possible, so we can improve our researching process together," he states.

What upsets Cramer is that Validea treats any of the stocks he writes about as picks or pans. "My whole goal with my column, as I've said a gazillion times, is to be a diary of what I do," he explains. "As I said today in my column, if you go and buy blah, blah, blah stock because you read that I'm long in it, I hope you lose money." If that's not enough, just read the disclosure at the bottom of his column: "Under no circumstances does the information in this column represent a recommendation to buy or sell stocks."

Well, that depends on your definition of "recommendation." What would you have thought when Cramer wrote about Intel (INTC) at 7:38 a.m. ET on January 13, 2000 that "we, like everyone else, remain long [in] it because, well, we have a habit of owning great companies that over the long term have delivered terrific results"? Validea researchers considered Intel a pick, in spite of the disclosure.

Cramer and others can argue that they warn their readers not to treat their stock mentions as picks, but they surely can't think that their readers are searching investment columns for tips on gardening. "You're supposed to be the expert," Ferry points out. "You're the one writing the article, and I'm coming to you for some quality advice or at least some quality insight because I respect your opinion or you were quoting somebody from a company like Sprint or AOL. I would rely on that information."

Validea's undertaking of holding experts accountable to the public they advise does have its fans among the professional class. Among them you can count Jon D. Markman, a four-bulb-rated personality and author of Online Investing--How to Find the Right Stocks at the Right Time (Microsoft Press). "It's never been done before. It's very ambitious," says Markman, who receives a top rating for his on-line column, "Supermodels," found at MSN MoneyCentral. "I'm not sure whether they can do it successfully or not, but it's fantastic that someone is actually taking a stab at it."

Ambitious is an understatement. Ferry hopes to hire an additional 45 researchers and developers by midsummer so that he can add to the existing coverage, expanding into TV and radio programs, broker reports, newsletters, and even computerized financial models like the stock screens you can find at MSN MoneyCentral and Quicken.com. "We want to hold them accountable as well," he explains.

The professionals, of course, aren't the only ones dispensing advice. Amateurs of varying degrees of competence and legitimacy spout off in financial chat rooms throughout the Internet. Discovering who among them is worth listening to may be even harder than evaluating the pros. Enter iexchange.com, a nifty little gathering place for nonprofessionals eager to share investment advice--often for a price--while competing for a place among the site's top 10 best analysts. Okay, it's an on-line karaoke bar for stock-pickers. But boy, can these pickers sing, though their repertoire is a little heavy on small-cap and technology issues, with a sprinkling of penny stocks.

One of the best is Troy Pillow, "ashlyn-rose" to the iexchange investment community. On the day I spoke with him, his 68 percent average return was good enough to place him number 5 among 336 analysts with 10 or more picks; a few days later, however, he fell to number 17, with an average return of only 33 percent. All the returns on iexchange are calculated from the day the pick is made to the day it closes--the date the picker specifies or when it hits its target price, whichever comes first. For example, ashlyn-rose picked AOL at $70 on January 7, 2000, and declared that the shares would hit $100 by February 7. Looks like that number 17 ranking is going to take another hit.

All these folks take their research and rankings very seriously. "I read literally a thousand pages of investment material a week," declares Pillow, a factory worker from Paragould, Arkansas. "I also get a lot of information off of iexchange.com; I check news articles and check the company summaries; and if I can, I'll find the e-mail address of somebody at the company and question them."

The iexchange info Pillow refers to consists of investment reports prepared and posted by the analysts who use the site. These contributors "come from all walks of life," says David Eisner, president and CEO. "There are professional traders and newsletter writers who post on our site. There are retired computer salesmen, doctors, and guys who sit at home and day trade."

Recent reports ranged in size from 20 words of wisdom by "Sure Pick" on Taiwan Semi (TSM) to 329 words by "dukie 1993" on the virtues of Cisco Systems (CSCO). The first report sells for $1.25; the second one is a steal at 5 cents, especially considering that dukie 1993 averages 13 percent more return on his selections than Sure Pick.

Iexchange also offers a couple of innovative investment metrics to help interested investors decide whom to trust. In addition to average (and annualized) returns, the site tracks each analyst's Directional and Predictive Accuracy. "Ebay_of_Pigs," for example, scores 80 percent on the directional front and 100 percent on the predictive. In other words, 80 percent of his picks--both open and closed--moved in the direction he specified, while all his closed picks hit his price targets.

Iexchange has competitors. Clearstation.com, for one, offers Top 10 Recommender Lists for the day, week, month, and year. You can check out the portfolio of each recommender, as well as subscribe to a recommender's list, so that every time he or she--does gender matter in a world inhabited by people named bandit1 and geep?--makes a move, you can make it too.

Why are all these people willing to share their secrets for turning research into gold, and endure having the results assayed? "I play fantasy sports," explains Warren White, a.k.a. "SmallCapAnalyst," currently ranked 13th on iexchange's list of best analysts. "It gets real competitive to try and win the league. I guess if you have the ability to accomplish something, it's interesting to try and do it in front of people." Try telling that to James J. Cramer.

(In the interest of honest disclosure and serious tooting while there's something to toot about, Validea.com ranked me as the 11th-best stock-picker over a three-month period and a five-bulb analyst. I've been afraid to show my face at iexchange. Those guys are good!)

Gregory Taggart is a regular contributor to this magazine.