SEC: Spreadsheets Don’t Count as Backtest Data
September 5, 2012
Two Excel spreadsheets and a set of manual calculations for which no copies exist do not constitute “backtesting” of an investment strategy against the kind of beating securities can take in bear markets, the Securities and Exchange Commission said in a complaint Wednesday.
The federal regulator charged that a San Diego financial advisor, author and radio personality falsely claimed that his “Buckets of Money” investment strategy had calculations had been empirically “backtested” over actual bear market periods.
But the SEC said that the claims made by Raymond J. Lucia in investment seminars about the strategy had ‘scant, if any, backtesting.’’
The SEC said:
Respondents have admitted that the only testing they performed involved: (1) some calculations Lucia performed manually in the late 1990s, copies of which no longer exist; (2) a so-called “backtest” for the period from 1966 to 2003, memorialized on a two page Excel spreadsheet, which was performed by an RJL employee in 2003 (the “1966 Spreadsheet”); and (3) a so-called “backtest” for the period from 1973 to 2003, memorialized on a two page Excel spreadsheet, which was performed by the same RJL employee, also in 2003 (the “1973 Spreadsheet”).
Backtesting is supposed to use substantive amounts of historical data as the basis of calculations for how an investment or trading strategy would “have performed had it actually been used in a prior time period,’’ the SEC said.
Lucia and his company, Raymond J. Lucia Companies, allegedly made the backtesting claims in “a lengthy slideshow” at the seminars, designed to win clients.
“Lucia and RJL left their seminar attendees with a false sense of comfort about the Buckets of Money strategy,” said Michele Wein Layne, Regional Director of the SEC’s Los Angeles Regional Office. “The so-called backtests weren’t really backtests, and the strategy wasn’t proven as they claimed.”
According to the SEC’s order, a backtest must utilize actual data from the time period in order to get an accurate result. And, in the “two cursory spreadsheets” that Lucia cited as the basis of his backtesting, the calculations relied on a 3 percent inflation rate.
This was not a historical rate and made “results look more favorable” for the “Buckets of Money” strategy.
According to the SEC’s complaint:
When historically accurate inflation rates are used in the alleged 1966 and 1973 backtests, an investor using the BOM strategy would have exhausted his or her assets by 1986 (if retiring in 1966) or by 1989 (if retiring in 1973), which are far worse outcomes than are presented by Respondents’ slideshow, which bases those outcomes on backtesting purportedly evidenced by the 1966 and 1973 Spreadsheets.
The slideshow presentation that Lucia and RJL used during the seminars failed to disclose the flaws in their alleged backtests and was materially misleading, the SEC said.
