Richard C Yeh

A heuristic for retirement savings

2010-03-07T16:31:00.009-05:00

Most persons should save at least a quarter, perhaps as much as half, of their post-tax (i.e., "take-home") income.

Saving for retirement feels complicated, because there are so many decisions to make. The purpose of this essay is to enumerate a few rules of thumb I shall use for simplifying these decisions. I think there may be some social benefit in promulgating these heuristics if they produce more-sustainable retirement outcomes.

Thaler and Sunstein (Nudge, 2008) make several useful points in this area:

Economists disagree on exactly how much people should save for retirement.
Humans are lazy.
Default choices or settings are extremely powerful.
Many people, when surveyed, say that they would like to save more for retirement; and then fail to do so.
Tversky and Kahneman demonstrated how associations and references affect our judgement.

References and Reality

What is retirement? Television commercials portray bliss with vacations and Viagara. I suspect the average reality is quite different. A few years ago, at a hotel in Clearwater, FL, an old woman said, "Retirement isn't all roses." Indeed, retirement is just permanent unemployment. And when you are unemployed, the key drivers that determine whether you will remain solvent are:

How much money you have, and
How quickly, and for how long, you'll deplete that money.

The first rule-of-thumbs I shall offer will be for part (2). Part (1) is left as an exercise to the reader.

Companies selling financial products offer (but do not guarantee) to help you realize your dreams. Many provide some sort of calculator to help you estimate how much to save. While these calculators range from simplistic to sophisticated (some even with Monte Carlo simulations to obtain a distribution on investment returns), the biggest problem is that they offer no transparency or guidance on assumptions --- and the assumptions are critical. The default settings on several calculators imply that income needed in retirement can be less than current income, and your investment growth rate exceeds inflation by several percentage points. Both can be dangerous assumptions.

On the income point, if you can hardly afford to take vacations now, there is no way that all the senior-citizen discounts will accrue to allow you to take any vacations on less income. If you maintain a car to commute to work, are you going to wait for the bus to visit the public library and supermarket? If you expect to finish paying a mortgage on your home, you should not forget to plan for increasing house-repair expenses, such as a new roof or water heater; if you rent, your housing costs are not going to decrease. If you are not one of the few C-suite executives, tenured professors, or government- or union-pensioners eligible for subsidized health insurance for life, you should start studying Medicare-related issues now. Otherwise, consider that private health-insurance quotes for a 60-year-old couple today range from $5,000 to $28,000 per year; and, in the past ten years, health-insurance premiums have increased at double the rate of inflation. There goes the prescription for Viagara. In summary, I think you should plan to spend about the same amount of money each year in retirement as now. (Moshe Milevsky: Your Money Milestones (2010) cites Modigliani for this idea of smoothing consumption.)

Source: http://www.ehealthinsurance.com/, quotes for family health insurance for male born 1/1/1950 and female born 1/1/1952 in zip code 10022, retrieved 2/15/2010, ranged $400-2,300 per month
Source: http://money.cnn.com/2009/09/15/news/economy/health_insurance_costs/index.htm

On the investment growth assumption, most persons should not expect to realize inflation-adjusted returns over 30 years in excess of the +6% (net of inflation but not taxes) annualized historical S&P 500 rate. And what of taxes? The historical inflation-adjusted rate may be appropriate for estimating pre-tax-with-tax-deferral (401(k) and traditional IRA) dollars, or post-tax-with-tax-free-growth (Roth 401(k) and Roth IRA) dollars, but may be as low as +3% or +4% in a taxable account. I shall provide an example in a postscript. I am not going to discuss the suitability or the moral implications of owning stock.

Savings Pipeline

Now, how many years should you plan for? For Generation-Xers and Generation-Yers, life expectancy at birth was about 70-75 years; and there is a significant chance that you'll outlive the typical lifetime. I am roughly 35 now, and I think it's safe to estimate that I'll work 30 years and then live 30 years more.

In this case, my savings in year 1 of my career will grow (or lay dormant) for 30 years, to be spent in year 1 of retirement. My savings during year 2 of my career will grow (or lay dormant) for 30 years, to be spent in year 2 of retirement. And so on. With this pipeline, I calculated the below table to estimate the amount I need to save, given various inflation-and-tax-adjusted investment returns.

If you expect your savings to grow, compounded, by this percent each year after subtracting inflation and taxes,	then, to have the same annual budget in retirement as now, assuming a schedule (years of work/years of retirement) of
	30/30	40/20	45/15	48/12	50/10
	your savings rate should be:
-4%	77%	72-53%	68-38%	64-29%	61-23%
-2%	65%	52-43%	45-31%	40-24%	35-20%
0%	50%	33%	25%	20%	17%
+2%	36%	18-25%	12-20%	9-16%	7-14%
+4%	24%	9-19%	5-16%	4-14%	3-12%
+6%	15%	5-13%	2-12%	2-11%	1-10%

To read this table, let's imagine you can invest in a 30-year certificate of deposit (CD) that returns 0% above inflation and taxes. (This product doesn't exist, but you may be able to simulate it with I bonds or TIPS. I am not going to discuss the suitability or the moral implications of owning government debt.) Then, for every $1.00 you put into the CD, you can withdraw $1.00 (today's dollars) 30 years from now. If your current post-tax income is X, you can simulate a future budget now by saving 50% of X (leaving 50% of X). If you are comfortable with the 50% you can spend today, then your lifestyle will not need to suffer much adjustment when limited to the saved 50% available in 30 years. If you save less of your income now, you are leaving yourself less to spend in retirement. You can vary your assumptions by considering:

Perhaps you will plan to work for 40 years and retire for 20 years. Then you can use two years of savings for each year of retirement, and (based on the same 0% tax-free, inflation-adjusted growth assumption) you could start by saving 33%. Why not 25% --- half of 50%? If you save 25%, then you are giving yourself a budget of 75% during your work years, but only 2 * 25% = 50% for each year of retirement, which violates the rule-of-thumb I suggested above. The other columns of the table show the needed savings rates for two, three, four, and five years of savings per year of retirement. Where there are percentage ranges, they indicate the saving rates needed in the first and last years of the non-retirement period. If you are saving 15% now, you are on a frontier hoping for a +6% annualized return on the 30/30 (years of work/years of retirement) schedule, or planning for the 50/10 schedule and with a 0% annualized return.
Perhaps you will have access to investments that return more than 0% after inflation and taxes. Good luck with that.
Perhaps you will receive Social Security payments or a pension. Good luck with that.

Changes in these assumptions can modify the savings fraction by perhaps a factor of two or three, but the conclusion is still robust: you should save a substantial fraction of your income.

To summarize, I am using two heuristics to estimate retirement savings:

Retirement income requirements are the same as now.
Using the savings pipeline and a conservative investment returns assumption, I must save a quarter to a half of post-tax income.

Pre-tax or post-tax?

In the above discussion, I've tried to use all post-tax, inflation-adjusted numbers. This is partly because it would be difficult for many persons to save 50% of their pre-tax income.

When it comes to retirement investing under United States laws, one often has three choices: (1) pre-tax funds with tax-deferral (401(k) or traditional IRAs), (2) post-tax funds with tax-free returns (Roth 401(k) or Roth IRAs), and (3) post-tax funds with taxable returns (depending on choice of investment vehicle). The benefits of (1) and (2) are that taxes are a one-time cut, either (1) at the time of withdrawal or (2) before investment; while in case (3), taxes not only take the the same one-time cut, but also reduce the realized return each year. For example, if over 30 years, your investment returns 4% each year with 3% inflation, the inflation-adjusted return will be 1% = 4% - 3%; if taxable each year with a 25% tax rate (on the 4% earned), your real return would be zero, but if taxed only at the end with the same 25% tax rate, your real return would total 26%, or about 0.77% per year. The disadvantages of (1) and (2) are that they come with restrictions on the amounts you can save and the conditions under which you can take a penalty-free withdrawal. The only difference between (1) and (2) is whether you expect your future tax rates to be higher or lower than now, and if you aren't sure, you can hedge by dividing your saving between them.

Competition for mind-share

2009-05-28T01:55:00.002-04:00

At the encouragement of a friend, I have decided to give blogging another try. I recall that daily writing as a high-school senior helped sharpen my thinking; and I hope it will again now.

I have changed the subtitle of this blog from "ideas scrapbook" (where I was using it as a bookmarking tool) to "competition for mind-share". Resolved: we must publicize nascent ideas clearly so that they can be criticized. This may be one personal blog out of thousands, but I hope to bring my own brand of logic, judgement, compassion to improve the signal-to-noise ratio in the blogging universe. I welcome comments.

If writing in this blog sharpens my thinking and makes me a better person, then it will have been successful. If this blog collects a significant following, then it will have been wildly successful.

Checking out 37signals

2008-12-29T23:25:00.002-05:00

Getting Real
On Blogs: < http://gettingreal.37signals.com/ch15_Keep_the_Posts_Coming.php >

Using their Campfire product. < http://filtrate.campfirenow.com/ >

Learning Ruby.

Antarctica

2008-07-19T22:51:00.001-04:00

Watched "Encounters at the Bottom of the World". Also saw <
http://www.emilystone.net/uploads/Cold_Comfort.pdf >

quantum error correcting codes and their decoding

2008-07-16T08:39:00.001-04:00

------Original Message------

From: seminars@cnls.lanl.gov

Sender: cnls-seminar-list-bounces@cnls.lanl.gov

To: cnls-seminar-list@swan.lanl.gov

Sent: Jul 16, 2008 8:00 AM

Subject: [CNLS Event] CNLS Seminar: Good quantum error correcting codes andtheir decoding

Date: 07/23/2008 2:00 pm - 3:00 pm

CNLS Conference Room (TA-3, Bldg 1690)

Good quantum error correcting codes and their decoding

David Poulin

University of Sherbrooke

Error correction is an essential ingredient for quantum communication and computation. To achieve good error suppression, current coding techniques typically encode each logical qubit in a very large number of physical qubits. It is possible to suppress errors with much less resources, for instance by encoding into a random code space, but decoding such codes is typically an NP-complete problem. In this talk, I will present some of our efforts at constructing good, efficiently decodable quantum codes, including turbo and sparse codes. I will also discuss the current bottlenecks in constructing provably good probabilistic quantum coding schemes.

Martin Feldstein: oil prices and short-term elasticity of demand

2008-07-03T12:57:00.001-04:00

We Can Lower Oil Prices Now
By MARTIN FELDSTEIN
July 1, 2008; Page A17

Although most experts agree that financial speculation was not responsible for the surge in the global prices of food and energy, many people remain puzzled about the source of these remarkable price rises. Economics offers a simple supply-and-demand explanation and reason for optimism about the future of commodity prices. In the case of oil, economics also suggests how policy changes today that affect the future could quickly lower the current price of oil.

We all know that rising incomes in China, India and the Gulf states have increased the demand for oil and many other commodities. But how could the modest, one-year rise of these demands lead to 100% increases in the prices of oil and other commodities? Let's take a look first at perishable agricultural commodities.

In the short run, there is little scope for increasing the supply of corn in response to a global increase in demand. For demand and supply to balance – for the market to clear – the price of corn must rise.

If the demand for corn were very price-sensitive, a relatively small increase in price would reduce global demand by enough to offset the initial rise in demand. However, since demand is actually quite insensitive to price in the short run, it takes a very large price rise to bring global demand into line with supply.

Here is a simplified picture of what happened in the past year. The quantity of corn demanded by high-growth countries rose gradually, increasing eventually by an amount equal to, say, 10% of the previous total global level of corn consumption. Since the supply of corn did not increase, the price had to increase enough to reduce corn consumption in other countries by 10%. If it takes a 10% increase in the price to reduce the quantity of corn demanded in the first year by just 1%, it would take a 100% increase in the price of corn to offset the initial 10% rise in the quantity of corn demanded.

In reality, the picture is complicated by the substitution in both supply and demand among different agricultural commodities, and by the role of the corn ethanol program. But the basic explanation holds: With a very low short-run price sensitivity of demand and little scope to raise supply in the short run, even a relatively small increase in corn demand by the high-growth economies can lead to a very large short-run rise in the price of corn.

Fortunately, the price sensitivity of both demand and supply will increase with time. This implies that the rising demand from China and other countries may eventually be accommodated with a price lower than today's level.

The situation for oil is more complex, but the outcome for prices is potentially more favorable.

Unlike perishable agricultural products, oil can be stored in the ground. So when will an owner of oil reduce production or increase inventories instead of selling his oil and converting the proceeds into investible cash? A simplified answer is that he will keep the oil in the ground if its price is expected to rise faster than the interest rate that could be earned on the money obtained from selling the oil. The actual price of oil may rise faster or slower than is expected, but the decision to sell (or hold) the oil depends on the expected price rise.

There are of course considerations of risk, and of the impact of price changes on long-term consumer behavior, that complicate the oil owner's decision – and therefore the behavior of prices. The Organization of Petroleum Exporting Countries (the OPEC cartel), with its strong pricing power, still plays a role. But the fundamental insight is that owners of oil will adjust their production and inventories until the price of oil is expected to rise at the rate of interest, appropriately adjusted for risk. If the price of oil is expected to rise faster, they'll keep the oil in the ground. In contrast, if the price of oil is not expected to rise as fast as the rate of interest, the owners will extract more and invest the proceeds.

The relationship between future and current oil prices implies that an expected change in the future price of oil will have an immediate impact on the current price of oil.

Thus, when oil producers concluded that the demand for oil in China and some other countries will grow more rapidly in future years than they had previously expected, they inferred that the future price of oil would be higher than they had previously believed. They responded by reducing supply and raising the spot price enough to bring the expected price rise back to its initial rate.

Hence, with no change in the current demand for oil, the expectation of a greater future demand and a higher future price caused the current price to rise. Similarly, credible reports about the future decline of oil production in Russia and in Mexico implied a higher future global price of oil – and that also required an increase in the current oil price to maintain the initial expected rate of increase in the price of oil.

Once this relation is understood, it is easy to see how news stories, rumors and industry reports can cause substantial fluctuations in current prices – all without anything happening to current demand or supply.

Of course, a rise in the spot price of oil triggered by a change in expectations about future prices will cause a decline in the current quantity of oil that consumers demand. If current supply and demand were initially in balance, the OPEC countries and other oil producers would respond by reducing sales to bring supply into line with the temporary reduction in demand. A rise in the expected future demand for oil thus causes a current decline in the amount of oil being supplied. This is what happened as the Saudis and others cut supply in 2007.

Now here is the good news. Any policy that causes the expected future oil price to fall can cause the current price to fall, or to rise less than it would otherwise do. In other words, it is possible to bring down today's price of oil with policies that will have their physical impact on oil demand or supply only in the future.

For example, increases in government subsidies to develop technology that will make future cars more efficient, or tighter standards that gradually improve the gas mileage of the stock of cars, would lower the future demand for oil and therefore the price of oil today.

Similarly, increasing the expected future supply of oil would also reduce today's price. That fall in the current price would induce an immediate rise in oil consumption that would be matched by an increase in supply from the OPEC producers and others with some current excess capacity or available inventories.

Any steps that can be taken now to increase the future supply of oil, or reduce the future demand for oil in the U.S. or elsewhere, can therefore lead both to lower prices and increased consumption today.

Mr. Feldstein, chairman of the Council of Economic Advisers under President Reagan, is a professor at Harvard and a member of The Wall Street Journal's board of contributors.

Pricing Algorithms for Combinatorial Auctions

2008-06-21T00:00:00.002-04:00

Looks similar to Preston McAfee's area.

-----Original Message-----

From: seminars@cnls.lanl.gov

Date: Thu, 19 Jun 2008 15:04:45

To:cnls-seminar-list@swan.lanl.gov

Subject: [CNLS Event] Seminar: Pricing Algorithms for Combinatorial Auctions

Tuesday, July 8th, 2008

10:00 am - 11:00 am

CNLS Conference Room (TA-3, Bldg 1690)

Pricing Algorithms for Combinatorial Auctions

Robert (Bob) Day

University of Connecticut

Combinatorial auctions involve the sale of several

interrelated goods and allow bidders to bid on packages or combinations of

goods, rather than just individual items. In this talk I give an

overview of combinatorial auctions and their application to a wide variety of

scarce-resource allocation problems. After giving an overview

of applications and general complexity results, I will discuss payment

determination algorithms, in particular the \"core\" payment paradigm now

being implemented in the United Kingdom for spectrum license auctions. The

various algorithmic and game-theoretic properties of this payment

mechanism will be introduced.

LANL Host: Cory Hauck, CCS-2

Learning By Learning To Communicate

2008-06-20T23:53:00.001-04:00

Tuesday, June 17th, 2008

2:00 pm - 3:00 pm

CNLS Conference Room (TA-3, Bldg 1690)

Learning By Learning To Communicate

Jake BEAL, LANL Director's Postdoc Awardee

MIT, Computer Science and Artificial Intelligence Laboratory (CSAIL)
Human intelligence appears to be a product of cooperation among many
specialists. I show how a system of specialists can capture
cross-specialist knowledge through a struggle to agree on signals for
communicating with one another. This process of "learning by learning
to communicate" may help to explain how humans develop our unique
capacity for the "high-level" agile cooperation that permeates our daily
lives. I have created mechanisms for signal agreement that exploit the
phenomenon of "communication bootstrapping," in which shared experiences
form a basis for agreement on a system of signals. These mechanisms are
demonstrated using a vision specialist and a hearing specialist that
jointly observe a simulated four-way intersection. As they agree on
signals, the two specialists capture some dynamics of the simulation as
differences in how they interpret these signals.

BIO:
Jacob Beal is a postdoctoral associate in the Computer Science and
Artificial Intelligence Laboratory at MIT, where he recently completed
his Ph.D. under Prof. Gerald Jay Sussman. His research interests center
on the engineering of robust adaptive systems, with a focus on problems
of system integration for human-level intelligence and on problems of
modelling and control for spatially-distributed networks like sensor
networks, robotic swarms, and cells during morphogenesis.

LANL Host: Christof TEUSCHER, CCS-3 and Reid PORTER, ISR-2

A Remarkable Tornado Photo

2008-06-13T23:17:00.001-04:00

"The Lede: A Remarkable Tornado Photo"

An Iowa woman snaps a stunning close-up of a looming funnel cloud.

Kalman filters

2008-06-13T17:26:00.001-04:00

Consider these for noise rejection, especially when moving to continuous-time data.

TouchGraph

2008-06-13T12:34:00.001-04:00

http://www.touchgraph.com/

Link: tricks/techniques to accelerate normalized cross-correlation for image analysis

2008-06-09T18:48:00.002-04:00

http://www.cs.ubc.ca/~deaton/remarks_ncc.html

FFT Window references

2008-05-07T23:39:00.002-04:00

http://www.lds-group.com/docs/site_documents/AN014%20Understanding%20FFT%20Windows.pdf
http://www.bores.com/courses/advanced/windows/files/windows.pdf

The first link has a table with window characteristics:

Window / Best for these Signal Types / Frequency Resolution / Spectral Leakage / Amplitude Accuracy
Barlett: Random / Good / Fair / Fair
Blackman: Random or mixed / Poor / Best / Good
Flat top: Sinusoids / Poor / Good / Best
Hanning: Random / Good / Good / Fair
Hamming: Random / Good / Fair / Fair
Kaiser-Bessel: Random / Fair / Good / Good
None (boxcar): Transient & Synchronous Sampling / Best / Poor / Poor
Tukey: Random / Good / Poor / Poor
Welch: Random / Good / Good / Fair

A. O. Scott: Here Comes Everyboy, Again

2008-05-07T19:39:00.002-04:00

"Mr. Sandler did not invent the archetype of the overgrown man-child, which has been around at least since the silent era. ... Nor has Mr. Sandler been alone, over the past 15 years or so, in turning male infantile aggression into the basis of a lucrative and long-running movie career. His rivals and confreres have included Jim Carrey, Jack Black and Will Ferrell — all of them different physical and temperamental types, but all of them committed to a brazen and unyielding refusal of maturity."

Add to this list the television comedies Seinfeld, Friends, Everyone Loves Raymond, and most recently, King of Queens. I had wondered about the provenance of this archetype, and the exact nature of the thing that so repulsed me about those shows, until this essay put a name to the phenomenon.

CNLS Seminar: Inference in Hidden Markov Models:Ergodic Theory and Systems Theory

2008-04-07T17:58:00.001-04:00

Inference in Hidden Markov Models: Ergodic Theory and Systems Theory

Ramon van Handel

Caltech

An essential ingredient of the statistical inference theory for hidden Markov models is the nonlinear filter. The asymptotic properties of nonlinear filters have received particular attention in recent years, and their characterization has significant implications for topics such as the convergence of approximate filtering algorithms, maximum likelihood estimation, and stochastic control. Though there has been much progress in specific models, most results shed little light on the fundamental nature of the problem, and a general underlying theory has therefore been elusive. In this talk, I will describe new results in a general setting which are based on surprising connections with two other topics: the ergodic theory of Markov chains in random environments, and systems-theoretic notions of observability and detectability for hidden Markov models. I will also show that for a special class of signals these methods can be combined to obtain necessary and sufficient conditions for stability of the filter. This strongly suggests that these techniques do indeed capture the fundamental mechanisms that lead to filter stability.

DoG is used in wavelets

2008-04-03T23:27:00.001-04:00

Just noticed that derivative of gaussian is also called "DoG" when used as a wavelet basis function. But I think Daubechies are more popular.

Are you using those in your price models?

Maybe in the future. One of my coworkers is using Daubechies wavelets in his model; compared with ordinary moving-average smoothing, doing a wavelet transform, cutting off the spectrum, and then transforming back seems to offer better noise reduction with less information. You still need a power-of-two data length, like for fast Fourier transforms.

Recently, I have been using techniques from "robust statistics": medians, trimmed data, etc. This reminded me of our early experiences with median filtering. Now I think that, while the technique may be nonlinear, the point of experimental science is to make a reliable measurement. If the number itself is what we want to measure (or even ordinary arithmetic transformations thereof), then I say filter. I think the only case where I wouldn't automatically reach for a trimmed mean (where the top and bottom x% of observations are excluded before taking the mean) is if I were doing transformation to a different space, such as FFT. But for dynamic pulling experiments, I suspect FFT is a rarely-used technique.

You probably know that the reporting of a standard deviation is only meaningful if the data are close to normal. Especially in the kind of data I have, I am plagued by fat-tailed distributions, and find that ordinary means and variances are too sensitive to outliers. The robust alternative is the MAD (median absolute deviation).

CNLS Seminar: Nikolai Sinitsyn

2008-04-03T10:06:00.001-04:00

-----Original Message-----

From: seminars@cnls.lanl.gov

Date: Thu, 03 Apr 2008 07:00:02

To:cnls-seminar-list@swan.lanl.gov

Subject: [CNLS Event] CNLS Seminar: Nikolai Sinitsyn

CNLS Sponsored Talk

Date: 04/03/2008 2:00 pm - 3:00 pm

CNLS Conference Room (TA-3, Bldg 1690)

Nikolai Sinitsyn

Geometric phases in stochastic and dissipative systems

CCS-3/T-CNLS

Contact: Cory Hauck

A year ago, I gave a talk at the postdoc seminar in CNLS to report about the discovery of geometric phases in purely classical stochastic systems [1]. During last year, this subject has grown and, besides the original application to Michaelis-Menten reactions, the concept was used in several fields including domain wall motion in ferromagnetic nanowires [2], stochastic models in epidemiology [3], an analog of the Born-Oppenheimer approximation in stochastic processes [4], and electric current fluctuations in interacting nanoscale conductors [5]. This new approach to driven stochastic systems is thus universal. I will review the recent progress and possible future research directions on the subject.

References:

[1] N. A. Sinitsyn, and I. Nemenman, "The Berry phase and the pump flux in stochastic chemical kinetics", Euro. Phys. Lett. 77, 58001, 2007

[2] N. A. Sinitsyn, V. V. Dobrovitski, S. Urazhdin and A. Saxena, "Geometric control over the motion of magnetic domain walls", submitted to Phys. Rev. Lett., Technical report: LA-UR 08-0907, (2008)

[3] N. A. Sinitsyn, and I. Nemenman, "Universal geometric theory of stochastic pump and reversible ratchets", Phys. Rev. Lett. 99, 220408, 2007

[4] N. A. Sinitsyn, and I. Nemenman "Coarse graining stochastic biochemical networks", Technical report: LA-UR 08-0025, 2008

Database Monte Carlo (DBMC): A New Strategyfor Variance Reduction in Monte Carlo Simulation

2008-04-02T22:51:00.001-04:00

Monday, March 31st,2008
10:00 am - 11:00 am

CNLS Conference Room (TA-3, Bldg 1690)

DataBase Monte Carlo (DBMC): A New Strategy for Variance Reduction in Monte Carlo Simulation

Professor Pirooz Vakili
Boston University

A well-known weakness of (ensemble) Monte Carlo is its slow rate of convergence. In general the rate of convergence cannot be improved upon, hence, since the inception of the MC method, a number of variance reduction (VR) techniques have been devised to reduce the variance of the MC estimator. All VR techniques bring some additional/external information to bear on the estimation problem and rely on the existence of specific problem features and the ability of the user of the method to discover and effectively exploit such features. This lack of generality has significantly limited the applicability of VR techniques.

We present a new strategy, called DataBase Monte Carlo (DBMC), which aims to address this shortcoming by divising generic VR techniques that can be generically applied. The core idea of the approach is to extract information at one or more nominal model parameters and use this information to gain estimation efficiency at neighboring parameters. We describe how this strategy can be implemented using two variance reduction techniques: Control Variates (CV) and Stratification (DBMC approach can be used more broadly and is not limited to these two techniques). We show that, once an initial setup cost of generating a database is incurred, this approach can lead to dramatic gains in computational efficiency. DBMC is quite general and easy to implement -- it can wrap existing ensemble MC codes. As such it has potential applications, among others, in ensemble weather prediction, hydrological source location, climate and ocean, optimal control, and stochastic simulations of biological systems.

We discuss connections of the DBMC approach with the resampling technique of Bootstrap and the analysis approach of Information Based Complexity.

LANL Host: Frank Alexander, ADTSC

Liars' Poker

2008-02-11T22:59:00.000-05:00

After reading Michael Lewis's Liars' Poker, and Roger Lowenstein's When Genius Failed, I am struck with a thought: make a program to play Liars' Poker. Either decimal or dice systems. Bayesian estimation of probabilities. Python.

Citation software - the next generation

2007-11-26T14:25:00.000-05:00

Zotero < http://www.zotero.org/ >

CiteULike < http://www.citeulike.org/ >

Del.Icio.Us < http://del.icio.us/ >

AT&T Wireless thoughts

2007-11-26T10:07:00.000-05:00

Mobile-to-mobile lookup:

< http://www.wireless.att.com/cell-phone-service/mobile-to-mobile/lookup.jsp >

CNLS speakers for follow-up

2007-11-26T08:36:00.001-05:00

Brian Munsky, discretizing systems biology:

http://www.uweb.ucsb.edu/~brian_munsky/Main.html

Glenn Foster, septic shock

Manfred Warmuth, numerical methods

Mike Watson, Efficient Solution of Differential Equations with Chebyshev
Discretizations

Charles Reichhardt/T-13, rectified brownian motion in "Swimming Bacteria
and Maxwell's Demon"

Tim Wallstrom (T-13), "The marginalization paradox and the formal Bayes'
law", especially http://arxiv.org/abs/0708.1350

Unemployment benefits

2007-11-20T23:35:00.000-05:00

The on-line application at the New York State Department of Labor Unemployment Insurance web site is only available "between the hours of 7:30 am to 7:30 pm Monday through Thursday (Eastern Time), Friday, 7:30 am to 5:00 pm, all day Saturday, and Sunday until 7:00 pm."

Bank of America / 100 Corporate Place, Suite 403 / Peabody, MA 01960 // EIN 56-2058405 NY 562058405

212-583-8000

Lockheed Martin NV Tech, Inc. / PO Box 98521 / Las Vegas, NV 89193-8521 // EIN 88-0347976 NM 02-297731-007

Working with recruiters

2007-11-19T14:54:00.000-05:00

What is the value of working with recruiters?

Some have inside contacts (hiring managers).
Some have relevant information (company X has a hiring freeze).

Coverage is spotty, though, and big firms tend to have established paths through human resources.

after-Thanksgiving sales

2007-11-19T14:50:00.000-05:00

"Black Friday" advertisements can be found at http://bfads.net/ and http://www.fatwallet.com/. Mentioned in last Sunday's New York Times.

This year, there doesn't seem to be anything special.

Electronic Lab Notebook (ELN) thoughts

2007-11-19T09:53:00.000-05:00

My ideal electronic lab notebook would:

store everything in real time: I would like to paste in observations, measurements, complete data-processing histories, paper reprints;
allow search for: free-text, creation date range, citation date range, citation count
allow by-owner stateful highlighting of important things and collapsing of unimportant things: this would facilitate future review without permanently losing observations considered irrelevant at the time. An outline structure would probably be fine, but I would want to be able to collapse or expand any paragraph at any level.
allow annotated hyperlinks to prior paragraph-level observations and dates in this notebook and others, for backwards-referencing;
automatically count citations (i.e., this "observation was confirmed/rejected" or "protocol was reused" or "data was commented-on" on such-and-such entry) (the two points above combined mean that comments on previous observations, where the comments are actually new ELN entries, remain in-context);
I don't see that entries more than two hours old ever need to be edited again, so don't provide an "edit" feature.
treat protocols in a special way, somewhat like a source-code control or versioning system. When protocols are mature, it is easy for people to say: I used protocol ABC. However, over the course of years, there could be modifications to protocol ABC that could improve yield, save time, etc. Often, people will continue to say that they used protocol ABC. I would propose that every protocol's "permalink" should contain its version number, so that past links to protocol ABC would point to the historical ABC, not the new-and-improved ABC. Actually, this can be easily handled in the above scheme: just have the complete protocol as an entry; when the protocol is revised, then users of the newer protocol will link to the newer entries. This can be organized with a special "kprotocol" keyword, or a separate "Lab protocols" ELN where protocol developers have write access.

The GUI should facilitate easy linking, perhaps through endnotes for each entry.
When reading, the blog-like "Comment" button would become an "Add new entry to your own ELN" button, with the specified paragraph already referenced.
When writing, typing a special link string (such as double open braces) should bring up an in-page pop-up allowing search or selection of recently-cited and most-popular ELN citations. While we're at it, this should also allow web search and pasting of URLs or page snapshots.

Discovered that this is an old idea: http://pubs.acs.org/hotartcl/ci/00/jan/inet.html.

Income growth and house-price growth: phase portraits

2007-11-18T15:12:00.000-05:00

Derek Cheung mentioned last May that his brother observed income as a leading indicator for house prices in Hong Kong. Zhang Da Peng's blog is at http://dpz88.spaces.live.com/.

Now we have data to explore this in the United States.

Standard&Poors/Case-Shiller home-price indices for metropolitan areas are available at http://www2.standardandpoors.com/portal/site/sp/en/us/page.topic/indices_csmahp/0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0.html.

The Internal Revenue Service's Statistics of Income Division http://www.irs.gov/taxstats/index.html has state- and zip-code-level sums of adjusted gross income and exemption sums. One problem with the zip-code-level sums is the cost: $500 per year for the whole United States.

Phase portraits (y = momentum; x = position) can show the behavior of a damped driven harmonic oscillator in different regimes: simple undamped is a circle; underdamped spirals clockwise to the origin; resonance spirals clockwise outward. If we plot inflation-adjusted income growth on the x-axis and inflation-adjusted house-price growth on the y-axis, then counter-clockwise motion would mean income leads housing. I suspect driven or undamped behavior in the first quadrant, and damped behavior in the other three quadrants.

Essay idea: stickiness of housing and job markets

2007-11-18T15:01:00.000-05:00

The value of skills in the job market is always backwards-looking, just like comparable sales in a housing market. I have forgotten where this essay idea was going, but I think it was going to be a personal essay about going to work on mortgage-backed securities, learning about house-price appreciation, and then trying to find work elsewhere in finance.

"A great programmer might be ten or a hundred times as productive as an ordinary one, but he'll consider himself lucky to get paid three times as much. ... this is partly because great hackers don't know how good they are. ... it's also because money is not the main thing they want." (Paul Graham, "Great Hackers", July 2004, http://paulgraham.com/gh.html, in The Best Software Writing I, edited by Joel Spolsky (Apress, 2005))

Blogs and Wikis again

2007-11-18T14:54:00.000-05:00

Just finished reading The Best Software Writing I, edited by Joel Spolsky (Apress, 2005).

On pp. 197-198, I read of an idea of running an instant messenger and a wiki in parallel with a conference or conference. The link is: http://www.shirky.com/writings/group_enemy.html, about halfway down (search for "modes") Each presenter could have her or his slides on the wiki, and interesting questions and comments from the real-time chat could be added. The wiki would then become an annotated proceedings.

With big-enough screens, you could see everything: instant-messenger chatroom, wiki, and external reference with browsing history.

New mission for blog

2007-11-18T14:44:00.000-05:00

Previously, I had no use for this blog. Now I see that it can be a useful content management system for annotating my web-browsing history. It is more suited for my purposes than a wiki: I anticipate that I will generate only a handful of posts per week, so I don't need a big management or organization system. Since so many resources to be referenced are available from the web, this is more in-context than other forms of scrapbooking software. Finally, unlike many wikis, it's free for multiple users and I don't have to administer it.

For now, I will change the name from the solipsistic paean "A song I can sing in my own company" to "Richard C Yeh et al".

Test link

2007-02-04T00:12:00.000-05:00

Link to network share via UNC \\Dice\Test: file://///Dice/Test/. (Works in Internet Explorer 7, but not Firefox 1.5.0.9 and not Opera 9.10.)

Link to network share mapped as drive Z: file:///Z:/. (Same as above, if mapped. Example 1 doesn't require the mapping to be in place.)

Link to local file: file:///C:/ipconfig.txt. (Always fails, if web page is not a local html file.)

Try this now:
1. In Windows, go to Start -> Run... and in the dialog, type:
cmd /k ipconfig /all > C:\ipconfig.txt
This will create a file for testing.
2. Paste this into your web browser:
file:///C:/ipconfig.txt
This should display the file.
3. In the wiki, make a link to:
file:///C:/ipconfig.txt
View the wiki, and click on the link.

If nothing happens in step #3 (which is behavior different from step #2), then you are experiencing a problem described at: < id="9601">. I find that I can replicate this problem with Microsoft Internet Explorer 7, Mozilla Firefox 1.5.0.9, and Opera 9.10. I find that I can work around this problem in Internet Explorer, but not the other two.

Blogs and Wikis

2007-02-03T21:45:00.000-05:00

After a good friend mentioned his ideas of using blog and wiki technologies in his new lab, I couldn't resist trying these again, in the hope that I'll better understand the possibilities of these. At work, I've documented some oft-used database tables and queries on the group wiki pages, served by Atlassian Confluence, which also runs the technology bug/issue-tracking system. Atlassian also has a blog feature, which I am trying to use to highlight developments in my work.

Away from work, I tried to install my own wiki on a server where I have no administrative privileges, but there are many configuration questions that I cannot answer. I told my friend that there would be significant effort in setting up and maintaining such a system, and I am happy to hear that he found a solution at http://openwetware.org/.

I am always impressed at the efficiencies to be wrung from outsourcing. For example, I think the Google Apps for Education product in use at Arizona State University is a good example to follow. There is much duplication of information technology effort across organizations, and consolidating and outsourcing those efforts could improve the quality and reduce the cost. I recall considering the purchase of a network-attached storage device and backup system for a research group where I was a graduate student. There's no educational value in that exercise. IBM's attempt to sell compute time is also notable, but has a smaller market, which probably needs time to play and explore. Drucker would have applauded our giving the problem to an expert to handle.

References:

I am such a fool --- to worry as I do

2005-04-01T13:30:00.000-05:00

Why a weblog, and why now? A test of popular technology, to see if the shoe fits. An evaluation of a medium, to see whether it can be any more than a narcissistic, solipsistic indulgence.