Monday, July 2, 2007

The Importance of Ignorance

Ira's blog concerning statistics got me thinking about a couple of situations where ignorant decision making is very important. One is, of course, recreational gambling. Gambling could not exist if we had information so complete that the outcome would be a certainty. There needs to be sufficient information to allow computation of the betting odds, but not enough to allow computation of the outcome. The other is life, casualty and all other types of insurance.

Generally speaking, information is gathered in order to compute insurance and annuity odds. It is not surprising that premiums are dependent on the odds computed from this information. However, too much information is adverse, because the outcome becomes determinist rather than statistical. As an extreme case, imagine that genetic and medical evaluation could pinpoint a young married person's death from natural causes within a year or so. Some people could not obtain insurance except at exorbitant rates, while others would not bother to purchase insurance at all.

If this seems far-fetched, think about the fact that the government has already had to step in to enforce ignorance. We know that the risk of death for an Afro-American is statistically higher than that of the other ethnic groups. This piece of information must be excluded from a statistical calculation with respect to insurance premiums. Women live longer on the average than men. This piece of information must be excluded in the computation of annuity pay-outs, by order of the supreme court.

Another querky aspect is politically-correct or public relations effects on ignorance requirements. Automobile insurance companies charge lower rates for teenagers who have passed through a drivers education class in high school. Studies show that they have a lower accident rate. I had a hunch about this and contacted an insurance executive. It turns out that the lower accident rate can be completely accounted for by the fact that the better students take the elective drivers education course. The auto accident rates correlated just as well with grade point average. When that factor is eliminated, there is no difference between those with drivers education and those without. When I asked the executive why his company gave drivers education discounts based upon false causality. he responded that it was a matter of public relations that they not be perceived to be against education.
Not only can statistics lie, in some cases they MUST lie.

9 comments:

Ira Glickstein said...

Joel has made some great observations about recreational gambling and insurance.

For philosophical reasons I do not gamble, even on the "50/50" games some clubs use to get some money for their activities. I understand the way odds and probability work. If the "house" is taking a percentage, the more you gamble, especially small-stakes, the more certain it is you will lose more money that you win.

Some folks think that, by the "law of averages" (which they misunderstand completely) if you have bet and lost lots of times, wou are "bound to win" soon. They think that the odds of a given person winning go up the more he or she loses! That is what keeps them at the slot machines until they lose their last quarter.

The only "reasonable" reason I can think of for gambling is the "change your life" argument, and that only applies to very high stakes gambles. The idea is that a $2 or $10 expense to buy a $100M winner sweepstakes ticket will not change your life if you lose. You can go to McDonalds for your next meal instead of a sit-down restaurant and get the small ticket price back. Even if you eventually lose, you get to think about how you would spend the $100M prize and that may divert your mind from your Earthly troubles and give you pleasure worth more than the ticket price. AND, if you WIN the $100M, it will really change your life!

Insurance is a reasonable way to spread the risk. You buy it and hope you will never have to make a claim. I always take the highest deductable to reduce the premium to as low as possible. My reasoning is that, if I suffer a loss, I can easily afford to pay a thousand or two without seriously affecting my lifestyle. On the other hand, if I had a very large loss of a hundred thousand or more the was not insured, that would diminish my lifestyle greatly.

As for ignorance being essential in some cases, as Joel claims, I agree partly. Certainly, recreational gambling would go away if the public had a real understanding of the odds. Small-stakes gambling is a definite loser and makes no reasonable sense. The problem is that most gamblers are small-stakes and they are often betting their children's milk money or the rent.

As far as insurance goes, I believe the insurance companies should be allowed to use ALL available risk information and should set the rates to account for estimated risk plus some reasonable profit. I agree that race, per se, should not be allowed to be used, but socio-economic factors that correlate to risk should be allowed. For example, the chances of having your car window smashed vary greatly from place to place, pretty much based on socio-economic level. Similarly, young men (and now women) have more car accidents per mile driven than more mature drivers, as do very old people (but older people tend to drive fewer miles to partly make up for the differences.) Why should we not allow (and even encourage) insurance companies to charge clients according to the best possible estimate of the risk?

Ira Glickstein

Ira Glickstein said...

From: Howard Re: Imporance of Ignorance.

[Howard Pattee asked me to post this for him.]

Ira, I was reading past posts and your response to Joel's importance of ignorance about insurance bothered me.

You wrote:
“As far as insurance goes, I believe the insurance companies should be allowed to use ALL available risk information and should set the rates to account for estimated risk plus some reasonable profit. . . .Why should we not allow (and even encourage) insurance companies to charge clients according to the best possible estimate of the risk?”

Isn’t your free-market ideology a little one-sided? You imply that insurance companies are overly restricted in the use of customer risk information. Should the customer have no right to exclude personal information? How about the customer’s right to have all the relevant information about the insurance company?

Anyway, encouraging insurance companies to find the best risks is totally unnecessary! The industry is populated with the most astute statisticians, computer programs, marketing experts, and public relations psychologists they can buy. How they partition populations, diversify risks and confuse the customer is a secret highly competitive game.

My experience with life, health, auto, home, liability, and other insurance companies is that except for few like TIAA their sales pitches are unethical because they will not disclose the crucial cost and risk information the customer needs to make a rational choice. The fact is that there are very few laws that require insurance companies to reveal risk and pricing information, or that limit what information they can require from a potential customer.

Finally, I think your ideological advice “to use ALL available risk information” to charge customers is not sensible. To take Joel’s example, suppose your company A uses all available risk information (e.g., my entire life history including my gene sequence) and determines that I’m in a very small, bad risk group. It would charge me a high rate -- and it would not make sense to buy their policy. I would go to company B that uses coarser partitioning of the population (using less information) and consequently offers me a lower rate. On the other hand, if your company A says I’m a very low risk, I’ll go to company B for a life annuity which will be cheaper than the same annuity from A (because A expects me to live longer than does B).

I think if you don't have an unethical sales pitch and plenty of credulous customers you may go bankrupt. But of course there are lots of credulous customers and many unethical companies, so maybe you will get rich. Is this what you mean by a free-market?

Am I missing something?

Howard

Ira Glickstein said...

From: Ira, Re: Imporance of Ignorance.

Howard, thanks again for reading materials in this Blog and providing your Comments!

In Brooklyn, where I grew up, young people regularly smash windows of cars as they commit petty burglaries and they are seldom apprehended because police have bigger fish to fry. Some Brooklyn auto glass repair shops have been convicted of *paying* neighborhood youth to do so as a way to increase business.

In The Villages, FL, where I live now, such crime is unheard of. Our community is gated, there is video surveilance of cars that enter, and our population is mostly 55+.

Do you propose that my insurance rates for glass breakage should be the same as someone with the same car living in Brooklyn? Of course not! (Please correct me if I'm wrong about your opinion on this specific example.)

On the other hand, coastal Florida residents pay more for insurance against wind damage from hurricanes than inland residents and people living is some other states. Houses located in flood zones have higher rates for flooding and so on. Nothing wrong with that. Right?

All that is based on insurance companies competing for the best balance of risk vs what they can get in premiums. The whole idea of insurance is to protect you and me from catastrophic financial burdens in case we are struck by a large loss. Each of us is at a small risk of being hit with a large loss so we each pay a relatively small amount such that if one of us is hit, they will be made whole by the insurance company. Since insurance companies compete, and are regulated by the states, they use their actuaries to compute the best estimate of the risk and then set rates to cover the risks plus some profit.

Of course, there have been some unethical insurance companies that accepted the premiums but, when it came time to pay off, they reneged. Some have weasel words in their policies covering wind damage but not water damage so, when a house looses its roof to wind and is subsequently destroyed by mold due to rain water, there is a question of how much they should pay for, etc. In the opposite case, flooding destroys a foundation and then the weakened house is later damaged by wind. What water-damage-excluded policy cover?

On the other hand, some homeowners in flood plains do not buy flood insurance because of the (justifiable) high cost, and then, when the inevitable flood comes, they complain when their flood-excluded insurance policy doesn't pay off!

On the other hand (I'm running out of hands :^) some homeowners in a storm damaged area have been known to inflate their claims and, through the rush of insurance companies settling claims, they get away with it. There are private insurance adjusters who go to such areas and take a cut of the proceeds as they help homeowners scam the insurance companies.

I believe *all* information about insurance companies should be a matter of public record. I favor laws that require all consumer contracts to be written in plain English. I like it when the media (and now the Internet Blogosphere) reports cases of insurance companies reneging on their promises. State insurance regulatory agencies should inveistigate, publicize and fine or exclude companies that are found to break the law.

Your example of insurance company A using all available risk information while company B uses a coarser partitioning of the population is a great example -- FOR MY SIDE! By all means, if you are a high risk for dying soon (due to age, medical history, genetics, ...) by all means, buy your life and health insurance from company B and your annuity from company A. If you are at a low risk of dying soon, do the opposite. What is wrong with that?

Many people I have talked to seem to favor insurance premiums varying by region, based on crime and weather patterns that affect risk. They also seem to favor rates based on the age of the insured. On the other hand, their backs go up when genetic DNA markers are suggested as a means of measuring risk.

If I have "good genes" when it comes to life expectancy and someone else of the same age has "bad genes", why should we both pay the same rate for life insurance?

Is there something wrong with that? What am I not getting?

Ira Glickstein

Ira Glickstein said...

From: Howard Re: Importance of Ignorance

Howard Pattee asked me to post the following for him:

*************************
Ira,

I think you completely missed the simple point I was making. I was bothered by your statement that “encouraged” insurance companies “to use ALL available risk information” to charge customers.

This appears contrary to Joel’s thread “the importance of ignorance” which implies NOT using ALL available information.

My example was to show WHY one should NOT use “ALL available risk information” to charge customers.

You obviously agree with my analysis. Why you claim this is YOUR argument I cannot follow.

Howard

Ira Glickstein said...

From: Ira, Re: Importance of Ignorance

Thanks again Howard for Commenting further in this thread.

As I understood Joel's original posting that started this thread, he was suggesting cases where ignorant decision making was important, such as recreational gambling.

He then moved into insurance and wrote: "...It is not surprising that premiums are dependent on the odds computed from this information. However, too much information is adverse, because the outcome becomes determinist rather than statistical. As an extreme case, imagine that genetic and medical evaluation could pinpoint a young married person's death from natural causes within a year or so. Some people could not obtain insurance except at exorbitant rates, while others would not bother to purchase insurance at all. ..."

I agree with Joel (and also Howard's related comments) that, if DNA markers and other medical tests could predict the death of a young person within a year, life insurance companies would suffer because high-risk persons could not afford the premiums and low-risk persons might not buy insurance.

Insurance only makes sense as a way of spreading a small, random risk over a large population. If there is 1 chance in 10,000 that a home will be lost to fire this year, each of 10,000 homeowners could pay 1/10,000 of the value of their home into a fund and there would be money to pay the one homeowner who lost their home that year.

If we had perfect knowledge of exactly which home would be lost to fire this year, those of us with safe homes would not buy insurance and the homeowner with 100% risk couldn't get insurance.

Similarly with life insurance and annuities. Perfect advanced knowledge of exactly when each of us will die would destroy any reasonable insurance program.

We all agree on that! I think we also agree that insurance companies should be allowed to use age and geographic location in computing their rates.

Where we seem to disagree is on the use of highly personal information by insurance companies, such as medical history and, particularly, DNA markers.

Howard gave a great example of why a consumer would shop around for an insurance company that uses less information, if that would benefit them. A high-risk person would get a better rate for life insurance if the company did not use medical history and DNA. A low-risk person would get a better rate for an annuity if the company did not use medical/DNA data.

The best advice is to shop around for the best rates available from established, ethical insurance companies!

So, why do I favor the use of ALL information, including highly personal information, by insurance companies? It is because I am selfish! I have good health for someone my age, an active lifestyle, good genes based on my parent's longevity, and I live in a safe area and so on. The more information and the finer the partitioning of the population the insurance company uses, the lower my premiums will be! (I might feel the other way if my risks were higher than average.)

I also think it is better for the total society if healthy and safe lifestyles are encouraged by charging premiums based on the best possible estimate of risk.

We are far from being able to exactly predict the year of death for a given person or fire in a given home. (If we ever get to the point the predictions are near-cerainty on an individual basis, I may change my opinion.)

Ira Glickstein

Howard Pattee said...

HOWARD writes: I don’t think Ira understands my view of why his ideal insurance company that uses “highly personal information such as medical history and, particularly, DNA markers” is likely to survive. I agree that his ideal company sounds good for him, but I doubt it is good for the insurance company. First, we should assume that insurance company statisticians are smart, and that if this refined policy were profitable they would use it. I think companies do not use ALL the individual’s personal information for the simple reason that it is not as profitable as using less refined information and diversifying risks over larger non-correlated population subgroups.

IRA says, “I agree with Joel (and also Howard's related comments),” but then he repeats his original position: “So, why do I favor the use of ALL information, including highly personal information, by insurance companies? It is because I am selfish! I have good health for someone my age, an active lifestyle, good genes based on my parent's longevity, and I live in a safe area and so on. The more information and the finer the partitioning of the population the insurance company uses, the lower my premiums will be!”

HOWARD responds: Of course you favor finer partitioning because of your above-average good genes and good health; but remember, by definition of “above average” you are a minority and the “average health” majority will not buy from your refined-partition company because it must charge the majority more than a competing company that uses less information (coarser partitioning) and more diversification. Consequently you would be investing in a risky company that is not competitive unless its sales pitch can fool many customers (which unfortunately many companies can).

Ira Glickstein said...

From: Ira Re: Ignorance / Insurance

Howard: Thanks for continuing this discussion! As I see it, there are two issues with respect to insurance companies using ALL available information or only using PART:

1) What will maximumize profitability of insurance companies?

and

2) Public policy that prohibits use of certain personal information, such as race.

(1) INSURANCE COMPANY PROFITABILITY

Howard says "...we should assume that insurance company statisticians are smart, and that if this refined policy were profitable they would use it."

I agree they are smart, but their non-use of certain personal information is, IMHO, due to legal public policy considerations rather than profitability. (See (2) below for a discussion of public policy.)

Howard argues that an insurance company that used ALL available information would lose business because: "the 'average health' majority will not buy from your refined-partition company because it must charge the majority more than a competing company that uses less information (coarser partitioning) and more diversification."

I disagree with that analysis. Say Company "A" uses personal info to partition the population into three health segments: "Good", "Average", and "Poor" and sets their rates accordingly. Their rates for the "Average" segment will be the same as the rates for Company "B" that does not use fine partitioning, so they will not lose business in that segment. Their rates for the "Good" segment will be *lower* than Company "B", so they will gain a higher percentage of that part of the population. Their rates for the "Poor" segment will be higher than Company "B", so they will lose business from that segment (good ridance to poor risks). Overall, I think Company "A" will have higher profitability than Company "B" despite their lower rates because they will assume lower risks and have lower payouts. Astute members of the "Good" partition will flock to Company "A" to save money.

(2) PUBLIC POLICY ISSUES

If insurance companies discriminated by race, you would object strongly (and so would I :^). Indeed, according to Joel, the law forbids such discrimination despite statistical justification: "... the government has already had to step in to enforce ignorance. We know that the risk of death for an Afro-American is statistically higher than that of the other ethnic groups. This piece of information must be excluded from a statistical calculation with respect to insurance premiums. Women live longer on the average than men. This piece of information must be excluded in the computation of annuity pay-outs, by order of the supreme court."

Look at http://www.quickquote.com/cgibin/termLifeQuest.pl and you will see it is legal public policy in the US to discriminate against older people and men and tobacco users and consider weight and height in computing Term Life Insurance rates. On the other hand, according to Joel, it is not legal to discriminate against women with regard to annuities!

If medical science came up with a race-neutral and inexpensive DNA marker test that predicted death risk within five years with a 90% confidence, which I think will be possible in the near future, would you allow insurance companies to make use of that test if they chose to do so?

Ira Glickstein

Howard Pattee said...

From: HOWARD Re insurance

I agree with Ira’s analysis. My example was too simple. The point I was trying to make is about the optimum amount of information and the corresponding partitioning of the population. My only claim is that there is an optimum profitability and minimum risk (for the company, not the client) somewhere between Ira’s “ALL possible personal information” and NO personal information. My guess is that finding this competitive balance is largely trial and error because so many other sales pitches and public relations type variables are concerned.

I can’t answer Ira’s question whether I would buy insurance based on my personal DNA life expectancy, because I would need to know more about reliability of the company that would sell such refined policies. It is not obvious that this is a profitable strategy. Many people, including myself, have a feeling that revealing too much personal information is intrinsically risky.

Ira Glickstein said...

From: Ira Re: Insurance. Thanks for your Comment Howard!

You wrote in part: "I can’t answer Ira’s question whether I would buy insurance based on my personal DNA life expectancy ... Many people, including myself, have a feeling that revealing too much personal information is intrinsically risky."

How about law and public policy?

I'd like Comments from you and others on the following:

Assume some substantial insurance companies proposed to institute a standardized DNA test, well-supported by scientific evidence, as a means of assessing risk (and rates) for their life and health insurance policies. Assume further that some privacy advocates objected. If you were a senator or congressperson, or a Supreme Court justice, would you vote to allow such a business practice? Why or why not?

Ira Glickstein