Tag Archives: workplace wellness

Are Your Vendor’s Claims Valid? (Part 2)

The first installment covered regression to the mean. This installment features the fallacy of using non-participants as a control for participants.  This “control” fallacy led the Food and Drug Administration to reject this methodology more than half a century ago. 

And, as we’ll see through examples below, correctly so.

The news of the fallacy and its rejection never reached the employee health services industry—or maybe it reached the industry altogether too well. Either way, vendors of wellness, diabetes, disease management and orthopedic programs routinely compare participants with non-participants, or measure just on participants alone.  Buyers don’t insist on controlling for participation bias, largely due to lack of understanding. Vendors not validated by the Validation Institute (VI) rarely offer to control for participation bias. Such a control would undercut their own performance claims, because participants always outperform nonparticipants. 

Indeed, one of the most dramatic savings figures was achieved simply by separating employees into participants and non-participants, without even giving “participants” a program to participate in.

Participation bias is even more invalidating in employee health services than in drug trials. The latter usually require only taking a pill and tracking results. The former require very active participation. Further, those who initially volunteer and then drop out are never counted as participants. Often, the dropout rate is never even reported. The result is what’s known in the industry as “last man standing” programs, because the only people whose outcomes are counted are the initial voluntary participants who stuck with the program the entire time. 

This study design is a recipe for massive invalidity. Not surprisingly, it has been proven four times that 100% of the alleged outcome of a program using this study design is attributable to the design, rather than to the intervention itself. 

This explains why VI-validated programs – programs that self-select to apply for VI validation because they actually accomplish something – make such modest claims, as compared with invalid vendors. It’s because modest claims are what they actually achieve…but modest valid claims trump massive invalid claims.

“Accidental” proofs of study design invalidity

The beauty of the first two proofs below is that they constitute what a litigator would call “declarations against interest,” meaning that the perpetrators’ own statements invalidate their own arguments. The wellness promoters who conducted these studies accidentally proved the opposite of what they intended to prove, without acknowledging it in the first case, or realizing it in the second.

 These two cases, discussed at length here, are summarized below:

  1. Using the same employee subjects, a program measured outcomes both ways: through a high-quality randomization and also through participants-vs-non-participants; 
  2. As mentioned, participants were separated from non-participants but not offered a program to participate in.

In the first case, a large group of employees without a diagnosis of/history of hospitalization for diabetes or heart disease was divided into:

  1. Group A, to whom invitations to participate would be offered;
  2. Group B, employees “matched” to the invited group using demographics and claims history, for whom nothing special was done.

See also: Are Your Healthcare Vendor’s Claims Valid?

The population was separated before any invitations were issued to Group A, making this a valid — and extremely well-designed — comparison. The “invited” Group A then included both participants (about 14% were willing to submit to the program, of which almost a quarter dropped out, leaving 11%) and non-participants.

The intervention was to use people’s DNA to tell them they were at risk for diabetes or heart disease, and then coach them. Because there were no hospitalizations or ER visits specific to those events beforehand as part of the study design, it would be arithmetically impossible to reduce the relevant hospitalization rate of 0. And yet “savings” of $1,464 per participant was claimed for the first year for the “last man standing” group of the 11% of Group A invitees who actually completed the program, vs. those Group A invitees who declined the invitation.

A cynic might say this massive savings figure was chosen because the program itself cost $500…and a program needs to show an ROI well north of 2-to-1 to be salable.

Using the valid randomized control methodology, the participants, dropouts and non-participants were then recombined into the full “invited” Group A…and compared with the control Group B. Though no cost comparisons were offered, there was essentially no difference-of-differences between these two groups in any relevant clinical indicators. While all changes in both groups were fairly trivial, the latter three trended in the “wrong” direction for the Group A vs. the Group B control.

Along with the fact that there were no relevant hospitalizations to reduce in the first place, the near-total absence of change in clinical indicators makes it impossible for any savings to be achieved, let alone $1,434 per participant, perhaps the highest first-year claimed savings in history. 

This excerpt courtesy of the Validation Institute. For the smashing (and needless to say, hilarious, this being the wellness industry) conclusion, click through here.

Did Biden Just Kill Wellness Programs?

After a decade of criticism from me (here on ITL, in particular) and a growing number of others, workplace wellness died last week. It had been on life support since Jan. 7. 

The bottom line: Brokers and advisers should now be aware both of the professional risks they are taking by pitching clinical wellness programs and of the career risk they are subjecting their clients to by encouraging use of the programs. Most clinical wellness programs now technically violate the Americans with Disabilities Act (ADA). 

You might be wondering why this is the first you are hearing of it. That’s because this is new news, as of last Thursday. Here is a timeline:

  • Jan. 16, 2018: Federal court case “vacates” the expansive, corporate-friendly EEOC rules allowing large incentives and penalties used in clinical wellness programs and directs the EEOC to write new rules accordingly.
  • July 16, 2019: AARP and unionized Yale employees sue Yale for imposing such penalties. 
  • Jan. 7, 2021: The EEOC finally writes the new rules. As directed by the court, these rules no longer allow employers to subject employees to large forfeitures (penalties or foregone incentives) for refusing to submit to clinical wellness programs. The three Republican commissioners, at the request of the U.S. Chamber of Commerce and Business Roundtable, carve out a large loophole for increasingly unpopular and possibly harmful outcomes-based programs. (Those are the programs where you get heavily fined for not losing weight.)
  • Jan. 8-20, 2021: The dog doesn’t bark in the nighttime: The Federal Register does not publish the proposed rules for public comment. The absence of this routine step suggests these rules are already generating blowback.
  • Jan, 21, 2021: The Biden administration shakes up the EEOC, promoting the two Obama appointees, both of whom have strong pro-employee biases, and demoting the Republicans.
  • Jan. 21, 2021: The Biden administration freezes all proposed rules from every agency from publication for comment in the Federal Register. This indicates a thorough realignment of priorities is in the works.

Here are three ways to know that this timeline means the end of programs that, like most, focus exclusively on risk assessments, screens and coaching.

First, the ADA requires that clinical inquiries and exams be “voluntary.” Absent rules defining “voluntary,” incentives can’t be imposed. Very few employees would voluntarily allow their employers to hire unlicensed wellness vendors to “play doctor,” especially with no law proscribing the vendors’ use of that data. (HIPAA does not apply to standalone wellness companies.)

Second, the new EEOC leadership alignment eliminates the chance that the EEOC will re-publish the existing rules. If indeed the EEOC publishes new rules once new rules are green-lighted for publication in general, the agency will specifically allow only minimal incentives. Penalties? No way.

See also: 3 Tips for Increasing Customer Engagement

Third, that 2019 Yale lawsuit has yet to be decided. The judge has been sitting on the summary judgment briefs for more than a year now, presumably waiting for the rules to be promulgated. With no rules, Yale is now in clear violation of the ADA, so the decision should come down within weeks — in favor of the plaintiff employees. This decision will resonate with class action attorneys everywhere, as punitive programs will be “low-hanging fruit” for them. The decision will unleash a flood of claims by 2022.

Here’s how to keep your clinical program intact

The rules – and the EEOC’s jurisdiction in general – apply only to clinical programs. You can still offer activity-based programs and still tie up to 30% of insurance dollars to those programs. The reason? Activity-based programs are governed by the Affordable Care Act (ACA) only, not the ADA. They include physical and mental activities, as well as seminars, quizzes, volunteer work and more.

The difference between the two program types is best illustrated by an example: You can make employees wear FitBits to track their steps, but you can’t track their pulses. (It turns out tracking steps doesn’t accomplish anything anyway.)

There is a way to maintain your clinical programs. (Why anyone would want to is a different issue, covered at length here.) You can offer them side-by-side with your activity programs but allow employees to fully satisfy program requirements and earn the full incentive (or avoid the penalty) without submitting to any clinical test or inquiry. 

Suppose an employee needs 500 “wellness points,” and a risk assessment and a screen are worth 250 apiece. “Lunch ‘n’ learn” nutrition seminars are worth 50 points each, but employees are currently capped at 5 such seminars. That program would be noncompliant. Raising the cap to 10 or upping each seminar to 100 points solves the entire problem by allowing employees to earn the full incentive by sitting through all those sessions.

Employers concerned about the time commitment for something like that could offer health literacy quizzes instead. [Note: My company offers those.] This simple one-page flyer checks the ADA/EEOC compliance box by offering both quizzes and conventional screenings. It also encourages the highest-risk employees to choose screens, while a large majority of employees would learn more about health, healthcare and health benefits through quizzes.

As a belt-and-suspenders protection against EEOC exposure, Quizzify also provides indemnification to employers. With or without health literacy, your own wellness vendor should reconfigure its program to both comply with the ADA and to indemnify you and your client if it doesn’t. 

Especially with indemnification, expanding the number of activity-based offerings to allow employees to hit their points target while avoiding clinical programs solves this new problem. The challenge will be to make those non-clinical activities useful.

If you want to explore this topic further, Insurance Thought Leadership is co-sponsoring a seminar on this very topic on Monday, Feb. 1 at 1pm EST. You can register here.

A Workplace Wellness Skeptic Lets Loose

This an excerpt from an interview that HIStalk conducted with Al Lewis, JD, the author of several books on healthcare outcomes, the operator of the website, They Said What? Because the Wellness Industry’s Pants Are On Fire, and the founder and CEO of Quizzify. The full interview can be found here

Tell me about yourself and what you do.

I am CEO and quizmeister-in-chief of Quizzify, which is a an employee health literacy company. As we say, wiser employees make healthier decisions. However, I believe we are having this conversation because of my personal blog, which is called, “They Said What?” in which wellness vendors, diabetes vendors and related vendors are critically analyzed to in fact show that they usually don’t achieve what they claim to achieve.

You’re offering $3 million to any company that can convince an impartial panel that its program can save employers money. Do you have concerns about having to pay up?

None whatsoever. The entry fee is $300,000, and, believe me, it’s worth [the risk] with this impartial panel of five judges, of which I only get to appoint one and the burden of proof is on me. They don’t have a chance, which explains why nobody has tried to take me up on it.

Is it lack of knowledge or intentional deception that motivates wellness companies to sell services to employers without having sound science behind them?

Confucius put it very well. He said, and in those days it was all gender-specific, that, “When a man makes a mistake and it’s pointed out to him and he doesn’t correct it, he is telling a lie.” So at this point, these folks know they are lying. They have made the gamble, and it’s a good gamble, that vastly more people are going to read their ads than are going to read my website. So what they do, and they’ve gotten very good at this in the last couple of years, is simply ignore my postings instead of responding to them so as not to create a news cycle and a whole discussion.

Is the available science good enough that they could do it right if they really wanted to?

I would say that, for wellness generally, it is mathematically impossible to save money. There are not enough wellness-sensitive medical events. Even if you were to reduce 100% of them, you could not pay for most wellness programs. I’m not going to say it’s impossible, but it has clinically never even gotten close to that 100%. The typical reduction in risk is 0%, somewhere between minus 2% and plus 2%, while you would need a mathematically impossible 100% to 150% reduction to break even.

Most vendors are counting on the fact that most employers have absolutely no idea how many of their employees go to the hospital every year for diabetes. I could tell you if you like, unless you want to take a guess. Out of 1,000 people under the age of 65, how many go to the hospital with a primary diagnosis of diabetes in the insured population?

I’ll say two.

Actually, that’s very close. It’s more like one. Occasionally, I run health and wellness trivia contests at conferences. How does the radiation in the CT scan compare with the radiation in an X-ray? But I also throw in that specific question. If you added all the diabetes events and all heart attacks together in a typical employer population, what would the rate be per thousand? In fact, it would be two, if you put both of those together. The guesses that I get are usually somewhere between 20 per thousand and 200 per thousand.

What about the perception of the incidence of chronic disease in general?

It’s not my take, it’s the world’s take. Because I do this show of hands thing, I do these trivia contests all the time. The employer benefits community thinks it is between about 20 and 200 of these events per 1,000 employees. Which of course makes no sense whatsoever. This is just what they say because they get bombarded with information talking about all the people who have diabetes and all the expensive chronic disease. Let’s take those two things one at a time.

A lot of people do have diabetes. They may not even know it. It’s not going to become an issue for them for many years after they find out. If in fact an employer intervenes, they may possibly be able to control it. But what the [employer is] doing is saving Medicare money down the road because virtually nobody goes to the hospital with diabetes before the age of 65. Yet employers want to start paying for medication for these folks, so it’s a net increase in cost.

And then your other point of chronic disease. I’ve written extensively on this fallacy that 86% of cost is chronic disease. If you read… carefully, you’ll find that they are saying that 50% of adults have chronic disease. Now if you’re defining chronic disease that broadly, you’re including a whole lot more things besides the things that a wellness vendor can get to. You’re including arthritis. You’re including hypertension. Who doesn’t have hypertension?

If you put all that together and say, “Let’s count every dollar that someone with hypertension spends on healthcare….” So. someone with hypertension breaks [a] leg, you count that. You probably don’t even get to 86%, but most of that is also going to be in the over-65 population. In the under-65 population, the major drivers of costs are birth events and musculoskeletal.

The wellness vendors have done a great job of moving the goalposts. It used to be they would say, “You’re going to get a three-to-one financial return.” Then they started saying, “You’ll get a one-to-one return.” Now they’re saying, “There is really no financial return, but the employees will be healthier.”

If you actually look at the health of the employees … I’m not going to name names, except to say that there are a handful of vendors, generally the ones validated by the Validation Institute, that get more than a trivial improvement in health. There are other vendors — and I don’t mind naming names; Interactive Health and Wellsteps come to mind — where employees actually get worse as a result of these programs.

If that’s the case, won’t those companies eventually get fired for failing to deliver?

Some number of them are getting shown the door, but new employers are coming in. The problem is that the vendors have figured out how to measure outcomes fallaciously in such a way that most employers and most consultants aren’t going to catch them. They compare participants [with] non-participants, for example. It’s been proven up, down, sideways, backwards, forwards and eight ways to Sunday that every iota, every dollar of savings in a participant versus a non-participant comparison is due to the mindset of the participants versus the non-participants and not to the program.

How do I know that? There are several data points. Studies have benchmarked those things and found exactly that. But the most dramatic one is a company called HealthFitness Corporation that did a wellness program for a company called Eastman Chemical. They separated the groups into participants and non-participants in Year Zero. But due to a whole bunch of incompetence and delays, they didn’t get the program started until Year Two. By the time they started the programs, the participants had already dramatically outperformed non-participants.

The funny part about that is that my nemesis, the Snidely Whiplash to my Dudley Do-Right or the Lex Luthor to my Superman, was stuck with this, so he moved the goalposts. He said, “Oh, we overlooked that. That was our bad. We weren’t competent enough to realize that the program had actually started in Year Zero, not in Year Two. Therefore, you don’t know whether it’s due to the participants or non-participants.”

That turned out to be a big enough lie. And I don’t mind saying, oh, I’ll say on the record, Ron Goetzel is a liar. He can go ahead and sue me. The difference between him and me is that, if he calls me a liar, I’ll have him in court the next day. [Editor’s Note: We have emailed Goetzel to see if he wants to respond or offer a general defense of the economics of wellness, as he once did via an article we published. If he does so, we will update this article. To our knowledge, he has not yet responded to the original HIStalk article, published last week.]

They put out a graph that shows suddenly that the program started in Year Zero, not Year Two. The people who actually did the program got upset enough with that. If you go back and look at the website now, they have in fact replaced the lie with the truth, which is that the program started in Year Two after dramatic savings had already been found.

You’ve made the case that the simplest way to measure a workplace wellness program’s success is to ask the people who signed up if they participate regularly and see benefit from it. Do most programs fail even that basic test?

There is a tool put out by the Validation Institute that is the most elegant tool for measuring the cost-effectiveness of programs that I’ve ever seen. We are big supporters of it. You ask employees two questions. How much did you use something? You may not even have to ask them that because you already know. Then, did you find it useful? Then you multiply the number of times somebody used something times the usefulness they found. That gives you an engagement score as your Y axis. On the X axis is the cost of the program. You plot the engagement score against the cost of the program and you can tell in a single graph how cost-effective your programs are as viewed by employee use, employee engagement.

For the rest of the interview, click here to go to histalk2.com.

$2 Million Reward if Wellness Works!

Does wellness save money?

I say no.

The wellness industry — specifically its trade association, the Health Industry Research Organization (HERO) — says yes.

We both can’t be right. The difference is that I am backing up my conclusion with a $2 million reward, up from last year’s paltry $1 million offer, for showing that wellness works.

See also: What Trump Means for Workplace Wellness  

Beyond that $2 million, I would also send a $1 million donation to the Boise School District to atone for the highly unfavorable coverage it has received about its program, coverage apparently so biased that the CEO of Boise’s vendor, Steve Aldana, called the award-winning STATNews journalist who wrote it a “lier.”

Screen Shot 2016-11-15 at 9.47.06 AM

To win the $2 million reward for yourself and the $1 million for the school district, you just need to prove (using the more-likely-than-not civil standard of proof), the following (to bend over backward to be fair, I will start out by offering to use only materials prepared by your side):

  1. During this millennium, the wellness industry has reduced hospitalizations by enough to break even, using the government’s Healthcare Cost and Utilization Project database. For this one, I will concede in advance that the wellness-sensitive medical event methodology (“potentially preventable hospitalizations”) as described on pages 22-23 of the HERO Outcomes Guidebook is the one to use. (HERO and I agree that non-hospitalization expenses increase.)
  2. The vendor anointed in 2016 as the “best” vendor, Wellsteps, indeed did reduce the costs of the Boise School District by about a third (as the company claimed), specifically by making the employees sufficiently healthier to support that savings (as the company claimed). For this one, I will concede in advance that the raw data collected by Wellsteps is accurate. In other words, we are both working off Wellsteps’ own published reports.

Here are the rules. This is a binding legal offer, as any attorney will tell you.

Panel, Venue and Judges

We each pick two panelists from Peter Grant’s “A-List” of the leading 260 health economists and policy experts (this is an invitation-only email group where health policy and health economics concerns are addressed and debated) that are unaffiliated with either the wellness industry or with my company, Quizzify, and together they pick a fifth.

The parties will convene in Boston for a 2.5-hour finalist presentation, featuring:

  • 10-minute opening statements, in which as many as 15 slides are allowed;
  • 30-minute cross-examinations with follow-up questions and no limitations on subject matter;
  • 60 minutes in which panelists control the agenda and may ask questions of either party based on either the oral or the written submissions;
  • Five-minute closing statements.

Entry Fee and Award

I give you a lien on $2 million as soon as you put $200,000 in escrow to cover the costs of the program, for panelist honoraria, venue, etc., as well as for wasting my time with your quixotry. If I win, I will make a $100,000 in-kind donation to the Boise School District to help compensate them for the fees the district wasted on its wellness program.

Length and content of Submissions

Each side submits up to 2,000 words and five graphs, supported by as many as 20 links; the material linked must pre-date the award application to discourage either side from creating linked material specifically for this contest.

Publicly available materials from the lay media or blogs may be used, as well as from any of the 10 academic journals with the highest “impact factors,” such as Health Affairs, published within the last five years.

Each party may separately cite previous invalidating mistakes made by the other party that might speak to the credibility of the other party.

Either side may cite an unlimited number of “declarations against interest” made within the last five years — meaning comments made by the other party so prejudicial to their own position that the other party would have said them only if they believed these statements to be true. Example: If I said, “Wellness definitely saves money” (except when I said that as an April Fool’s gag a few years back), you could cite that. There is no word limit on these.

See also: There May Be a Cure for Wellness  

Each party can then rebut the other party in writing with up to 2,000 words and five graphs as well as 20 links.

Additionally, we both take a lie detector test. Each side will present the polygraph operator with five questions, and all 10 questions will be asked of both parties. Results are then sent to the panelists.

What if you want to claim the award?

Send $1,000 via Paypal to alewis@dismgmt.com to hold your spot. I will set up an escrow account at Bank of America. Once we both sign the escrow papers, you send the $200,000 to that account, and I’ll give you first lien on $2 million of asset

What Trump Means for Workplace Wellness

Assume, reasonably, that voters chose Donald Trump to be the next president because they feel big business and government are in bed together. If indeed they are, workplace wellness is their sex toy.

There is nothing, certainly in healthcare and possibly anywhere, that more embodies the complete disdain for the average worker than the joined-at-the-hip partnership between big business and government known as workplace wellness.

That claim might seem extreme, but put yourself in the shoes of the average worker. You used to have a good health benefit. But then, following the passage of the Affordable Care Act, your benefits were reduced, and you were put in a high-deductible plan. True, your benefits might have been reduced anyway due to the increasing cost of healthcare, but coincidence to the average person smells like causality.

See also: The Value of Workplace Wellness  

A benefits reduction sounds like a wage cut. However, you are told you can earn some or all of it back. All you have to do is allow your employer (and, yes, it isn’t really your employer, but it smells like your employer) to pry into your personal life with a questionnaire; poke you with needles to do blood tests that over a 15-year period have proved useless at reducing the country’s heart attack and diabetes event rates; and prod you, in violation of all guidelines, to go to the doctor when you aren’t sick.

You (remember, “you” are still the worker) are then left with this Hobson’s choice: You must throw yourself at the mercy of an unregulated wellness vendor that – if last month’s C. Everett Koop award to Wellsteps for being the “best wellness program in the country” is any indication – is more likely to harm you than benefit you, while invading your privacy and sucking up your time.

As an employee, what recourse do you have? Basically none. The Equal Employment Opportunity Commission has no provisions against “voluntary” workplace wellness programs, and the word “voluntary” has now been defined to include even programs with non-compliance penalties that might exceed $2,000. The net result: You can be forced to pay a large fine for refusing to participate in a voluntary program, and there’s nothing you can do about it.

You can’t sue for malpractice because wellness vendors aren’t clinical professionals, and you can’t complain to the licensing authority because wellness vendors aren’t licensed. You can’t claim they violate the industry code of conduct because – unlike everything else, including war — wellness has no code of conduct: The wellness trade association has stonewalled on the code of conduct, which embraces only the simple notion that wellness should respect the dignity of workers and not harm them.

Should you opt to maintain your dignity and not violate clinical guidelines, by declining to be part of a wellness program, you may lose four figures in compensation just by wanting to be left alone to do your job.

Don’t take my word that this is how employees feel. Simply read the comments by employees to any article on wellness.

Meanwhile, what is the government doing? Simple. The government is carrying the Business Roundtable’s (BRT) water. The Senate is in the BRT’s pocket, holding “hearings” that are basically just ads for the BRT. And the president put the EEOC on a short leash after the BRT threatened him.

As members of the BRT, and their like-minded compadres at the U.S. Chamber of Commerce, corporations are gleeful. They can cut benefits and “offer” employees the opportunity to earn them back, or just fine employees directly. One vendor, Bravo Wellness, even dogwhistled to employers that they could get immediate “savings” by fining employees.

What happens now, and what should you do?

Wellness is likely to become a touchstone for all that is wrong with the Affordable Care Act, because, almost uniquely in the ACA, the wellness provision has basically no upside. (Disagree? Show I’m wrong, and claim the $1 million reward I’ve offered to anyone who can show wellness has broken even this century.) The American Association of Retired People (AARP) is already shining a light on wellness via a lawsuit, and the effort may make it much more difficult for the wellness industry, the BRT and the Chamber to hide behind the EEOC.

As representatives of the employers who may very well be abusing employees (and not knowing it, any more than the Boise School District realized it was being snookered by Wellsteps until the problem was exposed by a leading healthcare journalist — even though the invalidity and ineffectiveness of the district’s wellness program was perfectly obvious to Wellsteps’ colleagues on the award committee), you should get ahead of this curve. Drop punitive wellness programs, or programs with low participation (which reflects low satisfaction). Or swap out programs that “do wellness to employees” for programs that “do wellness for employees.” The difference is fairly self-evident. Are employees lining up, or do they need to be coaxed? Are there big bribes or fines involved? Is the program something you yourself would do without an incentive?

See also: ‘Surviving Workplace Wellness’: an Excerpt  

You shouldn’t need to wait for the law to change to make changes yourself now. “Pry, poke and prod” programs were a bad idea to begin with, and the passage of time and rise of populism hasn’t made them any better.


The editorial viewpoint in this article, though reflecting my opinion, is colored by my leadership of Quizzify. Quizzify does not “pry, poke and prod” employees, but rather just enhances their knowledge base in an entertaining way. Not just theirs – even yours. Play the sample game on the site and see for yourself. We hope to benefit from the likely retreat from government support for intrusive and ineffective wellness programs in the new administration. On the other hand, you are free to publish opposing comments or viewpoints. Join the conversation, even if it means hollering at me by quoting people who know they are wrong claiming savings they know are fabricated.