US Focus

Asking AI to Pick a Health Plan? One Test Found an $836 Average Overpay

Aditya Kumar JhaAditya Kumar JhaLinkedInAmazon·November 6, 2026·10 min read

Using AI for 2027 open enrollment? One test's picks averaged $836 a year above the best plan. What to verify, and a safer way to use chatbots.

The average HealthCare.gov shopper will see about 100 plans from five or six insurers for 2027, according to CMS. Benchmark silver premiums are up 13% on average after a 26% jump the year before, per KFF. It is no surprise that people are asking a chatbot to sort this out. KFF found 32% of U.S. adults used AI chatbots for health information in the past year, and an eHealth survey found 62% of Medicare beneficiaries who plan to review coverage this fall intend to use AI tools to compare plans. (Sources: CMS, KFF open enrollment preview, KFF poll, eHealth)

One published benchmark suggests AI is useful here, and not trustworthy on its own. In it, the best current models available to free users picked a near-optimal plan 57% of the time. On average, their picks cost $836 a year more than the best plan. That is a big improvement on the previous generation's 36% and $1,890, and still not a number to bet your deductible on. The trick is to give AI the math and keep the decision. Here is how.

Insight

Quick summary: Don't ask a chatbot which health plan to buy. Ask it to compare official plan documents you paste in, across a healthy year, a typical year and a worst-case year. Verify doctors, drugs and final prices on HealthCare.gov or Medicare's Plan Finder before enrolling. Leave your name, Social Security number and medical records out, since HIPAA generally doesn't cover consumer chatbots. For 2027 HealthCare.gov coverage starting January 1, enroll by December 15. Medicare open enrollment ends December 7. This is general information, not insurance, medical or financial advice.

The deadlines to check first

CoverageWindowDate people missCheck here
HealthCare.gov statesNov. 1, 2026 to Jan. 15, 2027Dec. 15 for a Jan. 1 start; Dec. 16 to Jan. 15 starts Feb. 1HealthCare.gov
State-run exchangesVaries by stateIdaho reportedly ends Dec. 15; some others run later in JanuaryYour state's exchange
MedicareOct. 15 to Dec. 7, 2026Dec. 7, with changes effective Jan. 1, 2027Medicare.gov Plan Finder
Employer plansSet by your employerYour employer's deadline; ask HRYour benefits portal

These dates are a good first test of a chatbot. A 2025 federal rule would have ended open enrollment on December 15 starting with 2027 coverage, but a federal court in Maryland vacated that provision in June 2026, the government has appealed, and CMS lists January 15 for HealthCare.gov. An AI drawing on older articles could tell you the window closes a month early. (Sources: CMS, BenefitChamps, Healthcare Insider, Becker's)

What the $836 test actually measured

The number comes from Khris Dai, an actuary and the founder and CEO of Visuary AI, writing in Contingencies, the American Academy of Actuaries' magazine, with a related preprint on SSRN. She ran 1,564 responses from OpenAI and Anthropic models across five ACA-compliant plans and 12 claims-based consumer profiles. A pick counted as near-optimal if it landed within $250 a year of the best plan in a claims-based pricing engine. This is one author's benchmark, from the founder of an AI company, and the Contingencies article does not name the exact models. Read it as a useful signal and not a final verdict. (Sources: Contingencies, SSRN)

  • Free-tier models: 57% near-optimal for the best current models, against 36% for the prior generation. Mean regret, meaning extra annual premium plus out-of-pocket cost, was $836 against $1,890.
  • Reasoning effort mattered: the same model hit 56% with high reasoning effort and 33% without, with regret of $940 against $1,780.
  • Conditions and medications helped: accuracy reached 61% across four current models and 73% for the strongest.
  • Too much detail did not help: adding visit counts and care episodes brought no overall gain. For medium users, accuracy fell from 72% to 28%, and models tended to underestimate out-of-pocket costs.
  • Prices were the unlock: given market-level prices for expected services, the strongest model reached 90%.
  • Hidden priorities: in 80% to 86% of top recommendations from the most advanced models, the main goal was the lowest expected annual cost. Protection against a costly year usually came second.

That last finding matters most. A plan that is cheapest on average can still be the wrong plan for someone who can't absorb a bad year, and a chatbot may never tell you it made that trade. Dai also warns that responses are probabilistic, so the same question can produce a different pick. She gives an example of plan paperwork itself misleading: a Summary of Benefits and Coverage suggested a $50 mental health copay from the first visit, while the filed plan design required meeting the deductible first.

Why 2027 is a bad year for stale answers

Insurance rules moved quickly, and AI answers can lag behind them. Louise Norris of healthinsurance.org and medicareresources.org tested AI tools this summer and found errors delivered with confidence, according to the Association of Health Care Journalists. One tool used outdated poverty guidelines and told her she qualified for Colorado marketplace subsidies when her income actually pointed to Medicaid. Another missed Minnesota's Basic Health Program completely. Here is what to double-check. (Source: Association of Health Care Journalists)

  • The subsidy cliff is back. Enhanced credits ended after 2025, and for 2027 coverage premium tax credits generally stop at 400% of the poverty level: $63,840 for one person or $132,000 for a family of four in the 48 contiguous states, based on 2026 guidelines of $15,960 and $33,000.
  • Repayment caps are gone. The IRS says that beginning with tax year 2026 you must repay the full amount of any excess advance credit. A low income guess can now become a tax bill with no ceiling.
  • Out-of-pocket ceilings rose 13.2%. The ACA maximum for in-network costs is $12,000 for one person and $24,000 for a family in 2027, up from $10,600 and $21,200. Plans can set lower limits.
  • Insurers moved. As of late October, KFF counted 11 carriers leaving at least one state and 7 entering a new one, and Cigna is leaving the individual market in all 11 of its states. Your old plan may not exist.
  • Eligibility narrowed. Starting in 2027, KFF says premium tax credits are limited to lawful permanent residents, Cuban and Haitian entrants and certain Compact of Free Association residents among immigrants.
  • Medicare drug costs shifted. The maximum standard Part D deductible rises from $615 to $700, and the out-of-pocket cap from $2,100 to $2,400.
  • Medicare Advantage limits rose. The maximum in-network out-of-pocket limit goes from $9,250 to $9,850, even as CMS projects the average premium falling from $14.37 to $12. A cheaper premium and a bigger potential bill can arrive together.

CMS also says more than 75% of HealthCare.gov purchasers will have access to a plan at $50 or less in 2027 and that buyers will see about 35 HSA-eligible plans on average. That can be great news. It can also steer you toward the lowest monthly price and away from the question that counts: what happens in your worst year? (Sources for this section: CMS, HHS, IRS, KFF, Milliman, Association of Health Care Journalists)

The safer way to use AI: a 5-step routine

1. Shortlist on the official site first

Start with real 2027 prices, not a chatbot's memory. HealthCare.gov lets you preview plans and estimated prices before logging in, and search your doctors, hospitals and prescriptions while comparing. Each plan links a summary of benefits, brochure, provider directory and covered drug list. Pick two to four finalists. Medicare shoppers can do the same on Plan Finder, which CMS says shows premiums, costs, benefits, formularies and Summary of Benefits documents. (Sources: HealthCare.gov, CMS)

2. Give AI the documents, not your identity

Plan summaries are public, so pasting them in costs you no privacy. Your medical chart is another matter. KFF found 41% of people using AI for health information had uploaded personal medical information, or 13% of all adults, while 77% of adults worry about the privacy of what they share. Fortune notes that OpenAI does not describe its consumer health product as HIPAA compliant, since consumer health apps are not covered by HIPAA. OpenAI says connected records and Apple Health data in Health in ChatGPT aren't used for training or ad targeting, but that is a company promise and not a HIPAA duty. For plan shopping, you rarely need more than an age, a county, a rough income and a medication list. (Sources: KFF, Fortune, Thurrott)

3. Ask for three years, not one answer

This is the step that fixes the hidden-priorities problem. Copy this prompt and adapt it:

Pro Tip

Prompt: "Using ONLY the attached 2027 plan summaries, build a table comparing each plan's yearly premium, deductible, out-of-pocket maximum and my cost for these services: [list]. Calculate total annual cost for a healthy year, a typical year with [your care] and a worst-case year hitting the out-of-pocket maximum. Name the plan year and page for every figure. If something is missing, write UNKNOWN. Do not recommend a plan until you list what you assumed."

Here is why that table matters, using two made-up plans for illustration only. Plan A costs $300 a month with an $8,000 deductible and a $10,000 out-of-pocket maximum. Plan B costs $450 a month with a $3,000 deductible, 20% coinsurance and a $7,000 maximum.

Illustrative yearPlan A totalPlan B totalWinner
Healthy year ($500 of care)$4,100$5,900A, by $1,800
Typical year ($5,000 of care)$8,600$8,800Close to a tie
Worst case (hits the maximum)$13,600$12,400B, by $1,200

Neither plan is simply right. Plan A suits someone with savings who rarely needs care. Plan B suits someone for whom a surprise bill would be a crisis. Real plans have copays and exceptions that this simple math ignores, which is why you want AI working from each plan's actual document.

4. Add prices, then run it twice

Since prices drove the biggest jump in the Dai benchmark, supply estimates for your likely services if you have them. Don't drown the model in clinical detail, since that didn't help in her test. Because answers vary, ask the same question in a fresh chat or on a second model. If the picks differ, find out which assumption changed. That disagreement often tells you more than either answer. (Source: Contingencies)

5. Verify the three things AI can't see

Before you enroll, confirm your doctors are in network, your drugs are on the formulary at the tier you expect, and the price matches your own application. AHCJ's review stresses that chatbots may not know a person's doctor network or formulary details and that wrong answers sound just as confident as right ones. Call the doctor's office too, because directories can lag. For income, use your best realistic estimate and update it if things change, since the repayment cap no longer protects you. (Source: Association of Health Care Journalists)

Chatbot vs. the free alternatives

OptionBest atWeak spotCost
General AI chatbotExplaining terms, building cost tables from documents you supplyMay miss current rules, networks and formularies; answers varyFree or subscription
HealthCare.gov toolsReal prices, doctor and drug lookupDoesn't explain your trade-offs in plain languageFree
Marketplace call centerApplication questions, at 1-800-318-2596Won't choose for youFree
Local assistersIn-person help, found through HealthCare.gov's local help toolLimited capacity after Navigator funding cuts, per KFFFree
Medicare Plan Finder and SHIPOfficial Medicare comparison plus unbiased counselors who don't sell plansHeavy demand during open enrollmentFree

A human can matter more than usual this year. KFF reports a pause on new agent and broker registration on HealthCare.gov's platform until February 2027, and an eHealth survey found 61% of Medicare beneficiaries would trust a person over AI if the two disagreed. eHealth is a broker that sells plans, so weigh that figure accordingly. Watch for scams as well: 58% in the same survey suspect they have been targeted by a Medicare-related scam. (Sources: KFF, eHealth, HealthCare.gov, NCOA)

If you're a student or under 30

HealthCare.gov says you can usually stay on a parent's plan until 26, even if you marry, change school status, live apart or aren't claimed as a dependent. On a Marketplace plan, coverage can generally last through December 31 of the year you turn 26. Catastrophic plans are mainly for people under 30 or with a hardship exemption, and KFF says a broader expansion of eligibility is paused for 2027. A tip for students who go to school far from home: ask whether a parent's plan network works near campus. That is our suggestion and not a rule, but it is exactly the sort of detail a chatbot can't verify for you. (Sources: HealthCare.gov, KFF)

Frequently Asked Questions
01Can ChatGPT or Claude pick my health plan?

They can help, but don't let them decide alone. In one benchmark, the best current free-tier models picked a near-optimal plan 57% of the time, and their picks averaged $836 a year above the best plan. Use AI to compare documents, then verify.

02Is it safe to upload my medical records to an AI chatbot?

It's rarely necessary for choosing a plan. HIPAA generally doesn't cover consumer AI products, so protections depend on company policy and other laws. Share a medication list and rough details instead of records.

03What's the deadline for 2027 coverage?

On HealthCare.gov, enroll by December 15 for January 1 coverage. Open enrollment runs through January 15, with coverage then starting February 1. State exchanges vary, and Medicare's window closes December 7.

04Why might AI give me the wrong subsidy estimate?

Subsidy rules changed after 2025. AI may rely on old poverty figures or expired rules. One tester found a tool that used outdated guidelines and missed that she qualified for Medicaid.

05What should I never paste into a chatbot?

Social Security numbers, insurance member IDs, full medical records and login details. A plan's public summary of benefits is fine.

06Is AI better for Marketplace plans or Medicare?

The benchmark above tested ACA-compliant plans, so it says little about Medicare directly. Experts quoted by AHCJ warn that state Medigap rules and formularies are easy for AI to miss, so use Plan Finder or a SHIP counselor.

Running the same plan comparison on more than one model is the simplest check in this guide. LumiChats gives you 40+ AI models in a single subscription, so you can see where they disagree on your public plan documents. Keep your personal records out of it, and let HealthCare.gov or Medicare.gov have the last word. This guide is general information, not insurance, medical or financial advice. For your own situation, talk to a licensed agent, an assister or a SHIP counselor.

Was this article helpful?

Found this useful? Share it with someone who needs it.

Free to get started

Claude, GPT-5.4, Gemini —
all in one place.

Switch between 40+ AI models in a single conversation. No juggling tabs, no separate subscriptions. Pay only for what you use.

Start for free No credit card needed
Aditya Kumar Jha
Written by
Aditya Kumar JhaLinkedIn

Published author of six books and founder of LumiChats. Writes about AI tools, model comparisons, and how AI is reshaping work and education.

Keep reading

More guides for AI-powered students.