Is ChatGPT a better judge of probability than doctors? - discussing case studies vs RCTs as reliable indicators of efficacy
Can case studies with few data points but high efficacy outperform "gold standard" large RCTs with anemic results? Can three stage 4 pancreatic cancer reversals count as efficacy of a novel protocol?
UPDATE: March 23, 2025 - added some sections as preamble on the dilemma in medicine of making RCTs (randomized controlled trials) the "gold standard" - it's use as a barrier to entry from competition from cheap generic drugs - and the camoflaging of dogma as "science" - and the lack of understanding of probability in medicine (and that many engineers even don't understand probability from first principles) - this lack of understanding leads to rote use of statistics in many fields - where the tools are passed down from supervisor to student as essentially dogma ("just use this - it works") - I argue that the obsession with RCTs over overwhelming anecdotal/case evidence is partly due to this lack of understanding of probability in medicine
.
UPDATE: March 23, 2025 - added stage 4 pancreatic cancer reversal example from Dr George Fareed and multiple examples from Dr William Makis
.
UPDATE: I posted this on many AI related sub-reddits - while many points were made, one point that was not realized - and that I believe jumps out from this article - is that while ChatGPT has been trained on the science - it has not been trained on the politics. ChatGPT may be a good judge of rare vs exceptionally rare events (which require attention/consideration) - yet ChatGPT has not been trained on the other compulsions of a real world oncology practice at a large hospital. It is quite possible, that once you add inertia, biases and filters to the AI model - sunk costs and compulsion to recover investment, insurer diktat, governmental pressures and incentives, and pressure to deliver hospital revenue projections - ChatGPT may also start behaving more like a human doctor - willing to ignore exceptional events as "flukes" if they divert from existing practice. You will then have a very realistic - less scientific, more dogmatic - and human AI model.
Just like the Challenger disaster exposed the politics in space travel - ignoring safety signals to comply with a politically imposed timeline - a similar dynamic exists in medicine. Lyme disease is ignored, H.Pylori was ignored, Semmelweis was ignored, long covid19 is ignored, vaccine issues are ignored - generic drugs are ignored, but a marginally effective, expensive drug is embraced.
.
.
SUMMARY:
I asked ChatGPT what is the significance of three successive stage 4 pancreatic cancer patients reversing their cancer within 6 months of a novel treatment protocol - and whether it measures up well against randomized control trials (RCTs) of traditional chemotherapy and radiation.
The answer was that reversal of these three is an exceptionally surprising event - and likely to be more effective than traditional treatments that have RCTs but report anemic results:
ChatGPT: "This means that if a single oncologist reports 3 consecutive cases of complete reversal within a 6-month timeframe, it would be a highly improbable event under normal expectations—strongly suggesting that a novel treatment protocol may be responsible rather than random chance."
.
.
The dilemma in medicine - RCTs as dogma
The dilemma in medicine:
- RCTs (randomized controlled trials) are necessary for weakly effective drugs to show efficacy
- RCTs are a necessity for approval of weak drugs
- yet public has accepted RCTs ALWAYS trump anecdotal evidence/case studies
Lack of understanding of probability fuels this illusion.
.
.
RCTs as barrier to entry
RCTs cost a million dollars or more to conduct - which means only patentable drugs are going to get an RCT.
Cheap generic drugs which are being repurposed for cancer will not find the funding for an RCT.
Pharmaceutical industry and medical institutions (that benefit from return on drug patents) - have a financial incentive to create a moat around their products - to create a barrier to entry for competition from cheap generic drugs.
The emphasis on RCTs as "gold standard" - and only acceptable evidence - creates that barrier to entry.
Pharmaceutical industry can surpass this barrier - but a cheap generic drug cannot (no financial backer).
This dynamic creates a world where drugs cannot be accepted if they are cheap - only when they are expensive.
.
.
Lack of understanding of probability
Beyond the financial incentives to keep focus on RCT-capable drugs and off the cheap generic drugs - there is another factor.
Probability is hard to understand even for engineers.
Disciplines like psychology, medicine - in order to cope - adopt certain ways of talking about it.
Which become dogma - as they aren't arguing from first principles.
But perpetuating linguistic logic "RCTs trump anecdotal evidence" - perpetuated as foregone conclusions.
.
In this climate, logic does not prevail.
But emotion and dogma.
Even generic drugs which seem to have a reasonably obvious benefit, and no downside - are shot down for having no RCT.
Many "educated" people in medicine and in institutions play along with this charade.
.
NOTE: I provide an example - the use of Ivermectin for reversing post-day8 residual anosmia for COVID-19 patients is well recognized, and has literature supporting it. All it requires is a 3 day course to see if it will reverse the anosmia. And a 3 day course of Ivermectin is known to be safe. Yet many doctors will refuse to consider it. Even when they know there is no downside, and that the only tool officially in their hospital's arsenal is "olfactory training" - which is costly, takes time and has only a statistical benefit (i.e. cannot guarantee for a particular patient that it will work for them).
.
Many doctors who object to this situation, refer to Penicillin - how it was universally accepted without needing RCTs.
But cannot phrase it in probability language to counter the critical mass of industry players saying they will only accept "gold standard" RCTs - and "this is how science is done" and such phrases.
These are all signs that a dogmatic approach is being used - but is being called "science".
I try to show here how three stage 4 pancreatic cancer reversals using a particular protocol - are a statistically significant event.
.
.
Stage 4 cancer reversals using novel protocols
The motivation for this article was that I was seeing way too many stage 4 cancer reversals among doctors who are beginning to use the metabolic approach (low carbs, intermittent fasting) and Fenbendazole/Ivermectin/Mebendazole based treatment protocols for cancer.
(for low carbs/intermittent fasting - see Dr Thomas Seyfried in my previous post - and for Fenben/IVM/Meben see Dr William Makis in my previous post)
Not only that, I was seeing nearly all YouTube videos about cancer or Fenben/IVM/Meben have comments from people reporting reversal of cancer - alone or in combination with traditional chemotherapy. And there are sometimes stage 4 cancers - including stage 4 pancreatic cancers in the mix.
Prior to the pandemic, there were already large Facebook groups (usually private) devoted to Joe Tippens protocol (Fenben, CBD oil, Curcumin) - which have many adherents - suggesting a market acceptance of this cheap therapy.
This raises the question - are there otherwise this many stage 4 cancer reversals with traditional chemotherapy and radiation therapy?
According to ChatGPT, a typical oncologist is unlikely to ever see a stage 4 pancreatic cancer reversal in his lifetime. And there are typically zero case reports of a stage 4 pancreatic cancer patient reversing their cancer in any given year.
The anecdotal reports mentioned on social media also nearly always report that they took Fenben/IVM on the side - and when they reversed their cancer, their oncologist was shocked, but expressed zero interest in understanding what happened. Occasionally these reports talk of an oncologist who quietly tells them “to continue whatever you are doing”. This is the state of curiosity in medicine - as the compulsion to entrain physicians to hospital protocols - also leads to an abandonment of curiosity - as that curiosity is not actionable - the doctor cannot do anything with that curiosity or is disincentivized from looking at anything beyond the traditional protocols (I do not examine the profit motive in this article). Many oncologists fail to even mention Vitamin D as a factor (see my previous article).
As seen in the pandemic, there is a culture of physician compliance to hospital protocols - even in the face of overwhelming anecdotal evidence to contrary treatments. And a culture of entrainment exists - that gravitates towards chemotherapy drugs and radiation even when the results are anemic - they however must be vetted by "gold standard" RCTs to pass muster.
(I address the efficacy of traditional chemotherapy in my previous article - and this article below also highlights the statistics for stage 4 cancers - which suggests traditional approaches have poor efficacy)
.
.
The case of generic drugs and lack of RCTs
.
Generic drugs often have lots of case reports supporting their use for cancer and other diseases.
However, they usually lack randomized control trials (RCTs) because these cost millions of dollars and everyone in the pharmaceutical industry knows that a generic drug is unlikely to get an RCT, since generic drugs are not expected to bring return on investment.
Some of these protocols have been in use for many years - and a large segment of the public has now direct experience of their efficacy. As a result any random video on cancer on YouTube has numerous comments from users about cancer reversal using these protocols among friends and family.
Yet doctors continue to say - that without RCTs for generic drugs - there can be no tolerance for their use.
.
.
Stage 4 pancreatic cancer reversal examples
In the examples below, the protocols being used are metabolic approach plus Fenbendazole/Ivermectin or Ivermectin/Mebendazole type protocols.
Here is an example of a stage 4 pancreatic cancer reversal from Dr George Fareed:
https://x.com/GeorgeFareed2/status/1902879429354983510
Dr George Fareed
>Stage 4 pancreatic cancer patient now has CA19-9 normal. txted: “Ive had no chemo for last 8-9 wks. We decided today no more IV chemo. We r switching to oral. Chk tumor markers once a month & next ct scan in two months. No surgery”
>@PierreKory
>@Fynnderella1
.
Image of tweet:
.
Masimaux has compiled a list of stage 4 cancers that Dr William Makis has reported:
https://x.com/masimaux/status/1899052409021685965?t=4MPmqT8ZBeI9rb4ZcD9XNw&s=19
>Dr. Makis H/T!
>
>I identified 16 cancer patients (9 with stage 3/4) on the @MakisMD's timeline who went into complete remission with ivermectin, fenbendazole, mebendazole...
>
>Mind-blowing!
>
>Read the article here:
>
.
.
Radiation oncologist Dr William Makis has tweeted on the stage 4 pancreatic cancer reversals he has seen:
https://x.com/MakisMD/status/1899392089537810866?t=sKRJC-vmIsVF-a7YbOnoqA&s=19
>NEW ARTICLE: IVERMECTIN and FENBENDAZOLE Testimonial - 57 year old man with Stage 4 Pancreatic Cancer sees 87% drop in CA19.9 from 10951 to 1438!
>...
>I have posted a number of Stage 4 Pancreatic Cancer cases in the past 6 months that have dramatically improved or are cancer free.
>
>With repurposed drugs, we are moving towards a reality where previously "incurable cancers" could become curable.
>
>With repurposed drugs, Stage 4 Pancreatic Cancer will be considered curable.
>
>But it's not for the faint of heart.
>
>Ivermectin 120mg/day was over 1.5mg/kg/day
>Fenbendazole 1332mg/day is higher than what Stanford University published
>
>You cannot be afraid of high doses and you may have to struggle a little bit with some transient side effects.
>
>But the results shock even the most experienced oncologists.
.
Image of tweet:
.
.
Chemotherapy and Radiation therapy
.
On the other hand we have RCTs for each of the approved chemotherapy options and for radiation therapy.
And this option is much more appealing to most doctors - who are bound by their hospital protocols. Their familiarity with the traditional approaches only - means that patients are not informed about newer protocols, and are mostly guided back towards the familiar - even though efficacy may be poor.
An example of a drug that had anemic efficacy, but was controversially still approved, is the Alzheimer's drug Aducanumab (Aduhelm):
.
The Aducanumab Controversy: Accelerated FDA Approval Leads to Agency Distrust
Anastasia Climan
July 19, 2024
.
Chemotherapy and radiation approaches have low efficacy for stage 4 cancers - yet most large hospitals will not suggest alternate approaches, but will shepherd the patient to these traditional treatments.
Even when the alternate approaches have few side effects, and thus little downside to their use either alone or in combination with traditional chemotherapy or radiation.
.
.
Asking ChatGPT for help with probabilities
.
In this article I examine whether these doctors are misinformed about the probabilities involved.
And whether there can be greater evidence for benefit with these case studies than the competing "gold standard" RCTs.
I ask ChatGPT to guide me through the questions:
- how rare are stage 4 cancer reversals?
- how many stage 4 cancers are there in the US in any given year?
- are the anecdotal reports of stage 4 cancers noticeably higher in number than we would otherwise expect?
- how many stage 4 cancer reversals can a typical oncologist expect to see in his lifetime?
- and can a doctor observing three stage 4 pancreatic cancer reversals within 6 months of treatment with a novel protocol assume he has the evidence that beats the "gold standard" RCTs for traditional chemotherapy drugs?
.
NOTE: we will not explore the financial aspects affecting treatment decisions - patented drugs cost more and are more profitable for hospitals than generic alternatives - or the fairness of doctors who ask for RCTs of generic drugs - knowing full well that the industry does not devote money to RCTs of generic drugs - RCTs cost millions of dollars and there is no expecatation of payback with generics.
.
.
Asking ChatGPT
.
Question:
What percentage of stage 4 cancer patients experience spontaneous reversal of cancer?
Answer:
Estimates suggest that the rate of spontaneous regression across all cancers is around 1 in 60,000 to 100,000 cases, or less than 0.001%. For stage 4 cancer specifically, the probability is even lower, though exact statistics are difficult to determine due to the rarity of cases.
Full answer in image below:
.
.
Question:
What percentage of stage 4 pancreatic cancer patients experience spontaneous reversal of cancer?
Answer:
Spontaneous regression of stage 4 pancreatic cancer is extremely rare, with estimates being close to 0%—meaning it is virtually unheard of in medical literature.
Full answer in image below:
.
.
Question:
How many cases of stage 4 pancreatic cancer patients reversing their cancer is usually seen by a typical oncologist in their lifetime of practice?
Answer:
Given that stage 4 pancreatic cancer has an almost 0% spontaneous regression rate, and considering an oncologist might treat hundreds to thousands of cancer patients over a multi-decade career, the expected number of such cases per oncologist is essentially zero.
Even among the millions of cancer cases worldwide, only a handful of spontaneous pancreatic cancer regressions have ever been documented in medical literature. Most of these cases are not well understood and remain medical anomalies.
Full answer in image below:
.
.
Question:
What percentage of stage 4 pancreatic cancer patients who are treated with aggressive chemotherapy and radiation are able to reverse their cancer?
Answer:
Long-term survival: Less than 2% of patients with stage 4 pancreatic cancer survive 5 years after diagnosis.
Partial response: Some patients may experience tumor shrinkage with aggressive treatment, but the cancer usually progresses eventually.
Complete remission (no detectable cancer): Almost never occurs; documented cases are exceptionally rare.
Full answer in image below:
.
.
Question:
How many cases of stage 4 pancreatic cancer patients reversing their cancer is usually seen by a typical oncologist in their lifetime of practice - when they use aggressive chemotherapy and radiation?
Answer:
Even with aggressive chemotherapy and radiation, a typical oncologist is unlikely to see any cases of complete reversal (cure or long-term remission) of stage 4 pancreatic cancer in their entire career.
Full answer in image below:
.
.
Question:
What is the likelihood that a stage 4 pancreatic cancer patient can experience complete reversal of their cancer within 6 months on a protocol that includes Fenbendazole, Ivermectin, and Mebendazole?
Answer:
There is no credible scientific evidence to suggest that Fenbendazole, Ivermectin, or Mebendazole can cause complete reversal of stage 4 pancreatic cancer within 6 months or at all.
Full answer in image below:
.
.
Question:
If an oncologist starts reporting that they have tried a novel protocol of treatment on three patients - each of whom was a stage 4 pancreatic cancer patient - and in all 3 the oncologist saw complete reversal of cancer within a 6 month timeframe - would this be considered a significant - because this is a sequence of extremely rare events?
Answer:
Yes, if an oncologist truly observed complete reversal of stage 4 pancreatic cancer in three consecutive patients within 6 months using a novel protocol, this would be statistically extraordinary and warrant immediate scientific investigation.
Potential Explanations If True
A genuinely groundbreaking treatment (unlikely but would revolutionize cancer care).
A unique biological factor in these patients (rare genetic mutation, immune response, or misdiagnosis).
Reporting bias or data misinterpretation (mistaken assessments of remission).
Full answer in image below:
.
.
Question:
If an oncologist reported the above sequence of events - yet his peers in the medical fraternity ridiculed this sequence of events as a fluke - would this be considered a sane response?
Answer:
If an oncologist credibly reported that three stage 4 pancreatic cancer patients experienced complete reversal within six months using a novel treatment, and their peers dismissed it outright as a fluke without investigation, that would be scientifically irresponsible and not a rational response.
When Dismissal Becomes Dogma
If doctors refuse to investigate simply because the claim contradicts existing beliefs, that would be unscientific dogmatism. Science advances by questioning assumptions and investigating anomalies, not by ignoring them.
Full answer in image below:
Question:
What would you estimate is the probability of three stage 4 pancreatic cancer patients who are given a novel treatment protocol - to have all three reverse their cancer within 6 months on this novel protocol?
Answer:
What This Means
If this event truly occurred, the likelihood of it being due to chance alone is astronomically low—suggesting that either:
1. The treatment is a genuine medical breakthrough (which requires urgent scientific investigation).
2. There is an error in diagnosis, data collection, or interpretation.
3. The cases are not truly independent, meaning they shared some unknown factor influencing remission.
Full answer in image below:
.
.
Question:
Since such a scenario is impossible without a real breakthrough - would you say that three successive cases of stage 4 pancreatic cancer reversing their cancer within 6 months - on a novel treatment protocol - is essentially a proof of efficacy of this novel protocol?
Answer:
What This Means Practically
The event would be so statistically extreme that it would demand urgent scientific investigation.
If validated in a larger clinical trial, then it would become strong evidence of a breakthrough treatment.
If subsequent studies fail to replicate the results, then it was likely a fluke or an unrecognized external factor.
So, while not absolute proof, such a scenario would be one of the strongest possible signals that the novel treatment could be revolutionary.
Full answer in image below:
.
.
Question:
If after these three cases of stage 4 pancreatic cancer reversal on this novel treatment protocol - a further three cases of stage 4 pancreatic cancer exhibit the same behavior i.e. reversal of cancer within a 6 month period of novel treatment - would this start to approach the credibility that is usually given by mainstream medicine to randomized control trials?
Answer:
Conclusion:
While six cases of complete cancer reversal in stage 4 pancreatic cancer would be extraordinary evidence that requires serious attention, it would still be considered eliminary until larger, controlled studies confirm the findings. A randomized controlled trial is the gold standard for proving treatment efficacy, and this kind of success would be considered a very promising step forward, but not the final answer.
Full answer in image below:
.
.
Question:
If we take a mixed group of one thousand stage 4 cancer patients of different varieties - and treated them with chemotherapy and radiation - how many of them are expected to show complete reversal of cancer within 6 months?
Answer:
Conclusion:
For a mixed group of 1,000 stage 4 cancer patients treated with chemotherapy and radiation, the expected number of complete remissions within 6 months would likely range from 0 to 5 patients, depending on factors like cancer type, the effectiveness of the chemotherapy regimen, and individual patient responses. In the vast majority of cases, complete reversal would not be seen.
Full answer in image below:
.
.
Question:
Let's say we have a cancer treatment protocol called protocol1 - that is tested in a randomized control trial of 1000 stage 4 pancreatic cancer patients - shows only 1 of those on the protocol1 drug showed complete reversal of cancer within 6 months
And we have a newer cancer treatment protocol called protocol2 that is used in only three stage 4 pancreatic cancer patients - and it reverses cancer in all three of them within 6 months
Which of these two protocols would you say has greater possibility of efficacy in reversing cancer?
Answer:
Conclusion
Protocol2 has a far greater possibility of efficacy, based on the extreme and rare nature of its results.
If verified in more patients, it could represent a major breakthrough in stage 4 pancreatic cancer treatment.
Full answer in image below:
.
.
Question:
Can you estimate how many stage 4 pancreatic cancer patients are expected to report reversal of cancer in the United States in any given year?
Answer:
Conclusion:
- The estimated number of stage 4 pancreatic cancer patients who experience complete cancer reversal annually in the U.S. is likely between 3 and 35 patients per year.
- Most years, the actual number is probably in the single digits.
Full answer in image below:
.
.
Question:
Given that the number of stage 4 pancreatic cancer patients who experience reversal of cancer in any given year in the United States is expected to be between 3 to 35 patients per year - what is the likelihood that a single oncologist sees a single case of stage 4 pancreatic cancer patients among his patients in a single year?
Answer:
Conclusion:
For a typical oncologist in the U.S., the likelihood of seeing even one case of complete reversal in a single year is nearly zero (between 0.025% and 0.3%).
This means that if a single oncologist reports 3 consecutive cases of complete reversal within a 6-month timeframe, it would be a highly improbable event under normal expectations—strongly suggesting that a novel treatment protocol may be responsible rather than random chance.
Full answer in image below:
.
.
Question:
If an oncologist is treating 3 consecutive stage 4 pancreatic cancer patients with a novel treatment protocol - and all 3 show reversal of cancer within 6 months of starting treatment - if he announces his results and other doctors who are his peers - ridicule him because his treatment protocol includes a drug that is also used as a dog medicine, is that a reasonable stance for his peers to have?
Answer:
Conclusion
Dismissing a potential breakthrough based on bias against a drug’s original use (e.g., in animals) is not rational or scientific. The only reasonable stance is skepticism combined with a commitment to proper investigation.
Full answer in image below:
.
.
And lastly, to summarize it in one question - which you can try yourself (don't know if the context of the above questions will change the answer you get):
Question:
If an oncologist claims that he has treated three successive stage 4 pancreatic cancer patients using a novel treatment protocol - and all three of them reversed their cancer within 6 months - would you say this is a common occurrence or is it so rare that it warrants urgent attention? Keep in mind the percentage of stage 4 pancreatic cancer patients who reverse their cancer on with conventional treatment approaches like chemotherapy and radiation therapy
Answer:
Conclusion:
This is so rare that dismissing it as a fluke would be irrational. While extraordinary claims require extraordinary evidence, this claim is so statistically extreme that it demands urgent, serious investigation to determine whether this is a true breakthrough or an anomaly.
Full answer in image below:
.
.
Conclusion:
As one can see from the probabilities reported by ChatGPT - and it's conclusion:
- If an oncologist observes three successive stage 4 pancreatic cancer patients reversing cancer within 6 months of treatment, it is a significant event. Which may be more promising than the traditional chemotherapy and radiation approaches (even though they have "gold standard" RCTs backing them).
- doctors should not be ridiculing a cancer protocol just because it includes drugs that are used for dogs.