Editors' pick: Originally published Sept. 22.
Boys and girls, the bell has rung so please take your seats. In today's class, I'm going to teach you how to dig into clinical trial data to find the red flags and bad stuff biotech companies don't want you to see.
Northwest Biotherapeutics (NWBO) will be the case study for today's lesson. On Wednesday, the company presented an update from a phase I study of DCVax-Direct, a personalized cancer vaccine.
As you might expect, Northwest Bio described the DCVax-Direct study results in glowing terms. From the company's perspective, the cancer immunotherapy's prospects could not be brighter. The truth about DCVax-Direct is far worse, as I'll demonstrate.
I can see some of you rolling your eyes because you don't care about Northwest Bio. I understand. It's a penny stock, nearly worthless. But hear more out. The methods I'll teach you today to find hidden red flags in Northwest Bio's clinical data are applicable to any cancer company you might be interested in. So, pay attention, please!
Before we go hunting for hidden nasties, let's go over some background.
DCVax-Direct is a personalized cancer vaccine consisting of dendritic cells harvested from a patient's white blood cells. The harvested dendritic cells are grown in a lab, then re-injected directly into the tumor, where they are supposed to prime the patient's immune system to identify and kill cancer cells.
For this clinical trial, Northwest Bio enlisted 40 patients with 11 different types of cancer. All the patients had metastatic, inoperable tumors no longer responding to previous therapies. All 40 patients were treated with DCVax-Direct injected into a single tumor, after which they were followed for tumor response and survival.
Alright, let's start the lesson by examining how Northwest Bio described the DCVax-Direct study results in its press release. Press releases are usually the first (and sometimes only) source of information for investors.
According to Northwest Bio, "The top 20% of these [DCVax-Direct] patients have so far exceeded 2 years of survival and are still alive. The longest survivor to date has reached nearly 3 years."
These "results" -- I struggle to even call them that -- amount to meaningless hand-waving.
Twenty percent of 40 enrolled DCVAx-Direct patients means eight patients are still alive after two years. Without knowing anything specific about these cancer patients and their prior treatments, a two-year survival rate of 20% is a number devoid of context.
Cancer is a heterogenous disease. A lung cancer patient has a different prognosis and expected survival rate than a pancreatic cancer patient or a breast cancer patient. By examining the specific genetic mutations present inside cancer cells we now know that there are important differences within cancer types. Not all lung cancers are the same; not all breast cancers are the same.
And, of course, the stage of cancer is important. How many prior treatments did the patients receive? What were those prior treatments?
Without these important baseline characteristics, Northwest Bio's statement of the DCVax-Direct study results is uninterpretable. Disregard it.
The company, in its press release, then claimed, "The top 30% of these patients (including pancreatic, melanoma, lung, ovarian, sarcoma and other cancer) as a combined group have an average survival to date of 26.7 months, compared with an average of expected survival times of 12.3 months. Individually, these patients have also substantially exceeded their respective expected survival times."
Again, the company is grouping together patients with different cancer types, making interpretation impossible.
The top 30% of patients combined have an average survival of 26.7 months, said Northwest Bio. That equates to 12 patients of 40 enrolled -- all with different cancers. What's so special about the top 30% of the enrolled patients? Answer: Nothing. It's an arbitrary cutoff. Red flag!
What's the average survival of the other 70%, or 28 patients, enrolled in the DCVax-Direct study? Northwest Bio doesn't tell us. That's another red flag.
Northwest Bio tries to legitimize its arbitrary survival statistic by comparing it with an equally mysterious multi-cancer comparison of 12.3 months "expected" survival. Red flag No. 3.
Be very skeptical when you see any company grouping together survival statistics of different cancer types, especially when baseline characteristics of the patients are hidden.
Let's now turn to really bad DCVax-Direct efficacy data Northwest Bio tried to hide from us. This is the fun part of the lesson.
For this exercise, we'll need to examine two of the data slides from the study that Northwest Bio presented Wednesday at an industry conference. Thankfully, the company provided a way for everyone to download the slides here.
We're going to start with Slide No. 20 -- a swimmer's plot depicting the survival times for each individual cancer patient enrolled in the DCVax-Direct study. Each horizontal line represents a patient treated with DCVax-Direct in the study. Each patient's cancer type is depicted on the Y axis. The X axis measures survival in months.
This chart confirms what Northwest Bio told us in the press release. The top eight patients, or bars, (20% of the total) are all green, meaning the patients are still alive, and their survival times exceeds 24 months, or two years.
From this chart, we can look at individual patients and group them together by cancer type. We can then determine the median survival time for each of these cancer types. Northwest Bio could have easily provided us with this level of DCVax-Direct data but the company chose not to. You'll see why when we start to dig into the survival numbers. [Hint: They're ugly].
Seven patients with pancreatic cancer enrolled in the study and were treated with DCVax-Direct, according to the chart depicted on Slide 20. Their individual survival times (in months) are 34, 30.5, 17.5, 8.0, 5.5, 4 and 2.5.
[I'm estimating those survival times by drawing a straight line down from the end of each horizontal bar to see where it lands on the time axis].
The median survival time for these seven pancreatic cancer patients -- all treated with DCVax-Direct -- is eight months.
It's not perfect, but this data point provides a lot more insight into DCVax-Direct's efficacy than anything Northwest Bio disclosed.
I won't go through all the math again (and you can do your own calculations) but here are the approximate, median survival times for some of the other cancer types enrolled in the DCVax-Direct study:
Sarcoma: 15 months. Colon: Seven months. Melanoma: Nine months.
It's still hard to interpret these survival times without a control arm to use for comparison. How long would similar patients not treated with DCVax-Direct live?
Unwittingly, Northwest Bio provides us with a way to make that comparison. Take a look at Slide No. 21 from the company's presentation.
This is another swimmer's plot, but it compares what appears to be the survival time for the best-performing DCVax-Direct patient in each cancer type with the "expected" survival for that cancer type.
For instance, if you look at the top of the chart, the pancreatic cancer patient treated with DCVax-Direct is still alive at approximately 34 months. Right below that, the shaded box indicates that pancreatic cancer patients of this type, generally, are expected to live approximately eight months. [Northwest Bio provides a citation for the published study from where the "expected" survival times are obtained].
From Northwest Bio's perspective, this chart is dynamite. Across the board, the DCVax-treated patients are living longer than "expected."
But Northwest Bio formatted the data on this slide in this way because it makes DCVax-Direct look much more effective than it truly is. Comparing survival times by cancer indication -- and not with a single, cherry-picked patient -- is way more negative for DCVax-Direct.
Let's go back to the survival data we captured from Slide No. 20. Remember, the seven pancreatic cancer patients treated with DCVax-Direct had a median survival of eight months. On Slide No. 21, the expected survival time for pancreatic cancer patients not treated with DCVax-Direct is eight months, according to Northwest Bio.
There is no difference in survival time between DCVax-Direct and the historical "expected" survival for pancreatic cancer patients, according to Northwest Bio. That's a dire outlook for DCVax-Direct not highlighted by Northwest Bio.
In colon cancer, the average survival time for DCVax-Direct treated patients was seven months compared to the "expected" survival time of 15 months, according to Northwest Bio's own data. You could argue that DCVax-Direct hastened the death of colon cancer patients.
In sarcoma, median survival for DCVax-Direct patient was 15 months. Northwest Bio has two "expected" survival times for sarcoma patients -- 24 months and eight months. Call it inconclusive. You could also say it's totally unreliable.
In melanoma, DCVax-Direct patients lived a median of nine months compared to an expected survival of 15 months, according to Northwest Bio. Again, DCVax-Direct underperformed significantly, putting cancer patients at greater risk of death.
These are the troublesome DCVax-Direct data Northwest Bio doesn't want you to see. I showed you how to find them.
And with that, the lesson in how to hunt for red flags in cancer clinical trial data is over. I hope you'll apply these tools whenever your favorite (or not so favorite) cancer drug company announced new clinical trial data. Stay sharp because all too often these biotech companies like to play games.
Adam Feuerstein writes regularly for TheStreet. In keeping with company editorial policy, he doesn't own or short individual stocks, although he owns stock in TheStreet. He also doesn't invest in hedge funds or other private investment partnerships. Feuerstein appreciates your feedback; click here to send him an email.