See additional material below, regarding “truncation and imputation”.
Background:
Because they do not file insurance claims, direct primary care clinicians have no need to record CPT codes. A large fraction take things a step further and decline to use EHR that require or facilitate the recording of standard diagnostic codes. One DPC thought leader, quoted and linked below, has even declared ICD-10 to be “part of the disease process”.
But faced with calls for risk adjustment, DPC advocates have complained that they are short-changed because the lack of Dx-code-bearing primary care insurance claims results in under-reported risk for DPC patients who are managed so successfully that downstream claims are avoided. This gap lead Milliman Actuaries to use “Rx” risk adjustment, rather “Dx”, in their landmark study of direct primary care.
To remedy this “data donut hole” a DPC-affiliated analyst, KPI Ninja, purports to have developed a proprietary process (perhaps involving ArtificiaI Intelligence) for extracting synthetic diagnosis codes directly from EHR, so that these can be used to fuel “fair” risk-adjustment using standard population health risk software such as ACG® or CMS-HCC.
Whatever it might involve, there is no indication that KPI Ninja’s wholly-secret EHR-to-ICD-10 conversion process has been validated. But it is at least arguable that they — or anyone who has developed or cares to develop an alternative solution to the data donut hole problem— should be given an opportunity for validation.
Although it may be tempting to cast DPC advocates as trying to “both have their non-accountability cake and eat it too”, investigative integrity may require a process that is formally agnostic in regard to whether ICD-10 coding may cause disease. Ethical DPC practitioners who believe they would hurt patients by ICD-10 coding should be allowed to practice as they preach, while the research community should still be invested in resolving the data donut hole problem.
The Jeffrey Gold Hypothesis: Basis for Proposal
“[ICD-10] is nothing more than another layer of bureaucratic red-tape that does nothing to enhance the quality or cost of your care, but rather furthers the disease process. All it does is waste more of your physician’s and office staff’s time – time that should be spent working towards your care. . . .Luckily for us [Direct Primary Care] doctors, we have nothing to do with this nonsense.” Jeff Gold, MD, “ ICD-10: It’s Nice Not Knowing You” (Bold emphasis supplied.)
If ICD-10 codes appearing in EHR are useless surplusage that only hinder working PCPs, it follows that FFS-PCP EHR that contain ICD-10 are completely adequate for patient care, even if the ICD-10 codes are simply treated as the trash they are said to be. In this respect then, FFS-PCP EHR sans ICD-10 codes must contain the same complete and necessary medical information as D-PCP EHR.
The proposal provides an opportunity to evaluate, conjointly, both Dr. Gold’s hypothesis and the ability of KPI Ninja or other entities to accurately convert EHR to ICD-10.
Proposal:
The Institute for the Study of Direct Primary Care will seek at least one research partner who can supply an adequate volume of anonymized EHR and corresponding claims data, and such other partners as are appropriate.
The Institute for the Study of Direct Primary Care and its research partners shall compile, post, and make digitally available at least six public archives, to be released on three dates certain, spaced as set out herein.
Initial Archive Release (Training Sets):
Training Set Archive 1 shall be a set of certified collections of anonymized EHR from fee for service PCPs complete with such diagnostic codes as they may bear.
Training Set Archive 2 shall be the same set of certified collections of anonymized EHR from fee for service PCPs, but with any direct indication of ICD-10 codes or similar codes expunged using a fully disclosed electronic process.
Training Set Archive 3 shall be the set of certified collections of ICD-10 codes extracted from the actual claims history of the same patients whose EHRs comprised the other Training Archives.
Training Set Archive 4 shall be the set of certified collections of prescription drug classes extracted from the actual pharmacy claims history of the same patients whose EHRs comprised the other Training Archives.
All four training archives shall be released on day one.
Second Archive Release (A Test Set):
The Test Archive shall be a set of certified collections of anonymized EHR from fee for service PCPs, but with any direct indication of ICD-10 codes or similar codes expunged, using a fully disclosed electronic process.
The test archive shall be released ninety days after day one.
Third Archive Release (Solutions Sets):
The Predicted Codes Archive shall be the sets of ICD-10 codes predicted, by submitting entities, from the Test Set and submitted prior to the date of the third archive’s release. Each participant shall be identified.
The Actual Dx Archive shall be the sets of certified collections of ICD-10 codes extracted from the actual claims history of the same patients whose EHRs comprised the Test Set.
The Actual Rx Archive shall be the set of certified collections of prescription drug classes extracted from the actual pharmacy claims history of the same patients whose EHRs comprised the Test.
The three solutions archives shall be released one-hundred and twenty days after day one.
Subsequent Releases (Paired Test and Solutions Sets)
Upon the request of any entity, additional test and solutions sets should be issued.
Subsequent Releases (Training Sets)
Based on additional information, included the results obtained and feedback from submitting entities, The Institute for the Study of Direct Primary Care may refine or redefine the process for expunging coding data from EHR and issue additional training sets.
Major Caveat:
Even a validated conversion process will need to be coupled to additional mechanisms to assure that D-PCPs do not (a) “upcode” by larding EHRs with information intended to mislead or (b) engaging in other dishonest self-reporting processes. Put otherwise, DPC EHRs need to be subject to audit or equivalent, just as ICD-10 codes by insurers or ACOs are monitored by CMS.
Creditable validation must come from an entity demonstrably independent from any direct care provider.
°°°°°°°°°°°°°°°°°°°°°°°°°°°°
Alternative solution: truncation and imputation
There are vast data sets that have been used to validate various risk adjsustment methodologies. I suggest the following alternative strategy for filling the Dx donut hole.
Complete patient claims records with diagnosis codes for a given patient may contain a a number of Dx codes that appear only in primary care claims, a number that appear only in downstream care claims, and a number that appear in both primary care and downstream care claims.
I hypothesize that for a group of individuals a “truncated” set of Dx code harvested comprising only those from downstream care will be significantly predictive of overall risk at least over a wide range downstream claims levels.
I suggest testing that hypothesis and measurement of the correlation between truncated risk score and overall risk levels. This should yield a formula for predicting overall risk from downstream risk Dx codes alone.
Now, in terms of the data donut hole, the implicit presumption of DPC advocates is that they would be shortchanged by this method because they are more proficient at resolving risk before downstream claims are incurred. Assuming, arguendo, that this is a fair point, that does not render truncated data useless; DPC advocates would be free to argue that are somewhat short-changed by incomplete risk adjustment, but still point to any gains in DPCs direction that can be demonstrated by adjusted truncated scores as establishing a minimum bona fide gain.
At worst, the truncated claims data approach amounts to assuming that DPC is, for the sake of argument, neither better nor worse than FFS at treating cases that might entirely avoid downstream care. In effect, this approach imputes to DPC similar performance at this class of cases. This is a handicap, if DPC is indeed better; but shouldn’t DPC at least be able to show a partial win on partially adjusted downstream care figures. Is it conceivable, in other words, that DPC’s cost-effectiveness gains are real, but occur only when successful direct primary care fully avoids any of their patients carrying a risk factor into any form of downstream encounter. Surely, if direct primary care is better than FFS primary care, then within the universe of downstream care claims DPC must have some visible advantage. (Indeed, KPI Ninja’s Nextera report brags exactly this point at page 10; let’s risk adjust that claim.)
DPC Referee 