NCIt ID normalizes to a different NCIt ID

[korikuzma]

Describe the bug

ncit:C4207 (Juvenile Granulosa Cell Tumor) normalizes to ncit:C39947 (Juvenile Testicular Granulosa Cell Tumor)

Steps to reproduce

http://normalize.cancervariants.org/disease/normalize?q=ncit%3AC4207

Expected behavior

We thought it would normalize to ncit:C4207

Current behavior

It normalizes to ncit:C39947

Acceptance Criteria

Given I want to normalize ncit:C4207,
When I hit http://normalize.cancervariants.org/disease/normalize?q=ncit%3AC4207,
Then it should return ncit:C4207 as the normalized concept ID

Possible reason(s)

No response

Suggested fix

No response

Branch, commit, and/or version

version: 0.9.0

Screenshots

No response

Environment details

NA

Additional details

No response

Contribution

None

[korikuzma]

@jsstevenson Would you please provide an effort estimate?

[jsstevenson]

Ah, I think this is roughly the same bug I talked to @wesleygoar about before. I'll walk through what's happening and outline a few possible ways forward

1. user searches /normalize for ncit:C4207
2. we check for an exact concept ID match against a merged record. there's no merged record under that ID
3. we check for an exact concept ID match against an unmerged record. There is a regular NCIt record for that ID but no pointer to a merged concept so we keep looking.
4. we check for a label match (obviously fails)
5. we check for an xref match. DOID:6032 has an xref to ncit:C4207 (but also to ncit:C39947, which is where the problem arises). From here, we get a reference to a merged record anchored by MONDO_0003741
6. we return the normalized record for MONDO_0003741

*Reminder here that the normalization behavior differs for disease vs gene/therapy -- we just construct merged records off of mondo terms rather than creating normalized references for ALL terms

I see a few issues here that overlap with/might resolve this behavior. I think some of this is intentional and some is bug, but it's been a long time so I couldn't say for sure...

1. Our normalized record construction behavior seems discordant from our term resolution behavior. Specifically, that DOID xref doesn't get included in the actual record that you see, but it can still be used to reach it. That is a problem! Either we disallow that search behavior or expand what's included in the record construction (I think this is a bug but I don't totally remember what desired behavior should be)
2. that DOID record has inexact xrefs. C4207 is the parent juvenile granuloma term and C39947 is the child term specific to testicular granuloma which corresponds to DOID:6032, but the DOID term has xrefs to both. This may be on purpose (in which case we could put a property blocklist to filter those values, we've already done something similar in thera-py) or may be accidental (we could get it fixed with a GH issue, they're usually pretty responsive). (or maybe we could refuse to include 2 xrefs of the same source)
3. In general, it's a little bumpy to have the VarCat/ODS phenotyping pipeline resolve to NCIt terms and the Disease Normalizer anchor on MONDO terms. We could just change the latter to also anchor to NCIt. That would be kind of a bummer (less transparent/open process, less comprehensive disease coverage, subject to ideological intervention depending on who's the president...) but it would fix a lot of these problems. And it would also let us close Use Mondo ID instead of NCIt ID to designate normalized groups #176
4. We could at least make Disease use the same normalized record construction process as Therapy/Gene. We did it this way because we thought Mondo mappings were generally good enough already, but it would resolve this problem and probably simplify overall normalizer maintenance.

Nothing here is more than ~4 hours of dev work + a day or two of asynchronous PR review back and forth

[jsstevenson]

Oh, one other thing

we check for an xref match. DOID:6032 has xrefs to both ncit:C39947 and ncit:C4207. from here, we get a reference to a merged record anchored by MONDO_0003741

I think it's probably unintentional that DO has both of these mappings (it's not like a clear rule violation or anything but my guess is that they meant to delete the xref to one when they added the other) -- but also, I would think the odds are that the user who is typing in ncit:C4207 for a specific case might actually want C39947 or C7289 or C40434. Obviously not an expert and I can vaguely imagine cases where that's not true but it might be worth double-checking

[korikuzma]

@jsstevenson Thanks for the detailed info. I think all of us (you, me, @ahwagner , and @wesleygoar ) should have a discussion on the route we want to take.

[jsstevenson]

Our feelings post 2025-07-22 meeting

• VarCat switch to GET /search is good enough for now
• Add blocklist logic
• Groups w/ multiple records from same source should get a specific warning so that we can compile a big list, proceed from there

[github-actions]

This issue is stale because it has been open 135 days with no activity. This issue will be closed if no further activity occurs in 14 days.