C.W.K.
Stream
Lesson 02 of 05 · published

Project: Domain Expert

~28 min · project, domain-expert, medical, legal, compliance

Level 0Observer
0 XP0/43 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

The goal

Fine-tune a model to be a medical-coding assistant that correctly identifies ICD-10 codes from clinical notes. Same recipe shape works for legal, financial, telecom, manufacturing — any high-stakes specialized domain.

Critical considerations

  • Data source: annotated clinical notes with verified ICD-10 codes. Hand-curated by certified coders.
  • Data quality: every example reviewed by a domain expert. No synthetic shortcuts on safety-critical data.
  • Evaluation: exact-match accuracy on ICD-10 codes, F1 per category, edge-case behavior.
  • Safety: include "I'm not confident" examples — the model must learn to express uncertainty rather than guess.

Code

Medical coding example with explicit confidence·json
{"messages": [
  {"role": "system", "content": "You are a medical coding assistant. Given clinical notes, identify the appropriate ICD-10 codes. If uncertain, say so explicitly with 'Confidence: Low' and suggest what additional information would help."},
  {"role": "user", "content": "Patient presents with acute chest pain radiating to left arm, elevated troponin levels, ST-elevation on ECG."},
  {"role": "assistant", "content": "Primary: I21.0 (Acute ST elevation myocardial infarction of anterior wall)\nSecondary: R07.9 (Chest pain, unspecified), R79.89 (Other specified abnormal findings of blood chemistry)\nConfidence: High — ST-elevation, troponin elevation, and clinical presentation are classic STEMI indicators."}
]}

External links

Exercise

Pick a specialized domain in your work (legal, financial, medical, technical). Define a 5-question rubric for what 'expert behavior' looks like in that domain. Sketch the data collection plan: who annotates, how many examples, what edge cases, what 'I don't know' patterns. This is the doc you'd send to a domain SME before kicking off the project.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.