Skip to content

Commit 563e156

Browse files
mart-rmart-r
authored
CU-8694vxcyx MedCAT 1.12 support (#22)
* CU-8693v4mx1: Update ICD10 and OPCS4 mappings in SNOMED CDB preprocessing * CU-8694vxcyx: Remove deprecated method usage and reference * CU-8694vxcyx: Bump medcat requirement to 1.12
1 parent c61b47a commit 563e156

3 files changed

+34
-34
lines changed

‎notebooks/introductory/Part_4_2_Supervised_Training_and_Meta_annotations.html

+25-25
Original file line numberDiff line numberDiff line change
@@ -13799,7 +13799,7 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1379913799
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
1380013800
</div><div class="inner_cell">
1380113801
<div class="text_cell_render border-box-sizing rendered_html">
13802-
<p>To fine-tune the existing models we use the <code>train_supervised()</code> method from MedCAT. The method has the following options:</p>
13802+
<p>To fine-tune the existing models we use the <code>train_supervised_from_json()</code> method from MedCAT. The method has the following options:</p>
1380313803
<hr>
1380413804
<p><code>data_path</code> - Path to the JSON file exported from MedCATtrainer</p>
1380513805
<p><code>reset_cui_count</code> - Each cui has an internal counter that is used for weight decay, this will reset it.</p>
@@ -13823,11 +13823,11 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1382313823
<div class="prompt input_prompt">In&nbsp;[11]:</div>
1382413824
<div class="inner_cell">
1382513825
<div class="input_area">
13826-
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cat</span><span class="o">.</span><span class="n">train_supervised</span><span class="p">(</span><span class="n">data_path</span><span class="o">=</span><span class="n">DATA_DIR</span> <span class="o">+</span> <span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span>
13827-
<span class="n">nepochs</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
13828-
<span class="n">reset_cui_count</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
13829-
<span class="n">print_stats</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
13830-
<span class="n">use_filters</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
13826+
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">cat</span><span class="o">.</span><span class="n">train_supervised_from_json</span><span class="p">(</span><span class="n">data_path</span><span class="o">=</span><span class="n">DATA_DIR</span> <span class="o">+</span> <span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span>
13827+
<span class="n">nepochs</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
13828+
<span class="n">reset_cui_count</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
13829+
<span class="n">print_stats</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
13830+
<span class="n">use_filters</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
1383113831
</pre></div>
1383213832

1383313833
</div>
@@ -13859,10 +13859,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1385913859

1386013860

1386113861

13862-
<div id="6ae0b7ea-bdeb-4d3f-b83c-3342c9b13cd3"></div>
13862+
<div id="97a59b96-c389-409a-8457-732acfff3f84"></div>
1386313863
<div class="output_subarea output_widget_view ">
1386413864
<script type="text/javascript">
13865-
var element = $('#6ae0b7ea-bdeb-4d3f-b83c-3342c9b13cd3');
13865+
var element = $('#97a59b96-c389-409a-8457-732acfff3f84');
1386613866
</script>
1386713867
<script type="application/vnd.jupyter.widget-view+json">
1386813868
{"model_id": "6fd10f1692234019836a7b40e83b56dd", "version_major": 2, "version_minor": 0}
@@ -13881,10 +13881,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1388113881

1388213882

1388313883

13884-
<div id="226f4eb5-22f1-43d4-83ae-4f42f75a13ba"></div>
13884+
<div id="949bb97a-d72f-4166-b47d-d4ffe721894c"></div>
1388513885
<div class="output_subarea output_widget_view ">
1388613886
<script type="text/javascript">
13887-
var element = $('#226f4eb5-22f1-43d4-83ae-4f42f75a13ba');
13887+
var element = $('#949bb97a-d72f-4166-b47d-d4ffe721894c');
1388813888
</script>
1388913889
<script type="application/vnd.jupyter.widget-view+json">
1389013890
{"model_id": "9a5ab9cfecc242b7aaf0f140e87bdde6", "version_major": 2, "version_minor": 0}
@@ -13963,10 +13963,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1396313963

1396413964

1396513965

13966-
<div id="04326482-ffec-4b5c-8633-f5562a193720"></div>
13966+
<div id="39e15e56-fb47-4b4b-bc54-840980738477"></div>
1396713967
<div class="output_subarea output_widget_view ">
1396813968
<script type="text/javascript">
13969-
var element = $('#04326482-ffec-4b5c-8633-f5562a193720');
13969+
var element = $('#39e15e56-fb47-4b4b-bc54-840980738477');
1397013970
</script>
1397113971
<script type="application/vnd.jupyter.widget-view+json">
1397213972
{"model_id": "434496e448984f55925d22fad0349ada", "version_major": 2, "version_minor": 0}
@@ -13985,10 +13985,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1398513985

1398613986

1398713987

13988-
<div id="f6af9754-3d34-43f0-931c-ae2d0882d7b7"></div>
13988+
<div id="05772180-dbe8-451b-a09a-c4435304092d"></div>
1398913989
<div class="output_subarea output_widget_view ">
1399013990
<script type="text/javascript">
13991-
var element = $('#f6af9754-3d34-43f0-931c-ae2d0882d7b7');
13991+
var element = $('#05772180-dbe8-451b-a09a-c4435304092d');
1399213992
</script>
1399313993
<script type="application/vnd.jupyter.widget-view+json">
1399413994
{"model_id": "f7d1803b3c6c4197b6612c5fdf189746", "version_major": 2, "version_minor": 0}
@@ -14007,10 +14007,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1400714007

1400814008

1400914009

14010-
<div id="fdc1a5ce-011a-4f3f-a5e2-ac061f13c471"></div>
14010+
<div id="4a031d0b-0138-4bde-ab47-a769a96bec8d"></div>
1401114011
<div class="output_subarea output_widget_view ">
1401214012
<script type="text/javascript">
14013-
var element = $('#fdc1a5ce-011a-4f3f-a5e2-ac061f13c471');
14013+
var element = $('#4a031d0b-0138-4bde-ab47-a769a96bec8d');
1401414014
</script>
1401514015
<script type="application/vnd.jupyter.widget-view+json">
1401614016
{"model_id": "c8d633f579de438a916d9ef3de9d8fe0", "version_major": 2, "version_minor": 0}
@@ -14029,10 +14029,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1402914029

1403014030

1403114031

14032-
<div id="c247a603-12d5-4cff-b8df-58babf41c20c"></div>
14032+
<div id="9b3536c0-957e-4873-97cf-0e5b64fd2cb4"></div>
1403314033
<div class="output_subarea output_widget_view ">
1403414034
<script type="text/javascript">
14035-
var element = $('#c247a603-12d5-4cff-b8df-58babf41c20c');
14035+
var element = $('#9b3536c0-957e-4873-97cf-0e5b64fd2cb4');
1403614036
</script>
1403714037
<script type="application/vnd.jupyter.widget-view+json">
1403814038
{"model_id": "de6c01c6983041e2b972f6008caefaea", "version_major": 2, "version_minor": 0}
@@ -14051,10 +14051,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1405114051

1405214052

1405314053

14054-
<div id="fe3fd5be-68e4-440a-9093-c4a51d88b939"></div>
14054+
<div id="0c50692e-679a-4580-a98c-d9b7c5a7b647"></div>
1405514055
<div class="output_subarea output_widget_view ">
1405614056
<script type="text/javascript">
14057-
var element = $('#fe3fd5be-68e4-440a-9093-c4a51d88b939');
14057+
var element = $('#0c50692e-679a-4580-a98c-d9b7c5a7b647');
1405814058
</script>
1405914059
<script type="application/vnd.jupyter.widget-view+json">
1406014060
{"model_id": "05132c907a874fe2a2eb9cb6c81da3b3", "version_major": 2, "version_minor": 0}
@@ -17502,10 +17502,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1750217502

1750317503

1750417504

17505-
<div id="993b7d0d-31c6-4cb9-8a67-229e48373814"></div>
17505+
<div id="0831575f-3f80-433f-8e19-3ac896196ba2"></div>
1750617506
<div class="output_subarea output_widget_view ">
1750717507
<script type="text/javascript">
17508-
var element = $('#993b7d0d-31c6-4cb9-8a67-229e48373814');
17508+
var element = $('#0831575f-3f80-433f-8e19-3ac896196ba2');
1750917509
</script>
1751017510
<script type="application/vnd.jupyter.widget-view+json">
1751117511
{"model_id": "00325922360c45009329d82ed6420f16", "version_major": 2, "version_minor": 0}
@@ -17524,10 +17524,10 @@ <h2 id="Fine-tuning-the-NER+L&#160;model">Fine-tuning the NER+L&#160;model<a cla
1752417524

1752517525

1752617526

17527-
<div id="cba1d441-64a6-4dcc-ab5a-5ae744aaff24"></div>
17527+
<div id="09e4a13d-2327-4e85-87d4-3d2777ffc588"></div>
1752817528
<div class="output_subarea output_widget_view ">
1752917529
<script type="text/javascript">
17530-
var element = $('#cba1d441-64a6-4dcc-ab5a-5ae744aaff24');
17530+
var element = $('#09e4a13d-2327-4e85-87d4-3d2777ffc588');
1753117531
</script>
1753217532
<script type="application/vnd.jupyter.widget-view+json">
1753317533
{"model_id": "d48e2f4d6dd3467fb3f17e0244b0e361", "version_major": 2, "version_minor": 0}
@@ -17926,7 +17926,7 @@ <h3 id="Train-MetaCAT">Train MetaCAT<a class="anchor-link" href="#Train-MetaCAT"
1792617926
<div class="prompt input_prompt">In&nbsp;[19]:</div>
1792717927
<div class="inner_cell">
1792817928
<div class="input_area">
17929-
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">mc</span><span class="o">.</span><span class="n">train</span><span class="p">(</span><span class="n">json_path</span><span class="o">=</span> <span class="n">DATA_DIR</span><span class="o">+</span><span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span> <span class="n">save_dir_path</span><span class="o">=</span><span class="s1">&#39;status&#39;</span><span class="p">)</span>
17929+
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">mc</span><span class="o">.</span><span class="n">train_from_json</span><span class="p">(</span><span class="n">json_path</span><span class="o">=</span> <span class="n">DATA_DIR</span><span class="o">+</span><span class="s2">&quot;MedCAT_Export.json&quot;</span><span class="p">,</span> <span class="n">save_dir_path</span><span class="o">=</span><span class="s1">&#39;status&#39;</span><span class="p">)</span>
1793017930
<span class="c1"># Saving the model this way will only save the model epoch with the best performance</span>
1793117931
</pre></div>
1793217932

‎notebooks/introductory/Part_4_2_Supervised_Training_and_Meta_annotations.ipynb

+7-7
Original file line numberDiff line numberDiff line change
@@ -664,7 +664,7 @@
664664
"id": "OTR2bcSZDP5V"
665665
},
666666
"source": [
667-
"To fine-tune the existing models we use the `train_supervised()` method from MedCAT. The method has the following options:\n",
667+
"To fine-tune the existing models we use the `train_supervised_from_json()` method from MedCAT. The method has the following options:\n",
668668
"\n",
669669
"\n",
670670
"---\n",
@@ -4353,11 +4353,11 @@
43534353
}
43544354
],
43554355
"source": [
4356-
"cat.train_supervised(data_path=DATA_DIR + \"MedCAT_Export.json\", \n",
4357-
" nepochs=1,\n",
4358-
" reset_cui_count=False,\n",
4359-
" print_stats=True, \n",
4360-
" use_filters=True) "
4356+
"cat.train_supervised_from_json(data_path=DATA_DIR + \"MedCAT_Export.json\", \n",
4357+
" nepochs=1,\n",
4358+
" reset_cui_count=False,\n",
4359+
" print_stats=True, \n",
4360+
" use_filters=True) "
43614361
]
43624362
},
43634363
{
@@ -6001,7 +6001,7 @@
60016001
}
60026002
],
60036003
"source": [
6004-
"mc.train(json_path= DATA_DIR+\"MedCAT_Export.json\", save_dir_path='status')\n",
6004+
"mc.train_from_json(json_path= DATA_DIR+\"MedCAT_Export.json\", save_dir_path='status')\n",
60056005
"# Saving the model this way will only save the model epoch with the best performance"
60066006
]
60076007
},

‎requirements-dev.txt

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
medcat~=1.10.0
1+
medcat~=1.12.0
22
pandas<2.0.0
33
seaborn~=0.11.2
44
pytest-xdist~=2.5.0
55
nbmake<1.4
66
nbconvert<6
77
jinja2<=3.0
8-
seaborn
8+
seaborn

0 commit comments

Comments
 (0)