Optimizing Cell Segmentation in Xenium with Baysor to Incorporate Interior-Protein Staining and Capture Unassigned Transcripts in Mouse FFPE Brain Tissue

[alipirani88]

Hi @VPetukhov

We performed Xenium with cell segmentation on mouse FFPE tissue using the basic mouse panel and an add-on custom panel. After cell segmentation, we noticed that many transcripts were not assigned to any cells, even when they fell within areas stained by the interior–protein marker. Given that many brain cell types have long processes, this likely contributes to the challenge. As shown in the attached image, several regions stained with the interior–protein marker (green) are not defined as part of any surrounding cells, yet they contain numerous unassigned transcripts. It appears that the interior–RNA marker is currently being used to define cell boundaries. We’re wondering if there’s a way to optimize cell segmentation using Baysor to better capture these transcripts and incorporate the interior–protein stain to improve the definition of cell boundaries? Or any other way to optimize the Baysor run to consider these long brain cell processes and define them into a cell?

Here are the config details for the baysor run

baysor run -x x_location -y y_location -z z_location -g feature_name -m 200 -p --prior-segmentation-confidence 0.8 --scale-std 0.8 --scale 5 --save-polygons GeoJSON -o /rsrch5/home/neuro_rsrch/apirani/Zahra_output-XETG00091__0018297_20240912__204104/output-XETG00091__0018297__Region_1__20240912__204104/Region_1_Mut_WholeCBM /rsrch5/home/neuro_rsrch/apirani/Zahra_output-XETG00091__0018297_20240912__204104/output-XETG00091__0018297__Region_1__20240912__204104/Region_1_Mut_WholeCBM_2_cells_stats_filtered.csv :cell_id_int