I am using FP-DSA on mac m1 - macos 12, facing issue "clGetPlatformIDs" while trying to hit FP Models on WSI

[lomshabhishek]

�[0m�[1mWARNING [274890450752] Unable to open X display. Disabling visualization.�[0m
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Traceback (most recent call last):
File "/usr/local/bin/fastpathology", line 11, in
load_entry_point('fastpathology', 'console_scripts', 'fastpathology')()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 490, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2854, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2445, in load
return self.resolve()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2451, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/opt/fastpathology/dsa/cli/fastpathology/deploy.py", line 1, in
from .utils import enable_fast_verbosity, download_models, run_pipeline
File "/opt/fastpathology/dsa/cli/fastpathology/utils.py", line 1, in
import fast
File "/usr/local/lib/python3.8/dist-packages/fast/init.py", line 22, in
fast.ImageFileImporter.create('') # Trigger splash, GL context initialization etc.
File "/usr/local/lib/python3.8/dist-packages/fast/fast.py", line 13671, in create
return _fast.ImageFileImporter_create(filename)
RuntimeError: clGetPlatformIDs

[andreped]

That is likely because the pyFAST wheel installed is not built for Silicon. I can add a fix for that after lunch :]

pyFAST recently added Silicon-support, so I have not had the time to test this.

[lomshabhishek]

Great! Thanks

Looking forward.

[lomshabhishek]

Also can you guide me a bit on how can I add more custom models to it

[andreped]

I think this can be resolved by running pyFAST completely headless, which is possible. I would need to rewrite this slightly either using the runPipeline FAST CLI or modifying my own custom run method (see here). My run method is basically a simplified runPipeline, so doing the same in Python should be possible.

Also can you guide me a bit on how can I add more custom models to it

Right now, adding your own custom pipelines (and models) is not supported. At the time of developing this proof-of-concept plugin, I did not have dedicated funding nor time to do it.
I only made the tool compatible against the models available in the FAST DataHub, and only two methods as well (somewhat hardcoded).

If you wish to be able to add custom pipelines and models, it would be better to add a FastPathology-like solution, where the user could add their own FPL files and upload their own models. I do not have time to do this right now, but I may in the near future.

If you wish to add support to a single model yourself for testing, you will need to make your own run method (see here). Note that you will need a mechanism that downloads the model of interest and sets the path of the model in your own custom FPL (see here). Definitely not straight forward, but should be possible to do.

Might be that I have time this weekend to do an iteration. Stay tuned! :]

[lomshabhishek]

is there any FastPathology like solution or maybe alternate to DSA which has support to add custom models
or any other way apart from adding plugin, like maybe calling REST APIs from within the solution to process region of interest.

[andreped]

is there any FastPathology like solution or maybe alternate to DSA which has support to add custom models
or any other way apart from adding plugin, like maybe calling REST APIs from within the solution to process region of interest.

To the best of my knowledge, no.

But I mean, it should be possible to add generic FPL support. Im on the final stretch of my PhD and do not have much time to spare for open-source activites at the moment. Otherwise, I would have made an attempt at implementing a plugin that does exactly that right now.

I will have to update the FP plugin soon anyways to test some of my other custom models. Might put more effort into making the plugin more useful for your application. Will let you know!

[andreped]

Small update, @lomshabhishek!

I have been sick the entire week, so I have not had the time to work on this. I still am sick, but I took a look at getting FAST's runPipeline to work headless. I managed to get that working. You can see the thread here.

That means that I can adapt the code to using this solution, which likely will be a lot more generic. I might have time to look at this tomorrow, but I think I will have to rest to fully recover.

[andreped]

Hello! Regarding the original issue, I believe the very latest release of pyFAST now supports ARM CPUs (M1/M2 chip):
https://github.com/smistad/FAST/releases/tag/v4.7.1

I updated the repo to use the latest pyFAST version, so this should (in theory) work: 656e2a5

Could you try to rebuild the docker image from scratch (without cache), to see if that resolves the issue?
https://github.com/andreped/FP-DSA-plugin#-installation

pyFAST should work on a headless machine, but I am unable to debug this exact problem myself, as I do not own a macbook with ARM CPU.

---

For adding generic FastPathology-like support for adding your own custom pipelines and models, it is likely better to create a separate issue to track that further and/or open a Discussion tab.

[andreped]

Hmm, I managed to reproduce this issue using the latest pyFAST version on my macbook with regular CPU. So the current solution does not seem to work on macOS overall.

[andreped]

I think this is due to use building the docker image on top of NVIDIA CUDA Ubuntu 20.04, which results in a different OpenGL version than the one available on macOS:
https://github.com/andreped/FP-DSA-plugin/blob/main/dsa/Dockerfile#L2

It is not to run on macOS for development, but the end solution will likely be ran on a Linux server, so I do not think I will spend much time adding macOS support to this pipeline. You can make an attempt if you'd like.

Added info about this issue to the README: 55a32cd

I will rather focus on adding generic FPL support.

[lomshabhishek]

Hi @andreped I faced same issue on linux as well not sure why is that the case
Here's the dockerfile

FROM nvidia/cuda:11.0.3-cudnn8-runtime-ubuntu20.04

ARG DEBIAN_FRONTEND=noninteractive

# Update and upgrade the system
RUN apt-get update && apt-get upgrade -y

# Install essential tools and libraries
RUN apt-get install -y \
    apt-utils \
    curl \
    wget \
    vim \
    git \
    xz-utils \
    python3-pip \
    ocl-icd-libopencl1 \
    opencl-headers \
    clinfo \
    build-essential \
    libopengl0 \
    libgl1 \
    libusb-1.0-0-dev \
    libcairo2 \
    memcached \
    libglib2.0-0 \
    libsm6 \
    libxrender1 \
    libxext6 \
    cmake

# Clone the OpenCL-SDK repo (GitHub method for OpenCL-SDK installation)
RUN apt-get install -y libstb-dev libsfml-dev libglew-dev libglm-dev libtclap-dev ruby doxygen
RUN git clone https://github.com/KhronosGroup/OpenCL-SDK.git --recursive \
    && mkdir -p OpenCL-SDK/build \
    && cd OpenCL-SDK/build \
    && cmake -D CMAKE_INSTALL_PREFIX=/opt/opencl-sdk -B . -S .. \
    && cmake --build . --config Release --target install

# Set environment variables for OpenCL-SDK
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/opencl-sdk/lib
ENV C_INCLUDE_PATH=$C_INCLUDE_PATH:/opt/opencl-sdk/include
ENV CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/opt/opencl-sdk/include

# Install histomicstk necessary for communicating with HistomicsUI
RUN pip3 install histomicstk --find-links https://girder.github.io/large_image_wheels

# Set working directory and setup environment
COPY fastpathology .
ADD . /opt/fastpathology/dsa
WORKDIR /opt/fastpathology/dsa/cli
ENV PYTHONPATH=$PYTHONPATH:/opt/fastpathology/dsa

# Install fastpathology python package
RUN git clone https://github.com/andreped/FP-dsa-plugin.git \
  && cd FP-dsa-plugin/dsa/fastpathology/ \
  && pip install -e .

# Install additional Python packages
RUN pip install --upgrade pip
RUN pip3 install pyfast==4.7.1 opencv-python tqdm h5py huggingface-hub numpy tensorflow
RUN pip3 freeze > requirements.txt

[andreped]

Strange that Linux does not work for you. It worked fine for me on my Ubuntu 18.04 desktop, even though the container was running with Ubuntu 20.04.

Only thing I changed was the pyFAST version. You can try reverting back to 4.6.0, but I doubt thats why (here).

I do not have access to the Linux desktop atm, so I will have to run some tests on my Windows laptop later. Will have to do so after work.

[abhisheklomsh]

@andreped While doing multiple iterations I have accumulated a lot of entries under the analyses tab wherein old images are there, what is the best way to remove them?

[andreped]

@andreped While doing multiple iterations I have accumulated a lot of entries under the analyses tab wherein old images are there, what is the best way to remove them?

@abhisheklomsh I'm not sure what you mean, can you send a snapshot? Are you talking about WSIs you added to your local database through Digital Slide Archive?

Next time, for new topics or questions, please make a new Q/A in the Discussions tab (see here).

[abhisheklomsh]

Here like it shows only dsarchive/histomicstk, after using different image tags and even names for images I have got multiple drop downs, not in this image but under the same drop down.

[abhisheklomsh]

@andreped While doing multiple iterations I have accumulated a lot of entries under the analyses tab wherein old images are there, what is the best way to remove them?

@abhisheklomsh I'm not sure what you mean, can you send a snapshot? Are you talking about WSIs you added to your local database through Digital Slide Archive?

Next time, for new topics or questions, please make a new Q/A in the Discussions tab (see here).

Sure slipped from my mind, apologies for the same.

[andreped]

Sure slipped from my mind, apologies for the same.

Can you make a new Discussion now, and we can take it there instead?

[andreped]

@abhisheklomsh We actually experience something similar to what you are right now, @abhisheklomsh, but sadly I don't personally have a Linux desktop machine to debug it on.

Did you manage to get it working on Ubuntu?

The reason why it did not work for you on Ubuntu, might be because you were missing nvidia-docker2, as described as a prerequisite here.

[lomshabhishek]

@abhisheklomsh We actually experience something similar to what you are right now, @abhisheklomsh, but sadly I don't personally have a Linux desktop machine to debug it on.

Did you manage to get it working on Ubuntu?

The reason why it did not work for you on Ubuntu, might be because you were missing nvidia-docker2, as described as a prerequisite here.

I still encounter this while trying to run it on ubuntu or when I try to dockerise the installation process.

188.2                  from /app/FAST/source/FAST/Tests/Benchmarks.cpp:2:
188.2 /usr/include/GL/glext.h:54: note: this is the location of the previous definition
188.2    54 | #define GL_GLEXT_VERSION 20190805
188.2       | 
188.4 [100%] Building CXX object CMakeFiles/testFAST.dir/source/FAST/Tests/DataHubTests.cpp.o
196.0 [100%] Linking CXX executable bin/testFAST
196.6 [100%] Built target testFAST
196.6 WARNING [140047897357952] Unable to open X display. Disabling visualization.
196.6 QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
196.6 
196.6      - Powered by -     
196.6    _______   __________   
196.6   / __/ _ | / __/_  __/   https://fast.eriksmistad.no
196.6  / _// __ |_\ \  / /               v4.7.2
196.6 /_/ /_/ |_/___/ /_/       
196.6 
196.6 ERROR [140047897357952] Terminated with unhandled exception: clGetPlatformIDs
196.6 Aborted (core dumped)