kompute: improve backend to pass test_backend_ops

[slp]

This is a first batch of improvements on the kompute backend to be able to pass test_backend_ops by fixing some bugs and adding some missing features. Tested on Apple Silicon (M1 GPU) and AMD (Vega 8).

The next batch will extend test_backend_ops coverage by adding support for more operations.

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[slaren]

I tried running test-backend-ops under Windows with NVIDIA GPUs. The first GPU passed, but the second one hangs in the first op:

Testing 3 devices

Backend 1/3: Kompute0
  Device description: NVIDIA GeForce RTX 3080
  Device memory: 10053 MB (10053 MB free)
  [...]
  1919/1919 tests passed
  Backend Kompute0: OK

Backend 2/3: Kompute1
  Device description: NVIDIA GeForce RTX 3090 Ti
  Device memory: 24313 MB (24313 MB free)

  ABS(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  ABS(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  SGN(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  SGN(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  NEG(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  NEG(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  STEP(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  STEP(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  TANH(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  TANH(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  ELU(type=f32,ne_a=[128,2,2,2],v=0): not supported [Kompute1]
  ELU(type=f32,ne_a=[5,7,11,13],v=0): not supported [Kompute1]
  RELU(type=f32,ne_a=[128,2,2,2],v=0):

[slp]

I tried running test-backend-ops under Windows with NVIDIA GPUs. The first GPU passed, but the second one hangs in the first op:

Thanks for letting me know. I guess the most likely explanation is that the kompute backend has trouble initializing devices other than the first one. I'll try to find some hardware to fix it in a future PR.