Discussions

• 6 You must be logged in to vote

  💬

  [SYCL][Intel GPU] Long Term Features & Issues Tracking

  Intel GPU

  airMeng started Feb 2, 2024 in General

  

  

  

  

  126
• 15 You must be logged in to vote

  🙌

  Performance of llama.cpp on Apple Silicon A-series

  ggerganov started Dec 17, 2023 in Show and tell

  

  

  

  

  45
• 2 You must be logged in to vote

  🙏

  Llama-server functionality with rpc support?

  justinjja asked Jan 27, 2025 in Q&A · Answered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  Can someone explain to me, in detail, on what llama.cpp allocates memory?

  MislavJuric asked Oct 18, 2024 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  💡

  Speedup build cu files

  lexasub started Feb 1, 2025 in Ideas

  

  

  0
• 2 You must be logged in to vote

  🙏

  When I use the docker images, I get a platform mismatch what am I missing?

  jacebenson asked Jan 28, 2025 in Q&A · Closed · Unanswered

  

  

  3
• 1 You must be logged in to vote

  💡

  llama-bench backend/device selection?

  etasnadi started Feb 1, 2025 in Ideas

  

  

  1
• 1 You must be logged in to vote

  💡

  🚀 Optimized Bit-Packing: Replacing Subtraction with Zero-Cost Type Reinterpretation

  tfernd started Feb 1, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  Dual-CPU utilization - CPU inference only

  nekiee13 asked Jan 30, 2025 in Q&A · Unanswered

  

  

  2
• 2 You must be logged in to vote

  💬

  CUDA error with AMD GPU?

  mikeperalta1 started Jun 11, 2024 in General

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  After the directory change, what should be the change in this command

  ramanrewati asked Jan 31, 2025 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  Where does llama-server store it's chat history?

  BVEsun asked Jan 31, 2025 in Q&A · Answered

  

  

  2
• 4 You must be logged in to vote

  💬

  AI Frameworks + llama.cpp = ❤️

  documentation Improvements or additions to documentation help wanted Extra attention is needed 3rd party Issue related to a 3rd party project

  ochafik started Jan 31, 2025 in General

  

  

  2
• 1 You must be logged in to vote

  🙏

  how do I enable NVLink / peer transfers?

  recallmenot asked Jan 29, 2025 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  Building HIP on Ubuntu with RX 7800XT

  bartekrey asked Feb 1, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  <1 tok/sec with A100 with DeepSeek-R1-Distill-Llama-8B

  sb98052 asked Jan 31, 2025 in Q&A · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙌

  GBNF Function Calling Grammar Generator for llama.cpp to make function calling with every model supporting grammar based sampling. (most models, I only had problems with Deepseek Code Instruct)

  Maximilian-Winter started Nov 30, 2023 in Show and tell

  

  

  

  

  2
• 1 You must be logged in to vote

  💡

  Feature request: Llama Web UI: "Save all chats" button

  mashdragon started Feb 1, 2025 in Ideas

  

  

  0
• 10 You must be logged in to vote

  🙌

  Huggingface GGUF Editor

  CISC started Sep 1, 2024 in Show and tell

  

  

  

  

  5
• 2 You must be logged in to vote

  🙏

  Using -ctk q4_0 -ctv q4_0 with llama.cpp server throws flash_attn error for bartowski/DeepSeek-R1-GGUF

  AshD asked Jan 26, 2025 in Q&A · Closed · Unanswered

  

  

  

  

  3
• 3 You must be logged in to vote

  💬

  Performance of llama.cpp with Vulkan

  performance Speed related topics need feedback Testing and feedback with results are needed

  netrunnereve started Dec 18, 2024 in General

  

  

  

  

  54
• 1 You must be logged in to vote

  💡

  Predicting activation patterns using a smaller model in order to optimise memory usage.

  smntlbt started Jan 31, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to use FP8 for SYCL building?

  GermanAizek asked Jan 30, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  Does llama.cpp support CUDA through HIP?

  MarioIshac started Dec 19, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Virus detected in the release b4595 (llama-b4595-bin-win-cuda-cu12.4-x64.zip)

  manish-ahuja asked Jan 30, 2025 in Q&A · Unanswered

  

  

  0