본문으로 건너뛰기
AI NEWS JOURNAL
llama.cpp releasesAI NEWS / JOURNAL

b12552

The update includes fixes for OpenVINO CI, optimizations for model recompiles in the prefill stage, improvements to output handling during model reuse, and support for f16 SCALE and f16/bf16 TRI operations. It also addresses issues with static-shape views in MoE layers, enhances SSM_SCAN build checks, and adds Falcon-H1 to the list of validated OpenVINO models with noted limitations.

번역 서비스가 일시 중지되어 원문을 표시합니다.

<details open> ggml-openvino: fix OpenVINO CI, minor fixes and optimizations (#32590) * ggml-openvino: round the GPU KV length up to a power of two Review notes (follow-ups, not blocking): - convert_ggml_input_to_ov pads the KQ mask into a std::vector and then copies it into the ov::Tensor. Writing the padded rows straight into padded.data() avoids the second allocation and copy (up to 16 MB per call at a 16K context). - The padding is skipped when the mask is not contiguous, and the KV inputs are then sliced to more rows than the mask has columns. Masks are always contiguous today; a GGML_ASSERT would make that explicit. - compute_llm_params reads GGML_OPENVINO_STATEFUL_EXECUTION directly; is_stateful_enabled() is local to ggml-openvino.cpp. * ggml-openvino: Optimize model recompiles of prefill stage Optimize the model recompiles of prefill stage when attended size changed between prompts. This can save model recompile time for llama-bench and llama-server. * ggml-openvino: Optimize model recompiles of prefill stage Optimize the model recompiles of prefill stage when attended size changed between prompts. This can save model recompile time for llama-bench and llama-server. * ggml-openvino: Remove solution C and keep A as default * ggml-openvino: match the output count before reusing a compiled NPU prefill The static prefill model has a fixed inp_out_ids shape [1,1,1,output_len]. The new reuse check compared only the attention sizes, so llama-perplexity, which alternates 1 and 512 outputs per ubatch, reused the model compiled for 512 outputs on a 1-output ubatch and failed with "got [1,1,1,1] expecting [1,1,1,512]". Store the output count of the cached prefill model and require it to match. * ggml-openvino: Fix LFM2 context create on GPU and DSpark speculative decoding * ggml-openvino: remove leftovers from merging dev into the LFM2/DSpark fix - compute_model_outputs exported OUTPUT-flagged nodes twice: dev's block ran before the new one and still exported result_norm, so the result_norm exclusion had no effect. Keep only the new block. - compute_node_dynamic_dims checked is_activation_input twice in a row. Remove the second copy. - process_view_input_new created the dynamic Split on split_dim. With stateful execution the input drops the leading size-1 axis, so use split_axis, which already accounts for the rank difference. * ggml-openvino: hash the output flags into the model-cache fingerprint The fingerprint hashed the ops, names and weights but not the outputs, so a blob compiled with another output set was imported. With the result_norm change, an NPU blob written before it gave wrong output, and enabling llama_set_embeddings_layer_inp() reused a blob without the layer-input outputs. Hash each node's output flag and bump the cache version to 8. * ggml-openvino: support f16 SCALE and f16/bf16 TRI test-backend-ops now runs SCALE on f16 and TRI on f16 and bf16. Both translators built their constants as f32, and Multiply, Add and Select need all inputs in one element type, so building the graph threw for these cases. SCALE now converts a non-f32 input to f32, applies the scale and bias, and converts back once, as the CPU backend computes it. TRI creates its zero constant in the input's element type. f32 graphs are unchanged. * ggml-openvino: [Qwen3.6 A3B]fix static-shape ids view of the last MoE layer The same-stride single-dim Slice in process_view_input_new was followed by a Reshape whose target baked the static chunk size into the token axis. After the output-ids GET_ROWS the last layer runs on n_outputs tokens (1 in the prefill graph), so the Reshape failed shape validation for qwen35moe with FORCE_STATIC or on the NPU. Skip the Reshape in static graphs when the slice already keeps the rank; the dynamic graphs are unchanged. * enable mamba based models on OV CPU-GPU * revert changes in tests * docs : add Falcon-H1 to the OpenVINO validated models Falcon-H1 runs on CPU and GPU since Mamba SSM support was added. Add its row and the known limitations: multi-sequence batches fail, GPU accuracy is slightly lower than CPU, and the NPU fails with an element type mismatch. * ggml-openvino: simplify the SSM_SCAN build check and clean up - Drop the SelectiveSSM configure check and its guards: OpenVINO 2026.4.1 always provides the op. Drop the no-op LANGUAGE CXX source property. - Drop the semicolon after OPENVINO_OP("SelectiveSSM"). GCC 15 reports it with -Wextra-semi, which fails builds with fatal warnings. - Add ggml_openvino_is_static() for the repeated NPU or GGML_OPENVINO_FORCE_STATIC check. - Share the f16 and f32 KQ mask padding code. - Drop the redundant inline on in-class static functions. --------- Co-authored-by: Suhas Somashekar <suhas.somashekar@intel.com> Co-authored-by: yzhou51 <yaowei.zhou@intel.com> Co-authored-by: Ananth, Swathi <swathi.ananth@intel.com> Co-authored-by: Beheshti <nazanin.beheshti@intel.com> </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/69871722> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-arm64.tar.gz) - [Ubuntu 24.04 s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-s390x.tar.gz) - [Ubuntu 26.04 s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-26.04-s390x.tar.gz) - [Ubuntu 26.04 s390x (zDNN)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-zdnn-s390x.tar.gz) - [Ubuntu x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-vulkan-x64.tar.gz) - [Ubuntu arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-vulkan-arm64.tar.gz) - [Ubuntu x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-cuda-12.8-x64.tar.gz) - [CUDA 12.8 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-b12552-bin-ubuntu-cuda-12.8-x64.tar.gz) - [Ubuntu x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-cuda-13.5-x64.tar.gz) - [CUDA 13.5 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-b12552-bin-ubuntu-cuda-13.5-x64.tar.gz) - [Ubuntu arm64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-cuda-13.5-arm64.tar.gz) - [CUDA 13.5 libraries](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-b12552-bin-ubuntu-cuda-13.5-arm64.tar.gz) - [Ubuntu x64 (ROCm 10.0)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-rocm-10.0-x64.tar.gz) - [Ubuntu x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-openvino-2026.4.1-x64.tar.gz) - [Ubuntu x64 (SYCL FP32)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-sycl-fp32-x64.tar.gz) - [Ubuntu x64 (SYCL FP16)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-ubuntu-sycl-fp16-x64.tar.gz) - [Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-linux-arm64-snapdragon.tar.gz) - [setup guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md) **Android:** - [Android arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-android-arm64.tar.gz) - [Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-android-arm64-snapdragon.tar.gz) - [setup guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md) **Windows:** - [Windows x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-cpu-x64.zip) - [Windows arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-cpu-arm64.zip) - [Windows arm64 (OpenCL Adreno)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-opencl-adreno-arm64.zip) - [Windows x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-cuda-12.4-x64.zip) - [CUDA 12.4 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-bin-win-cuda-12.4-x64.zip) - [Windows x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-cuda-13.5-x64.zip) - [CUDA 13.5 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-bin-win-cuda-13.5-x64.zip) - [Windows arm64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-cuda-13.5-arm64.zip) - [CUDA 13.5 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b12552/cudart-llama-bin-win-cuda-13.5-arm64.zip) - [Windows x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-vulkan-x64.zip) - [Windows arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-vulkan-arm64.zip) - [Windows x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-openvino-2026.4.1-x64.zip) - [Windows x64 (SYCL)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-sycl-x64.zip) - [Windows x64 (ROCm 10.0)](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-bin-win-rocm-10.0-x64.zip) **openEuler:** - [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23705) - openEuler x86 (310p) - openEuler x86 (910b, ACL Graph) - openEuler aarch64 (310p) - openEuler aarch64 (910b, ACL Graph) **UI:** - [UI](https://github.com/ggml-org/llama.cpp/releases/download/b12552/llama-b12552-ui.tar.gz)

자료 출처

원문 읽기 ↗

관련 주제

← 전체 소식 이어 읽기