Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

b12480

The Llama UI design system, including tokens, surfaces, fonts, and base components, has been moved from a temporary repository to ui/lib/ui, where it is now published as @ggml-org/llama-ui. Llama Chat now imports all UI elements—including model components, constants, enums, and helpers—from this centralized library, eliminating duplicate code. The update includes updated branding with Llama Sans and Llama Mono fonts, improved submenu rendering, enhanced elevation for chat form elements, and integration with the llama-brand package for colors and logo. The UI is now consistently applied across the application, with builds and deployment configurations adjusted accordingly.

b12445

The update introduces support for split_k in conv3d operations using Vulkan, leveraging a 256x128 shape (coopmat2-only) when beneficial and reusing the mul_mat split_k reduce shader.

b12451

The web UI for Llama.cpp has been relocated from tools/ui to ui/apps/chat as part of a structural refactoring to separate the component library (ui/lib) from the applications built on it. The app is now officially named Llama Chat, with its package renamed to @ggml-org/llama-chat, while maintaining private status. Documentation, help text, comments, and other references have been updated to reflect the new name, though legacy identifiers remain unchanged for backward compatibility.

b12456

The update addresses three issues in the OpenCL implementation of the bin flash-attention prefill path: releasing a previously unreleased WMM flash-attention mask fallback image, excluding q8_0 K/V at head size 512 from the WMM kernel due to incorrect results, and preventing the use of the WMM path beyond device 3D image limits to avoid clCreateImage failures.

b12474

The llama.cpp release b12474 introduces the application of output projection weight scale and provides precompiled binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, supporting various backends such as CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

b12475

The Llama Chat UI has been upgraded to SvelteKit 3, integrating its built-in service worker for improved update detection and offline functionality. The migration includes updates to Vite 8, TypeScript 6, Storybook 10, and ESLint 10, along with the replacement of deprecated packages and resolution of npm audit issues through dependency overrides, including pinning minimatch to version 11.0.5 to address a security vulnerability. Changes also align with current Svelte patterns, such as using $state instead of stores and adopting new syntax for actions and class bindings.

b12421

The llama.cpp release b12421 includes updated binaries for multiple platforms including macOS, Linux, Windows, Android, and iOS, with support for various backends such as CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon. The documentation has been reorganized, moving per-model TTS content to /docs and linking it from other pages. Some builds, like macOS Apple Silicon with KleidiAI and openEuler variants, are currently disabled.

b12428

The update includes the addition of four missing features in jinja, along with applied suggestions and a fix for an error message. This release is part of the llama.cpp project, offering binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, supporting various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

b12431

The update addresses issues with reasoning effort levels by ensuring rejected levels are properly handled during parsing, preventing 500 errors on API endpoints. It maintains support for template-ignored levels and aliases while trimming the output_config.effort test to a single level.

b12432

A performance improvement in llama.cpp addresses the issue of an O(n_cells) scan during each sequence removal, optimizing efficiency by reducing computational overhead.

b12433

A fix was implemented to prevent the MUL_MAT_ID operation from aborting on AMD devices when using F32, F16, or BF16 data types with an odd number of columns in src0, particularly affecting rank 1 LoRA on MoE expert weights. The issue arose because MMVF read src0 in pairs and required an even ne[0], so the fix restricts MMVF usage to cases with even ne[0], defaulting to fallback for other shapes. This resolves issue #30576.

b12438

The release includes support for json sort_keys=true in jinja, with various precompiled binaries available for macOS, Linux, Windows, Android, and other platforms, including CUDA, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon backend options.

b12385

The qwen2vl model in llama.cpp was producing incorrect results due to NULL values being passed for FFN weight scales, preventing per-tensor NVFP4 scales from being applied. This issue has been resolved by correctly passing the scales to the build_ffn function, consistent with how qwen2 handles them.

b12397

The update introduces a change to the template soft_max_f32_parallelize_cols_single_row to store column elements in registers when feasible, reducing global memory traffic from five passes to two, thereby improving performance on CUDA-enabled systems.

b12406

The latest release of llama.cpp introduces support for Inkling 2, including text and vision capabilities. This update includes text inference implementation, GGUF conversion for text models, vision multimodal projection support, still-image encoding, flash attention with folded bias and sinks, and improvements to tensor mapping and testing. The changes also address model conversion, checkpoint handling, and compatibility across multiple platforms, with support for CPU, GPU (CUDA, ROCm, Vulkan), and specialized hardware like Snapdragon and OpenVINO.

b12410

The release includes a fix to clear the slot cache when the global LoRA configuration changes, ensuring consistency in model loading and inference. Prebuilt binaries are available for multiple platforms including macOS, Linux, Windows, Android, and iOS, supporting various backends like CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

b12413

A fix has been implemented to correct the placement of begin/end of image tokens in the Inkling 2 model within the chat functionality, along with the addition of a test to ensure stability.

b12362

The latest release of llama.cpp (b12362) includes optimizations for grammar matching through left factoring, improved handling of character class boundaries, prevention of infinite expansion, and enhanced code clarity with ASCII comments and deduplicated logic. It reverts prior changes related to AST refactoring and alternative limits, while introducing shared immutable rules and removing performance benchmarks. The release supports multiple platforms including macOS, Linux, Windows, Android, and iOS, with various backend options such as CUDA, Vulkan, ROCm, OpenVINO, and SYCL.

b12364

A warning is now issued when the --spec-draft-p-min parameter is used with probabilistic draft sampling, as the draft's top probability does not reliably indicate how much of the draft the target model would accept under rejection sampling. The warning is skipped for drafters that do not use p_min, such as n-gram and DSpark.

b12377

The update refactors DFlash-based speculative decoding methods in llama.cpp by restructuring code organization, sharing components across models, splitting decoder graphs per head, and introducing dedicated classes for DFlash2 and LiliCorr drafts. It also adds support for new draft types, improves documentation, and removes unused code.