Ggml v1.0.0
There was an error while loading. Please reload this page.
ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quantized matmuls and MoE fusion on CUDA/Metal, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, new WebGPU kernels, expanded Hexagon op coverage, and scheduler improvements that avoid uploading unused inputs.
417a024 ggml : bump version to 1.0.0 (#1690) 771b3d9 requirements : fix numpy for python 3.14 (#1691) f154959 sync : llama.cpp ec04faf fix(hexagon): make host build work with clang-cl on Windows ARM64 (llama/32011) b45738e examples : adapt to removal of ggml_diag_mask_inf 1514439 sync : llama.cpp 5f29555 ggml : prune (llama/29801) 076861c hexagon: add Q5_0 and Q5_1 support (llama/32094) 63ec52d Title: CUDA: few-row MMQ for BS 2-24 (llama/31533) c6fe796 ggml : route GATED_DELTA_NET to the SpacemiT RVV kernel (llama/31897) 4ae6e72 vulkan : Intel FA kernel for prefill. (llama/29357) b22071c ggml-cpu: use the 4x4 simd_gemm tile on ARM NEON (llama/32128) ba8b013 CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (llama/26048) 05f605a ggml-backend : prefetch only inputs computed by earlier splits (llama/32100) b8a5b87 ggml : dispatch SpacemiT Q4_1 32x256 through the HP kernel (llama/31929) 7a8cef5 sycl: make oneDNN FA cache thread-safe and memoize runtime failures (llama/31138) 6dfea1a sycl : fix AOT for more than 4GB (llama/31873) 012ff44 sycl : use cached device-id in set_main_device (llama/32013) 2f9d47b CUDA: add cuda debug buffer pool (llama/31919) 522d257 opencl: add bin kernel kernel_gemm_noshuffle_q4_0_f32_32b_trans_ila_a7_bin (llama/30947) 0f9dcb7 CUDA: skip empty MoE tiles in MMQ stream-k fixup (llama/32067) 6bed9a8 opencl: add decomposed FA for Adreno (llama/32046) a357e7b metal : prevent out-of-bounds reads in SOLVE_TRI kernel (llama/32103) c36ccd8 vulkan: support batched tensor copy API (llama/31418) 1bffc59 backend : re-instantiate CUDA graphs when the cluster dims of a kernel change (llama/31284) 8153155 ROCm/HIP: fixed BLAS workspace instead of rocBLAS allocating one per call (llama/31838) ad7d200 cpu: accumulate flash attention V in F32 (llama/29810) 7f4a2db webgpu: add PAD_REFLECT_1D support (llama/31877) 0f51454 vulkan: allow dequantized KV optimization with CM2 (llama/31526) eb4de49 vulkan: allocate the device fault vendor binary buffer (llama/32008) 6b7f47a cuda : use MMVF for BF16 matrices of few rows at small batch sizes (llama/31959) 77d01f1 cuda : add sm70 FlashAttention config case for DKQ=256, DV=256, ncols=64 (llama/27997) bd37ba0 cuda : fattn mma head sizes 40 72 (llama/30285) 161ca15 CUDA: Add OCP FP8 support (llama/28413) 06faf80 sycl : fixed sycl CLAMP to actually support strided view (llama/31784) a986181 cuda : remove unused ids and type_src0 parameters (llama/32018) 55104b2 llama : Add FP8 KV-cache support (llama/28392) 0753641 ggml : add MUL_MAT src1 precision tests and gating (llama/31896) 1cd16bd cuda : remove unused quantize_cuda_t typedef (llama/31884) 4a18cdd cuda : halve VMM pool reservation size and retry on cuMemAddressReserve failure (llama/30542) 1c9ac9d CUDA: skip graph compat check when graph is being reused (llama/32004) 999c725 ggml : fix FreeBSD Vulkan build by setting _XOPEN_SOURCE=700 (llama/31701) 47bc1d8 CUDA: move graph and concurrent stream state out of common.cuh (llama/32001) df70e12 cann: fail early when checking if buffer type is supported (llama/31270) e447eb5 sycl: Add device-side MoE scheduler (llama/30156) 9a1c582 vulkan: add tiled-transpose concat shader for Qwen models (llama/30591) 8b3c8cf ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes (llama/30415) f860f1d vulkan: Divide GROUP_NORM workgroup to 2D to avoid maxComputeWorkGroupCount limit (llama/31930) eb5571a vulkan: Fix out of bounds reads/writes on MoltenVK (llama/31890) 5a9a264 vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm (llama/30829) 5610f41 sycl: un-gate oneDNN fused SDPA for Alchemist at head_dim=64 (llama/31574) 487cadf hexagon: fix 64-bit FastRPC handle truncation on close (llama/30952) be37c9c ggml-zdnn: accept non-zdnn tensors, invalidate compute tensors after every op (llama/31824) 294d5fc ggml : implement the backward pass for TRI (llama/30712) e18c2c6 ggml-backend: don’t upload inputs that are not read (llama/31419) e84d36b metal : prevent out-of-bounds writes to the Metal FA output buffer (llama/31662) 0dd184f metal : make ggml_tri select instead of scaling by the comparison (llama/30711) b6d482c opencl : include climits (llama/31935) 62de780 RPC: Improve loading times (llama/29413) a7276d0 ggml : support YaRN truncation config on openvino, sycl, vulkan, webgpu, rpc (llama/30802) da89459 sycl: cache device-id (llama/31126) fb57085 tests: bound the test_rope tolerance by the f32 angle resolution (llama/31874) e43d847 opencl: add f32 MUL_MAT_ID (llama/31783) d1c1e4b ggml-zdnn: fix release binary not detecting backend lib (llama/31905) 7e7b6e9 mtmd: refactoring: remove clip.h, consolidate code into mtmd (llama/31865) 5a0e536 cpu: enable SpacemiT IME2 MXFP4 dense repack (llama/31776) 7f4b25f llamafile: impl bf16 and q8_0 vxe (llama/31866) f22fc6a ggml : update CUDA/Metal radix Top-K to make tied selections deterministic (llama/31084) 47b8b4f ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (llama/31818) b524960 webgpu : add COUNT_EQUAL support (llama/31578) 75a4f5a ggml : switch to F32 src1 in the CPU backend (llama/31854) 6cddab2 llama, mtmd: follow the reference precision of ggml_prec_set_acc calls (llama/31851) 7353275 opencl: improve q8_0 and MoE small-n verification for spec (llama/31831) 8f1404e hexagon: support for automatic loading of libhtp without legacy env vars (llama/31781) 91c102a sycl: exact rounding in CPY to Q4_1, Q5_1 and Q8_0 (llama/31465) 8cf32bc sycl: refuse device buffers that would exceed VRAM (llama/31285) d969b07 hexagon: n_hvx and n_threads cleanup (llama/31857) 5746ebd llama_moe_cache: run missed experts on CPU, add tuning script (llama/30966) a49f174 sycl: add f16 support for SWIGLU_OAI (llama/30857) c6a0780 metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs (llama/31852) 1e5c3e0 webgpu : Implement group norm kernel (llama/31490) 37ecdca vulkan: Add ‘readonly’ to decodeBuf declarations. (llama/31833) c08538b CUDA: fix shared memory bank conflicts in the lightning indexer (llama/31803) 07b8362 opencl: add arange (llama/31829) f55a504 meta : raise the compute view limit for buffers with few tensors (llama/31447) 2870133 hexagon: fuse post-norm residual add into RMS_NORM_MUL (llama/31822) a135ec6 hexagon: conv_2d optimizations and support for conv_3d (llama/31814) e3d3633 tests : qualify isnan and signbit with std:: (llama/31791) a7d3330 tests : override hardware_concurrency in CI (llama/31797) ab7ca9c opencl: support repeat for f16, i32, i16, bf16 (llama/31782) 9290392 webgpu: add batched tensor API (get_tensor_n and set_tensor_n) (llama/31473) f594fed vulkan: detect ReBAR support, avoid BAR allocations if unavailable (llama/31658) 38e4be2 CUDA: fix Maxwell build (llama/31787) 8bd6b61 opencl: add q6_k multi column tiled GEMV for speculative verification and refactoring (llama/31758) d3ce63d hexagon: Implement set_tensor_n and improve repacking/tiling robustness (llama/31744) 9b1880f ggml-cuda: sanitize post-bias NaNs in MoE selection (llama/31005) 7e607b8 vulkan: Implement test-fusion interface (llama/30769) 2188d13 CUDA: use highest compiled arch in get_mmvq_mmid_max_batch (llama/31679) 114ce5e vulkan: RDNA3/4 Large-Tile Split-K Optimization (llama/30280) e074347 metal : fuse DeepSeek-V4/V4.1 and GLM MoE routing (llama/31615) f0f5b42 CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (llama/29050) fa8c617 CUDA: linearize MoE data layout for MMQ/MMF (llama/31047) 1690d2e vulkan: FWHT kernels for block widths above 512 (llama/29772) 40278c4 vulkan : check combined K+V f16 scratch size in FA dequant path (llama/30487) 2dccfdb sync : llama.cpp 25b845e ggml : support truncate=false for YaRN (llama/30206) accf2a1 sync : llama.cpp b2ce926 metal : use mul_mm for 32 <= K < 64 (llama/30982) 1a5f477 webgpu : add MEAN support (llama/31591) 2ab6d6c vulkan: Re-enable large coopmat matmul configuration on AMD proprietary driver 26.10.1+ (llama/31587) 19df6cf tests: fix CI flakes of TOPK_MOE caused by near ties (llama/31426) a0df99d metal: keep fusable node groups inside one command buffer (llama/31688) 3a996db opencl: enable f32 and q5_k multi column GEMV by default for Adreno X2 (llama/31694) d75184e opencl: support non-contig sigmoid (llama/31687) c634501 opencl: add DSV4_HC_PRE and DSV4_HC_POST (llama/31623) 0816137 Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ (llama/31296) 4ac0dae opencl: improve decode perf with FA (llama/31633) 4f681f2 webgpu: return maxBufferSize from get_max_size and avoid whole-buffer bindings in memset/clear (llama/31348) cd6d2b7 docs : sync ggml_type enum in gguf.md (#1685)
There was an error while loading. Please reload this page.