Local AI offers organizations greater privacy, cost control, and data ownership, but running models locally does not eliminate security risks.
Research presented in connection with DEF CON 34 identified 10 vulnerabilities in llama[.]cpp, a widely adopted inference engine underlying many local AI applications.
Key takeaways
- Researchers identified 10 vulnerabilities in llama[.]cpp, including use-after-free, integer overflow, and out-of-bounds memory access flaws affecting multiple trust boundaries.
- Two llama-server vulnerabilities received CVSS scores of 9.2, highlighting the potential risk to organizations exposing local AI inference services to untrusted traffic.
- Local AI does not eliminate security risks. Keeping prompts and sensitive data on organizational infrastructure still requires securing the inference engine, models, APIs, dependencies, and supporting infrastructure.
- Organizations should identify where llama[.]cpp is deployed and reduce exposure, including hardening internet-facing APIs, validating untrusted inputs, patching vulnerable components, segmenting AI workloads, and monitoring for suspicious activity.
Understanding llama.cpp and local LLM inference
Large language model (LLM) inference is the process of loading trained model weights into memory, processing input tokens through the model, and generating output one token at a time.
With remote inference, prompts and data are transmitted to an external provider. Local inference instead performs these operations on infrastructure controlled by the user or organization.
This distinction is important for organizations handling intellectual property, classified information, proprietary financial strategies, or other sensitive data that cannot be transmitted to third-party AI providers.
What is llama.cpp and how does it support local AI?
Llama[.]cpp has emerged as a significant component of this ecosystem.
Written primarily in C/C++, the library loads GGUF model files, manages inference contexts, performs tokenization and model computations, and provides interfaces used by other applications.
It serves as a backend or integration layer for products including Ollama, LM Studio, Jan, and GPT4All.
Its widespread adoption also means vulnerabilities in the underlying engine can potentially affect numerous downstream implementations.
Llama[.]cpp vulnerabilities expose critical memory safety flaws
Cyera researchers audited llama[.]cpp with particular attention to three trust boundaries:
- Android Java Native Interface (JNI) integrations
- The llama-server HTTP lifecycle
- GGUF model metadata processing
The research identified vulnerabilities involving familiar memory-safety problems, including use-after-free (UAF), integer overflow, and out-of-bounds access.
VulnCheck subsequently allocated CVE-2026-43622 through CVE-2026-43632 for the reported findings, excluding CVE-2026-43625 (CodexBar vulnerability).
As of a June 2026, assessment of build b9445 and gguf-v0.19.0, researchers reported that five of the 10 vulnerabilities remained unpatched.
Among the most significant were two server UAF vulnerabilities, CVE-2026-43631 and CVE-2026-43632, each carrying a CVSS score of 9.2.
How concurrency flaws create memory safety risks in local AI
Several findings demonstrate how concurrency can transform ordinary memory-management mistakes into security vulnerabilities.
Android JNI concurrency creates use-after-free risks
In an older Android JNI integration, multiple operations shared global native pointers.
A background thread could perform inference while another thread freed the same context.
Without adequate synchronization, the first thread could subsequently access memory that had already been released.
Researchers reported exploiting this condition by reclaiming freed memory and manipulating subsequent program execution.
Llama-server flaw exposes another use-after-free condition
A similar lifetime-management issue affected llama-server.
When the –sleep-idle-seconds feature was enabled, idle teardown could release model or vocabulary resources while another worker continued processing a request using those resources.
This created another UAF condition potentially reachable through the server interface.
These findings highlight an important security consideration for local AI, as keeping prompts on-premises does not automatically ensure that the underlying inference stack is trustworthy.
How to mitigate the llama[.]cpp vulnerabilities
Organizations using llama[.]cpp should first determine where the library exists within their AI environment, including applications that incorporate it indirectly.
Harden llama-server and API exposure
Internet-facing llama-server deployments should not enable –sleep-idle-seconds when accepting untrusted HTTP traffic until the associated UAF vulnerabilities are confirmed to be fixed.
APIs should avoid unrestricted exposure on 0.0.0.0 and instead use authentication with a reverse proxy or comparable access-control layer to limit unauthorized access.
Secure llama.cpp integrations and untrusted inputs
Android developers using the older llama-android[.]cpp JNI wrapper should migrate away from versions preceding build b7446 and review native components for unsafe concurrent access.
Applications using llama_batch_init should validate any untrusted input before passing it to native code.
Likewise, applications should avoid restoring state or KV-cache data from untrusted sources until the related memory-safety vulnerabilities are resolved.
How to secure local AI and open-weight models
Local inference offers meaningful privacy and control advantages, but those benefits introduce responsibility for securing the runtime itself.
Memory corruption, unsafe object lifetimes, malicious model files, and exposed inference APIs can create attack surfaces comparable to those found in traditional native applications.
Organizations adopting open-weight AI should therefore treat inference engines as security-sensitive infrastructure and incorporate controls such as:
- Maintain an inventory of local AI models, inference engines, dependencies, and other components.
- Restrict network exposure and segment AI workloads from sensitive production systems.
- Enforce least-privilege access and strong authentication for inference APIs and administrative functions.
- Validate and scan model files, state files, dependencies, and other artifacts before deployment.
- Apply security patches promptly and monitor upstream projects and dependencies for newly disclosed vulnerabilities.
- Sandbox inference workloads using containers, virtual machines, or other isolation mechanisms where appropriate.
- Log and monitor AI system activity, and regularly test incident response plans to ensure teams can contain and recover from AI-related security incidents.
Together, these measures can help organizations reduce exposure to AI-related threats while building resilience.
Bottom line
The llama[.]cpp vulnerabilities highlight an important reality of local AI: keeping sensitive data within an organization’s environment can improve privacy and control, but it does not eliminate security risks.
Organizations must secure the inference engines, models, dependencies, and infrastructure processing that data to reduce exposure and build more resilient local AI deployments.
Zero Trust can extend these protections by enforcing least-privilege access, continuously validating trust, and limiting the potential impact of a compromised AI workload or component.