← Back
Frequently Asked Questions
Everything you need to know about private, offline LLM inference on iOS
How do I install this SDK in my iOS project?
Add the SDK via CocoaPods or Swift Package Manager. After installation, import OfflineLLM in your view controller, initialize with let llm = OfflineLLM(modelName: "your-model"), and start generating inference. Complete setup takes under 5 minutes for a basic implementation.
What LLMs are supported?
We support quantized versions of popular open-source models: Llama 2 (7B, 13B), Mistral (7B), and smaller models optimized for mobile. Models are pre-quantized to 4-bit or 8-bit precision for on-device inference. Custom model support available for enterprise customers.
How much storage and RAM does the SDK require?
A 7B quantized model occupies 3.5-4.5 GB on disk. Runtime RAM usage peaks at 2-3 GB during inference depending on model size and batch configuration. Smaller models (3B) fit comfortably on most modern iPhones with 1.5-2 GB peak RAM.
How does performance compare to cloud APIs?
Local inference trades absolute latency for privacy and reliability. First token arrives in 500ms-2s on modern iPhones (vs. 100-300ms cloud). Throughput is lower, but you eliminate network latency, API rate limits, and cloud provider outages. Ideal for offline-first apps.
Will running LLMs drain battery?
Inference is CPU/GPU intensive. A 5-minute session consumes 3-5% battery on iPhone 14 Pro. Sustained heavy use (e.g., 30 minutes) drains 15-25%. We provide APIs to check battery state and gracefully degrade inference quality during low-battery conditions.
How do I update model weights?
Models can be bundled at build time or downloaded on first launch (with user consent). We provide a model-update API that patches weights incrementally without re-downloading the entire model. Typical update size is 200-500 MB for the latest weights.
Can I fine-tune models for my use case?
The SDK doesn't support on-device fine-tuning. Instead, fine-tune on your server and quantize the result to 4-bit, then ship the tuned weights with your app. For production fine-tuning workflows, contact our enterprise team.
What happens when there's no internet?
Everything works offline. Model weights and inference run entirely on-device, with zero network calls. Users can chat, generate text, or run inference without any connectivity. This is a core design principle of the SDK.
How do you handle privacy and data retention?
Zero data leaves the device. Inference runs entirely locally; no telemetry, logs, or user inputs are sent to our servers unless explicitly enabled by the developer via opt-in analytics. All model weights are encrypted on disk. No backups, no cloud sync, no third-party access.
Does this work with SwiftUI and UIKit?
Yes to both. The SDK provides native Swift APIs compatible with UIKit and SwiftUI. Example SwiftUI integration is included in our documentation. Inference runs on background threads automatically; UI updates are thread-safe.
What iOS versions are supported?
iOS 14+ (iPhone XS and later). Neural Engine acceleration available on A13 Bionic and newer. Older devices fall back to CPU inference with graceful degradation in token throughput.
How do I handle inference errors gracefully?
The SDK provides error enums for out-of-memory, model-not-found, and timeout scenarios. Wrap inference calls in try-catch blocks and implement user-facing fallbacks (e.g., suggest reducing context length, freeing memory, or retrying after a delay).