Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Using EXO_OFFLINE to prevent runtime internet access.
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors.
- Verifying model weight integrity with SHA-256 checksums and signed model cards.
- Running EXO in air-gapped networks without HuggingFace dependencies.
Dashboard and API Access Control
- Installing and configuring reverse proxies (nginx, Caddy) with TLS termination.
- Implementing role-based access control for the EXO dashboard and REST API.
- Using macOS keychain or Linux pass to store secrets for API authentication.
- Restricting administrative endpoints to specific source IP ranges.
Cluster Isolation and Network Security
- Segmenting EXO clusters with EXO_LIBP2P_NAMESPACE and VLANs.
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports.
- Preventing unauthorized device discovery and rogue node injection.
- Encrypting libp2p traffic between nodes when RDMA is not available.
Model Governance and Provenance
- Building an internal model registry with approved model lists and metadata.
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside source checkpoints.
- Enforcing that only specific HuggingFace repos or internal artifacts can be loaded.
- Documenting model lineage, license terms, and acceptable use policies.
Audit Logging and Compliance
- Configuring EXO log forwarding to immutable audit trails (SIEM, WORM storage).
- Correlating API call logs with user identity and timestamp.
- Capturing model instance creation, deletion, and inference request events.
- Generating periodic compliance reports for internal and external auditors.
Threat Modeling and Incident Response
- Identifying threats: data exfiltration through model outputs, prompt injection, side-channel leaks.
- Implementing prompt monitoring and content filtering pipelines.
- Creating incident response runbooks for cluster compromise scenarios.
- Isolating affected nodes, preserving forensic logs, and rebuilding clean environments.
Physical Security and Hardware Boundaries
- Securing Thunderbolt ports against unauthorized RDMA cable connections.
- Using secure enclaves and Apple Silicon hardware attestation where applicable.
- Controlling physical access to clustered Macs and shared storage.
- Documenting hardware lifecycle and decommissioning procedures.
Regulatory Considerations
- Mapping EXO deployments to GDPR, HIPAA, and SOC 2 requirements.
- Maintaining data residency by keeping inference on-premise.
- Documenting vendor supply-chain risks (MLX, EXO, model weights).
- Preparing for AI governance frameworks such as EU AI Act Article 53.
Requirements
- Experience with EXO or another local LLM runtime.
- Understanding of Unix filesystem permissions and networking ACLs.
- Familiarity with TLS/SSL certificate management and basic encryption principles.
Audience
- Security engineers.
- Compliance officers.
- AI infrastructure administrators managing sensitive data.
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
