Custos
A private AI assistant that answers questions about a company’s internal documents. The embedding model runs locally and the vector store is self-hosted, so document content never leaves the environment. Answers are scoped to what you are allowed to see, and every one of them cites the source it came from.
The gate holds after the model is compromised. In red-team testing a poisoned document convinced the assistant to send an email, and it genuinely tried. The email never went, because tools with real-world effects cannot execute themselves — they return a single-use token that only a human confirmation redeems. Detection eventually fails against an attacker with unlimited attempts. That is the argument for putting the control underneath it rather than in front of it.