Andrew Bolaji · Engineer

Selected work Each one ends on the decision that made it work

One argument runs through all seven: the guarantee belongs in the structure, not the interface. A paywall the client can bypass was never enforced, a compliance record you can delete is not a record, and a safety rule written into a prompt is a request. Every “hard part” below is the place where that rule actually lives.

Applied AI · Security

Custos

Live

A private AI assistant that answers questions about a company’s internal documents. Your corpus never leaves your environment to be indexed or stored: the embedding model runs locally and the vector store is self-hosted. Only the passages retrieved for a single answer reach the generation model, and generation is designed to be swapped for a local one. Answers are scoped to what you are allowed to see, and every one of them cites the source it came from.

The hard part

The gate holds after the model is compromised. In red-team testing a poisoned document convinced the assistant to send an email, and it tried. The email never went, because tools with real-world effects cannot execute themselves. They return a single-use token that only a human confirmation redeems. Detection eventually fails against an attacker with unlimited attempts. That is the argument for putting the control underneath it rather than in front of it.

The permission filter runs before retrieval, so the model is never handed a document the caller may not see.

Custos answering an HR question with every email address masked

HR query, general access. Every email in the answer comes back masked.

PythonFastAPIQdrantpgvectorsentence-transformersClaudeReactDockerTerraform

Try it live ↗ Deployment evidence ↗ 250 tests · unauthorized action rate 0 across 8 documented threat classes, each with a named control and a passing eval 61 adversarial evals gate every commit in CI; 73 total, including 12 live-model evals run before releases. Applied end to end in a live AWS account from a 34-resource Terraform module — ECS behind a load balancer across two availability zones. The permission gate was verified through the load balancer, not only in tests.

Applied AI · Voice

Porter

Live

The agent in the callout at the top of this page. Retell AI for voice, Claude and GPT for reasoning, Twilio for telephony. Because a system prompt is the product and can regress silently, it ships behind a containerised eval harness: 43 scripted scenarios, deterministic replay, CI-enforced on every push, across extraction, urgency classification, guardrail adherence, escalation precision and recall, and adversarial prompts.

The hard part

The first run failed, and the eval was wrong, not the agent. Three of the failures turned out to be false positives in my own scorer, including a regex that flagged “we don’t do landscaping” as an invented service. Trusting the scorecard would have meant fixing a prompt that was not broken. A harness you have not audited is just confidence with extra steps.

Retell AIClaudeOpenAITwilioPythonFastAPI

Call it 515-608-4723 → Metrics: 43 scripted call scenarios, run live against the model five times: 43, 43, 42, 43, 43. Median 43 of 43. CI gates every push on replayed recorded responses, so the gate costs nothing per commit.

Web · Compliance

Roster

In production

Operations and compliance software for regulated housing operators, built to a municipal ordinance with mandatory incident reporting and a five-year records retention minimum. Applicant intake, resident records, QR check-in, and a shareable inspector view.

The hard part

Hard DELETE is blocked at the database level. Not discouraged, blocked. A compliance record you can destroy is not a compliance record. Public form IPs are SHA-256 hashed, check-in photos are EXIF-stripped in the browser so GPS never reaches the server, and every anonymous endpoint runs through a function that whitelists the exact fields it can return.

React 19TypeScriptSupabasePostgres / RLSZod

316 tests · 21 migrations · running live

Data · Cloud

Reckon

Deployed

A business intelligence platform. Three data sources — an AI voice agent’s call records, Stripe, and MongoDB — into a warehouse, cleaned and modeled with dbt, served through an API to a dashboard, with an MCP server copilot that answers questions in plain English from any MCP client, including Claude Desktop. Infrastructure provisioned with Terraform and deployed to Kubernetes on AWS.

The hard part

Bad data stops the pipeline, not the dashboard. Data-quality and freshness tests run inline with the transformation, so a failing test stops downstream models from materializing. Stale numbers never reach a screen someone is about to make a decision from. The copilot follows the same rule from the other direction: it refuses to answer on stale data rather than guessing, and its limits live in a read-only database role instead of in the prompt, so even a fully manipulated model cannot reach a raw record.

52 Terraform-managed resources on real AWS. The whole environment builds from nothing and rebuilds the same way every time.

PythondbtDagsterRedshiftMongoDBFastAPIMCPRechartsMetabaseOpenTelemetryTerraformEKSHelm

Metrics: 52 Terraform-managed resources on real AWS · VPC, EKS, Redshift, S3, ECR, IAM and networking, all provisioned from code

Also built Junta, a two-sided roofing marketplace built for the site owner, where escrow had to be built against Stripe Connect’s automatic daily payouts rather than on top of them. Setting the connected account to manual payouts is the single load-bearing config in the whole system. Vouch, an iOS and Android app with a server-enforced paywall, 452 tests. Chants, where vote counters are recomputed from ground truth and set to an absolute value, because Cloud Functions fire at least once and not exactly once.

Client work Constraints I did not choose, still in production

Built for the owner of a private telehealth practice, a psychiatric nurse practitioner. Patient intake, calendar and booking, payment collection and transactional email, integrated with SimplePractice, the electronic health record system the practice was already running on. In production and still in use.

What made it different

Every other thing on this page, I chose the constraints. Here I did not. The intake and clinical wording had to sit inside the practice’s HIPAA obligations, the coverage information could only list the insurance plans she actually accepts, and the whole flow had to hand off to the EHR she was already paying for rather than the one I would have picked. That is the job: the customer’s environment is the specification, and you deliver into it rather than around it.

IntakeSchedulingPaymentsTransactional emailSimplePracticeHIPAA-aware copy