Skip to content

Home

About me

So AI can be safer, I evaluate its performance. One cool way to do this is to figure out the hidden story inside a human expert's mind when they evaluate performance in their field.

Knowledge-dissection projects

  • τ-Discernment — What does a user latently require before an agent takes a consequential action? I lift each user's implicit requirement, buried in task prose, into a typed, provenance-grounded object a grader can score (UserPreflightRequirements) — surfacing failures a task-completion grader misses. github.com/borisdev/tau-discernment
  • NoBSmed — What makes a medication or supplement recommendation questionable given the evidence, missing evidence, and relevance to the patient?
  • HealthBench audit — What makes a "doctor-approved" answer or grading rubric unreliable? I found fabricated citations and 29 possible patient-harm issues in OpenAI's medical-AI benchmark family. github.com/borisdev/nobsmed-healthbench-audit
  • HealthBench EBM failure patterns — Where do frontier models fail on evidence-based medicine, and which failures are worth fixing? I ran GPT-5.2 and Claude Opus 4.8 on an EBM-verified slice of HealthBench and clustered the errors into seven canonical recall/precision failure patterns — each with anchored examples, a root-cause hypothesis, and a distinguishing experiment. github.com/borisdev/healthbench-ebm-verified
  • Muni-Resilience-Bench — How much does a specific resilience project (wildfire hardening, flood control, seismic retrofit) lower a city's borrowing cost? An open benchmark for the missing, auditable translation from a project's risk reduction to citywide fiscal impact. github.com/borisdev/muni-resilience-bench
  • Wolf Games — What makes an AI-generated murder-mystery storyboard coherent across interpolated scenes?
  • PhD thesis — What makes an inequality metric capture nuanced gaps among social groups in the same city? I modeled those relationships as weighted edges.
  • SimpleLegal — What makes "Called the State Senator" lawyer-level work rather than an administrative task? The classification depended on the relevance of the output, not the wording of the task.

Selected technical impact

  • Audited OpenAI's HealthBench medical-AI benchmark family — surfaced fabricated citations and 29 possible patient-harm issues: github.com/borisdev/nobsmed-healthbench-audit
  • Built an AI agent-evaluation framework for Sindri.ai using Temporal.
  • Helped relaunch a stalled legal-billing AI feature by eliciting lawyer expertise and redesigning the rubric for 11 billing flags.
  • Built story-scene prediction for a gaming startup led by a Law & Order producer.
  • Contributed an experimental causal-graph agent to LangChain: langchain-ai/langchain#6255
  • Built backend systems for factory analytics, people analytics, and Tableau geospatial services.
  • Embedded subjective concerns into statistical analysis of income inequality in my social-science PhD: escholarship.org/content/qt8br7d5df

Non-tech fun points

  • Climbed Cotopaxi (about 19,300 ft).
  • Survived bodyboarding Mexpipe.
  • Taught geospatial data to students in Medellín, Colombia.
  • Taught kids snowboarding.
  • Managed an international restaurant team.
  • Counseled severely emotionally disturbed children.

Elsewhere

Writing

Essays on the patterns I've seen recur. Some are primarily a way to organize my own learning; I'm not an expert in everything I write about — especially the compiler-design piece.

# Title Topic
1 Beyond RAG: How Chomsky's I-Language and Compiler Design Converge on Knowledge Graphs LLVM-style IR, Chomsky's I-language, BFO ontology, grammar-first design
2 What Is Knowledge Engineering, Really? A working definition built around elicitation, evaluation, and 0→1 modeling in messy domains
3 Fine-Tuning LLMs Will Restructure Your Data Science Team How fine-tuning replaces annotation pipelines and the NN-optimization role; the new "fine-tuning analyst"
4 Why Domain-Specific Language AI Features Fail The customer-discovery process for niche language AI, and why a Lean Startup approach is required
5 Language AI Evaluation 101: Know Your User Why simplistic Ground Truth produces misleading accuracy metrics; cognitive empathy as the iteration loop
6 Hyper-Local Community Funding: A DAO Alternative to CDFIs Local digital tokens and DAOs as a delivery mechanism for under-served-neighborhood capital
7 Inequality with a Spatial View A note from my 2014 dissertation: the same income data can read as inequality going down or up depending on whether you keep the spatial structure. A spatial view is a graph
8 CV: Knowledge Engineering in Messy Domains The IR-compile pattern across clinical trials, legal billing, maritime construction, narrative gaming, and geographic inequality