Insights
Thinking that helps you make better technology decisions
Analysis of AI systems and the software around them, from people who build and deploy them for real businesses. No hype, no listicles, and every claim sourced.
AI Agent Time Horizon Measurement Is Not a Uniform Capability Scale
Human task duration remains a useful anchor for agent capability, but a new audit finds that equal gains in time horizon need not represent equal gains in difficulty.
AI Agent Payment Integration Needs Transaction Limits
AI agents should earn authority in stages, not receive a general permission to buy. Safe payment integration separates recommendations, cart changes, checkout initiation and payment behind explicit limits and verifiable consent.
AI generated training data needs weighting, not a blanket ban
AI-generated web text can improve a data-starved model, then become harmful as stronger human corpora grow. Separate provenance and validation slices before deciding what to filter, reweight or retain.
Measure AI ROI by workflow outcomes, not usage
Measure AI ROI against changes in workflow cost, speed, quality and revenue. Usage data shows adoption; it does not show whether the business gained more than it spent.
AI agent consistency testing needs repeated runs
Average pass rates hide tasks that work once and fail on an identical rerun. Measure Pass^k, action disagreement and resulting state before calling an agent reliable.
Agent context compaction needs validated state, not free-form summaries
Agent context compaction can preserve false claims and carry unauthorised instructions into later turns. Typed state, provenance and independent validation make those transitions auditable.
Answer Engine Visibility Measurement Without False Confidence
Measure AI answer visibility with a fixed panel of buyer prompts, repeated runs and several models. Report citations, mentions, prominence, recommendation position and negative recommendations separately.
AI agent sandbox security must include package infrastructure
Blocking the browser is not containment. Package registries, build workers and shared artefact stores can still provide agents with communication, code execution and indirect internet access.
