Skip to content
AI NEWS JOURNAL
AI, IN PERSPECTIVE

Search AI news

Search headlines, summaries and articles to connect the dots.

Newest first

Introducing eve 1.0

Vercel announces the general availability of eve 1.0, an open-source framework for building production-ready agents. The release includes performance improvements, expanded integrations, enhanced observability via OpenTelemetry, and new features like multi-tenant support, persistent memory, multiplayer sessions, and lower workflow costs. eve enables developers to build agents with customizable workflows, native UI integrations with frameworks like Next.js, and secure, scalable deployments on Vercel. Early adopters report significant gains in development speed, reduced codebases, and lower AI costs.

Agents are users, so your software needs evals

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These tests run multiple trials, score outcomes, and help identify failures across steps like searching cafes, checking budgets, and placing orders. Using frameworks like Harbor, teams can define tasks with specific instructions, environments, verifiers, and known-good solutions. Metrics such as success rate, Pass@k, and Pass^k reveal how reliably agents perform, while trace analysis uncovers root causes of failure. The process includes setting a goal based on observability, customer feedback, or roadmap needs; defining measurable metrics; designing task sets covering common and edge cases; running evaluations at scale using tools like Vercel Sandbox and AI Gateway; and analyzing traces to iterate improvements. Automated trace analysis enables scaling, and repeated evaluation supports continuous improvement, regression detection, and product optimization.

WTF are evals? How to test agents and the software they use

Agents interact with software through a funnel similar to humans: Discovery, Selection, Access, and Success. To measure whether agents complete user tasks correctly, end-to-end (E2E) evaluations are essential. These evaluations assess the full sequence of steps an agent takes, ensuring outcomes meet user requirements. Using frameworks like Harbor, developers can design tasks with defined environments, verifiers, and known-good solutions, then run trials across variants to measure success rate, pass@k, pass^k, efficiency, time, and cost. Analyzing traces reveals root causes of failures, enabling targeted improvements. Iterative evaluation helps catch regressions, test changes, and guide product development, turning evaluations into ongoing infrastructure for building better agent-driven products.

Vercel Container Registry now supports 2 GB compressed image layers

Vercel Container Registry now allows compressed image layers up to 2 GB each, while maintaining a total image size limit of 15 GB. It supports standard Docker workflows like docker push, docker pull, and docker tag, automatically creating repositories on push. Images can be made public or shared with up to 100 teams via the dashboard or CLI. The Vercel CLI v58.3.0 or later enables management of repositories, images, and tags, and GitHub Actions integration uses GitHub OIDC for secure CI logging without long-lived credentials. Failed background optimizations show error statuses in the dashboard, with retry options upon re-push.

How Mercor ships internal apps and billions of production tokens on one platform

Mercor reduced batch job processing time from 12 hours to 2–3 hours by using Vercel AI Gateway, which provided increased inference capacity and failover routing across providers. In its first 10 days on the platform, Mercor routed 18 million requests and 63 billion tokens without hitting rate limits. The company also leveraged Vercel Passport and its internal Audiences product to provision secure access for hundreds of business teams and over 2,000 projects in seconds, enabling rapid deployment of client-specific applications while maintaining strict usage controls and automated offboarding.