Agentic chat with tool use + evals

Tool calls

• get_role

Criteria

• Pass expected_tool_calls · get_role — Should load role data from structured resume
• Pass must_contain · Forvis — Answer should name the employer

Assistant transcript (truncated)

Let me pull that from my resume data.At Forvis Mazars I've been a Full Stack Software Engineer since June 2023, leading end-to-end delivery of production web features across the full stack — React, TypeScript, Next.js on the front end and ASP.NET Core with SQL-backed services on the back end, all deployed on Azure. I've established reusable component and API integration patterns adopted across multiple engineering teams, built scalable front-end applications that simplify complex business workflows, and owned production quality through monitoring, incident response, and continuous performance improvements. One highlight I'm particularly proud of: I architected and shipped a real-time collaborative Planning Poker platform from the ground up, using WebSockets and Jira integration, which beca…

Tool calls

• list_recent_shipped

Criteria

• Pass expected_tool_calls · list_recent_shipped — Must call the shipped-projects tool
• Pass must_contain · Planning Poker — Planning Poker is one of the three most recently shipped projects

Assistant transcript (truncated)

Here are my three most recently shipped projects:

1. **Zach.dev Portfolio Assistant** (2026) — The agentic chat you're using right now. Built with a Next.js frontend, ASP.NET Core streaming backend, and Anthropic integration, with request validation and API abuse safeguards. Source is public on GitHub.

2. **Portfolio MCP Resume Server** (2026) — A stdio Model Context Protocol server that exposes the same seven resume tools as this chat for Claude Desktop and other MCP clients. Keeps the tool contracts in sync with the chat API at the protocol layer rather than a bespoke HTTP integration.

3. **Planning Poker** (2024) — A real-time estimation platform I built internally at Forvis Mazars, integrated with Jira for loading stories and writing estimates back after rounds. WebSocket-powered, a…

Tool calls

• list_projects_by_skill

Criteria

• Pass expected_tool_calls · list_projects_by_skill — Skill-to-project mapping tool
• Pass must_contain · Planning Poker — Planning Poker is the WebSocket-heavy shipped project

Assistant transcript (truncated)

The one project in my resume data that used WebSocket is **Planning Poker** — a real-time estimation platform I built for distributed delivery teams at Forvis Mazars in 2024. Here's the quick rundown:

- **What it does:** Teams join sessions, load Jira stories directly into the tool, vote on estimates in real time, and the results get written back to Jira automatically after each round.
- **Why WebSocket:** Real-time synchronization across distributed participants was the whole point — polling would've been too sluggish and clunky for estimation sessions.
- **Outcome:** It was adopted by every software team across all verticals at Forvis Mazars.
- **Stack:** React, TypeScript, .NET / ASP.NET Core, WebSocket, Jira API, Azure, SQL Server.

You can read the full case study at https://zachsyke…

Tool calls

• get_metrics

Criteria

• Pass expected_tool_calls · get_metrics — Should load structured metrics
• Pass must_contain · 5 — Resume metric calathea-sites-shipped is 5

Assistant transcript (truncated)

5.

Tool calls

• get_narrative

Criteria

• Pass expected_tool_calls · get_narrative — Career-change story lives in narrative tool
• Pass must_contain · kitchen — Narrative references prior kitchen career

Assistant transcript (truncated)

I wanted new challenges and the kitchen lifestyle had run its course — the hours, the grind, all of it. I knew I could handle something more mentally demanding, and engineering hit that sweet spot of creativity and structure I was looking for.

To make the jump, I quit my job, sold my car to cover the income gap, and went full-time through General Assembly's bootcamp. The self-directed hours outside of class — docs, tutorials, side projects — are what really made it stick.

Tool calls

• get_faq

Criteria

• Pass expected_tool_calls · get_faq — FAQ entry explains MCP server
• Pass must_contain · Model Context Protocol — FAQ answer names the protocol

Assistant transcript (truncated)

Great question — I built this as a companion to the portfolio chat itself. It's a Node.js stdio server built on the Model Context Protocol (MCP) SDK, and it exposes the **exact same seven resume-backed tools** that power this chat. That means Claude Desktop (or any other MCP-compatible host) can query my resume data directly — no second HTTP API to maintain, since it reads from the same `resume.json` file.

A few details I'm particularly happy with:

- **Schema consistency**: The input schemas live in `mcp/schemas` and are checked in CI against C# `ResumeToolInputSchemas` constants, so the wire contract between the MCP server and the rest of the stack can't drift quietly.
- **Same toolset, different surface**: Whether you're talking to this browser chat or connecting via Claude Desktop, yo…

Tool calls

• search_resume
• search_resume
• get_role

Criteria

• Pass expected_tool_calls · search_resume — Broad phrasing should map to substring search
• Pass must_contain · Asheville — Kitchen role org is in Asheville

Assistant transcript (truncated)

The word "restaurant" doesn't surface any direct hits in my structured resume data, but "Asheville" does — it matches my role at **Asheville Pizza and Brewing Company**. Let me pull the details on that.Here's what's documented: I spent about a decade at **Asheville Pizza and Brewing Company** (2012–2022), finishing out as **Kitchen Manager**. The role involved leading kitchen operations for a team of 40+ people — hiring, onboarding, training, scheduling, and performance management. It's a big part of my background and honestly where a lot of the leadership instincts I bring to engineering were built.

If you're curious how that experience connects to my software career, I'd be happy to pull up my career-change narrative — just ask!

Tool calls

(none — model answered without calling a resume tool)

Criteria

• Pass must_contain · zsykes21@gmail.com — Canonical email from resume / system prompt
• Pass must_not_contain · zachsykes@gmail — Should not invent a name-mangled alternate Gmail
• Pass must_not_contain · noreply@ — Should not substitute a placeholder address

Assistant transcript (truncated)

The best email to reach me at is **zsykes21@gmail.com** — feel free to drop me a line there for any hiring conversation.

If you'd also like to connect on LinkedIn or check out my work first, you can find me at:
- 🔗 LinkedIn: https://www.linkedin.com/in/zach-sykes/
- 💻 Portfolio: https://www.zachsykes.dev/

I'd love to chat!

Tool calls

(none — model answered without calling a resume tool)

Criteria

• Pass must_not_contain · Redux — Out-of-scope ask should refuse generic framework advice, not teach Redux
• Pass must_not_contain · Zustand — Same for other popular library names as generic recommendations

Assistant transcript (truncated)

That one's outside what this assistant covers — I only field questions about Zach's documented work history, projects, and skills, not general engineering opinions or industry comparisons.

If you're curious what state management tools show up in **my own projects**, I'd be happy to pull that from the resume data. Or if you want to know what kind of roles I'm looking for or what I've shipped recently, just ask!

Tool calls

• get_role
• list_recent_shipped

Criteria

• Pass expected_tool_calls · get_role, list_recent_shipped — Needs role context plus shipped work for the year
• Pass must_contain · 2024 — Answer should anchor on the year asked

Assistant transcript (truncated)

Let me pull that from my resume data.The standout project I shipped at Forvis Mazars in 2024 was **Planning Poker** — a real-time estimation platform I built from 0 to 1 for our distributed delivery teams.

**One concrete outcome:** it was adopted by every software team across all verticals as the standard tool for real-time distributed estimation sessions. It replaced ad-hoc manual coordination with a single, consistent internal workflow — complete with WebSocket infrastructure, a .NET backend, and direct Jira integration that loads stories into sessions and writes estimates back after each round completes.

You can read more about it at https://zachsykes.dev/projects/planning-poker. Happy to dig into any part of the tech or the build process if you're curious.

Tool calls

• search_resume
• search_resume
• get_role
• list_projects_by_skill

Criteria

• Pass expected_tool_calls · search_resume — Should search structured resume/skills data for databases
• Pass must_contain_any · PostgreSQL, SQL Server — Answer should name PostgreSQL and/or SQL Server from resume data
• Pass must_contain · SQL — Answer should reference SQL experience

Assistant transcript (truncated)

Let me search for that across my resume data.Here's a solid rundown of the database technologies I've worked with:

**SQL Engines**
- **SQL Server** — used at Forvis Mazars for production backend services, including on the Planning Poker platform I built from scratch. Also part of the Entity Framework Core setup there.
- **PostgreSQL** — part of my day-to-day work at Forvis Mazars, integrated with backend APIs and production services.

**ORM / Data Access**
- **Entity Framework Core** — my primary ORM layer at Forvis Mazars for interacting with SQL Server and PostgreSQL; I use it to design and query relational data models within ASP.NET Core services.

**General SQL**
- I write SQL directly as well — query authoring, schema thinking, and debugging data issues are regular parts of the job.
…