Design intent often erodes somewhere between the approved mockup and the shipped product. Nine checks, each paired with the artifact to request and the person who should own it, help you judge before signing whether a build partner can carry that intent through engineering and quality assurance.
Every product design agency shows you its best work. Case studies with a clean Figma file and a hero shot of a dashboard that apparently shipped without incident. None of that tells you what happened in week nine, when an endpoint returned a shape nobody designed for, and somebody had to choose between changing the screen or changing the backend.
That choice is where launches are won or lost. Portfolios document outcomes, and you're buying a process. Nothing in a case study reveals how defects were triaged or whether the design systems on display were ever reconciled with the code that rendered them. So the question worth asking is narrower than "is this good design?" It's what an agency does after the design is approved.
Each signal below is something you can ask for during an interview or find in a proposal. Treat the artifact as the evidence and the named owner as the proof it's real.
Ask what "done" means, and you'll learn most of what you need. In a strong engagement, design and engineering agree on testable completion conditions before build starts, and quality assurance writes its cases against those conditions. The criteria live in the product requirements document and get referenced in build reviews.
The Project Management Institute (PMI) found in 2014 that 47 percent of unsuccessful projects missed their goals because of inaccurate requirements management. In the same year, its requirements management report found that only 46 percent of organizations had a formal process for validating requirements. Ask for three things:
A sample set of acceptance criteria from a shipped feature, with the pass conditions written in language a tester can act on
The name and role of whoever approves criteria when design and engineering disagree
One example of a build review where a feature was sent back because it failed a stated criterion
Feasibility is a design input. When an engineer reviews flows while the visual decisions are still soft, the expensive rework happens on a whiteboard. When the review comes after approval, it happens in Jira.
Figma’s 2025 survey of designers and developers found that 84 percent of designers collaborate with developers at least weekly. It also found that differences in assumptions were the most commonly cited challenge (52 percent) when working together. Assumptions get cheap to correct early. Request the feasibility review records: who attended and what changed as a result. Praetorix Foundry runs senior-led product design, engineering, and quality assurance as one accountable team, so engineers review flows before visual decisions are locked. A product design agency that can’t show you when engineering first saw the work is telling you it saw it late.
Tell us about your idea. 24-hour response. NDA-friendly.
Book a Discovery Call →A stakeholder walkthrough tells you whether the client likes the design. A task-based test tells you whether a person who has never seen the product can finish the thing they came to do. These are different activities, and only one of them predicts launch behavior.
Jakob Nielsen’s 2000 article, drawing on his research with Tom Landauer, put the proportion of usability problems found by a single participant at roughly 31 percent. That is why five participants per round, repeated across iterations, beats one large study at the end. What you want to see is the chain: findings written as observed behavior and a revision log that connects a specific problem to a specific change. The researcher or design lead who ran the sessions should own both and be able to walk you through them. If the findings exist but nothing changed, testing was useless.
Ideal-path screens are the easy half. The expensive half is everything that happens when the data hasn't arrived, or the request failed, and something has to be recovered.
Baymard Institute’s 2024 e-commerce checkout benchmark found that 31 percent of sites provide no inline field validation at all, so users discover errors only after submitting. A separate 2023 finding from the same research program: 98 percent of sites use generic error messages, even though the backend knows exactly what went wrong. Those are design failures before they are engineering failures. Ask for annotated flows and component-state documentation covering:
Loading and empty states, including what the user reads while waiting
Error states (field, request, and system errors), with the copy and the next action for each
Permission-denied and recovery paths, with the exact copy for each
The lead product designer should own this documentation, with a front-end engineer confirming each state is built as specified.
Contrast checking is where accessibility work starts and a lot of agencies stop. The 2026 WebAIM Million scan found detectable WCAG failures on 95.9 percent of the top million home pages, with low-contrast text on 83.9 percent, and the same six error categories have topped that list for seven years running.
Contrast is the easiest of those six to fix, which is what makes it a weak signal on its own. Look instead at behavior. Does the component spec say what receives focus when a modal opens and what a screen reader announces when validation fails? WCAG 2.2 success criterion 2.4.11 requires that a component receiving keyboard focus is not entirely hidden by author-created content, such as a sticky header or a cookie banner. That is a question nobody answers from a static frame. Ask for accessibility criteria written per component, one example showing the same component across input methods, and the name of the accessibility lead who signs them off.
Tell us about your idea. 24-hour response. NDA-friendly.
Book a Discovery Call →A designed button and a coded button with the same name and different behavior is a defect generator that produces one bug per screen. In Sparkbox’s 2022 survey of 183 design system maintainers, 37 percent named parity between design and code a top challenge, just behind technical debt. Zeroheight’s 2026 report found only 7 percent of respondents described their system as fully adopted across all teams.
So treat parity as something to be demonstrated. Ask for a published Storybook instance showing states as they’re actually coded or a reconciliation record listing discrepancies found and how each one was closed. The design system lead should own it, with a front-end lead confirming the coded components. Agencies that maintain design systems well keep that record because they need it themselves. Agencies that describe design systems in the abstract have a Figma file and a hope.
Content, data access, and third-party application programming interface (API) limits each have a lead time, and each one blocks a screen that looks finished. The dependency an agency discovers during handoff becomes your delay.
The same 2014 PMI report found that 5.1 percent of every dollar spent on projects and programs is wasted due to poor requirements management, which is largely a story about things nobody confirmed in time. A product requirements document that lists features without listing what those features depend on is half a document. Ask for a dependency register from a past engagement showing the owner of each item and the fallback if it didn’t arrive. Fallbacks are the tell. Teams that have been burned write them down.
A tidy Figma file is not a handoff. Without states, annotations, dependency owners, and a build review, it is a picture of intent. Design quality assurance belongs in staging throughout implementation.
The cost of late defects is well documented. A 2002 NIST-commissioned RTI study estimated the annual cost of inadequate software testing infrastructure in the US at $22.2 to $59.5 billion. Catching a spacing error against a design system token in staging costs minutes. Catching it after release costs a hotfix. Ask for build review records: what was flagged and who signed off.
Something will break. The 2024 DORA report put change failure rates at around 5 percent for elite performers and roughly 40 percent for low performers, so the question is not whether something breaks but how fast the failure is classified and who owns it.
Atlassian's three-tier model is a reasonable benchmark: a SEV 1 is critical with a customer-facing service down for everyone, and SEV 2 affects a subset. Ask for a sample triage process covering:
Severity definitions with examples drawn from a real release, not a template
A named owner per severity level and the expected response window for each
The cadence of the post-launch review and how long the agency stays on it
Score each candidate on the same nine signals and ask one question of every answer: did they hand you an artifact and a name, or a philosophy? Documented beats confident. An agency that can't produce a dependency register or a filled-in product requirements document after two conversations won't produce one in week nine.
Praetorix Foundry works this way: senior designers and engineers on one accountable team, no junior handoffs. Book a discovery call and bring the same scorecard you’d use on any product design agency.
Tell us about your idea. 24-hour response. NDA-friendly.
Book a Discovery Call →Use the same nine categories, then record whether each artifact came from a comparable product, team size, and delivery stage. Mark unverifiable claims separately instead of treating them as failures. This gives you a fair comparison while showing where an agency’s process has not been tested in a relevant setting.
The contract should name who approves design changes, who resolves disputes, and how revisions affect scope and timing. It should also require documented review of staging builds and define post-launch support. These terms prevent responsibility from ending at handoff when implementation exposes a problem.
Ask the agency to explain when the artifact was created, who used it, and what decision followed. Request a redacted version with dates, roles, and linked outcomes visible. A credible product design agency can describe the artifact’s place in delivery without exposing confidential client information.
Include the people who will approve requirements, build the product, test releases, and support customers. Ask each person to review the evidence from their own work area before selection. Their questions often expose gaps that a procurement or executive review misses, especially around ownership and post-launch support.
Yes, if it records decisions at the point they matter. A small team can use a short acceptance checklist, one dependency register, component notes, and a staging review log. You need evidence of ownership and follow-through, not lengthy documents that nobody updates.
Tell us about your idea. 24-hour response. NDA-friendly.
