Why Your Data Isn't Changing Your Business Decisions

Introduction

Katrina Wong, Chief Marketing Officer at New Relic, has spent 25 years and seven exits leading marketing at companies including Twilio Segment, Salesforce, Zuora, and SAP — and she now markets to developers, an audience notoriously hostile to being marketed to. Her perspective on turning data into decisions is grounded in a hard reality: 78% of technology leaders report more production incidents after deploying AI-generated code, even though 93.5% say the code looks higher quality during review. Marketers face the same trap. More data, richer signals, and faster analysis don't automatically produce decisions you can trust.

The Three Stages of Data Maturity

Wong describes marketing data evolving through distinct stages, and most teams are still stuck in the first one. Stage one is mechanical — uploading a trade show list sounds boring until a failed upload costs you half your leads. Stage two is asking questions of the data on demand. At New Relic, that meant building Gemini Gems so sales reps could self-serve value pillar guidance instead of pinging product marketers on Slack before every call. Stage three is agents surfacing what you need before you ask.

Turning Qualitative Signals Into Quantitative Insight

The most immediately useful application Wong described is win/loss analysis at scale. The old approach — interviewing a handful of reps and customers each quarter — carries a sampling problem and human bias. Analyzing thousands of recorded sales calls with AI produces a different answer, and New Relic runs both in parallel. In one case, humans reported a pricing and packaging issue; the AI analysis showed it was actually usage of a specific feature. Triangulating the two gives marketing an unbiased layer it didn't previously have.

Automation Bias Is Your New Data Quality Problem

New Relic's research found only 62% of code is now being reviewed while production failures rise — evidence of what psychologists call automation bias. The same dynamic applies to marketing reporting. Wong's team replaced human pacing calculations in their Tableau-Snowflake-Salesforce forecasting stack with AI, but still fact-checks against the source of truth. Her position: when the number is revenue, a confidence score short of 100 isn't good enough, and human-in-the-loop stays.

Governance Is Now a CMO Responsibility

Wong caught New Relic's image library drifting off-brand because most graphics were AI-generated — a small error compounding invisibly. Her operating rule for the team is blunt: **"be a good neighbor. If you see something, say something."** She also flagged a governance problem most teams haven't hit yet: multiple agents writing to the same Salesforce field, with no human aware of the conflict. The CMO role has expanded from hiring capable people to policing output quality across agents.

Personalization at the Individual Level, Not the Persona

Recorded calls plus AI analysis let New Relic run ABM down to the individual rather than the account or persona. On a recent European retail deal, one stakeholder wanted industry strategy and AI i ovation context while others wanted a specific product capability validated. In a conference room, a human notetaker captures maybe half of that. Wong's framing is that better primary data — not faster content production — is what makes the follow-up collateral effective.

Original Research as the Developer Trust Strategy

Marketing to engineers means no jargon, no fluff, and moving from 101 to 301 fast. New Relic's answer is using its own observability data as a research asset. The company coined "agent debt" for the gap between AI-written code volume and rising production incidents — a catchy name substantiated by proprietary data. Industry data reports are among their highest-generating tactics, briefed to the customer advisory board rather than gated behind a form.

Conclusion

The unlock isn't more data — it's building the verification habits that make data trustworthy enough to act on. Fix the mechanical pipes first, run AI analysis alongside human interpretation rather than instead of it, assign explicit governance for agent output, and use your proprietary data to serve the community before you sell to it. As Wong puts it, the question is always **"what value are we truly providing to our customers in exchange for their time?"