Software engineer
- Location
- San Francisco
- Type
- Full time
- Compensation
- $150k-$200k
- Posted
- 2 days ago
Join a team building useful software. Work closely with engineering and design to turn new ideas into products.
Find roles through people you trust, and message a verified founder or page admin about your interest.
The ERDM (Enterprise Reporting and Data Management) Business Analyst is responsible for ensuring that Sales & Finance AI solutions operate with the highest levels of trust, accuracy, control, and compliance.
We are seeking a detail-obsessed, data driven expert to serve as the guardian of accuracy for our Sales & Finance AI chat assistants. You will design validation frameworks, partner with data engineering teams, and ensure every metric the AI surfaces - especially Bookings and Revenue - is accurate, consistent, and trustworthy. You will lead incident response on any data accuracy or hallucination even
The role also serves as the bridge between Finance users and technical development teams. This role is responsible for translating business requirements into AI-enabled solutions and ensuring that Finance AI capabilities align with business goals and leadership needs.
Requirements & Solution Definition:
- Gather and document Finance business requirements through interviews, workshops and feedback; write clear user stories with testable acceptance criteria
- Ensure requirements reflect metric integrity standards and executive-readiness expectations
AI Quality Assurance:
- Develop and maintain AI testing frameworks: test scenarios, expected outputs, edge cases, pass/fail criteria
- Define quality standards for AI responses: accuracy, completeness, formatting, tone, contextual relevance
- Validate AI outputs against Snowflake data using SQL; confirm Bookings and Revenue match official Finance reporting
- Run UAT, regression and release testing for every feature, prompt update, data refresh or model change
- Coordinate business sign-off with Finance stakeholders before production release
Hallucination Prevention:
- Design validation tests for fabricated data, incorrect metric associations, misleading trends and unsupported narratives
- Catalog high-risk use cases: multi-step calculations, YoY comparisons, segment breakdowns, forecast vs. actuals
- Monitor response quality through periodic spot-checks and audits
- Escalate suspected hallucinations to the Data Accuracy Specialist (IC4) and team lead; work with IT/Engineering on fixes and guardrails
Executive Use Cases:
- Support design and testing of executive use cases and prompt frameworks for C-suite and SVP users
- Verify that curated queries, guided flows and personalized dashboards produce accurate, polished outputs
- Identify ways to make the executive experience more intuitive, actionable and insightful
Adoption & Change Management:
- Develop adoption strategies, from awareness through daily active use
- Lead user onboarding: guided walkthroughs, sandbox environments, office hours
- Deliver training (live demos, workshops, drop-in sessions) for analysts, managers and directors
- Manage communications: release notes, feature announcements, known issues, adoption progress reports
- Collect and synthesize user feedback through surveys, interviews and usage analytics
Required Qualifications:
Industry: Business Analyst, Database, Finance, Financial Analyst, SQL, Technology