Senior/Staff Applied Scientist

Description

The company builds a Work AI platform that helps employees find knowledge, use AI assistants and run AI agents across an organization's tools and data, with a focus on secure, open and customizable infrastructure.

The platform offers search, an AI assistant and agents on top of more than 100 enterprise SaaS connectors, with a choice of LLMs and public APIs, and is used by customers in over 50 industries.

The company has more than 1,000 employees in over 25 countries.

You would join the Data Organization, which brings together product data science, applied science, data engineering and business intelligence teams.

The product is used daily inside tools such as Microsoft Teams, Zoom, ServiceNow, Zendesk and GitHub.

Responsibilities

• Work with product data science and engineering teams to find better methods, tools and processes for online A/B experiments that support sound decisions in every product area.
• Build and maintain the company's A/B experimentation platform, shaped by feedback from stakeholders.
• Write code or evaluate vendors to put these methods into production at scale, in a form that engineering, product data science, product management and design teams can use easily.
• Run end-to-end evaluations of key use cases such as document and URL uploads, including building evaluation sets, defining criteria and choosing methods to interpret results.
• Split end-to-end evaluations into finer tests of individual tasks and skills, such as summarization, analysis and generation, multi-step reasoning and planning, tool selection and use, and coding and system design.
• Create and own best practices, tools and processes for evaluation problems such as query intent classification, applying sound statistics to LLM randomness, and industry benchmarking.

Requirements

• You have 5+ years of experience with a Master's degree, or 3+ years with a PhD, in statistics, mathematics, computer science or another quantitative field.
• You have strong skills in statistics and/or machine learning, and you have used them to make practical product, tooling and process improvements that serve business priorities.
• You are highly proficient in Python, enough to maintain a source-controlled internal library that many colleagues depend on.
• You communicate concisely and precisely, both in writing and out loud, and you write clear technical documentation.
• You are proficient in SQL and the modern data stack, including source-controlled dbt pipelines for ETL and ELT.
• You are skilled at defining product KPIs and guardrail metrics, building dashboards, and analyzing raw data to find strategic insights.
• You have experience in B2B SaaS.
• You have worked on ranking, on building and maintaining A/B experimentation platforms, or on ML measurement problems, or on several of these.
• You care about using AI to make data teams more productive and to help non-data professionals get more value from their company's data.

Conditions

This is a hybrid role, with 4 days a week in the Mountain View or San Francisco office.

The standard base salary range is $180,000 to $330,000 per year. Final pay depends on location, level, job-related knowledge, skills and experience. Some roles may also be eligible for variable compensation, equity and benefits.

The interview includes a short AI-focused exercise or discussion about how you think about, design and use AI in your work. You may refer to any tools, platforms or workflows you already use, and prior experience with the company is not required.

The company is committed to an inclusive, diverse workplace and does not discriminate on the basis of gender, ethnicity, sexual orientation, religion, civil or family status, age, disability or race.

Benefits

• Medical, vision and dental coverage.
• Generous time off.
• A 401k plan, with the option to contribute toward long-term savings.
• A home office improvement stipend when you join, plus annual education and wellness stipends.
• Regular company events and healthy lunches provided daily.
• Competitive compensation.