You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Collinear.ai

MTS - Engineering (Data Infrastructure)

  • Office
  • 3-6 years

Salary

Not stated

Similar roles pay $80K - 165K a year · our estimate

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company builds the systems that turn very large, messy datasets into material it can build on. The data is often terabytes in size and comes from archives, spreadsheets, email, PDFs, and scanned documents.

How quickly this data gets processed sets how quickly the company can deliver to frontier labs.

This is a hands-on role that sits between algorithms, distributed systems, and data quality. Some of the hardest problems, like connecting related records across millions of files, resist simple splitting, and you will set the approach for solving them at scale.

Responsibilities

  • Create parallel pipelines that handle multi-terabyte collections of archives, spreadsheets, email, PDFs, and scanned documents
  • Design graph-based systems that connect records for the same person or company across millions of files
  • Build fast search for strings and patterns across large, varied datasets
  • Find and remove performance bottlenecks, and decide how to divide work that does not parallelize cleanly
  • Transform data for later use, including replacing sensitive fields consistently across all files
  • Set the metrics for data quality and build review tools so the team can find and correct errors without reprocessing everything
  • Evaluate new datasets and remove low-quality data before it enters the environments

Requirements

  • You have 5+ years of experience building data-intensive systems that run in production
  • You have processed large datasets in parallel using tools such as Apache Spark, Ray, or Dask, and you know when a custom approach is the better choice
  • You have a deep understanding of graph algorithms and have applied them to transform large volumes of data
  • You have built efficient string matching and pattern search at scale, including fuzzy matching or indexing
  • You balance accuracy, speed, and cost well and can explain those tradeoffs clearly
  • Experience with entity resolution or record linkage
  • Experience with NLP or LLM-based information extraction
  • Experience with OCR or document processing, including poor-quality scans and handwriting
  • Experience with a systems language such as Rust, C++, or Go
  • Experience working with regulated or sensitive data, such as financial or healthcare records

Where you’d work

From the office

About the company

Collinear.ai

  • Industry: AI

Offices in San Francisco, United States, Sunnyvale, United States

Also hiring in United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 19 checks run
  • 3 red flags

Still hiring?

13 checks

2 red flags

How crowded?

6 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Spark
  • LLMs
  • NLP
Type
Full-time
Industry
AI
Specialty
Data Engineering
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Data & Analytics
Specialty
Data Engineering
Experience
5+ years
Tech stack
  • Spark
  • LLMs
  • NLP

Location

Work model
Office
Region
United States
Offices
  • San Francisco, United States
  • Sunnyvale, United States

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason