You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Intern, LLM Inference Acceleration Architecture

  • Office
  • 0-3 years

Salary

Not stated

Similar roles pay $100K - 170K a year · our estimate

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The team studies memory-centric system architectures for next-generation AI inference, where computation happens close to the data. Hardware and software are designed together to speed up large-scale LLM inference, and the results feed into future memory and accelerator products of the company.

In these accelerators, each memory device works on its own data and sends back only small partial results, which are then merged. As attention moves from grouped-query attention (GQA) to multi-head latent attention (MLA) and sparse top-k attention, the local work per device shrinks while cross-device merge and reduction take up more of decode latency. Using the team's in-house performance model and simulator, you will examine merge and reduction designs, including where the reduction occurs, how devices connect, and the precision of exchanged partial results, then propose an architecture that works well across modern attention workloads.

You will build skills in architecture-level performance modeling and design-space exploration for distributed memory-centric AI accelerators, in how GQA, MLA and sparse top-k attention map onto distributed memory and shift the balance between compute and communication, and in hardware-software co-design of interconnect and collective reduction hardware for LLM inference.

Responsibilities

  • Own a research study from start to finish: workload modeling, design-space exploration, and an architecture proposal, working alongside architects and performance-modeling engineers.
  • Present your findings to the wider organization at the end of the summer.
  • Add GQA, MLA and sparse top-k attention to the team's performance model and simulator for LLM decode.
  • Measure how each attention variant shifts work between local computation and cross-device merge/reduction traffic, and identify when merge becomes the bottleneck.
  • Survey reduction placement (centralized, in-network or hierarchical), interconnect topology, link bandwidth and latency, and partial-result precision.
  • Study token-selection patterns in sparse attention models, and design distributed top-k selection and data placement that cut communication and load imbalance.
  • Make first-order estimates of area and power for proposed reduction hardware, and weigh them against performance gains.
  • Check the numerical accuracy of reduced-precision merging against full-precision reference implementations.
  • Meet with your mentor and architecture engineers weekly, and share results through team meetings, a final presentation and a technical report that may inform future accelerator architecture and lead to publications or patents.
  • Take on other duties as assigned.

Requirements

  • Have at least 1 academic quarter or semester remaining.
  • Show a strong background in computer architecture, including memory systems, interconnects or networks-on-chip, and performance analysis.
  • Understand Transformer inference well, including attention variants (MHA, GQA, MLA), the KV cache and the decode phase.
  • Be proficient in Python and/or C++, with hands-on experience using or building architecture simulators or analytical performance models.
  • Preferred: pursuing a Master's or PhD in computer engineering, electrical engineering, computer science or a related field.
  • Preferred: familiarity with sparse attention (e.g., DeepSeek Sparse Attention, NSA, Quest), online-softmax merging as in FlashAttention or FlashDecoding, or in-network reduction.
  • Preferred: publications in venues such as ISCA, MICRO, HPCA, ASPLOS or MLSys.
  • Adapt your communication style to each situation and to the diverse global norms of the team.
  • Bring curiosity and resilience, and look for data to build understanding.
  • Build relationships, offer support humbly, and welcome other people's approaches.
  • Propose new ideas proactively and adapt quickly to change.

Conditions

Daily onsite presence at the San Jose headquarters, in line with the Flexible Work policy.

Reports to the Director.

Hourly base pay ranges from $28 to $68 USD for all roles at this level across all US locations and functions. Actual pay varies by work location and may depend on job-related knowledge, skills and experience.

The company is committed to an inclusive culture and a diverse workforce.

Benefits

  • Medical, dental and vision coverage, plus an inclusive rewards plan that supports your whole self.
  • Relocation stipend if you live 50+ miles from your new office.
  • Monthly housing stipend to help with a lease for your summer internship.
  • Charitable giving match and frequent volunteering opportunities.
  • On-demand wellness apps and paid therapy sessions.
  • Onsite café and gym, plus virtual classes.
  • A flexible environment that lets you find your own balance.

Where you’d work

From the office

Relocation offered

About the company

Company hidden

Office in San Jose, United States

Your chances

Still hiring, moderately crowded, and you'd be among the first.

  • 17 checks run
  • 4 good signs
  • 2 red flags

Still hiring?

11 checks

Actively hiring

In its favour2

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence

How crowded?

6 checks

Moderate

In its favour2

  • In its Early Window: not on the big job boards yetStrong evidence
  • In the office in San Jose: only people nearby can take itSlight evidence

Against it2

  • Entry level: the most applied-to level there isModerate evidence
  • Offers relocation: people from other countries apply tooSlight evidence
72 roles like this are in their Early WindowFound before the big job boards, while the crowd hasn’t arrived

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details12 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • C++
  • LLMs
Seniority
Intern
Type
Internship
Specialty
ML
Region
United States
Pay period
Annual
Show 6 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Seniority
Intern
Experience
0-3 years
Tech stack
  • Python
  • C++
  • LLMs

Location

Work model
Office
Region
United States
Office
  • San Jose, United States
Relocation
Offered

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Internship

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason