(USA) Principal, Software Engineer

Description

The Private Cloud Storage Engineering team runs some of the biggest Ceph storage clusters anywhere, spread across several sites worldwide.

The team supports critical business operations, customer-facing platforms and innovation projects.

The work sits where distributed storage, Linux internals, networking and cloud infrastructure meet, with a focus on scale, speed and recovery from failure.

You will work alongside computer, networking, security and cloud engineers, and the team values automation and observability.

Responsibilities

• As a hands-on technical leader, you will help architect and run some of the largest Ceph clusters in the industry, holding hundreds of petabytes of critical data.
• Drive the design, scale, performance tuning and automation of distributed storage across several global production environments.
• Work with computer, networking, security and cloud teams to deliver highly available storage that meets strict latency, throughput, resiliency and scalability SLAs.
• Shape technical direction, mentor engineers and take part in open-source storage communities.
• Architect, deploy and manage large Ceph clusters in global production, using CRUSH map design, erasure coding and replication to keep data available, durable and scalable.
• Design, build and maintain Ceph CSI and Cinder CSI drivers that give containerized workloads on the company's Cloud Native Platform persistent, resilient, high-performance storage.
• Find and fix performance problems across the storage, Linux and networking layers using perf, blktrace, bpftrace and tcpdump.
• Create automation for provisioning, lifecycle management, capacity growth, monitoring and daily operations with Ansible, Terraform and Python.
• Run root cause analysis on major production incidents, coach engineers on storage architecture and good practice, and contribute to the Ceph and Kubernetes CSI open-source projects.

Requirements

• 15+ years in distributed storage and Linux infrastructure, with 10+ years of hands-on experience running large Ceph environments at petabyte scale.
• Extensive experience building, improving and supporting Ceph CSI and Cinder CSI drivers for persistent storage in cloud-native platforms.
• Deep understanding of Linux internals, kernel tuning, storage architecture and networking, including TCP/IP, BGP, RDMA, BlueStore, RocksDB, OSD architecture and erasure coding.
• Proven skill diagnosing and tuning performance with perf, blktrace, tcpdump, bpftrace and eBPF, plus infrastructure automation in Python, Ansible and Terraform.
• Hands-on experience with Kubernetes, OpenStack (Cinder and Swift) and public clouds, including AWS, Azure and GCP.
• Minimum qualifications, meet one option. Option 1: a bachelor's degree in computer science, computer engineering, computer information systems, software engineering or a related area, plus 5 years of software engineering or related experience. Option 2: 7 years of software engineering or related experience.
• Preferred: a master's degree in computer science, computer engineering, computer information systems, software engineering or a related area, plus 3 years of software engineering or related experience.
• Preferred: knowledge of inclusive digital design, WCAG 2.2 AA, assistive technologies and accessibility best practices, applied to the company's accessibility standards.

Conditions

The annual salary range is $143,000.00 to $286,000.00.

Annual or quarterly performance bonuses are included in compensation.

Stock may be included for certain positions.

Primary location: 1345 Crossman Ave, Sunnyvale, CA 94089-1114, United States of America.

The company has zero tolerance for illegal drugs and alcohol on the job, aiming for a safe and productive workplace.

Benefits

• Competitive pay plus performance-based bonuses.
• Health coverage: medical, vision and dental.
• Financial benefits: 401(k), stock purchase and company-paid life insurance.
• Time off: PTO including sick leave, parental leave, family care leave, bereavement, jury duty and voting.
• Also short-term and long-term disability, company discounts, Military Leave Pay, and reimbursement for adoption and surrogacy expenses, among others.
• PTO and/or PPTO can be used for vacation, sick leave, holidays or other purposes; the amount depends on job classification and length of employment and meets or exceeds paid sick leave laws where they apply.
• Company-paid education for full-time and part-time associates in company and Sam's Club facilities, from high school completion to bachelor's degrees, including English language learning and short certificates, with tuition, books and fees fully covered.
• Some benefits have eligibility rules that depend on job classification and length of employment; benefits can change and may follow plan terms.