Sr Principal

Responsibilities

• Build backend capabilities for observing and evaluating AI models, agents, and RAG applications before and during production.
• Design systems for testing, measuring, and benchmarking AI application accuracy, relevance, performance, and safety.
• Build scalable APIs and data-processing systems that contextualize AI telemetry and evaluation results.
• Shape the technical roadmap and architecture for AI Observability and Evaluation.
• Collaborate with product managers, engineers, and customers to deliver integrated features and developer workflows.
• Improve system observability, distributed-system performance, reliability, and cost efficiency.
• Mentor engineers and help maintain engineering standards.
• Own and execute projects from inception through delivery.

Requirements

• 12–15+ years of software engineering experience.
• Strong experience with one or more backend languages such as Go, Java, Python, or Rust.
• Experience building and operating highly available production systems.
• Experience with cloud infrastructure such as AWS, GCP, or Azure.
• Experience designing APIs using REST, gRPC, or GraphQL at scale.
• Experience working cross-functionally, operating autonomously, and using sound judgment.
• Experience owning projects from inception to delivery.
• Experience developing LLM-powered product features is desirable.
• Experience using AI coding assistants such as Cursor, Codex, or Claude is required.
• Effective collaboration and ability to work in a remote-first environment.

Benefits

• Remote role with a remote-first working environment and collaboration across geographies.
• Palo Alto Networks is committed to reasonable accommodations for qualified individuals with disabilities or special needs.
• Equal opportunity employment and consideration for legally protected characteristics.