Cyber
Deloitte Cyber understands the unique challenges and opportunities businesses face in cybersecurity. Join our team to deliver powerful insights to help our clients navigate the ever-changing threat landscape. Through powerful insights and managed services that simplify complexity, we enable businesses to operate with resilience, grow with confidence, and proactively manage to secure achievements.
Position Summary
Level: Lead Solution Advisor
As a Lead Solution Advisor at Deloitte Consulting, you will lead quality engineering for enterprise GenAI platforms and core AI services deployed on Azure/AWS. You’ll own the end-to-end test strategy, scalable automation and evaluation frameworks, performance/security validation, and CI/CD quality gates for distributed, multi-tenant AI services and APIs.
Work you’ll do:
- Define and drive QE strategy for GenAI services: test pyramid, coverage model, release gates, risk-based testing, and operational quality standards.
- Architect and implement automation at scale: API contract tests, integration/E2E workflows, data-driven regression, and hermetic test environments in containers.
- Test and operationalize LLM evaluation pipelines: scoring, drift/regression detection, rubric-based checks, RAG grounding/citation validation, multi-step agent/tool-call validation, and failure recovery testing.
- Own system-level and solution-scale testing across microservices, async workflows, and service-to-service integrations.
- Lead performance engineering: define SLAs/SLOs; run load/stress/soak tests; profile bottlenecks; validate caching, concurrency, and scaling; measure cost/latency trade-offs.
- Lead security validation: OWASP API Top 10, authn/authz/RBAC, rate limiting/abuse cases, prompt injection defense verification, sensitive data exfiltration checks, secure tool execution boundaries.
- Establish quality metrics and observability hooks with engineering (OpenTelemetry traces, eval metrics, dashboards), and drive incident learnings into regression suites.
- Mentor engineers/testers, raise automation standards, and influence architecture/design for testability and safety.
- Partner with product/engineering to define acceptance criteria, quality KPIs, and release management processes in Agile delivery.
The Team:
Cyber Operate teams manage clients' critical cyber assets either as a fully managed service or in partnership with clients. They deliver skilled talent, cutting-edge technologies, and robust processes to operate client cyber capabilities. This includes managing the identity lifecycle, security operations, threat intelligence, application security, business transformation, and ensuring continuous compliance.
Qualifications:
Must have skills/ Project experience/ Certifications:
- 5-7 years in QA/SDET/Quality Engineering with significant ownership of automation and release quality for distributed cloud services.
- Deep experience in public cloud environments (Azure and/or AWS) including networking concepts, IAM/security fundamentals, and cloud-native deployments.
- Strong programming and framework design in Python (Pytest); a plus depending on stack.
- Strong Linux/Bash skills and comfort debugging production-like environments.
- Strong experience with Kubernetes and containerized test infrastructure (Docker, Helm/manifests, cluster debugging).
- Proven experience with performance and scale testing (k6/Locust/JMeter; BreakingPoint/Spirent strongly preferred where applicable).
- Experience with CI/CD and DevOps toolchains (Azure DevOps/Azure Pipelines/GitHub Actions/Jenkins) and enforcing gates (coverage, reliability, eval thresholds).
- Proven experience testing AI-integrated systems: LLM regression, RAG validation, agent/tool-call workflows, safety/bias considerations, and evaluation methodologies.
- Strong understanding of API testing, contract validation, and non-functional requirements (resiliency, multi-tenancy, observability-driven debugging).
Good to have skills/ Project experience/ Certifications:
- Experience with Azure ML / SageMaker / GCP Vertex AI model lifecycle validation (deployment → monitoring).
- Familiarity with model governance/traceability.
- Experience integrating with Jira/ADO APIs, test management systems, and building quality dashboards.
- Familiarity with LangChain/LlamaIndex/Semantic Kernel, MCP-based tool integrations, or internal eval frameworks (e.g., Opik-like tooling).
- Cloud/AI certifications (Azure/AWS + AI/ML) strongly preferred.
Education:
- Bachelor’s degree in (any of) Engineering, Information Technology, Statistics, Data Science, Applied Mathematics, Computer Science, Physics, or related fields.
Location:
- Bengaluru/ Hyderabad/ Pune/Chennai