Cyber
Deloitte Cyber understands the unique challenges and opportunities businesses face in cybersecurity. Join our team to deliver powerful insights to help our clients navigate the ever-changing threat landscape. Through powerful insights and managed services that simplify complexity, we enable businesses to operate with resilience, grow with confidence, and proactively manage to secure achievements.
Position Summary
Level: Solution Advisor
As a Solution Advisor at Deloitte Consulting, you will ensure the quality, reliability, performance, and security of GenAI/LLM-enabled services running in public cloud (Azure/AWS). You’ll build automated tests for APIs and AI workflows (RAG, tool-calling), implement repeatable evaluation runs, and help the team ship safely via CI/CD quality gates.
Work you’ll do:
- Design and execute functional, integration, system, and regression testing for cloud-native microservices and public APIs.
- Build and maintain automation frameworks (Python + Pytest; Selenium/Playwright for UI where relevant).
- Validate LLM/RAG behaviors: relevance, grounding, consistency, hallucination checks, prompt/version change regression, citation/attribution tests.
- Create and maintain golden datasets (test prompts, expected outputs, labels/metadata) and run automated eval pipelines in CI and scheduled runs.
- Perform non-functional testing: latency/throughput, load/stress, resiliency/failure-mode tests (timeouts, retries, rate limits, idempotency).
- Execute security-aware testing for AI surfaces: prompt injection attempts, sensitive-data leakage checks, OWASP API risks, auth/RBAC validation.
- Work in an Agile/Scrum team; partner with engineers/architects to define acceptance criteria, SLAs/SLOs, and release quality gates.
- Use observability signals (logs/metrics/traces) to reproduce issues, triage failures, and drive fixes.
The Team:
Cyber Operate teams manage clients' critical cyber assets either as a fully managed service or in partnership with clients. They deliver skilled talent, cutting-edge technologies, and robust processes to operate client cyber capabilities. This includes managing the identity lifecycle, security operations, threat intelligence, application security, business transformation, and ensuring continuous compliance.
Qualifications:
Must have skills/ Project experience/ Certifications:
- 3–5 years in QA/SDET/test automation for distributed or cloud-native systems.
- Strong automation skills in Python (Pytest preferred); API testing proficiency (REST/GraphQL).
- Hands-on experience testing in Azure and/or AWS (deployments, networking basics, IAM concepts).
- Exposure to Docker and basic Kubernetes concepts (pods, services, logs).
- Solid Linux CLI skills; Bash scripting for test utilities and CI tasks.
- CI/CD experience: Azure DevOps, GitHub Actions, Jenkins, or Azure Pipelines.
- Working knowledge of performance testing tools (e.g., JMeter/k6/Locust; BreakingPoint/Spirent a plus).
- Understanding of GenAI concepts: prompting, RAG, embeddings/vector search basics, evaluation metrics.
- Strong debugging, communication, and documentation habits.
Good to have skills/ Project experience/ Certifications:
- Experience with Azure AI Search / Elastic / Pinecone / FAISS / Milvus (any vector retrieval).
- Familiarity with OpenTelemetry, Datadog/Grafana/Splunk/ELK.
- Exposure to NIST AI RMF / AI risk and safety testing patterns.
- Experience with test management (ADO Test Plans, TestRail, Zephyr/Xray) and Jira/ADO Boards.
- Certifications (nice-to-have): cloud (Azure/AWS), AI/ML, or quality engineering.
Education:
- Bachelor’s degree in (any of) Engineering, Information Technology, Statistics, Data Science, Applied Mathematics, Computer Science, Physics, or related fields.
Location:
- Bengaluru/ Hyderabad/ Pune/Chennai