Kafka Engineer || Pleasanton, CA || Onsite
esolutionsPleasanton, CA
Kafka Engineer || Pleasanton, CA || Onsite
esolutionsPleasanton, CA
APPLY NOW (opens in a new tab)today
About the role
Kafka Engineer
Location: Pleasanton, CA (onsite)
Platform Design & BuildDesign and operationalize a multi-environment Confluent Cloud footprint (Prod/Non-prod) including RBAC, Service Accounts, API Keys, Private Networking (VPC peering/Private Link), IP allowlists, and encryption. Define topic architecture standards (naming, partitions, replication, retention, compaction), consumer group strategy, and exactly-once/at-least-once semantics. Implement Schema Registry and Stream Governance (compatibility modes, lifecycle, breaking change controls).Stand up connectivity via Confluent Cloud Connectors Establish Terraform/IaC modules for repeatable provisioning (environments, clusters, topics, ACLs, connectors, schemas).Security, Compliance & Reliability Partner with Security to implement least-privilege, RBAC, secrets management, and compliance (PII, PCI/SOX/GxP as applicable).Define and enforce data governance (ownership, contracts, lineage, retention, DLP).Build SLOs/SLAs, error budgets, and incident response runbooks; coordinate with SRE for DR, multi-zone resilience, and capacity planning.
Observability & Operations:
- Instrument metrics, logs, traces across Confluent Cloud and clients (lag, consumer health, throughput, latency, error rates, dead letter queues).Integrate platform telemetry with Datadog/Prometheus/Grafana/New Relic/Splunk ; set alerts and automated remediation where feasible.
- Manage upgrades, quotas, cost optimization, and platform hygiene (ACL audits, orphaned topics/connectors cleanup).Application Onboarding & Enablement
- Partner with app teams to design event contracts, idempotent producers, retry & DLQ patterns, and backpressure handling.
- Provide reference implementations and client libraries/configs (Java,.NET, Python, Node) with secure defaults and resiliency patterns.
- Run an " onboarding desk ": intake, discovery, architecture review, capacity estimates, provisioning via Terraform pipelines, and go-live checklists.
- Conduct training, documentation, and office hours ; champion best practices for event-driven architectures.
Matching similar jobs
JOB OVERVIEW
Experience level
Lead
Location
Pleasanton, CA
Occupation
Computer Systems Engineers/Architects
Industry
Computer Systems Design Services
Posted
today
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)