Kafka, Redis And Hadoop Developer

Key Responsibilities: Design, develop, and maintain real-time data streaming applications using Apache Kafka, including KStreams and KTable. Develop scalable big data processing solutions using Hadoop and PySpark. Implement and optimize data storage and retrieval using NoSQL databases such as Redis, MongoDB, and HBase. Develop data processing and integration pipelines using Python, Scala, Java, and Bash/Shell scripting. Build reliable distributed systems with efficient data partitioning, fault tolerance, and high availability. Work with streaming technologies such as Apache Flink and Apache Storm. Leverage cloud platforms such as AWS, Azure, or GCP for big data processing and deployment. Apply data security and integration best practices, including Kerberos authentication. Perform data analysis and support predictive modeling using statistical techniques and machine learning algorithms. Develop and maintain visual analytics and reporting solutions using Tableau. Troubleshoot performance issues, optimize data pipelines, and ensure data quality and reliability. Collaborate with cross-functional teams to deliver business-critical data solutions.

Required Skills and Qualifications: Bachelor's or Master's degree in Science, Engineering, Computer Science, or a related field. 9–12 years of overall professional experience in software development and data engineering. Strong hands-on experience with Apache Kafka, Redis, Hadoop, and PySpark. Proficiency in Python and familiarity with Scala, Java, and Bash/Shell scripting. Strong understanding of real-time streaming, distributed computing, and big data architecture. Experience with NoSQL databases and cloud-based data processing platforms. Knowledge of data security, data integration, and visualization tools such as Tableau. Excellent problem-solving, analytical, and communication skills.