Senior Data Engineer

Devsinc is looking for a highly skilled Senior Data Engineer with 4–6 years of professional experience to design, build, and maintain scalable data pipelines and processing systems that support analytics, data products, and AI/ML capabilities. The ideal candidate will have strong hands-on experience with ETL/ELT pipelines, Python, SQL, Apache Airflow, Apache Spark, and Redis , along with the ability to develop reliable and maintainable workflows for large, complex, and continuously growing datasets. You will collaborate with Product, Business Intelligence, Data Science, and Engineering teams to deliver production-ready datasets and high-performance data solutions. Responsibilities • Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, validation, and delivery. • Build, schedule, and orchestrate production-grade data workflows using Apache Airflow . • Develop distributed data-processing jobs using Apache Spark . • Write efficient, reusable, and maintainable Python code for data processing, automation, and pipeline development. • Develop and optimize complex SQL queries for data transformation, analysis, and quality validation. • Design pipelines capable of processing large-scale structured, semi-structured, and unstructured datasets. • Implement Redis for caching, high-performance data access, and data-intensive application requirements. • Integrate data from APIs, relational databases, files, third-party providers, and other internal and external sources. • Implement data validation, monitoring, logging, error handling, alerting, and pipeline observability. • Optimize pipeline performance, data storage, processing time, and infrastructure costs. • Develop reusable data-ingestion and transformation frameworks instead of one-off scripts. • Troubleshoot pipeline failures, performance bottlenecks, and data-quality issues to ensure timely resolution. • Collaborate with BI, Product, Data Science, and Engineering teams to deliver reliable, production-ready datasets. • Establish and maintain data-engineering standards, technical documentation, and development best practices.