Data Engineer
Job Description
<strong><u>Responsibilities</u></strong><p><br></p><ul><li>Design, build, and maintain scalable and reliable data pipelines using Azure Data Factory, Azure Functions, and other Azure services.</li><li>Implement data storage solutions using Azure SQL Database, Azure Data Lake Storage, and Azure Cosmos DB.</li><li>Design and implement Modern Lakehouse Data Warehouse using Delta Lake and Spark, ensuring scalability and reliability of data pipelines and optimal data storage and retrieval performance.</li><li>Develop data processing and transformation solutions using Azure Databricks, Azure Stream Analytics, and Azure Data Explorer.</li><li>Design and maintain data warehouse solutions, ensuring optimal data storage and retrieval performance.</li><li>Ensure data quality, integrity, and compliance with data governance policies by implementing data validation, monitoring, and error-handling procedures.</li><li>Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver data solutions that meet business needs.</li><li>Create and maintain visualizations and interactive dashboards using Power BI to provide actionable insights for business users.</li><li>Optimize and fine-tune data processing and storage solutions for performance, cost-efficiency, and maintainability.</li><li>Develop and maintain technical documentation related to data engineering solutions, processes, and best practices.</li><li>Stay up-to-date with the latest Azure data engineering technologies, trends, and best practices to continuously improve data solutions and processes.</li></ul><p><br></p><strong><u>Requirements</u></strong><p><br></p><ul><li>Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.</li><li>Minimum of 3 years of data engineering experience, focusing on the Microsoft Azure ecosystem.</li><li>Experience in writing and optimizing complex queries using T-SQL</li><li>Strong knowledge of Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, and other Azure data services.</li><li>Proficiency in Python for data processing and manipulation, with experience in data warehousing concepts and technologies.</li><li>Experience with ETL/ELT processes, data modeling, and data integration best practices.</li><li>Power BI or similar data visualization tools for creating interactive dashboards and reports is a PLUS.</li><li>Strong analytical and problem-solving skills, with an ability to quickly learn and adapt to new technologies.</li><li>Excellent communication and collaboration skills, with a demonstrated ability to work effectively in cross-functional teams.</li></ul>
