Senior Big Data Platforms Engineer
Job Description
<p><strong><u>Job Summary:</u></strong></p><p>The Senior Data Platform Engineer takes on the responsibility for data modeling and administration of a distributed in-memory database cluster based on Apache Ignite, a set of “SQL Server” and “SQL Server Integration Services” servers, and a set of Windows web servers. The Senior Data Platform Engineer will work closely with a team of other data engineers who are responsible for Extract, Transform and Load (ETL) and write their extracted data to the Ignite database. The Senior Data Platform Engineer will also collaborate with a team of full-stack web developers to build data-driven applications.</p><p>The ideal candidate will have hands-on experience in Linux and Windows server administration and in implementing distributed systems and highly available database clusters. The candidate should have expertise in Apache Ignite distributed database, or other distributed database technologies such as Apache Cassandra, Redis, etc., or Big Data technologies such as HBase, Impala, Google Bigtable, etc. The candidate should also have hands-on experience in SQL Server and SSIS administration.</p><p><br></p><p><strong><u>Job Responsbilities:</u></strong></p><ul><li>Design and implement solutions for regular and distributed databases.</li><li>Design, develop and maintain data platform server architecture, including partitions, backups, indexes, and caches.</li><li>Plan, monitor, and troubleshoot the data platform servers to ensure availability, reliability, and performance.</li><li>Optimize and maintain the data model to improve performance, including scaling, tuning, indexing, and block size adjustments.</li><li>Collaborate with stakeholders to analyze data requirements and design, develop, and maintain efficient data pipelines.</li><li>Ensure data governance and security by following best practices and implementing policies.</li><li>Develop and maintain technical documentation and reports that describe the performance of the database clusters.</li><li>Work alongside the data engineers responsible for ETL to ensure data is delivered efficiently to the target database.</li><li>Collaborate with the web developers to design and develop high-performance data-driven applications.</li><li>Adhere to technology best practices.</li><li>Mentor junior members in the team.</li></ul><p><br></p><p><strong><u>Job Requirements:</u></strong></p><ul><li>Bachelor’s degree in Computer Engineering, Computer Science, Information Systems, or a related field from a reputable university.</li><li>Minimum 3 years of relevant experience.</li><li>Strong experience in SQL, database technologies (e.g., relational databases, NoSQL) and data modeling.</li><li>Strong experience in distributed in-memory databases and can tune to extract optimal performance.</li><li>Hands-on experience in Linux administration and shell scripting.</li><li>Hands-on experience in SQL Server/SSIS administration</li><li>Understanding of distributed cache theory and practice.</li><li>Hands-on experience with at least one of: Ignite, Cassandra, Redis, HBase, Impala, Bigtable, or a similar technology.</li><li>Proven experience in high availability and disaster recovery technologies.</li><li>Expertise in performance tuning, memory management, and distributed computing concepts.</li><li>Hands-on experience in implementing Apache Ignite is highly preferred.</li><li>Programming skills in languages such as Python, Java, Scala or .NET is preferred.</li><li>Familiarity with CI/CD tools and DevOps, DataOps, or MLOps is preferred.</li><li>Familiarity of containerization tools and frameworks (Docker, Kubernetes, etc.) is preferred.</li><li>Strong analytical and problem-solving skills, with the ability to analyze complex data-related issues and propose effective solutions.</li><li>Excellent troubleshooting and debugging skills.</li><li>Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.</li><li>Attention to detail and a strong commitment to delivering high-quality work within established timelines.</li><li>Ability to work independently and manage workload effectively.</li><li>Strong communication and leadership skills to collaborate with stakeholders across different departments.</li><li>Ability to multitask and prioritize workflow.</li><li>Passion for learning new technologies and methodologies.</li><li>A team player with a positive attitude and ability to work well in a fast-paced and dynamic environment.</li></ul><p><br></p>
