- Job TypeFull Time
- Experience4 years
- Job FieldData Science / Research
The Data Integration Engineer is responsible for:
- Designing/optimization of KRA’s data warehousing and big data platforms.
- Leveraging on appropriate technologies to deliver robust processes for real-time/near real-time data ingestion, job automation and model deployments.
- Creating reproducible lean engineering processes for better memory and space management of data management solutions.
- Proactively building and implementing services, including end to end monitoring of data pipelines and platforms, scripting and automation of data lifecycle and quality processes.
- Problem resolution for recurrent incidents escalated by support teams around data analytics solutions.
- Co-ordination and supervision of assigned development teams.Development or enhancements to existing data services in line with procedures and standards.
- Identifying different sources of data and building a roadmap for real-time / near real-time data collection.
- Responsible for Data Integration into the Enterprise Data Warehouse and Big Data Platforms in projects, and supporting business teams in data quality automation.
- Responsible for planning, research, design and implementation of new data analytics platforms and technologies to address the organization’s analytics demands including big data platforms.
- Responsible for automating big data lifecycle management, big data storage systems, data security and data governance.
- Responsible for design and implementation of processes to ensure data reliability, efficiency, quality, and continuous improvement.
- Responsible for eliminating tool redundancy and ensuring timely data availability.
- Responsible of performing analytics infrastructure sizing based on requirements and design in projects.
- Responsible of creating data pipelines using both proprietary and emerging technologies (like Apache Nifi and Kafka among others).
- Identification of the correct analytics technology stacks to use as per project requirements.
- Technical responsibility for working with business in identification, development, piloting and scaling of ML and AI use cases.
- Working closely with BI support and application support teams to make sure that all the big data applications and pipelines are highly available and performing as expected.
- Reviewing design and architecture to guarantee service availability, performance and resilience.
- Reviewing application development tasks allocated to supervised staff to ensure that they are accomplished within the set requirements and that they meet highest standards of quality.
- Ensuring that solutions built comply with quality assurance (including fixing of functional and non-functional issues) and release guidelines; and have the requisite documentation.
- Planning for solution demos for delivered solutions/enhancements to get stakeholder feedback and for adoption.
- Reviewing analytics domain coding standards and recommends/implements improvements.
- Bachelor’s degree in Computer Science, Information Systems, Information Technology, Analytics or other related fields from a recognized university.
- Data Warehousing Solutions design, setup and optimization.
- Big Data Platforms design, setup and optimization.
- Structured and Unstructured Database Systems.
- ETL / ELT Jobs design, optimization and tooling.
- Machine Learning and AI an added advantage.
- Data Architecture and Design an added advantage.
Work experience required
- Four (4) years of hands-on experience of which one (1) should be at Supervisory level working with Java and experience in Python, R, SQL and Scala in the analytics field within a busy environment processing large and high velocity data sets.
- Proven experience in design, development and implementation of big data processing architectures and data ingestion techniques.
- Demonstrated experience in big data querying techniques and tools.
- Working experience in solutions development and delivery using agile frameworks will be an added advantage.
Functional and Technical Skills
- Hands-on experience in implementing, managing, monitoring and administering overall Hadoop infrastructures as well as development and monitoring of Hadoop jobs.
- Hands-on experience in structured and unstructured databases administration and development.
- Hands-on experience supporting installation and code deployments into Hadoop clusters
- Experience in sizing & capacity planning of data platforms as per data requirements.
- Hands-on experience monitoring and reporting on Hadoop resource utilization and troubleshooting.
- Hands-on experience in doing data backup and recovery tasks.
- Experience in data lifecycle management (including data retention and purging strategies)
- Experience in ETL, big data jobs, data streams processing and database performance optimization.
- Hands-on experience in installation and performance maintenance of Apache Nifi, Apache Kafka, Airflow and ML flow.
- Experience with ML/AI model deployments will be an added advantage
- DevOps and infrastructure automation experience (containerization technologies, Ansible, etc.) will be an added advantage.
- Working knowledge of Linux/Unix and Windows operating system platforms.
Behaviours and Competencies
- Demonstrated analytical, technical, and problem-solving skills, with high-levels of creativity and a practical approach that is self and principle-driven.
- Ability to balance the long-term and short-term implications of individual decisions and effective at driving short-term actions that are consistent with long-term goals.
- Ability to interact confidently with users to establish real problems and explain the solutions while prioritizing competing work commitments and delivering on time.
- Excellent written and verbal communications skills, able to distil complex technical concepts into simple terms, with strong persuasion skills to gain support for and establish principles, standards, and change.
- Excellent relationship building, teamwork, and collaboration skills that enables the provision of effective support and guidance across programs as well as negotiation.
- Vendor and technology neutral –driven primarily by long-term business outcomes rather than personal preferences.
- Be resilient, focused, results oriented and a team player.
All applications should be submitted online by 11th July 2022.