The data engineer will be the key interface in operationalizing data and analytics on behalf of the business unit(s) and organizational outcomes. This role will require both creative and collaborative working with IT and the wider business. It will involve evangelizing effective data management practices and promoting a better understanding of data and analytics. The data engineer will also be tasked with working with key business stakeholders, IT experts, and subject-matter experts to plan and deliver optimal analytics and data science solutions.
Additionally, data engineers will also be expected to collaborate with data scientists, data analysts, and other data consumers and work on the models and algorithms developed by them in order to optimize them for data quality, security, and governance and put them into production leading to potentially large productivity gains.
- Build Data pipelines
- Collaborate across departments
- Educate and train
- Participate in ensuring compliance and governance during data use
- Become a data and analytics evangelist
- Drive Automation through effective metadata management.
This will include (but not be limited to):
- Learning and using modern data preparation, integration, and AI-enabled metadata management tools and techniques.
- Tracking data consumption patterns.
- Performing intelligent sampling and caching.
- Monitoring schema changes.
- Recommending — or sometimes even automating — existing and future integration flows