I am currently recruiting for an experienced bid data, Data Engineer to be responsible for setting up, expanding and optimising the data and data pipeline architecture for my client. We are looking for someone who can build data pipelines from the ground up, has extensive data manipulation skills and essentially lives and breathes data!
My client are a global technology company with offices based in Dublin, this is a permanent position paying a competitive salary and benefits.
Data Engineer - responsibilities will include:
- Creating and maintaining data pipelines
- Assembling large / complex data sets to meet functional and non-functional business requirements
- Identify, design and implement process improvements, with ongoing automisation and optimisation
- Work with key stakeholders
- Keep data separated and secure with responsibilities of understanding and carrying out processes which ensure best practice and GDPR, and other data legal requirements
Skills & Experience Required:
- Skilled in Big Data technologies such as - Azure, Hortonworks, Kylo, Spark, Scoop, NoSQL
- Proven experience in using advanced SQL to deliver value
- Experience of building and optimising big data pipelines, architectures and data sets
- Requirements capturing
- Experience in automisation
- Optimisation of extraction, transformation and loading of data from multiple data sources
- Excellent interpersonal skills, must be able to liaise with multiple stakeholders
- A successful history of manipulating, processing and extracting value from large disconnected datasets
Experience of the following software/tools is highly desirable:
- Hadoop, Spark, Kafka etc
- SQL, NoSQL, MySQL
- Data pipeline and workflow management tools - Kylo, Talend Azkaban, Luigi, Airflow etc
- Experience with Azure or other cloud services
- Stream processing systems - Storm, Spark-Streaming etc.
- Experience with object orientated/function scripting languages - Python, Java, C++, Scala etc.
- Experience with Jupyter
Must be self-motivated, highly organised individual, able to build strong relationships with stakeholders, ability to clearly document information with excellent communication skills
Applicants must be eligible to live and work in Ireland without sponsorship.
If you are interested in this permanent opportunity please submit your application on line or email direct to Salome.Maybanks @ projectpeople.com
Project People is acting as an Employment Agency in relation to this vacancy.