Big Data Engineer
-
Location
Unknown
-
Sector:
-
Job type:
-
Salary:
up to £650 per day
-
Published:
almost 4 years ago
-
Expiry date:
2018-08-06
Working for the world leading online travel company
We are looking for a highly motivated, disciplined big data engineer with proven understanding & experience on big data technologies such as Hadoop, Spark, Hive, Teradata.
You will have a real passion for innovation and an appetite to challenge existing thinking. You will be a great communicator equally comfortable with discussing and explaining technical problems and solutions with analysts & engineers.
Role Description
• Set up & maintaining of Big data analytics applications in cloud such as EMR, RedShift, Qubole, R Studio and more.
• Developing tools to automate usage monitoring, system health, executing upgrades and building multi-region cloud solutions for these applications.
• Working with vendors to fix platform performance and stability issues.
• Help users with training & best practices to write efficient queries & processes to analyze the data in the most efficient & performant manner
• Evaluate new tools & technologies in cloud.
• Collaborate in program & system architecture discussions to identify a suitable quality solution to the business requirement.
• Implement suitable architecture to provide maximum efficiency and cost optimization.
Experience
• Strong knowledge & experience of configuring, optimizing & supporting production Hadoop platforms
• Deep understanding & expertise in writing and optimizing Hive & Spark queries and jobs
• Application support experience
• Experience in Java, SQL, R & Scripting (Python or Shell)
• Experience with building monitoring around tools availability, usage & performance
• Experience working with Cloud is desirable; AWS, GCP
• Automation tooling such as Terraform would be beneficial.
We are looking for a highly motivated, disciplined big data engineer with proven understanding & experience on big data technologies such as Hadoop, Spark, Hive, Teradata.
You will have a real passion for innovation and an appetite to challenge existing thinking. You will be a great communicator equally comfortable with discussing and explaining technical problems and solutions with analysts & engineers.
Role Description
• Set up & maintaining of Big data analytics applications in cloud such as EMR, RedShift, Qubole, R Studio and more.
• Developing tools to automate usage monitoring, system health, executing upgrades and building multi-region cloud solutions for these applications.
• Working with vendors to fix platform performance and stability issues.
• Help users with training & best practices to write efficient queries & processes to analyze the data in the most efficient & performant manner
• Evaluate new tools & technologies in cloud.
• Collaborate in program & system architecture discussions to identify a suitable quality solution to the business requirement.
• Implement suitable architecture to provide maximum efficiency and cost optimization.
Experience
• Strong knowledge & experience of configuring, optimizing & supporting production Hadoop platforms
• Deep understanding & expertise in writing and optimizing Hive & Spark queries and jobs
• Application support experience
• Experience in Java, SQL, R & Scripting (Python or Shell)
• Experience with building monitoring around tools availability, usage & performance
• Experience working with Cloud is desirable; AWS, GCP
• Automation tooling such as Terraform would be beneficial.