Senior Big Data Engineer/Team Lead
We are looking for a Senior Big Data Engineer/Team Lead to team up with Sigma Software and apply the best engineering practices in daily work. We expect our ideal candidate be flexible and motivated to work in a fast-paced environment.
Our Customer, Oath, is one of the world's biggest advertising media conglomerates, owner of one of the largest advertising and video platforms with millions of users and billions of video plays.
Can you imagine that a system operates petabytes of data with more than 140 billion events daily and about 2M events per second? The system works with near real time processing, where delay is around 1.5 mins. We surely can! We welcome you to bring ideas on how to make this system even better.
Here is a perfect chance to work with a subteam that is responsible for Oath’s DWH, covering data injection from multiple sources, such as video views, ad impressions, viewability events etc., as well as efficient data organization and analytics based on that.
As a Senior Big Data Engineer you will:
- Lead a distributed team (7 locations around the world)
- Design, review, implement, test, and optimize ETL (batch and real-time) processes
- Design and develop big data applications from scratch
- Provide development support for existing systems
- Troubleshoot data and/or system issues
Qualifications you must have to join us:
- At least 3 years of development experience in Scala and Java
- Strong experience with Spark and its Streaming from Kafka
- Strong knowledge of the Hadoop Ecosystem
- Experience in Data Warehousing/RDBMS technologies and back-end reporting systems
- Strong scripting skills to perform data/file manipulation
- Experience with automated testing practices
- At least 1 year experience in team leading
- Fluent English
Will be a plus:
- Experience with microservices development