Middle Big Data Engineer
We are looking for a Middle Big Data Engineer to team up with Sigma Software and apply the best engineering practices in daily work. We expect our ideal candidate be flexible and motivated to work in a fast-paced environment.
Our Customer, Oath, is one of the world's biggest advertising media conglomerates, owner of one of the largest advertising and video platforms with millions of users and billions of video plays. Can you imagine that a system operates petabytes of data with more than 140 billion events daily and about 2M events per second? The system works with near real time processing, where delay is around 1.5 mins. We surely can!
Here is a perfect chance to work with a subteam that is responsible for Oath’s DWH covering data injection from multiple sources like video views, ad impressions, viewability events, etc., efficient data organization and analytics based on that.
- Performing data analysis
- Designing, reviewing, implementing, testing, and optimizing ETL processes
- Designing and developing big data applications from scratch
- Providing development support for existing systems
- Troubleshooting data and/or system issues
- Creating system documentation
- At least 3 years of development experience in Java/Scala
- Work experience with Spark
- Strong knowledge of the Hadoop Ecosystem
- Experience in Data Warehousing technologies and back-end reporting systems
- Strong scripting skills to perform data/file manipulation
- Experience with automated testing practices
- Fluent English
Nice to have:
- Experience with RDBMS solution