SENIOR DATA OPERATIONS ENGINEER*
THIS IS US
The Data Science department uses statistical modeling, machine learning and optimization techniques to continuously improve business processes at NEW YORKER. We work in agile project teams on a variety of use cases, including order optimization, pricing, distribution and logistics. We are a very ambitious and international team and are proud of our high standards.
THAT'S THE JOB
- Work closely with the data engineering and data platform teams to maintain, operate and improve the infrastructure and pipelines that run critical data science projects on our on-premise Cloudera cluster
- Monitor and operate (including on-call duty) key data science pipelines (that rely on Spark)
- Monitor and analyze runtime behavior, including runtimes, resource usage and distributed communication error rates across all data science pipelines
- Maintain operating systems on our Cloudera cluster
- Monitor and maintain our cluster and promptly react to errors and outages
- Install and maintain the software and development environments required by the data science team on our Cloudera cluster
- Support the network and infrastructure team in optimizing network boundaries between various systems used by data science
- Liaise with our suppliers and network and datacenter services team to ensure hardware issues are fixed promptly
- Initiate, align, implement and enforce a logging and monitoring strategy to make data science project runtime characteristics transparent
- Define, align and enforce proper HDFS usage guidelines, including maintaining HDFS via appropriate retention periods and monitoring average HDFS block size count and over-partitioning in pipelines
- Monitor and manage Impala databases and tables
- Mentor team members, accept mentoring by team members, constantly align your work with your users, exchange constant feedback with your peers through insightful code reviews and grow together as a team
THAT'S WHAT WE NEED
- A Master's degree in computer science or equivalent
- The willingness and ability to work in a team with strong communication and hands-on mentality
- The ability to think and work in a well-organized, structured, independent and self-driven manner while aligning proactively with your team
- A strong software development background and strong programming skills
- Five or more years of working experience as a DevOps engineer or in a similar position
- Extensive experience working with Hadoop clusters and with data storage (e.g. HDFS)
- Extensive knowledge in information systems theory (e.g. relational databases, distributed systems) and working with data (e.g. modeling)
- The ability to write efficient, well-tested code with a keen eye on scalability and maintainability in Python and bash
- Experience in software development practices and tools (e.g. Kubernetes, pull request reviews, CI/CD, agile as in agile manifesto not Scrum)
- Good Linux knowledge
- Speaking English fluently is a must-have, good spoken and written German is a plus
THAT SPEAKS FOR US
- A secure, permanent job in an internationally operating company in a future-proof industry that is characterized by growth
- Participation in shaping the digital transformation in an innovative company
- Permanent employment contract with 30 days of vacation, an attractive salary and further training opportunities/career model
- A structured onboarding and mentoring program in a great team with international colleagues, a “Duz Kultur” and open doors
- Numerous benefits, such as a 30% staff discount at our NEW YORKER stores, company and team events, and canteens with a diverse food selection
GET IN CONTACT
NEW YORKER Group-Services International GmbH & Co. KG
Contact: Aayush Sarma | Recruiting Specialist
P +49 531 2135 - 5421
www.newyorker.de/Jobs
Redaktionell empfohlener externer Inhalt
