1+ months

Staff Data Engineer

Walmart
Hoboken, NJ 07030
1388184BRReq ID:1388184BRCompany Summary:The Walmart eCommerce team is rapidly innovating to evolve and define the future state of shopping. As the worlds largest retailer, we are on a mission to help people save money and live better. With the help of some of the brightest minds in technology, merchandising, marketing, supply chain, talent and more, we are reimagining the intersection of digital and physical shopping to help achieve that mission.Job Title:Staff Data EngineerPosition Summary:The Data Platforms team at JET is looking for an exceptionally talented Data engineer with an outstanding track record reflecting achievements and skills in modernizing and improving the data infrastructure from the ground up. So, if you are passionate about working with very Large Data sets (structured/unstructured), building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges and building robust ETL pipelines then we would love to talk to you. Jet Data Environment Jets Event driven Ecommerce Microservices generate massive volume of data every second. Our Data Platform ingests and organizes this and other third party high speed, high volume data and enables strategic and operational decisions and reporting by Analysts and Data Scientist in every domain of Jet Ecommerce business. Our Data Platform supports Strategic and Operational Analytics and leverages Azure hosted Big Data clusters as well as a constellation of Big Microsoft SQL Server instances. Our platform leverages Spark, Hive, Kafka, Redis, Elastic Search, Cosmos DB and stores several petabytes of data.City:HOBOKENState:NJPosition Description: Play a pivotal design and hands on implementation role in improving the Data infrastructure in a project-oriented work environment. Influence cross functional architecture in sprint planning Gather and process raw data at scale - collect data across all business domains (our functional-first, event sourced, micro services backend) and expose mechanisms for large scale parallel processing Design, implement and manage a near real-time ingestion pipeline into a data warehouse and Hadoop data lake. Process unstructured data into a form suitable for analysis and then empower state-of-the-art analysis for analysts, scientists, and APIs Build efficient new Data Models and refactor existing ones. Partner with business to build right data models and analytics capabilities. Solve complex SQL and Big Data Performance challenges. Mitigate Risks in our data infrastructure by developing the best in class tools and processes. Implement controls, policies, processes and best practices in the Data Engineering space. Evangelize an extremely high standard of code quality, system reliability, and performance. Help us improve our database deployment and change management process. Provide reliable and efficient Data services as part of the database team. Work closely with the devs on development best practices and standards. Be a mentor.Minimum Qualifications: Degree in Computer Science, Information Technology, Math or related technical field preferred. Natural inclination for designing well thought out data solutions as well as solid hands-on implementation capability 4+ years experience in engineering data solutions using technologies including Spark (batch/streaming), Scala/Java (building data pipelines vs frameworks), Hadoop, Hive, HBase, Kafka, Spark, Oozie, Yarn. Solid hands-on experience in building data pipelines, deploying and managing Big Data infrastructure, establishing deployment and operational excellence of Big Data clusters. Experience in Relational databases, Data Warehousing, SQL, ETL and/or NOSQL databases will be sweet. Proven experience in Building or improving large scale data infrastructure from the ground up. This could include building Data warehousing stores, Data Architecture, Data Integration, building highly performant data movement pipelines, building tools and automation to facilitate Data and operational governance, Data Lineage, Automation and Monitoring. Proven Performance Tuning Accomplishments and Advanced tuning and trouble shooting skills in Large data environments on the Hadoop or RDBMS stack. Advanced knowledge of internals of at least one RDBMS (MSSQL preferred) and best practices. Data Ops experience including hands on skills in scripting language such as Python, Perl or Bash Proven experience in trouble shooting, problem determination and rapid problem resolution Experience and ability to work under high pressure in a complex technical environment.Category:Data Science / Machine Learning Brand:Walmart Labs

Categories

Posted: 2019-11-07 Expires: 2020-02-08

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

Staff Data Engineer

Walmart
Hoboken, NJ 07030

Join us to start saving your Favorite Jobs!

Sign In Create Account
Powered ByCareerCast