Recently, Amogh Antarkar embarked on a demanding project at H-E-B to improve business analytics and the customer experience.
The data engineer designed and implemented a data engineering service that integrates Google Analytics clickstream data across the supermarket chain’s apps and sites. It required Antarkar and his team to confront tricky factors, like the sheer volume of data the company generates and the multi-cloud nature of the platforms.
“It forced the team and I to architect a well-thought-out cloud native data processing pipeline,” he said.
An assignment like this is a perfect glimpse into some of the work that Antarkar does as a data engineer at H-E-B. But just as important, it reveals the opportunity to pursue ambitious projects at the creator of digital products for households that shop at H-E-B and its partners.
“This was possible because of the flat organizational team structure at H-E-B and the people-centric culture,” Antarkar said. “It gave me an opportunity to drive a key initiative and build a reusable and reliable engineering pattern from the ground up that had a positive impact in the engineering chapter.”
Built In Austin spoke with Antarkar to learn more about his experiences as a data engineer at H-E-B.
Give us a little insight into a typical day for you.
My day typically starts with a daily team sync and coffee. I spend time in design, architecture and development of the data engineering patterns and solutions. My role involves working closely with fellow data platform engineers and architects to design and model solutions for the business and data science teams that are our key stakeholders. Other times, I am modernizing data platforms and implementing frameworks to establish better standards. I occasionally sync up with leaders and product for sprint and roadmap-planning to collaborate on new initiatives.
For our tech stack, I use services on AWS and Azure because that is where the data lake resides. Python, Spark, Databricks, Datadog, Kafka, GitLab and CI/CD tools are some of the other tech we use. Many source systems reside on Google Cloud and on-premise tools, so we integrate with these as well. H-E-B is a large progressive company with an increasingly digital presence. Its engineering teams reside in multi-cloud environments with a variety of technologies and tools.
Tell us about a project you’re working on right now that you’re really excited about.
I designed and implemented a data engineering service that integrates Google Analytics clickstream data across H-E-B’s e-commerce apps and sites. This helps us listen to the pulse of the business and run experimentation to improve customer experience. The sheer volume of data generated daily made it an interesting large-scale data problem. Add in the multi-cloud nature of the platforms, the faster analytics, and the business and data science needs, and this project was demanding. It forced the team and I to architect a well-thought-out cloud native data processing pipeline.
A project like this was possible because of the flat organizational team structure at H-E-B and the people-centric culture. It gave me an opportunity to drive a key initiative and build a reusable and reliable engineering pattern from the ground up that had a positive impact in the engineering chapter.
In order to be a strong data engineer, you need a diverse skill set.’’
What’s the most important skill (hard or soft) a data engineer needs to be successful in their role?
In order to be a strong data engineer, you need a diverse skill set. This includes programming, SQL, cloud, data and software engineering, design and data science. In addition, an engineer needs to understand the business and have an ability to communicate with stakeholders effectively. Data is the core to everything. It is essential to be curious, and to learn about state-of-the-art technologies. Loving (data) is the answer!
