Senior Data Engineer

Macquarie Group
Melbourne, Australia
Mar 25, 2023
Apr 08, 2023
Job Function
Industry Sector
Finance - General
Employment Type
Full Time
Are you a talented Senior Data Engineer looking for your next challenge? Then you would love to be part of the team that is leading Data Integration at Macquarie. We are a group of data & software engineers building innovative cloud-based solutions. Our mission is to make all data the company has, available in all its services and systems and we do it based on the data mesh principles. To make it happen we use a tech stack that goes from Kafka and kafkaStreams, to GCP data stack including Pub/Sub, DataFlow, BigQuery, Spanner.

We operate in a collaborative and autonomous environment where we are empowered to do it right and continuously learn, becoming masters in our craft. This is a fantastic opportunity for someone who is passionate about big data technologies, enjoys solving problems and takes pride in delivering high quality solutions. You will work in a multi-talented, diverse team across multiple geographical locations and implement solutions in a fast-paced environment.

You'll bring your in-depth knowledge of data processing practices and a desire to work in a DevOps environment where you will have end-to-end ownership for developing, deploying, and supporting your solutions.

Your Core Skills:
  • As a Senior Data Engineer, you will have ideally led the successful delivery of data engineering projects, including gathering end-to-end requirements, data analysis, data engineering activities and data validation with stakeholders
  • Experience in data processing/streaming or data warehouse / analytics
  • Hands on experience in setting up Google Cloud Data Engineering
  • Strong hands-on experience in any programming (Python, Java, Spark)
Your Tech Related Skills:
  • ETL & data engineering using DBT
  • ETL & data engineering using leveraging multiple Google cloud like DataForm, DataFlow, DataProc, BigQuery
  • Knowledge on Data Modelling and reporting using Google Cloud BigQuery
  • Knowledge on building data pipelines leveraging GCP best methodologies
  • Strong understanding about Kubernetes, Docker containers and to deploy them in GCP
  • Experience in writing code to extract data from data sources including CDC, Outbox pattern implementations...etc
  • Experience in scheduling like Airflow, Cloud Composer etc.
  • Experience in CI/CD automation pipeline facilitating automated deployment and testing
  • Experience in data catalog and metadata management.
  • Experience with JIRA or any other Project Management Tools
Key Accountabilities and Priorities:
  • Build and operationalize real time and batch data pipelines including raw data acquisition, staging, integration of new data sources, cataloging, cleansing, batch and stream processing, transformation, and consumption
  • Independently work on assigned projects and foster a collaborative environment for a high-performing team
  • Quickly Understand and formulate application-level requirements pertaining to complex workflows, or integration with external applications
What's in it for you:
  • learn from the best engineers in Australia and work on market leading products
  • be part of a team that deeply values diversity and creates space for you to be your best
  • use the latest cloud technologies to tackle interesting and challenging group wide banking and finance problems
  • our work environment is inclusive with a strong focus on employee experience
  • flexible work options, including working from home
  • ongoing professional development and free technical certification
About the Corporate Operations Group

The Corporate Operations Group brings together specialist support services in Digital Transformation & Data, Technology, Operations, Human Resources, Business Services, Corporate Strategy & Solutions, and the Macquarie Group Foundation. We deliver service excellence to ensure Macquarie is open for business, deliver on transformational change, invest in our people and have deep relationships with our customers.

Our commitment to Diversity, Equity and Inclusion

The diversity of our people is one of our greatest strengths. An inclusive and equitable environment enables us to deliver innovative and sustainable outcomes for our people, clients, shareholders and communities. From day one, you'll be encouraged to be yourself and supported to perform at your best. If our purpose of 'empowering people to innovate and invest for a better future' is as inspiring to you as it is to us, please apply. With the right technology, support and resources, our people can work in a range of flexible ways.

We are committed to providing a working environment that embraces and values diversity, equity and inclusion. We encourage candidates to speak with a member of our recruitment team if you require adjustments to our recruitment process to support you, and the type of working arrangements that would help you thrive.

  • You need to sign in to save