Data Engineer

Location: Corporate Headquarters - The Woodlands, TX
Department: Operations Applications
Reports To: Executive Director of Software Development

Job Summary:

We are seeking a highly skilled and experienced Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines, optimizing big data workflows, and integrating Databricks with AWS services. You will play a key role in designing and implementing cloud-native data solutions that drive business insights and innovation.

Key Responsibilities:

• Proficient in Databricks SQL for executing complex ad-hoc queries on large-scale lake house datasets, enabling rapid data exploration, trend identification, and actionable insights for business decision-making.
• Skilled in leveraging Databricks notebooks to design and implement scalable data transformation workflows, integrating PySpark and SQL to cleanse, enrich, and prepare large datasets for downstream analytics and reporting.
• Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks.
• Architect and implement ETL/ELT workflows leveraging AWS services such as S3, Glue, Lambda, Redshift, and EMR.
• Optimize Spark jobs for performance and cost-efficiency in a cloud environment.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
• Implement CI/CD pipelines for Databricks notebooks and jobs using tools like GitHub Actions, Azure DevOps, or Jenkins.
• Ensure data quality, security, and governance using tools like Unity Catalog, Delta Lake, and AWS Lake Formation.
• Monitor and troubleshoot production data pipelines and jobs.
• Mentor junior engineers and contribute to best practices and standards.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 5+ years of experience in software/data engineering with at least 2 years working with Databricks and Apache Spark.
• Strong proficiency in Python, SQL, and PySpark.
• Deep understanding of AWS cloud architecture and services (especially S3, Glue, Lambda, IAM, Redshift, and CloudWatch).
• Experience with Delta Lake, Databricks Workflows, and Databricks SQL.
• Familiarity with data modeling, data warehousing, and data lakehouse architectures.
• Experience with infrastructure-as-code tools like Terraform or CloudFormation.
• Strong problem-solving skills and ability to work in a fast-paced, agile environment.

Preferred Qualifications:

• Databricks Certified Data Engineer or AWS Certified Solutions Architect.
• Experience with streaming data using Kafka, Kinesis, or Structured Streaming.
• Knowledge of MLflow, feature stores, or MLOps practices.
• Familiarity with data governance and compliance frameworks (e.g., GDPR, HIPAA).

Soft Skills:

• Excellent communication and collaboration skills.
• Strong analytical thinking and attention to detail.
• Ability to mentor and lead technical discussions.

What's In It for You?

You will be joining a high profile team and a great company known for its Servant Leadership and "work hard, play harder" culture. Competitive compensation package includes base salary plus annual cash bonus and equity grant (restricted share units) eligibility.

We also offer excellent family benefits including: medical, dental, vision, flexible spending account, long term and short term disability, life insurance, 401(k) retirement and unlimited opportunities to "Connect with Your Future."

Waste Connections is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to disability or protected veteran status.