The Data Engineering Jobs & Salaries Guide

Table of Contents

• What is Data Engineering?
• The Top Tools of Data Engineering
• The Data Engineering Career Path
• The Data Engineer Interview: What to Expect
• How to Learn Data Engineering at Jigsaw Labs
• Top Resources for Data Engineers

Every modern company depends on data to drive its decisions and products. With all of that data to capture and interpret, there is a huge demand for skilled data engineers. Jeff Katz of Jigsaw Labs demystifies the data engineering career path, breaking down what it takes to pivot into data engineering, the typical responsibilities (and salaries!) you can expect at any level of your data engineer career, and how to learn data engineering. Find out how Jigsaw Labs is preparing students for today’s data engineering field with a fresh perspective on tools, techniques, and skills and an externship that gives students real world experience.

Meet the Expert: Jeff Katz, Founder and Instructor of Jigsaw Labs

• After a brief career as a lawyer, Jeff became a software engineer in 2013, and has since helped over 350 students transition into software engineering.
• In 2018, Jeff Katz founded Jigsaw Labs, which teaches students to become data engineers in its 6-month, part-time course.  

What is Data Engineering?

In a modern organization, teams make decisions based on data. The sales team wants to target the best leads. Marketing wants to promote the best ads on the right platforms. The Operations team needs to understand how they’re gaining and losing money. All of that involves data. 

But of course, someone needs to capture and analyze this data, whether it’s from inside the company’s web application, or located in a third party tool, like Salesforce, Google Ads, or Google Analytics. The first step is designing a system that can collect this data repeatedly and accurately, so that it can be used for further analysis. That’s the foundation of a data driven organization, and that’s the job of a data engineer. 

It’s the data engineer’s job to collect data from various sources, change that data so it’s easier to use, and store that data in an analytical database. The data engineer needs to automate this to ensure the data is current, accurate, and standard throughout the organization.  From there, the data engineer may be involved in cleaning and analyzing the data to provide to the marketing, operations, sales, or product teams. 

Where does a data engineer fit into a team?

Once the software is developed to collect the data into an analytical database, how data engineers work with the broader company depends on the organization’s needs. 

• Data engineers may act in a data analytics role. They may measure things like Click Through Rate or perform marketing funnel analysis. You might expect someone from marketing or data analysis handling this task, but it requires technical skills to collect and analyze this data, and repeatedly generate these reports, and that’s what a data engineer does. 
• Another typical role for data engineers is machine learning operations (MLOps). For example, a data scientist or machine learning engineer may develop a machine learning model that recommends specific content to users.  But actually deploying that machine learning model so that it can make those recommendations accurately and quickly is a separate matter – it involves back end and cloud engineering skills, and that’s where a data engineer comes in. A data engineer may be used to operationalize the machine learning model so that it’s off of the data scientist’s computer and into production so it can help users.  
• Data engineers can act as back end and cloud engineers, as the data is often located in various cloud services, whether AWS or GCP.  To retrieve this data, data engineers need to lean on Python and SQL skills to select just the data necessary, whether from a database or an API.  

3 Traits of Successful Data Engineers

The typical data engineer has three main skills:

1. Must be a strong back end engineer. Data engineers need to be able to code in an organized, maintainable way. This means a strong foundation in Python and SQL at a minimum.  
2. Adaptable. Data engineering is a new field, and the tools used vary from company to company.  Data engineers should embrace and explore changes in the data landscape to help make their companies more data driven. This means learning new technologies and recommending the best path forward to their team. 
3. Empathy. Data engineers help teams throughout an organization make decisions. That means, before data engineers develop systems to collect or analyze the data, they should understand how the data will be used, and why it’s important, so that it is collected in a way that addresses the needs of stakeholders. 

The Top Tools of Data Engineering

There are a couple of main types of data engineers and there are some key differences in the skills and tools used.

Analytics Engineering

Analytics engineers use extract & load (EL) tools. These tools pipe data into a data warehouse automatically, after some initial setup. Once the data is in the data warehouse, analytics engineers use SQL to select, manipulate, and analyze that data.

These SQL queries can grow to be quite long, consisting of dozens of files. To organize these commands, analytics engineers use a tool called Data Build Tool (DBT). DBT is the standard way to organize these transformations. Once the queries have been performed, next is to use this data to develop dashboards in business intelligence tools, such as Looker or Tableau.

Data Engineering

Analytics Engineers mainly rely on their knowledge of third-party services to build data pipelines, and SQL to manipulate and query that data to connect to dashboards. But as the pipelines become more complicated, an out-of-box solution may not be available. In this case, companies will need data engineers to build their pipelines and organize their data.

Data engineers use SQL and Python to extract data, and cloud computing tools like Docker and AWS to deploy their data pipelines. Because the data needs to be repeatedly updated, data orchestration tools Airflow are used to repeatedly move data from one location to the other.    Data engineers may use web frameworks like Flask to make their data available from a website or API.

MLOps

One specialization in data engineering is to become a Machine Learning Operations practitioner. An MLOps practitioner is used to deploy a machine learning model, continually update (train) the machine learning model with new data, and to ensure that the machine learning model makes predictions quickly (when necessary) and without consuming too many resources. Doing so requires knowledge of web frameworks like Flask, SQL and Python, and cloud computing skills like AWS and Docker.  

Soft skills for Data Engineers

Data engineers are serving internal stakeholders, so it’s necessary for them to communicate and understand what the use case is and have that in mind when building the system. 

For example: It could be that you’re noticing teams using different metrics for what is considered an active user. It might be better for an organization if everyone is on the same page and it might not, but it’s the data engineer’s job to think about that. It’s important to have these perspectives and translate them into a data product to serve teams with the appropriate data.

The Data Engineering Career Path

Data engineering isn’t so much a linear career ladder as it is a jungle gym. Once you have learned the fundamentals of data engineering, you can choose a specialization, climb the ladder, or switch into other tech careers fairly easily. Another nice thing about this field is that you don’t need a certain amount of years of experience to progress. The technologies are new and if you learn them, there’s nothing stopping you!

A “typical” data engineering career path could look like…

• Begin your career as an analytics engineer or data engineer, and later transfer to an MLOps engineer role. There are many machine learning engineers that started off as data engineers.
• Begin your career as a data engineer and then pivot into being a back end engineer. 
• Begin a career in operations or marketing, since a lot of data engineering tasks involve helping those teams automate their workflows or capture data for better decision-making. 

Entry-Level vs Senior-Level Data Engineering Roles

The differences between junior and senior-level data engineers are similar to the differences between junior and senior back end engineers. 

• The entry-level data engineers will be using Python, SQL, and some cloud computing. 
• The mid-level data engineers work with large codebases and write sustainable code. They may have design pattern recommendations and be well versed in them.
• At the senior level, engineers are expected to make system design recommendations, where engineers know the tradeoffs between working with different kinds of databases and cloud computing services to design data pipelines in an efficient manner that meets the needs of various stakeholders.

Data Engineer Salaries

Data engineers at the entry-level typically make between $80K-$100K. Mid-level data engineers can make $115K-140K. Senior-level data engineers make anywhere from $170K-$200K.

What types of jobs are Jigsaw Labs students landing after graduation?

Graduates of Jigsaw Labs have gone on to become back end engineers, data engineers, Python engineers, and even machine learning engineers. They have landed roles at cryptocurrency companies, fintech companies like AmEx, and medical organizations like Memorial Sloan Kettering. A few of our students incorporated what they learned at Jigsaw Labs into their business and product development roles. Jigsaw Labs graduates have starting salaries of around $115K a year.

The Data Engineer Interview: What to Expect

In a data engineering interview, you can absolutely expect questions on SQL. You can also expect a soft question about the types of data pipelines you’ve built. Finally, they’ll ask about Python. Python questions arise slightly less often, but strong data engineers are strong Python engineers so you should not be surprised when it’s assessed. 

If you want to ace an interview, you’ll need to know SQL and be able to pass the technical assessments. If you want to wow your potential employers, show them that you’ve built an end-to-end data pipeline or written code that uses object oriented programming and Airflow that’s deployed on a cloud service like AWS. In the interview, you have to be able to discuss your reasoning behind your builds and be able to organize your thoughts. It’s important to be someone people want to work with and to be enthusiastic. 

Which careers transfer to data engineering well?

If you automated something in your last job – for example, maybe you wrote an Excel Macro because you found yourself doing the same process over and over – take that as a sign that you could switch over to data engineering. 

For example: One of our Jigsaw Labs students was a salesperson and his interest in data engineering started when he wrote a LinkedIn scraper to identify leads. In my former career, I was a lawyer and I wrote something that pulled information from various legal contracts (like the contract date, title, and signatories) so it could place this information into Excel for me. Things like that are great because they show you know how to think through a problem and deliver an efficient solution. 

Former marketers can find a good fit in data engineering because there are so many use cases for marketing. Every company has issues with selling their products, and if you have experience working in marketing then you will be attuned to the marketing team's needs as a data engineer. 

How to Learn Data Engineering at Jigsaw Labs

At Jigsaw Labs, we teach the full analytics engineering and data engineering stack and we are thorough about each skillset involved. 

When we teach SQL, we don't stop at relational queries; we also teach window functions, CTEs, and subqueries – and how to organize these queries in a large codebase with DBT. Students learn the tradeoffs of using different databases like Snowflake and Postgres. Students also learn modern tools like Fivetran and Meltano to organize the data.

Then it’s back end engineering with Flask, Python, Object Oriented Programming, and testing. After that, we teach cloud computing with Docker and AWS, and orchestration with Airflow.   

We are constantly iterating on our curriculum. So far we've thrown out a third of our curriculum every time we’ve launched a new cohort. We want to make sure we’re teaching exactly what employers are looking for now. Because our curriculum is closely aligned with the job market,  our graduates are in high demand from employers.

What can students expect to learn in the internship at Jigsaw Labs?

Halfway through the program, six hours of class time per week is devoted to internships. Some students work up to 20 hours a week in the internship, and do so with top companies. Our previous externships have included:

• Working with machine learning researchers out of Harvard to build a demo of the product to help show a use case.
• Doing machine learning work with a cryptocurrency startup. 
• Working with Meltano, a top data engineering company, to help them analyze data in their Slack channel, which they use for onboarding and support. 

Is there an ideal student for Jigsaw Labs’ Data Engineering Bootcamp?

In admissions, we are looking to see that a student is a good culture fit for the bootcamp and that we are confident they will land a job after graduation. Part of that is ensuring students respond well to feedback, collaborate with others, and that they’re curious about the subject matter. Halfway through the program, we offer an internship program, so we need to be able to count on students to be on time to meetings with partnering companies, and be engaged and respectful throughout the partnership.

Do students need to be able to code to apply to Jigsaw Labs?

To prepare for the bootcamp, we recommend that applicants complete our free training, which includes 10 readings and 4 labs. Once an applicant understands that material, they will have the technical skills to apply for Jigsaw Labs. From there, we’ll have a technical interview, which allows us to make a good prediction about your skills coming out of the program.

How does Jigsaw Labs support its students in their career journeys?

Like other bootcamps, we source our students interviews for various positions we think our graduates would be a good fit for. But after students graduate, we’re not done with meeting as a class.  We have twice weekly meetings in the evening to help students continue with their internships, prepare for technical interviews, and apply for jobs. This is important because students have many responsibilities when they graduate and often feel pulled in multiple directions. Meeting biweekly means students have instructors to help them balance their time, learn additional skill sets, and prepare and give feedback on interviews. All graduates are undergoing this same process, so it makes sense that they do so together so they can share their learnings.  

Top Resources for Data Engineers

For anyone starting in data engineering or looking to level up their career, we recommend:

• The DBT Slack channel
• Data engineering subreddits
• The writer and computer scientists Chip Huyen - They have so many great tips!
• The free meetups and data workshops hosted at Jigsaw Labs.

Find out more and read Jigsaw Labs reviews on Course Report. This article was produced by the Course Report team in partnership with Jigsaw Labs.