Path to Data Engineering — Day 1

Getting Started...

If today was the day when you decided to explore the route of becoming a data engineer and don’t have any guidance or you cannot afford to go to school, then not to worry! We will start with very basics and build it up day-by-day being on the path towards becoming a data engineer.

But first things first

What is Data Engineering?

With the explosion of data in the 21st century with smartphone, digital technology, popularity of social media, we started living in the world of Big Data. Now, everyone wanted to leverage the insights from this data and use it to either build new products or improve existing. This led to cloud computing where massive data can be crunched and further into Machine Learning and modern day AI.

Day 1 of Data Engineering 

The first 2 mandatory skills you would want to have are:

 1. SQL

2. Python

There are many free resources available where you can learn these skill sets. Let’s first focus on SQL. In order to practice SQL you need to have a relational database like PostgresSQL and a client or an IDE like DBBeaver. Further, you would need access to some free datasets. There are websites where you can download such data like www.Kaggle.com.

So, lets list them step by step :

Step 1:

Install Postgres or other similar database like MySQL

Step 2:

Install an IDE like DBBeaver or other similar like SQLWorkBench

Step 3:

Follow article to learn more about setting up and using PostgreSQL with DBeaver.

Step 4:

Create account in Kaggle, download a dataset and upload data in the postgres database. 

Step 5:

Now practice your SQL especially, joins, subqueries, ordering, aggregation, windowing functions, case when then, common table expressions etc… Use free resources like W3School

Step 6:

Once you feel good, you can practice your SQL skills in Leetcode .

Next up is Python. Let’s list step by step, how journey here would look like.

Step 1. You should have a google account, if not then create one.

Step 2. Use Google Colab for practicing Python

Step 3. Learn basics of Python programming if you are completely new. These basics include collections like dictionary, list, set, loops, enumeration etc… Use free resource like W3School to learn basics

Step 4. Once you have mastered basics in Python, learn these 2 Python libraries. Also good to learn about numpy but not mandatory to get started.

 1. Pandas

2. MatPlotlib

Awesome! If you are an absolute beginner,  Day 1 may take few weeks depending on the level of exposure, and there will be ups and downs. Stick to it and I know you will do it. Keep going, Keep learning. Next up we will see how Day 2 would look like. Stay tuned.

Listen to the podcast on same below