The difference between Data Engineers vs Data Analysts vs Data Scientists | What's right for you?

from personal experience I know it can be quite confusing to decide which exact data job you should go after data engineer data analyst or data scientist so I'm going to use this little diagram I created to explain the responsibilities and skills required for each of these roles and I'll also cover the potential salaries but before I begin I just want to know which role interests you the most before I go through my views on each and whether or not which role you grabit a towards two changed after watching this video please share and comment below once you're done watching so to understand what Dees Das and DS's do first let's look at how data flows through an organization first we need to collect the data and store it somewhere this can be SQL databases somewhere in the cloud like AWS S3 buckets or just plain old CSV files then we need to clean and transform the data to make it usable then we need to explore and analyze it it to extract meaning out of the data that we have and lastly we can perform some Predictive Analytics on our data sets data Engineers are responsible for the first step in the process as their job is to build and optimize systems for data analysts and scientists construct data pipelines and ensure that data is properly received and stored their main goal is to build a scalable infrastructure that makes data accessible to anyone who needs it a great data engineer should be able to connect hundreds if not thousands of employees to the data they need as a data engineer You' be developing constructing testing and maintaining the architecture implementing new ways to improve the data quality building data pipelines creating and integrating apis performing ETL operations or preparing data sets for data modeling you'd be working with lots of data so experience in Hadoop map produce hive or data streaming would be essential you would also need in-depth database knowledge whether it's SQL no SQL or maybe even graph databases and on top of these you would need to have strong programming and scripting skills of course now data analysts are mostly responsible for steps two and three in the organizational data flow as they need to pull analyze and interpret data to extract meaningful actionable insights that solve business critical problems and I must emphasize insights that solve business problems here as a data analyst you can create the most beautiful and icatching reports and dashboards if you don't know how to identify the data problems your stakeholders have and cannot deliver meaningful insights that your stakeholders can use to drive business actions then what's the point of those shiny dashboards a great data analyst is someone who can combine their domain knowledge critical and analytical thinking and their Tech technical skills to solve unpredicted multi-dimensional non-linear data problems on a day-to-day basis you'd be cleaning and transforming and analyzing data to find insights performing exploratory data analysis creating reports and dashboards and maintaining them or doing whatever ad hoc analysis that comes your way You' need to have strong Excel and SQL skills for sure and preferably some data visualization skills using a bi tool like Tableau or powerbi if you knew how to code in a programming language like python that would be a great Plus in terms of other skills you'd need to have strong analytical skills a decent math knowledge I'd say basic to intermediate statistics would suffice you'd also need great communication skills as you'll need to be able to connect with both Technical and non-technical audiences I made a video on how I'd become a data analyst if I had to do it all over again and I also made a data road map so feel free to check that them out I'll leave the links in the description below and last but not least let's go through what data scientists do they would operate mostly in the final stage of the flow process but their roles can often span from step two to four depending on the size of the organization specific team within the organization or budget usually at larger organizations your job role tends to be more specific so data Engineers would collect and store the data data analysts would then get the a hoc insight and then data scientists would do the experimentation but at many companies usually smaller firms like startups roles tend to overlap and you could be doing pretty much everything as a data scientist on a daily basis if you were to do the so-called traditional data scientist tasks you'd be developing machine learning algorithms building deep learning models doing AB testing identifying Trends and patterns to drive business decision making out of the three roles this one is the most most challenging in my opinion as on top of all the technical skills you'd also need to be very very good at math and statistics and I say this because I've worked in a data science team before and they all had at least a master's degree in a quantitative subject with many holding phds to be a data scientist you'd need to be great at programming you'd need to have in-depth knowledge of machine learning or deep learning you would need to know how to work with big data and you'd need to be extremely strong at mathematics and statistics knowing this it shouldn't come as a surprise that the average US salary for data scientist is over $124,000 which is just a fraction higher than the average data engineer salaries which come in at $123,000 both of these roles offer significantly higher paays than data analyst roles which makes sense as you can learn the necessary skills to become a DA much quicker than that of De or DS's Lots lot of data analysts actually move into data science and data engineering roles as their careers progress as they gain more knowledge and get more technical so to quickly recap data Engineers collect and store data and their main job is to make data accessible to those who need it within the organization if you want to be a data engineer you'd need strong programming scripting and database knowledge skills combined with experience with working with big data data analysts pull Wrangle and analyze data to create reports dashboards or atoc analysis to solve business critical problems if you want to be a data analyst you need Advanced Excel and SQL skills at least in terms of technical skills you would also need strong critical and analytical thinking and communication skills to work with business stakeholders to identify and solve business problems data scientists develop machine learning algorithms and build deep learning models this is the hard hardest role to get out of the three in my opinion as you need to be great at programming plus math plus statistics most people in this role have phds and master's degrees in a quantitative subject so that's my take on data Engineers versus data analysts versus data scientists if you like this video make sure to check out some of my other videos right here thank you so so much for watching and I shall see you in the next one thank you [Music]

Channel: Mo Chen

Views: 17,232

Keywords:

Id: eUrwxH9as_k

Length: 7min 39sec (459 seconds)

Published: Sun Oct 15 2023