DevOps Engineer vs Data Engineer - Which career is RIGHT for YOU? | 13 Key Differences Explained

now coming to the 13th and the most important part which you have been waiting for uh money money what's what's what's there for me what will come to my pocket hello friends welcome to itk Funday where we make it interesting for everyone I'm anul tiwari and in this video we will tackle a very common question which has been coming to me uh which is related to your career and that is uh sir I'm confused whether I should go for a devops Engineer carer Road or should I take a data engineer career path now both are very exciting and booming careers but in this video we will understand the differences each of these career path has and I hope by the end of this video looking into these differences you will be able to make up your mind which career path would be more aligned to your skills your strength and more importantly your passion so without further Ado let's get started so friends I did some research and I have found 13 key differences which I feel you should be aware of now there can be many others but I think these 13 will definitely give you an understanding of what is expected from each of these career paths okay so let's start with the first difference what is the first difference the first difference is of the career Focus the carer focus of a devops engineer is of building and maintaining the devops pipeline making sure that the code is deployed in production faster quicker and then the whole cicd pipeline has to be automated without any manual intervention if you want to learn more about devops again there are different resources I have also made videos on devops so you can check it out the core focus of data engineer on the other hand is primarily of building and maintaining data pipelines ensuring that the data loads the data Integrity the data quality everything is intact and complex data architectures they have to maintain because for every business there is a different data there's a different domain which you need to understand so as a data engineer while developing those data pipelines you have to keep all those things in perspective now this is a very small view of what it contains you know devops could take you 6 months uh to master same for data engineering so just understand the Cris and the gist of it for now the tools and the skills now when we talk about devops there's a plethora of tools right now in the market which you could learn but just to name a few you need to know Linux scripting python you need to know bash uh you need to know infrastructure on cloud gcp AWS Azure now it it it will be tweaked based on which particular project you are working but I'm just giving you an overview cicd tools there are many you need to understand git you need to understand Jen genkins anible terraform and kubernetes Docker for deployment of your container ERS so yes a very huge tool set is there but on the other side in data engineering I would't say that you would need to know a lot around tools it is less on tooling but more on the technology and the knowledge around data so in tools you need to understand what is SQL what is no SQL databases you need to understand the SQL Basics if you know plsql nothing like it it's it's really good you need to understand now python spark because all the data pipelines are U you know getting automated using U you know programming languages previously uh 15 years 20 years or maybe I would say 10 years um in the past U we used to have ETL tools and people used to just drag and drop and create the pipelines or you would say ETL uh pipelines um you know using the software but now the developers have started building the pipelines using Python and Spark framework uh but but still the tools are there Informatica is there uh you need to know data modeling that's an important thing you need to understand how you will structure the data as per the requirement uh databases on cloud so all the databases which you get on cloud slowly and gradually as you will learn through your projects you would understand uh how to use SQL Server you would understand how to use big query uh red shift synapse database on Azure all those and one very important is snowflake I have made a video on snowflake which has been viewed by many uh it it covers the basics of snowflake it's a database as a service data Weare housing as a service uh provided by this company really really booming right now now coming to the focus very important Point guys coming to the focus I feel um and it is all you know with a cavat that I feel maybe you you might have a different perspective on it but I feel devop is more of a broad kind of a knowledge you need to know width you need to know broadly what all things happen throughout a software delivery life cycle and for every particular phase you have different tools and Technologies which you have to know but then you would only Master a few of those but you need to know broadly every other technology as well where in in data you need to have deep knowledge there's there are no so many tools which you have to understand if you know SQL very well if you're very good with plsql if you're very good with building uh and creating data models and uh creating data pipelines then there are tools which you could Master very quickly so from from that perspective understanding of data is way more important than understanding of tools and Technologies those comes later in in data engineering understanding the data because data would continue to vary right now you are working for a sales department data is different tomorrow you will go for for a manufacturing Department uh data is different you might switch the industry you might go from a retail uh company to a media company data is different so the domain the data domain has to be understood by a data engineer very very quickly where the functional knowledge is also key okay so what are some carrier paths from where the generally the switch happens to devops engineer basically it is it often comes from software engineering and system admin background so generally the software Engineers who have been developing the code for a long time and want to move to the next level they prefer to go into a devops carrier frack and again with the system admin uh because they're already aware of Linux they are already aware of how to maintain the infrastructure they find devops very interesting because while they might be weaker on the dev side of it they're very strong on the Ops side of it so what they can do is they they they try to uh work on the dev side but they have the strength on the operation side and that's why system admins also prefer to come to uh devops engineer career path now these are couple but there could be many others I'm just giving you some uh view of what I think are the key differences here carrier switch for data Engineers generally come from data driven rules which is very obvious data analyst business analyst uh bi developer uh data operations dbas all these kind of of people who have been in touch with data they have this opportunity to get into Data engineering with a caveat that now if you want to become a data engineer gone were the days where where programming knowledge was optional now it is it is important for you to at least have basic understanding of uh you don't need to know objectoriented programming for example if you can understand okay these are the libraries these are the modules I have to use this is How I build a data pipeline in Python using data frames using Panda library then you can start it's it's not rocket science because you're not building a full full stack application right you need not to be that level of a coder but you need to know enough to build your data pipelines and with time you will obviously learn uh knowledge wise for devops you need holistic it knowledge because a devops pipeline touches every aspect of an IT industry okay starting from your development so you need to understand how the developers are developing code using vs code then how they are pushing it on git from GitHub repository how that will be pulled by genk then that data pipeline then your Docker and kubernetes has to be learned so there are many uh anible uh for configuration management terraform for infrastructure as a code so basically you need to know everything in it and that's why I think broader knowledge is more important here and uh yes it infrastructure I as I have already said that will be important from here as as we have already discussed deep knowledge in data warehousing is very important for data Engineers ETL and El knowledge extract transform and load elts extract load and transform El now we are using in data LS concept I have created a separate video on data pipeline watch it it will clear a lot of concepts for you again data modeling data legs batch and realtime data how you how you create different pipelines how you use Lambda architecture to combine the best of both the worlds realtime data badge data so all that knowledge is very deep and as you start to learn it you will become more proficient in data engineering side collaborations now collaborations means who all uh are the teams or people with whom a data engineer or a devops engineer work with so for devops engineer uh software developers SRE which is site reliability engineering if you don't know again there's a video for side reliability engineering go watch it it Ops teams these are the teams generally with which you know the collaboration happens again I'm not putting agile scrum master all those things are natural you have to connect with your scrum Master you might have to connect with the product owner all that I have I have skipped but you need to collaborate with them as well data scientist business analyst um you know stakeholders are are very important uh for for uh data Engineers because data is generally consumed by the senior stakeholders visualization team who are building reports and dashboards on powerbi business objects tab data operations team who are looking into the data infrastructure all those kind of teams uh generally data Engineers collaborate with coming to the seventh difference and the key challenge uh what I think is that for devops engineers the key challenge is to keep your devops pipeline ready and to be able to deploy any change in code into production into the working code in production uh in seconds that's what is the key challenge or key goal of a devops team is and obviously throughout that pipeline if there are any bugs there there are any vulnerabilities devops teams or devops Engineers have to fix those whereas in here I think the speed does matter but more than that the consistency and reliability of data matters most because you understand this guys that uh data if if you get faster data but the data is incorrect it is more dangerous okay that's why uh people prefer to have accuracy reliability and Security First and speed then comes again speed is an important parameter faster data loads are important now with AIML use cases we need faster Analytics as well but again the bottom line is data consistency data reliability data integrity and data security these four I feel are more important than data performance uh it's it's U it's an important thing but again this is what I feel so we are done with the seven sorry guys the board is small differences are huge no I'm kidding seven differences done now let's uh let's look at four remaining uh differences which will complete hold uh 13 list of differences so let's start with the difference number eight so guys let's Now cover the remaining four differences I think we covered seven there and now we have uh remaining how many 1 2 3 4 5 six sorry six differences we have six differences remaining and watch it till the end because in the end we'll talk about the most important 13th difference which which is the salary which I have cated from different sources so watch till the end so the difference number eight is demand so basically devops uh Engineers demand is booming a lot in Tech startups and that's why in India because India is now becoming a hot soil for new startups new technology startups there's a lot of demand for devops engineers because obviously you need to what what you need to do in a startup you need to build a prototype you need to quickly test it validate it launch it in the market test how it is working what is the customer feedback and all that cannot happen if you do not have have a reliable fast um devops pipeline so that's what uh where the actual demand is but this is just you know a summary devops is needed everywhere now but here it has the maximum demand wherein for data Engineers the demand is predominantly on in Industries which are very primarily data driven and which are dealing with vast amount of data for example Facebook for example Twitter all these kind of companies Amazon all these companies thrive on customers data and their uh the need for data Engineers would be way more than any traditional company and for example suppose you have you have opened your own company a tax startup okay I think you would be more focused on delivering softwares using devops than uh creating and building insights and analyzing the data doing AIML right at the beginning because for all that you need historical data to draw insights right and for that you need some time in the market and that's why U you know here devops wins the race when it comes to Startup culture now the ninth and a very important point which is the learning curve now the learning curve of a devops engineer is easier again this is my my understanding compared to a data engineer here you can have a very quick Hands-On exercise you can very quickly build and deploy a devops pipeline there are numerous tutorials if you go online you you will also find numerous courses various people are offering devops courses for 6 months for three months they will teach you everything for building devops pipeline you will build the code you will deploy it everything you will do very quickly and the fruits of Labor whatever you are doing you will be very excited about doing it because you will see the results very quickly and uh you will see that okay the moment we push the code into G automatically the pipeline got triggered the build was triggered then it was executed then it was deployed into a Docker container it what it got deployed into kubernetes all that kind of thing you will see happening and that will you know that will be very interesting wherein when it come to data engineering the learning curve is steeper because you need to work on specific good data projects as I said data varies from one place to another place from one Department to another department okay and that's why in order to call yourself a true data engineer you need specific different projects now you will have uh training courses and you have websites which offer these kind of trainings but I honestly feel that in order to truly understand what is data engineering you need to have live projects in a in a company and I I I I cannot recall any good data engineering certification or program yet uh there are Cloud certifications who provide you data engineering certifications yes there are but again compared to devops I think the learning curve is deeper and the longer Time and Labor is needed for actual fruits to be seen that's what I feel for um for data engineering you need to give yourself time to see tangible results but understand this very key difference while you can become a devops engineer very quickly you have to be on your toes all the time all the time because every new day a tool is coming every new day something is happening and all so you can never be relaxed of sorts wherein in here if you have buil your core competency around data if you have worked on certain tools more or less the work would revolve around your core data knowledge and how you work uh in data okay so that's why while you might take longer time to get to that level once you at that level you would be uh at a much easier or a calmer Place compared to a devops engineer again all my experience all my research feel free to to negate it feel free to say that no you feel differently and mention it in the comments I'm all open for any objections it's it's fair right because I'm just sharing whatever I know whatever I feel and you're not obliged to agree or disagree whatever you want you can do it but I feel that this will take more time compared to devops again then another thing which you have to consider before choosing a path is what exactly you like because in a devops kind of a role you you would need to do a lot of troubleshooting you need to you need to be in love with variety you need to be in love with the width of things you need to be uh passionate about the macro view of everything which is happening are you that kind of person because some people like variety but some people get exhausted if they have to continuously switch from one tool to another from this to that right so you have to understand what kind of personality you have and based on that you have to you have to take a call in data you need to understand the data challenges how you would collect the data process transform or store the data and it is more detail oriented you have to go into the depth of what are the needs of the business how you would create the data pipelines and all so while you might not have to deal with the macro you have to deal with the micro View and then uh the variety part will only come on the data side of it you don't have to learn so many tools at one time you don't need to you need to just focus on I have seen data Engineers who just have very good understanding of SQL and rest all revolves around that core knowledge okay visibility very important point and I think this is where data Engineers are bit ahead of devops because devops Engineers always work at the back end of things right so for example a CEO or a cxo or your senior stakeholders might not be in a direct Touch of a devops engineer yes your scrum Master would be your product owner would be for sure but here with data engineering the visibility goes a level above because you are dealing with the cosos and cxos data for example you are you are responsible for loading the data for uh for a sales dashboard which shows how much sales we have done for every quarter in particular region in particular geography what was the actual what was the forecasted what was the Target and whether we have met the target we are this you know the boardroom is sitting to analyze that data and see the performance of the company you tell me data engineer would be closer or not data engineer would definitely be closer to the senior stakeholders and more than that the developers the powerbi and Tableau developers who develop those reports they will be for even more closer which gives you a lot of visibility but I would I would not say it is a downside okay every every every role has ups and downs so I think this is where um devops sorry data Engineers have a slight Edge and here on the 12th Point devops Engineers have a big Edge because they don't need to worry about data engineering they don't they they can give a damn about data engineering they they are not at all required to understand data engineering if they want to they can okay but as as far as they know DeVos they are sorted but for data Engineers now that you know the now the journey is different they have to it is required and mandatory for them to understand devops because now we are moving into Data Ops data Ops is like devops so if you want to learn about data Ops let me know in the comments and I'll try to make a video on the same so now all the data deliveries are delivering as data products okay and it is being treated data is being treated as a product and in order to deliver a product efficiently you need a pipeline and that's what is being done using data Ops where you will need certain aspects of devops knowledge so it is required so yeah we are done with all the 12 and now coming to the 13th and the most important part which you have been waiting for uh money money what's what's what's there for me what will come to my pocket right okay so this data I'm going to share with you but with a caveat it is highly highly uh variable okay wherever I have done the research the numbers keeps on varying and when you will finally get a package it will depend on numerous different situations okay your previous package which company you are joining the need of that particular company your current knowledge how your interview went how well you negotiated with the HR all that boils down to that particular package which you get okay so don't don't get fixated on these numbers but these numbers are just average numbers which I have collected from certain websites like pay scale and glass door so uh yeah let's discuss those numbers now okay guys so coming to the final difference which is the salary I would say it is not at all a difference because it is such a neck to neck competition that salaries are all equivalent but if you have to give some have to make someone a winner then data engineers get slightly more than devops Engineers but you see the numbers and you would yourself say that the difference is not huge and by the way all these numbers have been collected so it will change it might be wrong at certain places so quot don't quote me on this in us the average salary of a devops engineer varies from $80,000 to $200,000 the case stands for, okay in UK it is from £50,000 to £80,000 and in India it it it uh varies from 4 lakhs to 15 lakhs now compared to data engineers data Engineers uh us salary is from $90,000 to $220,000 not much difference okay £55,000 in UK to 90,000 and in India it is 5 lakhs to 20 lakhs so basically you can see that the difference is not huge and that's why it is such a you know such these two roles are so so closely fought and always in demand so friends I hope you now understand the difference between a devop engineer carer path and a data engineer career path and what is the difference between the two and by looking into these 13 differences you might be in a better position now to understand which carer path is for you so with that said I really hope that this video added uh some value to your knowledge if it did I would request you to put a like button hit the like button hit the Subscribe button put your comments how you likeed this video what you would want to view next on the channel and until next time guys keep learning keep sharing all your knowledge and yes keep hustling bye for now

Channel: IT k Funde

Views: 3,479

Keywords: devops, dataengineer, career, careerguidance, careerroadmap, roadmap, interview, devops engineer, data engineering, data pipeline, etl vs elt, which career is good for me, salary comparision, key challenges, jenkins, terraform, ansible, kubernetes, snowflake, informatica, docker, learning path, step by step

Id: c3RJ27f22uc

Length: 24min 37sec (1477 seconds)

Published: Thu Apr 11 2024