Who's Solving The DEVELOPER SHORTAGE Crisis?

obviously the software industry depends on the ability to create ever growing amounts of software but the skills of software development are difficult to learn globally software development skills are in great demand there's a lot of talk about a shortage of software development skills and this seems to be a fact in most parts of the world this shortage has been around for a long time though and if anything seems to be getting worse rather than better so why is there a skill shortage now and what should we be doing about [Music] it hi I'm Dave Farley of continuous delivery welcome to my channel if you haven't been here before please do hit subscribe and if you enjoy the content today hit like as well according to estimates the population of software developers in the world is currently at 27.7 million this is up from 24.5 million in 2020 and it's is projected by to increase by another million uh by the end of next year projections predict that this is to nearly double to 45 million by the end of the decade although there have been many prominent layoffs in Tech firms in recent years despite this there remains a significant shortage of people with the skills that our employers are looking for us sources claim a record of over a million unfilled vacancies in technical jobs this year so what are the companies that employ People Like Us doing to cope with this mismatch between supply and demand well naturally they're looking for the easy solutions first and going to common sources of Solace like artificial intelligence no code and seeking to increase the use of Nearshore and offshore workers but maybe more significantly they're also adopting remote working strategies in general as the default hiring model which increases the talent pool from which they can draw making the pool of available developers on which they can draw essent Al Global this is a big change and was of course initially driven by the covid pandemic but is now widely seen as a tool to provide access to more people with the skills that the companies need increasingly geographically distributed teams are seen Less in the context of saving money as they used to be and much more now in gaining access to This Global Talent pool this is presumably good news for people with the skills because wages will be higher this will no doubt cause its own problems later because Employers in high wage economies will increasingly attract people from lower wage economies to apply making the developer shortage in lower wage economies even worse than it is today presumably if the economies of supply and demand hold it will mean a global pressure to increase wages for technical people let me pause there and say thank you to our sponsors we are extremely fortunate to be sponsored by Eco experts tricentis and transic all of these compan companies offer products and services that are well aligned with the topics that we discuss on this channel every week so if you're looking for excellence in continuous delivery and software engineering or maybe looking for a new job click on the links in the description below and check them out longterm you'd expect all of this stuff to balance out and as software development becomes more appealing as a job due to the increase in wages we'd expect to see an increase in the numbers of people training and joining the profession which does seem to be happening in many parts of the world University admissions for software courses are up software engineering courses are the most popular engineering courses of all in many geographies now but there does seem to be a problem still and it's not just that it takes a while for this kind of change to play out there's something more broken here than only that in an article about the shortage of software developers from a company called hatchworks they list four things that all contribute to the skills shortage they also mentioned a fifth but it didn't seem relevant to me so I'm going to leave that out of my my take on it the first that they list is industry growth clearly as more software is needed and we are to cope with this increased demand we can address that by either adding more programmers to write the code making programmers more efficient at writing it or finding someone else or something else to write it for us those are the only tools really at our disposal which are likely to have any effect and they're mostly the theme for the rest of this today the next barrier to growth in the list though is I'm not quite so sure I'd agree with that is the increasing complexity of software development I think that systems that we build today are certainly often more complex now but the tools and support for programmers are now dramatically better than they used to be so there's a tradeoff there for sure I'm also not con convinced that today's programming tasks are really more difficult than development was in the past it's certainly easier to build a basic crud system now than it was 20 or 30 years ago because of the tooling and the availability of examples I'd argue that the level at which software is more complex now than it was before is really more at the limits of performance and scalability and that's always been a fairly esoteric game for genuine experts building large scalable systems is easier and cheaper now than it used to be with the cloud and techniques like servus where I think there is some truth in this claim of increased complexity though is is different to simply saying that modern software is more complex and this is this difference really has two important parts to it one is that the scale today genuinely can end up being Global and the other is that we often shoot ourselves in the foot by not choosing sensible solutions that match the scale of the problems that are in front of us one way that systems were simpler in the past is that at least with the benefit of hindsight there was a kind of relative golden age of simplification we learned to be able to build simple basic systems fairly easily by following a very successful recipe the relational database back system relational databases introduced a simplified programming model that most people understood well enough to build basic systems I'd argue that the key here was in in dealing with concurrency in a way that made systems scalable enough but without relying on programmers needing a deep understanding of the intricacies of dealing with the extremely complicated problems of concurrency relational database Management Systems allowed systems to easily scale well enough to usefully work for hundreds and low thousands of users and through techniques like the ability to roll back transa transactions we effectively version controlled changes to data so that it didn't end up in a horrible un mergeable mess this meant that changes were safer and that you and I could be protected from the consequence of changing the same value in a database even though we were working in parallel with each other and changing the data in parallel so we ducked one of the genuinely tough problems in software the management of concurrent change to data as long as your system fit into this widely understood programming model development was reasonably easy at least compared to the Alternatives but if you did need to deal with the real consequence of concurrency for yourself rather than relying on the Excellence of the system programmers that built databases things were a lot more complicated what happened next was that we started to build systems that needed to scale much further than the few thousand concurrent users and the database model then ran out of steam it didn't work at those kinds of scales so people with bigger problems organizations like the huge web comp companies started to innovate with other more effective techniques that did work at this new scale these techniques were dramatically more scalable and became fashionable and famous in the software development World ideas like no SQL micr services and event-based systems spring to my mind the difficulty is that all of these programming models exposed us more directly to those really difficult problems that are at root to do with concurrency and scalability the problems that the rdbms is hid from us for a while software is easy as long as it's local single threaded and doesn't share data as soon as you introduce distribution concurrency and writable shared data though software is worldclass complicated and needs thoughtful deep expertise which is indeed rare actually I think that may be an interesting video for another day have we got the programming models wrong for the cloud let me know in the comments if you think that's an interesting topic so the two parts of the problem are that this increased scale of modern systems broke a model that had worked pretty well for a couple of decades and here's the really annoying part teams that didn't need to because they didn't have the problem of massive scale still brought in the much more complex programming model because that was what was fashionable at the time by choosing an approach that was based on the old-fashioned relational database architecture they may have found a better fit for their real problem rather than than trying to adopt the problems of global scalability for a system that wasn't if you're writing software for an internal use in a company of 10,000 people you don't need it to be globally scalable and the Technologies of global scalability come at a a cost they are more complicated to get right so yes modern software is sometimes more complicated than it used to be but often because we choose to make it more complicated because of fashion more than because it really needs to be the skills to cope with the complexities of leading scalability and performance have always been esoteric and rare writing a Flight Simulator game in the 1980s that worked on the hardware of the day or an operating system that managed concurrent processes wasn't easier then than building a global scale website is today it was just difficult in different ways partly because the other factor that added to the difficulty was that there was less help then than there is today nowhere to look up Solutions not really very much shared code at all to look at no stack Overflow no internet search developers knew what they knew from reading books and they had to remember it so let's not allow our Rose tinted glasses to undervalue how difficult software has always been to create which brings us to the next factor in the skills shortage a mismatch between education and Industry this was called out in the article knowing what to teach people to get them started in software development career has always been a difficult problem to solve but this is probably something that has got worse my impression is that software development has never been a very strongly academic subject I don't know if this is because of the nature of the subject but at least partly it's because of an accident in history I think and this is EXA exacerbated by the commercial imperatives surrounding software development and the need to create ever more software faster because of the constant pressure on everyone to deliver sooner what every employer is seemingly looking for is developing with proven experience of working on exactly the same type of problems ideally in the same industry using exactly the same tools as them it's no wonder that it's hard to find people so inevitably education is never going to produce graduates like that the result is that even with a good degree it can be very hard to find your first job and to gain that vital industry experience that everyone's looking for the step from graduate to Junior developer is a huge one I think that this is one thing that employers could do more to improve on the shortage of software developers that is invest more time effort and money to supporting people in their careers particularly people who are just starting out instead of trying to hire clones of your existing Workforce hire bright motivated graduates and invest in their learning develop in-house Apprentice like schemes this way you get to shape the next generation of developers to precisely fit your organization and teams as a civil engineer you don't get get to build a bridge alone on your first day out of University why do we expect the equivalent from software Engineers when I suggest this to organizations they often push back though with yes but as soon as we have trained them they leave for better jobs we'll come back to that topic later when we talk about competition for talented people but for now let's get back to the topic of education for software professionals software development is a difficult thing to do well and it takes a long time to learn to be good at it during in the early days of my career a large fraction of software developers were self-taught former hobbyists we cut our teeth playing with software doing all sorts of interesting sometimes silly silly things and learning a lot by from our mistakes if you'd like to see some of the sillier things that I did check out this very old video of mine I don't much like the description of software development as a craft but often when people say it's a craft I think what they really mean is that it's a creative thing and you learn to be good at creative things through practice more than Theory you don't learn to be good at writing or playing music without writing or playing music a lot this is true of other kinds of things too though as well as craft and I think that craft is only a rough approximation for what we really mean here this is just as true of engineering for example engineering is a deeply creative discipline too but it's different to craft because engine in engineering there are more effective guidelines that help help us to rule out some of the more dumb ideas along the way and so to make progress more efficiently and more effectively this is another way for us to make ourselves more efficient and so reduce some of the pressure of the skills shortage at present I don't think that university courses do enough to give people the core skills of our trade in this way though this is not really their fault though because as a trade we have not clearly identified these core skills and told the universities what it is that we need but I think that they maybe becoming clearer which is what my book modern software engineering is all about really there's a link in the description below to the book if you're interested in finding out more in universities I'd start by teaching problem solving skills our job is not to write programs it's to solve problems with software it's been my experience in employing graduates that there's a difference between the ability of graduates to get up to speed with professional development based on their course of study but it's not as you may suppose the difference between computer science and software engineering in my experience the the bigger difference is the difference between graduates with more formal Science Background like physics or chemistry who are better at problem solving the stuff that computer science and software engineering graduates have learned is helpful but it's the easier stuff to pick up on the job than these core tools of problem solving these are ideas like working incrementally in small controlled steps making progress by trying things out small experiments keeping a record of what you did so that you can easily step back and try again planning your next small step based on the data and evidence not just your hunches and working to control the variables so that you can more easily see what's happening when something changes and more broadly how to research a problem fundamentally how do you solve a problem when you don't know and can't just look up an answer I want I want to hire graduates with those sorts of skills these are the ideas that seem at the core of the skills of the best developers to me and these are the sorts of things that universities seem to be pretty good at teaching for other disciplines so while it's helpful to learn supporting theories and fashionable approaches that is not enough without lots of practice and the core skills of problem solving are much more fundamental and transferable not only from University into industry but from industry to to Industry in software development so for our profession to really grow we need to be better at teaching how to be a good software developer whether that is in universities on the job or more likely both the next in the list that the article called out was in the increasing competition for talented people when organizations push back on my suggestion of training inh house in reality they're pushing back against Market forces in in my experience good organizations actively train the Juniors who join them not just in their way of working but also in the broader principles of what it takes to be good at this stuff and yes after such training some people may choose to leave so find ways to help them to stay make the organization a good place to work support and help them to Foster their Ambitions and their career and even if you really must make contract Clauses that tie people in to stay for a time after their training is complete other professions do this sort of thing all of the time the most effective response to people leaving after training through though is to make employment in your organization good enough to appeal to and keep the people there now you have a motivated engaged Workforce so this is a double win I've seen this approach work several times and I've applied this approach myself where I've been in a position to do so to be honest though I don't really see any practical alternative because this is a Marketplace and talented people will be attracted to good organizations with interesting problems to solve so look for ways to foster both of those things the most common responses to the issue of De developer shortage though aren't much like any of these things and I I am somewhat skeptical about their ultimate Effectiveness it's obviously true that if the world needs more software we can either add more people with the skills to produce it or find ways to make the process of software production more efficient and that means easier this is really the corporate appeal of ideas like artificial intelligence code generation and low code Solutions but while both of these things can help a bit in some narrow fields of software development I don't believe that they are a general solution at least until AI is better at humans at everything and then they'll be running the businesses as well as writing the software as I've said before in both cases the current approaches seem to be focused on the problem being one of optimizing for Less typing but that misses the central point it is the problem solving that is the bottleneck not the typing professional software development is a process of exploration and Discovery and for that we need the mindset and the tools that allow us to try out our ideas and discard the bad ones not simply produce software based on bad ideas faster this is why I talk about the core of our skills being about the problem solving not the mechanics or even the detail of writing the code these problem solving skills are the core of our discipline and the sooner we recognize this and train people in those skills the sooner we can address both sides of the skill shortage being able to add to the pool of talent and make those talented people more efficient in the application of their skills thank you for watching and if you have been watching do consider adding your support to our team of patreon supporters who keep this channel going there's some great discussions currently going on on our Discord channel so do check those out if you join thank [Music] you

Channel: Continuous Delivery

Views: 172,394

Keywords: developer shortage, software developer shortage, developer jobs, engineering jobs, programming, software engineering job, get a software engineering job, software developer career, learn to code, software engineering career, computer science, Developer shortage crisis

Id: KNTWhNctGZ4

Length: 20min 12sec (1212 seconds)

Published: Wed Dec 13 2023