How do Systems on a Chip Work? Understanding Microchips. By. Branch Education Your smartphone has a variety of components that can do dozens of different things. and the brain at the center of all of these components and functions is the System on a Chip, or SoC for short. When you watch a video such as this one, phone a friend across the country, or navigate through a city using GPS, it may seem like magic, or alien tech, but in reality these feats of technology are all performed inside this SoC which is designed and manufactured through the hard work of countless scientists and engineers.Ā In this episode weāre going to explore each section of this SoC and then piece together the processes this system on a chip goes through when you do something as routine as snap a photo and post it online. These chips are unfathomably complex as there are between 5 and 10 billion transistors that make up the SoC and they all need to fit into an area the size of a penny. This is a complex topic, and there are many avenues that we could explore, but in order not to overload your brain with details, weāve decided to focus on the following aspects. First, weāll start with a few notes and caveats, and then weāll dive into the layout and key sections of the system on a chip. Next, weāll explore how data flows through the SoC using the example of what happens when you take a picture. Then weāll look at some of the complexity inside the central processing unit or CPU block of the SoC. And stick around until the end where weāll discuss how scientists and engineers design and manufacture these microchips, and weāll explore what a nanoscopic view of the SoC would actually look like compared to this visualization. So, for this video, grab a snack, sit down, and maybe take some notes if you feel like it because this is going to be one very thorough and incredibly detailed video. To start, this video is sponsored by Gerber Labs, a quick, reliable, and user-friendly provider of printed circuit boards. Next, this microchip is actually composed of two key parts. On top is the DRAM or the dynamic random-access memory, which is the working memory of your smartphone, and on the bottom is the SoC, the brains of your smartphone. FYI, the long-term memory, where all your music, apps, files, and the operating system are saved, is over here in the flash storage. We have a separate episode on that microchip, and you can find it here, but in this episode weāre going to focus on the SoC. This setup is called a Package on Package microchip or PoP Technically this is the microchip, and the separate layers of the DRAM and SoC are each called a die. However, the word chip has evolved to mean a variety of things so, while all of these rectangular components are microchips or chips, both the DRAM and system on a chip are also called chips even though they're just half of the overall package on package microchip. Moving on, we may make comparisons or analogies between smartphones and humans; for example we called the SoC the brain, or the camera can be considered similar to a human eye- however these are two very, very different systems, and while the analogies may be useful conceptual tools, the underpinning principles behind SoCs and human brains and how they work are entirely different. Finally, the companies that design these SoCs are incredibly secretive about the specifics of how they work and what the actual designs look like. It takes dozens of different companies to design the optimal layout for billions of transistors to perfectly execute the functions of a phone, and a separate set of companies to manufacture the billions of chips used by all our phones. So, in this video, weāll provide you with the best information we can, considering the companysā secrecy of intellectual property. As a result, this information will be mixed and woven together from different sources. For example, this is an LGV10 smartphone from 2015 and it uses a Qualcomm Snapdragon 808, but due to the limited availability of information on it, weāll have to use images and information from the Apple A12 and A13 microchip in the IphoneX and iPhone 11, as well as the Qualcomm snapdragon 865 in the Samsung S20. All of these microchips have different performance metrics and different capabilities, but fundamentally they work on similar principles. Okay, with all these notes covered, letās move on to understand how this System on a Chip works. A human brain has different sections for processing information received by your eyes and ears, a different section for speech, ones for controlling your taste, smell, movement, balance, involuntary functions, and so on. Rather similarly, this System on a Chip is divided into different functional areas. Here is a graphical representation called a block diagram. The key areas in the SoC are: the CPU containing multiple cores; the GPU which renders graphics; the shared memory cache which is usually around 4 to 8 megabytes; the digital signal processor which interfaces with things like the speakers, microphone, sensors and many other things; the display engine which communicates with the touchscreen display; the video processor which compresses and decompresses images and video and enables 4k video recording and playback; the image signal processor which processes pictures and video taken by the cameras; the modem which interfaces with the various wireless networks; the storage controller, which saves and loads information from the flash storage microchip; the memory controller which connects to the DRAM microchip located on top of this one; the security enclave, which executes encryption, and manages the public and private keys; a number of peripheral functions such as clocks, temperature sensors, debug ports, and general purpose inputs and outputs, an always on microcontroller unit; and finally the network-on-chip or NoC which arbitrates or manages data flow through the SoC, the DRAM and other parts of the phone. Furthermore, some SoCs may have on board power management circuits that complement the power management performed by separate chips outside the SoC. Additionally, some SoCs such as Appleās, have a neural processing unit or an NPU that is specialized circuitry that can execute machine learning algorithms far more efficiently than the CPU, both in terms of speed, and more importantly, power consumption. One confusing detail is that different generations of SoCs use different marketing names and mix multiple functions into different functional blocks. For example, hereās the block diagram of the snapdragon 808, versus the diagram for the snapdragon 865 and hereās the block diagram for Appleās A12 chip. Each companyās marketing team may decide to provide differing block diagrams to try to impress the consumers, which at times can certainly be confusing. Letās move on and explore a real-life example of how data moves around your smartphone when you take a picture. But before that, one quick note.Ā If you enjoy watching this video, or if you think itās a useful learning tool, make sure to like and comment on this video as well as share it with others.Ā With your help, we can get this video to over 10,000 comments and more than 100,000 likes.Ā But for now, letās get back to understanding what happens when you take a picture on your smartphone. First, the photons from the scene enter the cameraās lenses, flow through a color filter array, and hit the sensorās photodiode pixels. These color filtered photons are then absorbed by each photodiode and converted into an analog electrical current which is then converted into a digital 12-bit binary value. A 12-megapixel cameraās image has 12 million of these 12-bit binary values. Note, that each pixel is either Red, or Green, or Blue, and thus the overall image is considered raw and still has to undergo a number of image processing steps to turn it from a raw image into a recognizable picture, where the pixels each have a red, and green, and blue value. But before these processes take place, this image data must be stored somewhere, and as a result itās sent to the smartphoneās working memory or DRAM. To do this, the data travels from the camera and enters the SoC through a mobile industry processor interface or MIPI which can send or receive data at around 5 to 8 Gigabits per second. This image data is then routed by the Network on Chip arbitrator through the SoC to the memory controller and into the DRAM, which, as mentioned before, is located on the die above the SoC. There are two quick things to note: First, the data path between the SoC and the DRAM is shared by everything. In this scenario itās the job of the network arbitrator to prioritize the incoming sensor data of the uncompressed raw image so that no data from the camera is lost, and thus the arbitrator streams the raw image directly into the DRAM. Second, a 12-megapixel uncompressed raw image takes up around 24 megabytes However, the memory cache on the SoC is typically 4 to 8 megabytes and is shared among all the processes on the SoC. Therefore, when the image signal processor or the video processor works on a recently taken picture in order to make it viewable or to compress it, they can only work on small subsections of the overall image, leaving the entire image temporarily stored on the DRAM. This also happens while watching a video, playing a video game, or using pretty much any app. Okay, letās get back to the raw image data of the picture that was taken by the camera, and then sent to the MIPI routed through the SoC, and stored in the DRAM. The next step is for the image signal processor or ISP on the SoC to read the raw uncompressed image data and perform a number of image processing steps. These steps involve first correcting for darker pixels on the edge of the sensor due to lens shading. Then the ISP performs the demosaicing process which involves taking the image data and the pattern on the cameraās color filter, and calculating a red, green, and blue value for every pixel. Next the image signal processor denoises, sharpens, enhances the image, and color corrects it because the red, green, and blue filters on the camera donāt directly match the hue of the red, green, and blue pixels in the display, and finally the image signal processor tone maps the image. After all this we have a picture with 12 million pixels, each with an 8-bit red, green and blue value. To get the picture to promptly appear on your screen another series of steps are involved: the RGB image data is taken from the image signal processor, sent to the GPU where it gets overlaid into the graphics of the camera app and scaled to fit the screen The resulting RGB values get sent to the display processor, and then the image is routed to the display where itās converted into intensities of current in order to light up the corresponding pattern of red, green and blue pixels. And there you go- a picture taken by the camera and whoosh! displayed on the screen. But weāre not done yet.Ā How is the picture saved on your smartphone? Well, first the picture needs to be compressed, and to do that the uncompressed image currently residing in DRAM is sent to a dedicated video coding processor where it first gets converted from red, green and blue values into YUV, or luminance, blue chrominance, and red chrominance values. Next, this data undergoes a series of algorithms in order to remove information that is undetectable to the human eye and compresses the image into approximately a 3-megabyte JPEG format. This compressed image is sent back to the DRAM, and then routed to the smartphoneās flash memory for long term storage. Now, if you were to send this picture to a friend, the compressed image would get brought back into the DRAM, and then routed to the modem where itās divided and assembled into packets, then sent to the 4G, 5G or Wi-Fi microchip, converted into electromagnetic waves and sent to a cellular or Wi-Fi network. One thing weāve talked a lot about is the movement of data through the SoC. All of this data movement between the blocks runs over connecting wires on what is part of the network on a chip. This network has routers and switches for shared access to routes and targets like the DRAM. The routers and switches arbitrate the flow of data and act like digital traffic lights and the pathways act as a digital highway. These highways have different widths of wires based on what theyāre talking to and the amount of information that needs to be sent.Ā
Depending on required data rates, there are typically 128 or 256 wires running parallel to one another, carrying one bit at a time, and operating between 500 and 1,500 Megahertz. In order to avoid wasting power from your smartphoneās battery, the frequency and resulting data transfer rates across the network on a chip ramp up and down depending on the requirements of the applications youāre running and this feature is called dynamic frequency scaling. Before we explore the CPU and see how these chips are designed and manufactured, letās take a quick break and talk about this videoās sponsor, Gerber Labs. Every SoC eventually makes its way onto a printed circuit board along with other components that are put together to create a vast array of different circuits. You can start your own creative circuits by building a printed circuit board from Gerber Labs. Gerber Labs is founded by electrical engineers for electrical engineers. They make it easy to get boards with consistent quality, quick turnaround, and reliable shipping. Check them out with the link in the description and stick to the end of this video to receive a discount code for your order. Okay, so, thus far weāve glossed over the functionality of each of these blocks in the SoC, but now letās quickly take a look into the central processing unit or CPU section of the SoC. As you see there are multiple cores, and each core can run part of a program by executing instructions. If we focus on just one of these cores, we see that inside it there is another incredibly complex set of blocks that depict the different functional sections and the data flow between them.Ā There is a ton of information in this block diagram, and weāre planning on making an entire series to explain it all, but for now, here are the CPUās memory caches. The instructions such as add, multiply, load, store, compare, jump, and many more, flow this way, and the actual data being processed flows this way. And then over here are the blocks that actually execute the arithmetic, branching, and storing of data. Letās move on to a few additional details about this processor. All smartphones use a reduced instruction set computer, or RISC architecture, and almost all of these architectures are licensed from a company called ARM which stands for Advanced RISC Machines. Some companies like Apple license the instruction set architecture or ISA from ARM and use this ISA to design their processor cores in house whereas other companies like Qualcomm license complete blueprints which are technically called intellectual property cores from ARM. Qualcomm then integrates the ARM IP cores into its SoCs, sometimes keeping the design of the core as is, but more often modifying or customizing the ARM core to better suit Qualcommās design needs and, in the process, Qualcomm rebrands the name of the CPU. However, all this information still begs the question of, how do these SoCs get designed and manufactured. Instead of talking about the entire design process which you can find in the creatorās comments, weāre going to discuss two design principles of the SoC. The first principle is called hardware acceleration, and this principle deals with the fact that instead of having a single very powerful general purpose CPU with a lot of cores, this chip has a variety of special purpose blocks dedicated to performing specific functions. These special function blocks, or hardware accelerators, compute their tasks faster while using significantly less power than if the same operation were performed by the general-purpose CPU. For example, as we saw with the processes involved in taking a picture, there is an entire block of the SoC that's dedicated to processing the image, and then a separate block whose job it is to encode, decode or compress or decompress the image, and, if youāre watching this video on your smartphone, itās actively decompressing this video as you watch it. These sections are useful for taking pictures, but critical for recording 4K video, and saving battery life. Hardware accelerators are utilized for pretty much all computationally-intensive tasks performed by your smartphone The next design principle is that, because SoCs are used in smartphones, which operate on a battery, there's a huge focus on having the SoC consume as little power as possible.Ā One example of low power consumption design is in the CPU.Ā Instead of having all high performance cores, chip designers often follow a big-little design structure, where there are 2 or 4 big cores that are high performance cores but consume more power, andĀ 4 little cores that are lower performance
but are energy efficient, and your smartphone prioritizes using these lower performance energy efficient cores when possible. Furthermore, the design of the transistor, which is the smallest and fundamental building block of the SoC, has had its design evolved to both be smaller, but more importantly to consume less power.
As of 2020 the latest technology of transistors is called a gate all around field effect transistor. Letās finally move onto how these chips are manufactured. To start, the lionās share of all SoCs get manufactured by a single company called TSMC, or Taiwan Semiconductor Manufacturing company, and a smaller, yet increasing proportion of SoCs, are manufactured by Samsung. These microchips are manufactured on 300mm wide silicon wafers in factories called fabs.Ā In order to manufacture an array of chips, the silicon wafer has to go through a series of 120 to 160 processes or steps performed by dozens of different machines. This is the most common sequence of steps
for manufacturing microchips, As you see, there are only a dozen or so steps, but each of these processes are performed dozens of times in order to build an incredibly complex layout of transistors. But this video is getting too long, so weāll have to save the details of each step for a future video. For now, hereās what a nanoscopic view of the SoC looks like.Ā On the bottom is the silicon wafer, and the billions of transistors are implanted and built into and on top of the silicon. On top of the layer of transistors are layers of local interconnects that connect transistors to one another, and then on top of that are the global interconnects that connect sections of transistors. This is just a small fraction of the entire SoC layout. Let's zoom out in order to get a sense of the size of these components. I'm sure you've heard that transistors are incredibly small, and, Well, here's a single grain of fine table salt. Mind-blowing- right? It makes you thinkā¦ the potential of humanity to reach incredible levels of science and engineering is limitless. That pretty much sums it up for this quick overview of the SoC. We know we covered a lot, but we only just scratched the surface of how this semiconductor chip works and how they're designed and manufactured. As usual, there are more details in the creatorās comments which can be found in the English Canada subtitles. Again, it would help a ton, if you could take a second to like this video, subscribe to the channel if you havenāt already, comment below- either ask a question, or tell us what your favorite part was, and most importantly share this video with others. Weād also like to thank Gerber Labs for sponsoring this video. Get 15% off your order using the code Brancheducation15. There will be a link to their website in the description below Finally, weād like to thank all of our YouTube membership and Patreon Sponsors for helping us to build these in-depth engineering explanations. Thanks for watching. Always remember to consider the conceptual simplicity, yet structural complexity in the world around you.
My mistake was not getting a snack when the dude told me to. Great video! I recently started looking at apples new M1 chip and they briefly mentioned here that it uses the ARM instruction set. Computers are fun!
Watched the whole thing. Thanks for sharing!