This image you’re looking at was
rendered completely on a TI84 calculator. No tricks, no “I rendered it on my
PC and copied to the calculator.” This is the real deal, a raytracing engine that
runs on a calculator that supports textures, reflection, HDR, gamma-correct rendering,
dithering, and global illumination. Now, most people familiar with my older catalog
of videos probably know that a while back, I already wrote a raytracing program on a
graphing calculator. But that one had a few limitations that I’d like to address. Mainly,
my first raytracer wasn’t all that fast. It took almost 6 hours to render this image
here, which works out to about .0000485 FPS, and less than 5 pixels per second. To be fair,
raytracing is not something that can easily be pulled off in realtime. Each pixel involves a
whole boatload of computations to determine what is visible and how it should be illuminated. Even
now, you need a beefy GPU to pull it off. But, even for the cheap little CPU in these
calculators, 6 hours for this is just pitiful. The reason everything is so slow is because
TI-BASIC, the built-in language I used, is interpreted. In short, the processor
in the calculator doesn’t run my program, it runs a program that runs my program. This
makes sense for a calculator, since the whole process is sandboxed, and a lot harder to crash,
but for my code, it means every math operation I do involves more work than it really needs to,
and this overhead adds up. TI-BASIC also limits you to 15 colors over a fraction of the full
screen, and probably the biggest killer for a project like this: TI-BASIC only lets you use
single letter variable names. There are only 26 places to store values in the program, and the
code is practically unreadable because of it. This time, instead of TiBASIC,
I’ll be writing this project in C++ using the toolchain provided by Matt Waltz and the
other contributors to the CE Programming project. C is really nice compared to BASIC. It’s compiled
so the programs run directly on the CPU, code is a lot easier to document and reuse, and I can
poke around in memory basically anywhere I want, including writing to every pixel. Best of
all, I can write and test code right on my computer through emulation, which has the added
bonus of being faster than the real hardware. After taking a look at some of the sample code
in the toolchain repository, I figured the best way to get started would be to rewrite the
original version of the program, so I more or less ported it over. Here’s a fun fact, did you
know that these calculators have a 32,768 color 320x240 display? That’s on par with the Super
Nintendo, and a whole lot better than the TiBASIC graphical specs. Even this recreation of the old
program already looks a whole lot better than the original. As for speed? Well, the old version
took just under 6 hours to draw this image. The new version takes about 3 minutes. That’s
nearly 400 pixels per second or an 80x speedup. It’s a great start, but this video would be pretty
short if I just stopped here. We can go faster. At its core, raytracing is really just a bunch of
math to model how light travels through a scene, and to do all that rendering math in my
code, I’m using floating point numbers, which are like scientific notation but for binary
numbers. Floats are a lot better than integers for general purpose computation since they can work
on really big numbers and really small numbers, but the tradeoff is that generally math
operations on floating point numbers are a lot slower than integers, and on the Z80 processor
inside the TI84 this is definitely the case. There aren’t any hardware optimizations
for floats like you would find on any modern processor or GPU, which means that
even for something as simple as adding two numbers together: integers take 2 CPU
cycles, and floats take a few thousand. One of the reasons floats are so expensive is
because you have to account for the fact that the decimal in two floating point numbers might not be
in the same place. Before we can add two numbers, we have to make sure all their digits are lined up
properly, but if we can be sure that the decimals are always in the same spot, we can treat these
numbers like integers. This approach is called Fixed Point, since instead of floating around, the
decimal point is always in the same place. A great example is US Dollars. As long as we assume you
can’t have a fraction of a cent, you can write out any amount of money as an integer number of
cents rather than a fractional number of dollars. Fun fact time again! I actually lied about the
processor that these calculators use. I said that they use the Zilog Z80, which is an 8-bit
processor from all the way back in 1974. Well, that’s partially true. Up to and including the
Plus C Silver Edition, everything in the TI84 line was using a groovy 70s-era architecture, which,
fair enough, for what they do these calculators don’t need to be all that powerful. But, the model
I’m using here, the 2015 skinny-legend TI84 Plus CE finally gave the CPU an update. Now, it has
a Zilog eZ80 under the hood. A whole lot faster, plus a better instruction set AND 24-bit wide
registers. Yes, you heard me right, 24-bit, not something more typical like 16, 32 or 64, this
thing is 24-bit. Your guess is as good as mine. Anyways, I can make my fixed point numbers have
24 bits, 12 below the decimal, and 12 above. This can represent numbers as large as 2047.999,
and as small as .0002, not perfectly precise, but good enough for graphics. Plus, these
fixed point numbers are way fast. I was able to render one test scene in less than
a quarter of the time! An improvement to be sure, but still, not good enough…
Besides speed, one thing I really wanted to improve in this raytracer was the complexity
of the scenes it draws. The first one had a reflective sphere, a tile floor, and a
single shadow, all centered in the view. Simple scenes like that can offer some
mathematical shortcuts, but this time I want to make things more complicated. So let's
add a few walls, a ceiling, and another sphere. The whole idea for ray-tracing is that we take
some sight ray, and figure out what objects in the scene it intersects with. Depending on the answer,
we shade that pixel in the output a certain way. But, if we have an efficient way to find out
if a ray intersects something in the scene, we can use this information to figure
out other ways light is traveling. For example, if there’s something in the way of
a ray pointing from a surface to a light source, we know that that point will be in shadow.
There are two other properties of light that we’ll also want to model here. The first is Lambert’s
Law: light intensity on an object falls off the more a surface points away from the light source.
And the second is the inverse-square law, light intensity falls off the further away you get from
the source. The two of these have to do with the angular size of the surface from the perspective
of the light source. If we assume the lamp gives out equal energy in all directions, the bigger
the solid angle our surface takes up, the more light it’s gonna reflect. Turn the surface away
from the lamp, and the solid angle gets smaller. Move the object twice as far away, and now it
takes 4 times as much area to take up the same angular space. In both cases, the surface
will get less light, and appear dimmer. When you put these two effects together with the
shadows, things already look a whole lot better. But, computer renders have this awkward
tendency to look a little too clean, and you can’t get much cleaner than a room with
two perfect spheres and solid-color planes, so, let’s look into texturing. All we have to do is
to take an existing picture, and to project it down onto an object in the scene when rendering.
In our case, whenever we’re rendering a pixel on the floor, we’ll want to use the color from a
pixel in some image like these hardwood planks. Since the floor is an axis-aligned
plane, that task is pretty simple: just take the x and z coordinates where
we hit, use those to pick a pixel in the texture, and then interpolate between any
neighboring pixels to hide any jagged edges. But here’s something else that doesn’t look
too great. If you look closely at any surface, you’ll see bands of color rather than a smooth
gradient. As unnecessarily good as the display hardware in these calculators is, we are still
bound by 15-bit color. In the red, green, and blue channels there are only 32 different levels
of brightness to choose from, and so in areas where color is changing slowly, the transition
between the two closest colors stands out. Thankfully, dithering is an easy fix to this
problem. By mixing together a few pixels of each color along a transition, we can get the illusion
of a bigger palette at the expense of a little pixel noise. There’s no shortage of dithering
algorithms to choose from depending on if you want to prioritize speed, image quality, or noise, but
here, I’m doing arguably the simplest technique. Taking whatever error there is from the last
pixel, and just adding it onto the next one in the row. Even though there’s not much to it, this
technique helps a lot to smooth out the colors. While we’re on the subject of color, I gotta
confess, my renderer still is a little bit entirely wrong. My code all operates on the
assumption that light intensity in the renderer scales linearly with the RGB value I set the
pixels to. But in the real world, that’s not the case. There’s a great MinutePhysics video talking
about why this is, but long story short, standard RGB values are grouped so that darker colors
are very close together in terms of brightness, but brighter colors are more spread out. If
we’re assuming a linear mapping from actual intensity to sRGB, things are going to look
darker than they should. There is a conversion function between the two, but, well, that looks
hard. So instead, I added it as a lookup table. The nice thing about the lookup table, is
that I can use it to add Tone Mapping for HDR at no extra cost, by adjusting the
values according to this function. Now the renderer preserves detail in
areas that would normally appear too dim, or so bright that they look washed out.
Accurate color representation goes a long way towards photorealism, but there’s still
one glaring issue with this image. Anything in shadow isn’t just dark, it’s pitch black, and
in the real world, you don’t see this very often. Even if something isn’t being lit directly
by the sun, light can bounce off of something else into the shadow. This is called global
illumination, as opposed to direct illumination, and it shows up all over the place. Under trees,
along sidewalks, inside a room on a sunny day. But compared to direct illumination, global
illumination is a lot harder to simulate. Instead of just one or two light sources, the illumination
an object receives comes in from all directions. Professional renderers generally use
random sampling, since if you take the average of enough light samples, you’ll
eventually get close to the right answer, but, since I’m not in the mood to wait a week for
the render, I’m gonna use a different technique called radiosity. The radiosity algorithm computes
a lightmap for each object in the scene, which is like a texture that keeps track of how much
light is hitting the object at a couple points. First, we get the direct illumination for
every patch, and then we simulate light bouncing around the scene by adding up the light
contributed to each patch by all the others. Repeat that enough times to account for
the light paths with multiple bounces, and we’ll eventually converge on a more
accurate representation of how the scene is lit. I still render direct illumination individually
for every pixel, since the resolution of the lightmaps make shadows look a bit chunky. But,
I can also turn off the direct illumination completely and render the scene with just light
that’s reflected off the walls. One effect I think helps with photorealism is color bleeding.
You can see in the global image that the red wall contributes a bit of red light to everything else,
and so does the blue wall and the wood floor. When we look in the fully rendered image, this
means that the spheres have a bit of a colored tint on the sides. And this is something we can
also see in the real world. If a brightly colored object like this box reflects a lot of light,
things around it will have a similar color. It’s a subtle effect, but once you know
about it, you’ll see it everywhere… For one last cherry on top, it wouldn’t be
a raytracing demo without a mirror ball, so I threw together some code to reflect a
sight ray off of the one sphere, and then render the scene from that direction. This is one
of the things raytracing gives almost for free. We already have all the code necessary
to render sight rays from the camera, all we have to do now is change
the direction of the ray. And, there you have it. A raytraced scene with
reflection, global illumination, HDR, gamma correct rendering, point lights and textures, all
still on a TI84 calculator, in case you forgot. For fun, I tried my best to recreate the scene in
Blender and did a quick render on my computer to compare. For the most part the calculator isn’t
that far off. I noticed in some areas of the image, the Blender illumination is a bit different
from the calculator’s, notice how the mirror ball is dimmer here, and the back wall has a bit more
red reflecting from the side. It’s not perfect, but remember this is still night and day compared
to the graphics this calculator normally draws. On my computer, this render took a couple
minutes to perform using a modern multicore processor, but, how long does it take for the
calculator? Get ready for the quickness... 14 minutes. Granted, the calculator is performing
a simpler algorithm than Blender, but time-wise, we’re in the same ballpark. The original version
took 6 hours for a lot less. This version runs at roughly 90 pixels per second which is almost
20 times faster. Altogether we’ve got better results in less time. I’d count that as a win.
As always, there are still ways it could be improved. When writing some of the more complex
fixed point functions, I got a little lazy and just used the floating point library
instead of writing it myself in assembly. I mean, I wrote my own fast multiplication and
squaring routines in ASM since those get used pretty often, but for something like square roots,
I only use it a few times per pixel, so garbage like this is good enough for me. There are a few
redundant calculations my renderer performs which could probably be stored and reused, mostly
during radiosity. Cutting out all this work would make things faster, but, there are memory
limits to consider, and what I have works for now. It’d be really cool to add support for triangle
meshes to render more complex shapes, but that comes at the cost of way more compute time per
pixel. Other forms of lighting like directional light and surface lights would also make for
some interesting renders, as would refractive materials like glass. I might get around to it
in the future and give this video a sequel (no… three-quel? idk) so uh, subscribe or whatever.
But, if you’re the impatient type, the source code is all going up on Github, and you’re more
than welcome to add these or other features. I’ve already spent enough time on this as is.
So, in the interest of getting this video out, I’m gonna have to be happy with what I’ve written.
Ti84 calculators can now do raytracing faster, and with more photorealistic results. I don’t
know about you, but I for one am excited that we’re now one step closer to real-time
raytraced Minecraft on a calculator...
