As many of you guys may have noticed, my
videos have been getting less common over the last several months. Nothing’s
wrong, I’ve just been deep diving into the world of VRChat with every
waking, non-work moment of my life. At some point over the last two years, scrolling
over the pages of DigiKey, Mouser and LCSC, looking at all of the out of stock electronics,
it just became a lot less interesting to me. And this platform that I had just begun to dip
my toes into became much, much more captivating. I wanted to share with all of you
guys some of the most compelling, entertaining and intellectually moving themes
I’ve found in this online universe. From being able to meet other people like me to being able to
hang out in a virtual K-Mart with my mom, staffed with role-playing employees - while she was on
the other side of the country, to exploring the depths of what’s possible when thousands of smart
people are given a true sandbox to run wild in. If all you know of VRChat is Ugandan Knuckles
and Frederic Blue, I’d really like to show you some of the corners of this
cyberverse that I frequent. Strap in losers. We’re about to
go to Treehouse in the Shade. Treehouse in the shade is a VRChat map written by
1001 and SCRN. Unfortunately 1001 has passed away, but SCRN is still here and even active
in making shaders and doing ridiculous things with their avatars (more on that later). Everything in VRChat is a unity asset. And the
restrictions placed on what those assets can contain are very loose. W hich means people
have been able to write shaders which can be part of avatars and worlds. For those not in
the know, shaders are small programs that run on your graphics card instead of your CPU,
letting users customize the appearance of any 3D asset programmatically sometimes
just to give a different look and feel, sometimes for much, much more. They’re
naturally sandboxed, and bonkers fast. Anyway, there’s a website called shadertoy,
which is where people from around the world can work on shaders that and it runs inside of
your browser. 1001 and SCRN went through a lot of these programs and painstakingly adapted
them for VR. So when we go to this map, we see all of these beautiful scenes made of
these programs. In this map the only things made out of triangles are the players and the
treehouse, itself. Everything else is an implicit mathematical function programs that are running
on our GPUs. No modeling, no texturing, just code. Shadertoy is designed to operate in a little
window in your web browser. But, in VRChat, you can come to experience these things almost
first-hand. Have a sense of presence. While just little functions, the shaders feel real. Even
if you aren’t visiting VRChat with a VR headset (you can use just desktop mode), just being
able to explore the space with your friends can be somewhat immersive. Being able to
fly around this treehouse in this virtual world and be able to dodge through these
columns. It’s just amazing. It’s so much fun being able to explore these worlds in
a way that’s so open with your friends. If you’re a Quest user, you’ll only
be able to partake in some VRChat experiences before you connect
your headset to your VR-Ready PC but, as soon as you do or if you’re already a
PC VR user, I think for just about everybody, Treehouse in the Shade needs to be
one of their first VRChat experiences. People have been exploring
digital social platforms since the 1970’s! Originally starting with text-based
Multi-User-Dungeons or “MUDs” as they’re called. Nerds from around the world took to
understand what it means to socially relate over a digital medium and that medium evolved
in many ways on many different platforms. Graphics evolved, from 2D to 3D, but in 2014,
VRChat realized that when you bring VR into the equation, you get something really special.
It’s an entirely new avenue of immersion, both for you and those interacting with you. Much like the VR headsets of today, VRChat looks
and feels almost nothing like VRChat from 2014. They’ve had seven years to grow and learn with
their userbase with first-hand experience. It’s difficult for me to explain
why this is so captivating. People have always said “it’s like you’re
there” with each successive technology (radio, recordings, video and HD) but
with VR it’s like you’re really there. Many other platforms have come and
gone, rising and falling in popularity. Some were very compelling just in different ways. VRChat, however, stands alone insofar as
how weird the decisions they made seem. No one would engineer a platform this way. No one
would plan this glorious mess. No architect would design a system like this, but from the inside you
can start to see they were the right decisions. As they’ve iterated, they’ve found how many parts
of “conventional wisdom” learned in meat space don’t apply in cyberspace. Instead
of defining the platform, they more or less let it be defined by the hundreds of
thousands of users that came to populate it. It’s a wild-west level of freedom. Instead
of a sanitized corporate view, it’s messy, organically grown, from just a few users, to over
50,000 concurrent users and many times that in daily and monthly users which has now formed
an incredible multi-million dollar economy. VRChat has opted to build a system that makes
absolutely no sense from outside - and they leaned into it. Why so many anime and cat girls?
Why are mirrors the single most important feature of any world. Why is the default home world so
barren? Why can users upload almost anything they want and call it an Avatar? (Yes, my friend
Savanna’s avatar was a photo booth for a bit). Why would you let users use any shaders they
want and apply any look and style to their avatar when it doesn’t mish mash with the
world and makes an inconsistent art style? Why are worlds and activities less
important than the nuance in avatars? Why does talking to humanoid models
sometimes feel a little weird, but talking to a fax machine
feels perfectly normal? Why can’t users create and upload avatars
without spending some time on the platform? These all seem like obvious mis-steps,
or answers that don’t make any sense. Some do have clear answers, like for the last
one - users can’t upload without spending some time in VRC because they need see and
understand this world instead of come in with preconceived notions. And for the rest,
when you finally actually dig into the platform you can start to see why their answers
to these questions make a lot of sense. It feels difficult for me to show you
what presence feels like in a 2D screen or why I think this is so compelling.
It feels like trying to explain to an alien why they would want to come to this planet. It’s the sites, the sounds, it’s the people,
the culture, the art, the glimpse into humanity at this time and space. And it’s that
ephemeral that you can’t afford to miss. For instance, with the loss of 1001, SCRN
doesn’t have the map files. So, at some point an update will happen to VRChat which will
render Treehouse in the Shade unplayable. The worlds you can experience in VRChat
are an evanescent artifact of this time and place in cyberspace. So, I urge
you. Come join me. Sooner than later. One counterintuitive aspect of social interaction
is the importance of who you are over what you are doing. When first visiting VRChat, I dreamed up
all of the worlds I could create that I could play with my friends in and share experiences that
I could author. But, that’s a lot of me’s and I’s. And, I’ve gotten into it more, I’ve come to
understand that while worlds and activities have importance, users find their avatars far more
important than I could ever have imagined. Worlds might have been what got me into
VRChat, but it’s not why I come back. It’s the people I’m there for. The
late-night conversations and connection. If I wanted to go somewhere for the “place”
I’d play space pirate trainer, beat saber, pistol whip or alyx. But, VRChat has honestly
been dominating the time I’ve been spending in VR. A big part of that is the avatars. With
little miis running around the world, you don’t get the same sense of authenticity of
the person you’re talking to as when someone is comfortable in their avatar. It’s important to
get more than a faint representation of a person. When avatar fidelity fidelity can increase,
that authenticity can begin to bloom. VR Renders unimportant the juxtaposition
in art styles that would normally clash. Instead of seeing that clash - you see something
elsewhere that’s much harder to overcome. It’s when people don’t match their avatars.
And I don’t mean physically. As there are services like readyplayerme that can generate
avatars that reflect individuals real world selves if they choose. In practice, though, this
choice of avatar mechanism is rather unpopular. It is an amazing experience to talk to
someone who practically becomes their avatar. Whether they’re a bunch of barely
connected glow sticks, a nargacuga, beluga whale, hyena or one of the many cat girls. Not that avatars are even locked in. Even if it’s just for a moment going on another
level deeper and putting on a different character, or putting a costume on your current
avatar for halloween. It just matches. The rationale for why avatars matter so much
is drastically different for everyone. For some it may have its roots in dysphoria to
which people can find respite in VR. For others they may have technical
interests in what can be done on avatars. For most, I think it’s just … fun. The Italians figured this out in the
16th century with their masquerade balls. There’s something just FUN about
exploring other fictional identities. Ones that don’t reflect your body or even
within our sense of reality altogether.
The tests have been run time and time again,
with shockingly popular “mirror worlds” as they’re called, it shows that people care much
more about their avatars. Many of these worlds are nothing more than a mirror so people
can see their and their friends’ reflection. Mirrors are an incredibly curious phenomena.
Nothing about VR particularly screams “MIRRORS WILL BE IMPORTANT.” But in VRChat,
this phenomena has promulgated the platform. There’s a lot of theories behind this -
why “mirror dwelling” is so pervasive. I don’t have any hidden knowledge on the topic,
but I do like some of the theories I’ve heard. Many avatars include chords to be used with the
user’s fingers. Controllers have a lot of input, and this enables users to puppet their own faces
by moving their fingers around. This enables them to be much more expressive than visemes
alone, where the mouth moves to the user’s voice. Mirrors can provide these people a
physical kinesthetic sense where actual proprioception doesn’t exist. The users can
see themselves how others around them see them and can close the loop on their expressions. Some avatars provide a HUD which includes
a mirror so the player can see themselves. Another idea is that mirror augment the user’s
FOV, enabling users to see much more of their surroundings than they can see normally using a
headset. While you don’t get as much peripheral vision when you’re in a headset, when you
look at a mirror, you can see all of your friends at the same time while chatting with
them, not just those sitting across from you. Mirrors also add to realism for many through
phantom touch. For many, by seeing others around them interact with their avatar makes those
interactions, that touch feel even more real. For some people they’re just kind
of fascinated with their own avatar, it’s something they really love and are
comfortable with. By spending time in the mirror, it can sort of help that familiarity with
the avatar as being an extension of them. Idk? Whatever reason it feels
like people stick mirrors in bars. Another interesting area is full-body tracking.
Many people aren’t satisfied with the limitations of just having 3 point tracking (Head, and two
controllers) and instead opt for 6 point tracking, including the waist and feet, so that they
can have their avatar’s body match their body. Some people just just recline in a chair,
perambulating around without having to get up and walk… There are entire communities
of dancers in VR who just dance in VR. It can be a ton of fun just to let loose and go
wild with your friends dancing at a DJ party. Others infinity or “RP” or “Role Play” walk
by using their play space as their real-world play area. When they need to walk outside of
the bounds of their real-world play space, they rotate the mapping of their playspace between
the virtual world and the real world. By rotating, say, counter-clockwise, and walking, in a
clockwise circle, they end up walking in a straight line in VR and not be limited by
the space constraints of their play area. Avatars are highly customizable. Able to
use custom armatures for interesting motion, unique shaders to give it an artistic field,
and gizmos to show their friends or to use when navigating virtual worlds or make their visit to
these virtual worlds a little more interesting. For instance with the advent of AudioLink
by Llealloo Avatars can get audio input from the world and can have visual effects on their
avatar respond that to the the sound, so that the world’s visual effects and the avatar’s
visual effects can be synchronized. The fidelity of and what gets included with
avatars is highly dependent on the person. Many mutes add drawing to their avatars, people
who frequent the club scene may purposefully shoot for more jarring avatars. Furries
sometimes have effects to look extra plush. People within the flourishing deaf community
frequently have more articulate hands to be able to sign to one another or
have other accessibility features. Overall it’s remarkable to see what
happens when people are given the ability to express themselves to the
prodigious degree that VRChat enables. A shockingly large number of people have
installed blender and unity and embarked on a journey learning to take a base model and tweak
it. Adjusting textures, armatures, and appearance, reticulating splines and tweaking out. Or
starting to upload their first VRChat world. Very few people know how to do it all. Many are
just interested in some aspects and pay artists, one-off to work on their avatar in some areas
they may not be proficient in, themselves. Or maybe just learn to use Unity to add
assets they buy off of booth and gumroad. There is a massive industry surrounding content
creation in VRChat. Filled with artists, programmers, modelers, and just about everyone
else. With stores like booth selling everything from little crowns or masks to high-end digital
digital SLR cameras that actually function. To full on upload-ready avatars on
gumroad. The level of content creation is unbelievable. It’s ridiculous just how
many people are making VRC Content creation their full-time job. Even in spite of there being
large worlds where all the avatars are free. Dynamic Bone for instance, an add-on used
to provide a sproingyness to armatures went from a backwater tool to the best selling
addon on the entire Unity Asset store. The VRChat creator economy is booming. Many creators just learn these
tools to work with their own avatar. Getting a base model, adding the things they
want, picking a nice toon shader like poiyomi. Slide a bunch of sliders, tweaking
their avatar, to make their it theirs. Other folks, like me! Started with a strong
programming background and some shader experience. So, I found my base model, and paid an artist
to some of the more detailed modeling work that I don’t have an eye for, myself and used this
as a basis for all sorts of neat experiments. The community I normally run with are the shader
people, so the things that fascinate me the most are the flagrant use (And sometimes
misuse) of constructs to accomplish interesting technical feats. This is just a
tiny cross section of a much larger community, but it’s the one that interests me the
most, so I’m going to talk about it. Artists with no shader background explored
shader coding so that they can take their current avatar and add something really
special or cute like this shader bracelet. Professional geologists with
no game dev background who jump in and learn about everything from
shaders to the inner workings of unity, building content like an avatar that
transforms into the tram from half-life and going onto providing the
community with some incredible tools. Programmers who skip generation content
altogether for some of their avatars, like using a mathematically
defined signed distance field bee, avatars which transport anyone
nearby to their strawberry world. Some people with virtually no background in
programming at all before coming to VRChat were able to learn about shaders
and C# coding from the community and then share their first programming
projects with their friends. People who just wanted to build a ball pit
or pool as part of their avatar that they and their friends could play in, wherever
they went. And the impact they had on me, laying the groundwork for my
world, “CN’s ballpit and lofi.” Remember SCRN, the co-author of Treehouse in the
Shade? Well, they’ve been leveraging those shaders to run AIs on avatars on the GPU. Everything from
a chess engine with AI built out of nothing other than surfaces with render buffers and cameras
pointed at them, a robotic AI dog “Tupper” that can perform voice recognition by looking
at a representation of the player’s mouth, a transformer english-to-japanese translator,
to full on multi-layer convolutional neural network object classifiers that can use
a camera in the virtual world looking at objects and classifying them which bake the
entire neural network down to a 2D texture. The running gag is we, as a species
have made rocks to think. To be fair, we flattened them out and shot lightning into
them first to make them our computer chips. Well, SCRN takes it to another
level, making these thinking rocks in our computers imagine a whole new
world, where they take imaginary rocks, flatten them out, shoot lightning into
them, and made those metarocks think. Though most of my focus has been on avatars. There
is still a vibrant world creation community filled with incredible people. There are breathtaking
worlds some of which can unfold into deep storylines. Some have enormous care put into them
to make them extra social. And others yet have probed the depths of just how cozy can something
digital feel? But for me it’s the technical! Worlds are given a much more powerful
set of tools to work with than avatars. For instance, there is no mechanism
to execute normal code on the CPU with avatars. But worlds get access to much of the
Unity API and exposure to the VRChat API through a node-based language “UDON.” This development
was bolstered by a community developer who made “Udon Sharp” which lets world creators code in
plain old C#. One with an amazing journey through breathtaking avatar shaders, and incredible worlds
before they hit their stride making those tools. A wild me↑ shong wu↓ with no background in
shaders or games at all as best as we can tell, pushing past the limitations we thought existed,
performing linear cosine transformations to create hyper realistic lighting for worlds,
GPU-based audio effects and a tool to capture player motion as youtube videos to be able
to play it back, puppeting avatars for dance or education so users can see that motion in a
much clearer way than you could with just video. An owner of a real life club, who, when
the global :cough: events happened, turned their sights to VR and launched
a private club. One where you have to know someone to get in. Where they
can get to know you and your story. A military veteran making a map filled
with the words and feelings they’ve found to be most touching and meaningful.
Frequently hanging out in public instances, so you too can meet one of the most
fascinating individuals I know. Or even active VR technical artists
who know their way around the toolset, who want to dive deeper and go further into
the future with a community that will help them hone their craft and push the state
of the art beyond what many could imagine. Some people think that virtual platforms would
gain some benefit from generating all the tools in-house, and be the source sole source for
users to use. But, with VRChat, everything except the most core and fundamental components
that constitute VRChat avatars and worlds are made by the community surrounding it, not needing
to wait on programmers inside the corporation to decide upon things and adhere to release and
maintenance cycles. Everything from Udon Sharp, to AudioLink. World Creator Toolkit to CyanEMU
have been written by creators in the community. There’s countless worlds, and captivating
experiences to be had… But for me, content creation is what makes VRChat a “game.”
It’s the game of trying to figure out how to use a bunch of limited scuffed constructs in
a way to bring about some amazing effects. It’s a game played across discord servers,
late night discussions. Exploration by teams of people, each applying their skills and
abilities to further what’s possible in VR. Everyone’s interests are different and VRChat
has a home for them all. Even though what drew me in most was shaders. There are communities of
those who went out clubbing in the before times, musicians, or artists, modelers, designers.
Programmers and engineers. Furries (of all matter an ilk) and con-goers. Shut-ins, mini-golf
enthusiasts, people who are just lonely or looking to go on an adventure. Gamers, people interested
in sign language, and even a straight up wizard. I’d really like to share some of these with you
some day but for now, let’s start with my story: I’m an engineer, and because of inspiration from
Dr. Marc Olano, my CS435 graphics professor, I learned pixel shaders in college. Which led
me to make some of my first youtube videos, all the way back in 2007 and ones like No!
Euclid! But that was about it til late 2020. MAGFest wasn’t going to happen in 2021. I was contacted by some VRChat players
who wanted to make a party in VRChat. I didn’t think there was anywhere I’d be able
to fit. But they said “oh you can make shaders in VRChat” --- I thought they were kidding or
didn’t understand. But nope! Boy was I wrong. I dove in. Made an account. Strapped on the full body tracking. And Madman from
MAGFest greeted me to show me around. Without that I would have been lost. It would have
been difficult to find the communities that I grew with over the last year - and I urge you to find
a guide, or a friend with VRChat and if you can’t and assuming this video doesn’t go viral,
I’d love to show you around a little. One of the first people Madman
introduced me to was TCL. YES THAT TCL, except now my once viewer of my
live streams now, years later, works for VRChat. It was wonderful to see all the
wild things people could make, and talk to the folks who made the content I was
exploring! Friending people increased my trust rank and unlocked world and asset uploads. I came
to really love this platform and the people on it. I started writing effects for the worlds my
friends like Texelsaur and GPLord were making for various events, I found llealloo and was able
to port ColorChord over to AudioLink in VRChat, which is now included in some of the biggest
venues. I started documenting everything I thought was cool in “shadertrixx.” I worked
on making my ball pit “cn’s ball pit and lofi” which just hit spotlight. I’ve been
working on new object sync systems. I have so many stories and things
I want to make videos on now. And hopefully with this omnibus video out of the
way, I’ll be able to start making small videos. Show me your friends and I’ll show you your
future. I have found my friends I want to run with. I want to live in this future. It’s
this... cyberverse that we’re making right here and right now. Fundamentally, VRChat is about
connection. It’s a convention that just … never ends … where you can
meet new people, hang with old ones. I can go world-hopping with my mom who lives on
the other side of the country. I could show her the shaders I’m submitting to an upcoming art show
or the projects I’ve been working on recently. I can visit museums with friends that have
art that can’t be represented in our world. Or go on other escapades with my friends
that we never could have imagined. In social VR, having a virtual presence
produces a less toxic interaction than the anonymity of text alone. That’s not to say
public lobbies aren’t sometimes a cancer, but as you find your crew, you can
really have some gratifying experiences. When you run and push the boundaries,
you start to see that games aren’t fun because of their freedom, they’re fun
because your decisions matter. For me that was becoming part of the creator community
and working on projects. But others find communities of their own to invest in and
come to find that investment in themselves. VRChat lets you experience art in a way
that could never be seen before. It lets others show the concepts in their mind’s eye. I’ve loved exploring it. Finding what’s
touching. Finding what’s profound. To ask me why I think VRChat
is so compelling feels strange. It’s helped me find, and sing the song of
my people. I’ve shared just a few of them with you. Find your people. Run with them. Find
your song, and sing it at the top of your voice. Seeing friends who I made in VR, produce things
they found beautiful and find a fame of their own, to see them explore and grow in their knowledge
and personhood. I love VRChat for its connection. We don’t have to wait for someone who sits behind
a desk to tell us what our future is. We’ve already laid the groundwork, made the cities.
Come, and join us and let’s make this a thriving metropolis. With people like TinyTurtles here,
who I just found one day visiting my ball pit. I have only been at that for a
little bit more than a month, so I’m a complete noob when it comes to
shaders. I’m sure there’s a million ways that this could be made better and more efficient.
Obviously, I’m not a noob as a programmer. But, yeah this is my first time doing this. I’ve
only been editing avatars since the beginning of the year and I spent most of that time learning
how to get good at texturing. And then AudioLink came along and changed everything. And this is the
first time I really felt like I needed to start building my own shaders. I am older than people
think. This month I turn 50. And VRChat really reminds me very much of what the internet was
like in the very early 90’s. It was this sort of creative sandbox where you could do anything. Kind
of a wild west. And it feels the same in VRChat. There’s a whole world of creative possibilities
that just seem endless right now and I kind of. I think we’re at the beginning of something really
important. And, you know, when you’re 50, you have to speedrun the future. You can’t wait for it to
come to you. There’s so much creative energy here. And, you know, I just found this place
randomly once. I was just blown away by it. And yeah, this is fantastic. And, I’m
really glad to be here and a part of it.
