Some people make games using the Unreal Engine. That’s nice, but I wanted something a little different. That’s why I’m using the Real Engine.
(OK, I’m not really calling it “Real Engine”; Epic please don’t sue me).
What’s the Real Engine? Simple: instead of modeling a space in Blender and then texturing it, I’m building a diorama and taking some pictures of it. Then I import it into the point-and-click game engine I built and connect up the pictures into a space. If something needs to move, I either do stop-motion animation, or I photograph the components separately and animate it within the engine.
Here’s an example:
Why?
Why would I do a cockamamie thing like this? Well, it’s not the first thing I tried. The first thing I tried was reaching out to an artist whose work I’ve been enjoying for decades, who lives in my city, and who claimed to be inspired by Myst. He agreed to work with me on the project, which was just perfect.
Then I guess he thought better of it, and he ghosted me.
So I found another artist who wanted to work with me. He didn’t ghost me. He just didn’t do the work, and eventually told me that he wasn’t going to do the work. I think there was some sort of illness involved. So we parted ways. It’s too bad, because he clearly understood where I was coming from, and had some great ideas.
So at that point, I could have gone and found yet another artist. But if two artists had failed, why would I think a third one would do any better? As Einstein supposedly said (but didn’t actually), “the definition of insanity is doing the same thing over and over and expecting different results.” Also, I had to consider my emotions: it’s heartbreaking to think you have a partner, and then realize that, actually, you don’t. I didn’t want to put myself through that again.
Now that I’ve started working on this myself, I realize that it’s possible that I was asking too much of the artists. I’m going to end up spending three times as many hours doing the art as I’ve spent on the code, at least. I’ve had thirty years of experience as a programmer, and much less as an artist, so it’s not really a fair comparison. But it’s still the truth.
How long will this take?
To get started, I prototyped a single room. Actually, I only did one end of the room, because it was just a quick test. My prototype only took a few days. So I thought that maybe it would only take a few months to do a full game. I was wrong. Here’s the room:
Looking at that image, I hope you’ll see why I thought this was a good idea. But look a little closer, and you’ll see why I was wrong about the timing. There are four major issues (and some minor ones):
Light leaks. The edges aren’t well sealed, because I wasn’t thinking hard about light.
You’ll notice that you can see a nubbly gray thing behind the scroll niches. That’s a random blanket that I put there to block light. I actually needed to do a bunch more construction to hide that background correctly (replacing the blanket with duvetyne would help too). Anyway, the visual design of that scroll case could use some work — the niches should be diagonal, and go less close to the walls.
The table has tiny dowels for legs, and the top is plain construction paper. I would rather have something that looks a little more like wood, and legs that aren’t so spindly — even if it’s just faux finishing.
The windows don’t have glass in them, and in fact, the muntins (or cames, or whatever the bits between the glass are called in this context) are made of the same paper as the walls.
The ceiling is flat and boring.
The depth of field is not great, about which more later.
This is the same problem that AI art has: it’s fine on first glance, and then you get closer and it all falls apart. As John Salvatier notes, “Reality has a surprising amount of detail”.
The thing is: I could fix most of these — but fixing them basically means rebuilding the entire room.
So “a few days” becomes a few weeks or a month, and now I’m looking at a much longer timeline. Fortunately, nobody is going to scoop my idea, because nobody else is insane enough to build a game this way.
Prior Art
OK, maybe someone else is. There’s Harold Halibut. They’ve built real objects and then used photogrammetry to put them into a traditional 3d engine. It’s a neat idea, but my idea involves one fewer step.
Update: Also apparently Lumino City, Papetura, and The Neverhood use similar physical modeling techniques.
Why not Blender?
Why don’t I just use Blender like everyone else?
I hate Blender. I mean, I understand that it’s the standard thing that everyone uses, and I know that I would eventually get used to the UI. But almost every time I use Blender, I end up mangling the geometry in some way that requires either me to take a hundred years of clicky work to fix it, and in the end I wish I had remade the part from scratch.
I’m not much good at textures. I could just buy a bunch of textures, but getting them to look nice together would be tough. And I would still end up with a bunch of repetition, because there’s just no way a solo dev can get enough textures. Cyan manages it — when playing Firmament, I didn’t notice any texture repetition (I did notice that the music was an incomprehensible drone, except for the totally perfect VNV Nation track at the end). But this seems to scale with studio size. In this screenshot from Quern: Undying Thoughts, take a look at the wood texture on the dock. They get a little extra mileage by mirroring the texture on alternate boards, but it’s still the same texture:
Zadbox, which created Quern, is smaller than Cyan. Going down to a single-person studio, Haven Moon is a noble effort, but I got awfully bored of seeing the exact same brick wall in dozens of places.
I didn’t want to do that to my players. The whole point of games like this is to explore a new world. But if you’ve already seen all of the textures on the first island, then you’re not really exploring a new world.
By contrast, here’s what my staircase looks like (before I install it — it’ll be in a stone tunnel, so the details might be harder to see in situ):
Every stair is different. Some of the angles are a little off square, which is totally fine, since it’s supposed to be sort of beat up and ramshackle.
Why not AI art?
I’ve addressed this before. But also consider the following prompt:
Painting based on Goya’s Saturn Devouring His Son, but with a giant eagle eating a tiny headless monk in a brown robe. The eagle holds the monk’s in its giant claws. Crimson fluid leaks from the monk’s neck.
So, AI is bad at prompt following. But also, it would be really hard to have it do two images of the same room from different locations. I would probably have to generate one image, then convert it to a 3d model, and then use controlnet. I’m not sure this is less work than any of the other methods.
And even if I did build a gray box model to use with controlnet, Midjourney doesn’t even understand that if there are arches and columns, the arches have to be on top of the columns.
Process
So given that I’m going to use the Real Engine (Okay, okay, Epic, I’ll stop now), how will I do it?
What kind of space?
Myst and Riven are set on a series of islands. Haven Moon is set on a series of islands. Quern is set on a series of islands. Firmament is set in… well, I won’t say to avoid spoilers, but a small number of relatively enclosed spaces. In a game like this, any time the player tries to walk off the edge of the map and just hits an invisible wall, it’s immersion-breaking. That’s why there are so many islands.
So I needed space without a lot of freedom. I decided on an abbey high in the mountains. The mountains will be a matte painting, and most of the game will take place indoors. It’s not exactly an original setting for a mystery, but I think I’ve put my own twist on it. The setting then gave me enough inspiration to put together a story. And the story has then suggested certain aspects of the puzzles.
Roughing out
First, I drew a floorplan in Inkscape. (I’m not sharing any maps here because I don’t want to spoil the fun of exploring the space once the game is released). This just marked out the walls and doors of each room, so I could get a sense of how the pieces fit together.
Then I copied that floorplan into Blender, building up the volumes of the space. This let me more easily see how the roof lines would appear, and fly around to get a sense of scale. I guess some people can do this mentally from just a floorplan, but since I plan to have complex roofs, Blender was easier. I left some interior spaces out of the model, since they don’t have roofs.
Scale
Dollhouse scale is usually 1:12 (for a delightful note on this, search this page for “Mervyn O’Gorman” ). This makes the arithmetic easy for Americans like me who grew up with “customary units”: an inch is a foot. And it’s workable for interiors: you can fit a large room on your table, if you have a big table, and you can fit a phone camera into a small room and still get things in focus.
But if you want an exterior of the entire abbey (and you definitely do!), you either need a bigger workshop than I have, or you need to take a hint from model railroading and go to 1:48 scale, which the model railroading folks call “O scale”. Having two different scales is annoying, because it means that you can’t seamlessly blend — if you want a door from the interior at 1:12 to the exterior at 1:48, you can’t just put the two dioramas next to each other, you have to put a green screen on both doors and then later paste in to each the appropriate photograph of the other one.
I think I actually will end up using a third scale because 1:12 is too teensy for some of the really complex bits, but hopefully that’s just for a room or two.
For another wonderful article of scaling, see John McPhee’s article The Ships of Port Revel, in his collection Uncommon Carriers. “At scale the mallards are thirty feet long,” is a line I often think of as I work on this project.
Photography
I had initially planned on shooting the whole thing on my phone camera. There’s a problem with this: my phone is six inches wide, and the camera is at one end, so if I have a narrow hallway, I can’t have the camera centered unless I hold it vertically (which loses resolution, since the game will be designed for 16:9). Also, the phone camera is very smart, which means that I’ll be constantly fighting it to get consistent lighting. I know there are APIs for manual mode, but building a custom Android camera app would be a lot of work.
A SLR was right out; they are giant compared to the space I’m shooting, and even macro lenses typically have too-large minimum focal distances.
Keep in mind: the camera must operate inside an entirely enclosed space, and I need to see the pictures as I go in order to make sure I’m getting what I want, and I need to do this without picking up the camera to look at a screen.
So I decided to build a camera. Well, I bought a Raspberry Pi camera module (and then a second one when the first one wasn’t good enough), and built a tiny Flask app which runs on the Pi to take pictures and allow me to adjust the exposure and focal length over wifi. The Pi is powered by one of those phone battery packs, which is cheap and runs about all day before needing a recharge. It sits on some wooden blocks so that the photos are all from the same eye level. The camera is a bit noisy, but long exposures help a lot.
Actually, I ended up buying a third camera and a second Pi. The OwlSight only has a 68 degree horizontal field of view. My aspect ratio is 16:9 (1.77:1), but for most shots it’s actually 1.956:1, because I leave a little margin around the edges so that I can show it as you rotate, making the direction of rotation clearer. So I ended up having to move the camera very far away from the wall to get enough into view. With a second camera at a slight angle to the first, I can stitch two photos into a panorama.
I’m also doing focus stacking. The idea is that a macro lens has a relatively shallow depth of field. This is a distinctive look: things shot in macro look like they are shot in macro. If you want to make large things look like macro, you use tilt-shift photography to fake this effect. If you want the opposite, you use focus stacking: take photos at multiple depths of field, and choose the in-focus parts of each. The stacking happens before the panorama, since then I don’t have to worry about panotools picking the same points of correspondence for each image pair in the stacks.
The Pi doesn’t have enough RAM to easily run focus stacking software, so my Flask app just scps the files up to my desktop, and then runs the focus-stacking software there. Then it copies the composited image back for display. Then if I like the pic, I can press a button to copy it back into the game’s assets directory (named appropriately). I also save the images from which I built each focus stack, in case I want to do more editing later.
The whole thing is slightly janky, but there’s a trade-off between spending time improving the app and actually making the game. By nature, I’m very excited about working on tools. As an indie game dev, that’s a trap. Because of the long exposures at multiple focus levels, I do have a bit of time between shots to futz with the camera code and write the post that you are reading now.
Every room I build, I learn something. For instance, the OwlSight camera’s auto white balancing seems to blow out reds. I didn’t notice that until I tried to shoot some red-orange needle-felted balls. So I’ve added manual white balancing control.
The second room I built has a rough floor. This means that the camera stand (which is literally some random wood blocks that I had laying around) doesn’t sit flat. Or maybe the floor just isn’t level to begin with — there was definitely some warping when I applied the air-dry clay to the masonite, and I’m not sure gluing braces on later quite got it all out. Mostly, I don’t care if if the camera is a bit off level, but in one case, I have a machine on the wall that needs to be dead square to avoid annoyances during animation. So I built a camera leveler with another wood scrap. It has threaded inserts at the corners, so I can turn screws to adjust the height of the four corners of the cameras.
The third room ended up having insufficiently bright lighting. I bodged together a partial solution, but it’s still going to be the dimmest room in the game. Next time I’ll use brighter LEDs.
Architecture
Architecture is really hard. Consider the chapel: I picked the shape for it based on where it fit into the rest of the monastery. There are some subtle constraints — like, I really wanted one of its walls to be blocking the view from the door to the front hall, because otherwise that door would open into the garden, which is at O scale. And I did a flat angled roof because a gable roof would be hard to square with the six-sided floorplan. Also, I think a gable roof with elaborate decorative skylights might be weird, and skylights are the only hope for lighting a room of this size.
After examining the model in Blender, I decided that it could be a chapel. A bit 1960s looking, but OK. Unfortunately, this meant trapezoidal walls, and compound angles.
The skylight wasn’t going to be quite enough light, so I added a bunch of fenestration. I decided on stained glass, and tried several ways to make it look right. In the end, one of the windows has a complex pattern which I printed on acetate, and the others have simpler patterns which are alcohol-based ink on acrylic. The acetate is brighter, but the acrylic is good enough and comes in larger sizes.
Of course, the windows have to be high enough that you can’t see much out of them, because outside the windows is the garden, which is still a different scale. I guess I could have just planted some trees right outside, and built two scales of tree, but then I wouldn’t as much extra light. With high windows, the sky will still be visible. So I just painted a rough blue-white gradient on some cardboard, which, through the acrylic, is good enough.
If this were actually built in the 1960s, the windows would be aluminum rectangles. But rectangles are boring. I went with bird-inspired shapes, which are hopefully a bit Gaudi-esque. Birds are a big theme in this game, for reasons players will discover.
I made the chapel floor with “stone” tiles. To make tiles, I used poured paint on paper, cut into squares. This was a bad decision, because I had to pour like a dozen sheets to get enough squares, and then I had to glue them down (I used gray paint as the glue, so it could double as mortar). The good decision I made was to align it diagonally, which meant that I could hide some of the unevenness. But then I didn’t do a great job of getting straight lines, so I had to trim some of the tiles. Probably nobody will notice. Also the tiles peeled some, which means that the floor looks kind of cracked. Fortunately, the story of the game makes this not totally unreasonable, so I’ll just knock over some of the pews and throw some prayer books around, and say there some sort of catastrophe here.
The walls are covered with Thai kozo paper. But where the paper joins, it doesn’t look great, so I built some columns to cover those places up. The columns are foam covered with more kozo paper. For a couple of the interior corners, I used some wood framing. There are a lot of little decisions like this: I need to cover an edge or a corner, so I add a little trim, which ends up adding visual interest to the space.
Tools and materials
Premade dollhouse furniture
No. If I do that, my game will look like everyone else’s dollhouse. Also, in theory it’s copyright-infringing, although in practice, I’m unlikely to get sued. I tried to use some lighting marketed for dollhouses just to avoid wiring my own LEDs, but they ended up not being bright enough. Dollhouses usually have one open wall so that you can get Sandy Toksvig inside them, so lighting inside a dollhouse is for looks not for illumination. So I won’t try that again.
3d Printing
I want to avoid 3d printing as much as possible, because 3d printed stuff lacks that handmade look. Also, it means I have to use Blender. Well, when I’m lucky, OpenSCAD. I have needed a few prints. First I ordered online, then I borrowed my brother-in-law’s, and now my wife brought me home a 3d printer that someone in her office was just giving away for some reason (this is how the rich get richer).
The other bad thing about 3d printing is layer lines. I had some bench parts printed, and I thought I could cover up the layer lines with acrylic paint, but it it turned out that even two layers of gesso followed by two layers of paint didn’t solve the problem. When your camera is four inches from the subject, every little line shows. The actual solution is XTC-3D, which is a rather unpleasant product: expensive, short pot life, hard to apply thinly, and weird-smelling. But it does the job, which is the important thing.
Speaking of extremely fine details, my workshop is unfortunately also inhabited by two long-haired cats. Every time I see them, I think “Did he who made the lamb make thee, and if so, why did he make you shed so damn much?” Keeping the hair out of the dioramas is quite tricky, and I often find that I have to remove a stray hair and reshoot.
Clay
Even though I have a full pottery studio with a kiln, I’m going to use very little ceramic clay for this project. One reason is that the cycle time is high: I need to accumulate enough stuff to fill a kiln, and then run the kiln, and then glaze, and then run the kiln again.
Ceramics has a very distinctive look, which is mostly not what I want for this space. I do have plans for a couple of pieces — which, given the cycle time, I should probably start on.
Some of the door moldings are made from DAS air-dry clay. It’s nowhere near as nice to work with as real clay, but it’s surprisingly strong — you can pick up a piece of molding that’s 1/16” thick and 7” long by one end, and it won’t break. Try that with green stoneware clay and you’ll have a bad day. And you can cut it with a knife even when it’s dry. I also made a vase out of it, but getting something round at that scale with that clay is tricky. I should probably just use real clay for that.
Epoxy clay is really nice to work with, if you’re quick. I made doorknob parts with it, and filled in some of the cracks in the door moldings. It’s hard as a rock once it’s dry, so you don’t want to be too sloppy, but it remains sandable when hard.
Needle felting
Instead of glowing screens, I’m probably going to do most of the screen-like puzzles using needle felting. I think in my ideal world, every puzzle would have a reasonable and diegetic hardware manifestation, but that’s not quite how the puzzle design ended up going.
So far, it’s a lot of fun. I’m not anywhere near the level of Andrea Love, but I can make cool stuff. For my first animation, I made ten frames (for each of the three animations), then I used RIFE to add three frames in between each of my originals. I think this was especially helpful at the end of the animation, since my animation loops back to the first frame, but the felt didn’t end up exactly perfect. At 20 FPS, it still looks like stop-motion, which I think matches well with the handmade look of the rest of the game. I’ll probably reduce the frame rate of the other animations in the scene, and add a little jitter, to match.
(This video has a white rim which will be masked out in the actual game — doing alpha channels in video is tricky, so I’ve just written little shader which applies a mask at runtime)
Release
I’m really excited to be working on this project. I go to work every day happy, and I fall asleep thinking about what I’ll do next. I can’t wait to show you the world I’ve built.
High Mountain Abbey will be released once I have designed, constructed, and photographed a lot of dioramas. Hopefully 2025 but early 2026 is more likely.
Previous post: Middles updates, design, and prior art