The making of Shadow of the Colossus on PS4
Digital Foundry talks tech - and more - with Bluepoint Games.
Now known as the 'masters of the remaster', Bluepoint Games has a well-earned, solid gold reputation for delivering some of the best current-gen - and indeed last-gen - ports of gaming's most beloved properties. From Metal Gear Solid to God of War, from Gravity Rush to the Uncharted trilogy, Bluepoint's work has been uniformly excellent across the years.
However, with the recently released Shadow of the Colossus remake, the studio has pushed on to a new level. The developer has built upon its existing PS3 remastering efforts with a full-blown remake, reimagining Team Ico's original work with a release that captures and enhances the unique ambience of the PS2 classic, executed to today's triple-A gaming standards.
That said, despite some extensive pre-launch marketing, there's still a lot that we don't know about this game. What engine technologies did Bluepoint draw upon to create this remake? To what extent does original Team Ico codebase factor into the new game? And looking towards Shadow's stunning PlayStation 4 Pro implementation, how does the team manage to make its 40fps 4K mode look so good despite a 1440p base framebuffer? And conversely, looking at the performance mode, how did Bluepoint hit its 60fps target so consistently when so many have failed?
And there's more than just technical discussion too: with the team's staff roster expanded significantly for the Shadow of the Colossus, where does the team go next? Will we see a brand new, original game or will the team instead take push even further with another remake?
In this mostly tech-orientated interview, we talk with Bluepoint's president and co-owner, Marco Thrush, along with tech director Peter Dalton and produce Randall Lowe. And we've got options for you: carry on reading from here, or else watch our video below which contains all of the same content - plus a bonus, where Marco, Randall and Peter tell us about their favourite areas in the game (backed up appropriate video, of course).
We really got a kick out of this interview and learned a whole lot more about this remarkable game. We hope you enjoy it too.
Digital Foundry: So let's start with some details on the Bluepoint Engine itself - what it's capable of, how it's tailored for Shadow and other future possibilities you might see for this technology.
Peter Dalton: I think when you initially look at our engine and our technology, we spent a lot of time making sure that we can basically accomplish the task of running two engines side-by-side. One of the great things about some of the titles we've been able to work on is that they're great titles in their own right and so as we look at the game and we want to replace certain key pieces, we really tailored our technology to be able to extract certain pieces, put certain pieces of the game through our own technology but then also run the original game engine side-by-side. And so with that comes a lot of considerations from memory usage to performance to what kind of threading models and stuff we use to basically allow us to have the most amount of flexibility within each game.
Digital Foundry: What kind of overhead does that take to have the two codebases running together like that?
Peter Dalton: You know, each game is a unique snowflake, so each one creates its own problems. When you look at Shadow of the Colossus, if you look at the way we split stuff up on the PS4, the main core is pretty much doing all the simulation for the Shadow side and then we utilise all the rest of the cores for more of our rendering and some of our background physics processes. So it usually takes a fair toll on some of the primary processes and we've really worked really hard to make sure that our stuff is as lightweight and is optimal as possible, so that when we're running the other game side-by-side, it's not tanking our performance.
Marco Thrush: One thing also to keep in mind is that when we say we're running the two engines side-by-side, each engine kind of has their own area of responsibility, so whereas the original game's engine is really just running all the game logic, it doesn't have to deal with any rendering, any sound stuff, any file loading... all that stuff is done in our engine, so each engine essentially has their own area of responsibility and so you're not really doing a lot of the same work twice.
Digital Foundry: How easy or difficult is it to keep those two in sync?
Peter Dalton: I think, as Marco hit upon, the key is setting up areas of responsibility. For example, we use the Bluepoint Engine for all of the rendering and file loading and audio-type tasks and so it's going into the Shadow of the Colossus engine and we're moving and redirecting all of the file loading type aspects back into the Bluepoint Engine and so then once you have those discrete, clean areas of responsibility which takes quite a bit of time to get each engine into a spot where it fits those rules, then you actually get something that's manageable. And then there's a lot of profiling, a lot of analysis to figure out exactly what kind of core utilisation we're getting and how to organise things in such a way to maximise that.
Digital Foundry: So thinking of that, let's say you start a new project. You get this original codebase and you do a lot of work with Japanese games. I mean, is this all commented out in Japanese? Are you working in assembler code for PS2? Or what kind of languages are these titles written in?
Peter Dalton: It's a little funny that one of our most prized pieces of technology is to basically take all of the comments that are in the C++ code and convert them to English. That tool has paid for itself more than once.
Digital Foundry: So Shadow of the Colossus was a C++ game originally?
Marco Thrush: No, it was not actually.
Peter Dalton: It's much more of a C game that when we did the first remaster of it, bringing it to the PS3... a lot of the files - not all of it but a lot of it - was converted to be C++ compliant, which was really not that much work and then as we integrated it even further into the Bluepoint Engine for the PS4 remake, that required even further translation and clean-up.
Digital Foundry: So basically then, it sounds like the PS3 remaster served as your jumping off point for this new PS3 version of the game then?
Peter Dalton: Absolutely. In fact, each game has its own challenges but one of the key benefits of remaking Shadow the Colossus for the PS4 was that instead of using the original PS2 codebase and starting from scratch, we basically picked up where we left off on the PS3 and just moved straight on to the PS4, so the time to get it up and running on the base PS4 hardware was much quicker than, say, some of the other titles that we've worked on, like the Uncharted collection.
Digital Foundry: So before we jump away from the PS2, I'm just wondering whether any of you have any real experience of creating games on the PlayStation 2 and I'm curious, if so, what did you think of the work that Team Ico did back in the day?
Peter Dalton: You know, actually the first title I ever shipped was Hot Wheels Velocity X, which was a PS2 title, so I have actually shipped games on the PS2 and there's where I first ran into learning the ins and outs and the kinks and the gotchas. One of the things I do have to say is that when you look at the original engine source code, there's obviously areas where Japanese developers have a little bit of a different mind set, different practises than we exercise here in the States but there's a lot of their systems that actually hold up very well from some of their IK [inverse kinematics], their foot IK systems... the bone dangle bits that hang around Wander's waist, for example. You know, a lot of their AI logic. And so a lot of those things just required tweaks and we try to practice 'don't rewrite it unless it's necessary' and more of a mentality of 'bug fix rather than rewrite'. I think it pays tribute to the original engineers just how much of the original code is actually in that game.
Digital Foundry: One of the most impressive things about Shadow of the Colossus is the performance. Can you talk about your approach to hitting such a stable 60 frames per second on PS4 Pro?
Marco Thrush: Well, the key goal is to say from the very beginning that you're going to do it and then actually stick to it. It's really just due diligence. You can't just say at the end of the project, "You know, I wish this was running at 60fps." It's just not going to happen, so it's like, from the beginning up, building art content in a smart way so it is scalable to that degree, keeping your pipelines flexible, like generating LODs on demand, not hand-building them, so it allows you to rebalance how the LODs from meshes are and stuff down the road when you have more accurate performance numbers. That kind of stuff definitely helps. Other than that it's artists that care, programmers that care, everybody just caring on the team and wanting to make it as good as it can be and putting the effort in that it takes to get there.
Peter Dalton: To that point, well it does take discipline upfront. It did not happen by accident by any means. There was a lot of work that was done of running around different areas, profiling, looking at the different configurations for the final game. There's three different configurations - there's the base PS4, 60fps mode on the Pro kit and then also the 30fps at 4K, and so there was a lot of playing the game finding problem areas, going back to art, working through it you know then knowing that if we ever dipped you would call us out so we couldn't have that!
Digital Foundry: And you know, I did search long and hard and honestly the only place that I found dips was right in the temple area at the beginning. What's happening there specifically?
Peter Dalton: You know, I'm going to be completely transparent and honest with you because I respect you! So the dips that occur in there usually occur after you've beaten several Colossus, and you get the doves that are flying around… and the doves, the number of animated objects in there. And part of it has to do with legacy systems that perhaps maybe we should have been more aggressive about rewriting or pushing more into the Bluepoint Engine, but just the management of that number of dynamic objects, the original codebase struggles with.
Marco Thrush: There are some other issues... when you come back [to the temple] it starts loading a bunch of data, so the other threads are kind of busy too.
Digital Foundry: Yeah, I've noticed that it tends to occur most when you've come back from another area, especially after you've defeated a Collusus and if you go back to it a little bit later, it seems to smooth out a bit.
Marco Thrush: Correct, yeah. So that's kind of like the extra CPU overhead going off at the same time in the background. There's actually one other spot where the game slows down and that's the cutscene near the very end. But it's in the cutscene, no player control, so we kind of figure we're OK. But we love solid frame-rates - that's our seal of quality.
Digital Foundry: So, you mentioned the 4K mode. The pixel count there turns out at 1440p but the temporal anti-aliasing you guys are doing is just remarkable. It's extremely clean even at 1080p. Can you talk a little bit about what you're doing there? Is it like a temporal accumulation with samples that just kind of amount to 4K when you're at 1440p? What's going on there?
Marco Thrush: So it's exactly that. We're doing exactly what Ratchet and Clank is doing which is actually rendering a native 1440p frame and injecting it temporarily into a 4K buffer, so the actual final output that we're sending to the hardware with the UI and stuff is 4K and really just calling it 1440p is kind of lying a little bit, because over time the temporal jittering of the rendering will give you a higher-quality result than a single 1440p frame.
Digital Foundry: Yeah, I definitely agree with that. The only time it looks truly 1440p is immediately after the scene cuts and you don't have any sampling data from that point, but other than that it's very, very clean. I also wanted to ask about the way you handled the alpha texture AA like the trees, grass, horse hair and fur. Is that all included in that process? Because it's extremely nice-looking as well.
Marco Thrush: So, in addition to the temporal jittering at the injection into the 4K buffer, we also have standard temporal AA. So that's what we're using, but yeah, we're not doing anything like checkerboard rendering, we're just doing the native approach, which simplifies things a lot actually.
Digital Foundry: Another thing that really impressed me was the way indirect lighting and shadows are handled, especially when you're in a shadowed region, you still have a very nice sort of ambient shadow from the horse and Wander. Can you talk a little bit about that?
Marco Thrush: Yes, so that's a great trick we learned from the Last of Us guys, where they had ambient capsules that were approximating the shapes of the objects and they're just doing a screen-space raycasting against that to essentially get to soft ambient shadows, and then we do a multipass version of that, so we get just the direct normal occlusion and also a directional occlusion from the lighting model. So that's where you kind of get like these directional shadows as well when we have strong light coming from a certain direction.
Digital Foundry: And what about the post-processing? I'm a big fan of motion blur and the object blur that you guys are using is quite nice-looking, and I've even noticed that it seems to apply to things like the transparencies on the horse. What artistic principles does it follow, what kind of technique are you guys using here?
Marco Thrush: It's pretty straightforward motion vectors and compute shader stuff. I mean, we basically just look at a bunch of games that implement something really, really well and we try to meet that quality or exceed it and we just work diligently on just improving things. Like thinking through things like, "Hey, how can we make this better," and it's just a lot of dedication from the guys.
Digital Foundry: And it seems like you've adjusted the shutter speed to kind of vary between the 60 and 30fps mode.
Marco Thrush: You have the motion blur slider in the settings which if you go to a hundred percent all the way, that's essentially a hundred per cent shutter speed which in 60fps mode would be 16 point something milliseconds and for 30, it would be 33ms.
Digital Foundry: So on the animation work then, I want to talk about the physics themselves. You mentioned the bone charms around Wander's waist there, but the actual appearance of things like cloth as well, and some of the fur on the Colossi... How did you implement all of that and how much of that is shared or based on the original work?
Peter Dalton: When you look at the bone danglies around Wander's waist, those are actually 100 per cent the original code simulating those. When it comes to Wander's poncho, that's more of a custom solution. It's actually based on Nvidia's Apex cloth stuff, so part of it is that as we go through, we evaluate and look at what the state of it is. We knew that we wanted to fix up the cloth, we knew that we wanted to fix and change the way the horse tail and the mane react and so that's actually a different custom physics solution that one of our engineers here spent a couple of months on writing a solver for, and then we also applied cloth, and to some of the Colossus for some of the banners and other things that hang off of them.
Marco Thrush: And then the fur specifically, that's another custom solution which is all compute shaders that simulate and render the fur essentially. It generates the geometry for the GPU to then render.
Peter Dalton: So, the art staff would sort of go through on each Colossus and use texture maps to basically paint different fur values of where they want the fur, the thickness, how stiff you want the strands, the length, which direction in which it lays, how it clumps and then all that would be ran through a compute shader to actually generate that geometry on the fly.
Digital Foundry: So just out of curiosity, what did you guys think of the original implementation of the fur effect in Shadow of the Colossus where they kind of simulate a more modern technique by layering textures in a very unique way.
Marco Thrush: They actually use that all over that game. They used it to actually simulate really thick grass layers as well. Like, on the fourth Colossus they use that there as well. It's a great trick, especially when you use skinning texture then weight the different layers slightly differently, so you actually get a little bit of 'sway'. I mean, the PS2 was a fill-rate monster, so it's kind of like a technique for the time and that's what a 512-bit bus gets you - lots of fill-rate.
Digital Foundry: So you've seen code base, what was the most significant bottleneck that you could see from your perspective on PlayStation 2? What was it that could murder performance?
Marco Thrush: I think it's just the sheer amount of stuff they're pushing.
Peter Dalton: I think they really they needed more cores to parallelise stuff.
Marco Thrush: I don't think you could look at the code and say, "Oh, this piece of code was badly written". I mean, so yeah, some things aren't as great as they could have been if they could have invested unlimited amounts of time but I don't think anything works to the detriment of the game. Yeah I mean like, some of the choices... making the game look more cinematic by adding stuff like post effects, bloom, motion blur, those are all a fairly high fixed overhead on something like a PS2. These days you can like snuff it off, like 2-3ms here and there, but back then it used to take a little longer.
Digital Foundry: Were there any effects that you guys experimented with that had to be dropped due to performance budgets or did you nail everything going through it?
Peter Dalton: I think, for example, even the death effect on the Colossus, we went through multiple iterations of that here in the studio trying to figure that out. The original game does this capsule overlay that they fade in and so we looked at whether we wanted to do something similar to that approach, or do we want to do something just directly within the materials on the Colossus itself - like, how do we achieve that sort of look? That's most definitely one that went through multiple iterations.
Marco Thrush: A couple of examples, I can think of... Well, Dormin. His representation went through a lot of iterations for performance reasons. Originally, he was less soft-looking but we're still happy with where he ended up being, but some compromises were made. And then the only other one I can think of is during the final Colossus battle, it would have been nice to have a bit more rain but that's just like a wish list thing.
Digital Foundry: How did you come decide on the colour palette that was used and the way the sky is rendered? The cloud texture, for instance, is much more realistic because the capability is there to do it, but obviously Shadow of the Colossus is so well known for its art direction, so I can imagine it was quite challenging to come up with something that would please everyone.
Marco Thrush: Yes... [laughs] and if we really did please everyone that's a question of debate obviously, but yeah we try to make sure as many people as possible are happy. We realised very early on as more realistic looking art appeared in the game that a lot of the other things that were like holdovers from before, or a lot of things they didn't do before, we couldn't really get away with any more - like not having a realistic fogging model that properly shadows or even something as simple as not having footsteps in the sand. When graphics look realistic, you really missed that and it kind of feels like you're working on this rock-hard surface that just happens to look like sand. So, a lot of effort went into having to add little details like that in addition, and then the same thing with the sky. Having a realistic-looking world and then having just a simple repeated cloud texture, it just doesn't quite cut it, especially when even in the original game, you get very strong directional sun shadows but there was no actual sunlight source in the game, so we tried to avoid that that slippery slope of, "it looks unrealistic but it's still good enough".
Digital Foundry: Thinking of shadows actually, I did mean to ask. Are you baking the shadows or are these a hundred per cent real-time shadow casters?
Marco Thrush: There's everything. There are baked far distance shadows, you have real-time cascaded shadows for the sun. Obviously, spotlights have shadows, we have baked ambient shadows, we have real-time SSAO... there's a little bit of everything there and then there's the ambient capsule shadows that I was talking about before.
Digital Foundry: And while we're talking about lighting, could you also go into some detail regarding the volumetric lighting solution featured in the game, maybe some detail on your implementation as well as the cost on the GPU and the like?
Marco Thrush: It's probably about a millisecond or so. We're using a view space approach which is not too dissimilar from the Assassin's Creed Black Flag approach. they're essentially doing a 'froxel' volume here you have froxels that are view-frustum aligned and it injects the fog and the lights directly into those froxels and then it uses that to render. We actually use that as a trick to have our fog particles, that's how they can render as fast as they do. They can use the fogging system and directly render into it.
Digital Foundry: And how about the materials themselves? There's a lot of great stonework and dirt textures and with everything in here, it looks like you've adopted a full physically-based rendering workflow.
Marco Thrush: I would say that the textures are pretty much photorealistic, with the exception that they're slightly more exaggerated because that kind of fits the original art style. But we actually use photo-scanned materials and everything very much adheres to PBR workflows, and we have a very powerful layered material system where it generates custom shaders depending on the unique materials and how they're set up so it gives us a lot of flexibility so we basically have like an uber-shader and it can have as many layers as the programmer lets the artist have, basically. And code behind it just takes care of it, creating the optimal shader for everything. And yeah, I mean we even use runtime blending as well to inject into it to allow dynamically changing blending in-game. It like when you have the tenth Colossus, which people call Dirge, he's the sandworm. When he tunnels into the sand, the sand is raised through displacement but in addition to the sand raising, it actually switches to a different sand texture that's like a more softer sand and the sand at the ground level is more rippled.
Digital Foundry: So you mentioned earlier that the game is using sort of a mix of pre-calculated and real-time shadows throughout its world with the focus on real-time shadows. Would it be possible to, say, modify the sun position arbitrarily in real-time to change the mood in the game for instance?
Marco Thrush: At the cost of additional memory, in theory, it would be. There's actually a couple areas in the game where the sun does change direction [Laughs]. After you enter the initial area for Colossus eight, the sun actually changes direction by (I think) almost 90 degrees and we actually had to change the hallway leading into that arena and add an extra bend to it to hide that transition because obviously we have real-time sun shadows and in the original game, that was all just baked into vertex colours and they just painted in whatever they wanted, so we had to hide that transition in our version a little bit. And then, yeah, it's not really a sun but obviously during the final Colossus encounter, the moon is also at opposite direction, but the generals do actually rely on our sun being static and being a fixed direction for optimisation reasons as well.
Digital Foundry: As far as I'm aware, this is the first game where you guys actually have recruited artists into the studio to work on the project. Now that you have a full-on art team, do you think it would be possible to move on to an original game again? I mean, Blast Factor, I think, was the first game Bluepoint did but now your capabilities are far beyond that.
Marco Thrush: Well, we could but we're doing another remake!
Peter Dalton: You know, there's been an art staff involved on every title that we released from the Uncharted [Nathan Drake] Collection, and it's just, what is the size of that art staff that's required to accomplish our goals? And so with Shadow, the studio most definitely grew in both number of people and skill level - and so we continue to grow and expand and look at areas of weakness. One area that we've not explored very deeply is, "What does our design prowess looks like? What is our ability to build something original and to expand it, rather than just upgrading something?" and so we look at those things and we are constantly trying to enhance and strengthen the studio to get to a point where, yeah, doing original development or whatever comes our way, we're fully capable of doing it.
Marco Thrush: So, to elaborate on that, originally when we did Shadow [on PS3] there wasn't a whole lot of artwork involved. It was really just engineering and then on this remake now, we knew the code, all the engineers that are here, most of them had worked on that exact code base so we knew what we had to do. There wasn't any unknowns, which allowed the engineering staff to focus a lot more on making sure the art pipeline is as good as possible, the rest of the engine gets as good as possible. This project served us as a great point of growing the art team to the point where we can take on a full triple-A game major scope of art content. So now our next step is, let's improve the art pipeline, let's improve the engine, let's improve workflow for artists, let's grow on the art side some more to handle our next project because it's a bit bigger. And our next focus is, all right, let's work on design and add new stuff to get to the next remake, because now that can be our sole focus of making sure that's where we put the time and everybody else, they're already at a level where we can perform.
Digital Foundry: So bearing in mind the scope of this project, would you say it's the most challenging remaster you have done yet and if not, what else would qualify for that?
Peter Dalton: You know, challenging is hard to quantify because a lot of times it comes down to timeline, right? I would say as far as the remasters and remakes, it is the most challenging, it is the one we're most proud of. But they each have their own challenges. There's been other projects with shorter timelines and such that have been more physically demanding on the team.
Marco Thrush: It does really depend on who you asked. If you were to ask Art here, they would definitely say Shadow was the most challenging project, I think. From a pure engineering perspective, I think, Titanfall [on Xbox 360] might still rank up pretty high up there.
Peter Dalton: That was very challenging mostly from a timeline and manpower perspective.
Marco Thrush: Normally, you don't rewrite systems just to save memory. Well, on Titanfall, we had to write collision systems, animation systems, just to save memory which is something you don't normally have to do.
Digital Foundry: Well, looping back to Shadow of the Colossus then, are there any other visual elements or techniques that were implemented in the remaster that you guys are especially proud of that we maybe haven't touched on yet?
Marco Thrush: Something we're definitely proud of is the particles because a lot of this stuff, people don't actually realise that the dust that comes from the Colossi, for example, we didn't manually script up or place events down to say, "He's shaking now, dust comes off". It's actually all driven dynamically. We have emitters on the Colossus himself and they monitor the velocity of the mesh and know when to spawn particles. You have stuff where he throws his arm or Colossus Three swings has his weapon down and the dust comes off automatically from the weapon. We didn't have to actually script that up. If you look at some of the forest areas, I don't know if you've noticed this, but the leaves actually fall off the trees, land in the water, follow the flow of the water stream and eventually actually submerge in the water and fade out. So those kind of little details in addition... like the water itself, there's a lot of resources poured in the water and again, there's not manual art scripting involved. It's good art, good technology, all just working hand in hand together. You know, you have artists painting flow maps, foam maps and simulation does the rest, essentially.
Digital Foundry: And water is also used in a lot of different areas throughout the game. I noticed you fight one Colossus in a large body of water and then there's of course the smaller stuff like the streams and little pools of water that you're talking about with the moss growing on it. It's all really good stuff.
Marco Thrush: It's a lot of attention to detail, right? Colossus fur behaviour changing when he's underwater versus above the water are tiny little things and most of them, you will probably and hopefully never notice - but they're just there to not make it pull you out of the experience.
Digital Foundry: Yeah, I really appreciate those kinds of small details. It really does kind of help build a more compelling world to explore.
Marco Thrush: We almost put ants walking up a tree in the game…[laughs]
Digital Foundry: Ah, so I think Horizon has that!
Marco Thrush: They do! [laughs]
Peter Dalton: That and next time, our character will drip water as opposed to just getting wet!
Digital Foundry: I have to ask, what up with the coins?
Marco Thrush: There are no coins, they're relics! So, the relics were pretty much a nod to the greats such as Nomad of the Colossus, who, way past the original release of the game continued giving this title extra leg and extra life. It was a nod to those people to give them something else to find, to give them the reason to want to play the new game, other than just look at the art and not just have the new experience but giving them more replay value for that point and also maybe to convince new people to look more in depth at the beauty that is the environment.
Randall Lowe: We want more than just the visual reward of the game for exploring. We wanted there to be something more material at the end of it, to say, "Yeah, I've been over every corner of this place, I've seen everything". It's a bit of a trophy to hold up at the end of the day.
Digital Foundry: Did you think about maybe including areas that appear off-limits at first but are maybe climbable in the right way? I mean, I know that in the original demo version of the game you could climb all the way up to the top of the temple. It was extremely difficult but I wonder if you included anything like that or thought about it.
Randall Lowe: There were definitely a couple of places in the game where you could get to just by using some tricks with Agro that you've probably seen and so we wanted to make sure we preserved those things for everybody that knew how to do it or had interest in learning how to do it. There's a spot in a little canyon way between the forests and one of the rolling hills that was really well known for being able to jump out of the world and so we gave some more purpose to that rather than just letting you get outside behind the world geo and really dressed it up a bit and made it kind of special.
Digital Foundry: I think Shadow of the Colossus makes a great case for an open world game that's not too large, something that's still manageable and can be crafted entirely by hand. It has a lot more personality than some of the larger worlds I've experienced. I do appreciate that, it's great.
Marco Thrush: I mean, look at the person that discovered all the relics, right? If you looked at his stats in his video, by the time you get to the end, he had spent nine hundred-something kilometres on foot and six-hundred on the horse riding through the environment. It goes to show that there's clearly enough to see in the environment even if it's not large!
Digital Foundry: We're coming up to the end, so is there anything else you want to add before we jump off here?
Marco Thrush: We're really excited about our next project and we think you will be too when you find out what it is.
Digital Foundry: I suspect we will!
Marco Thrush: And we're hiring!