Limiting the scope

Something that happened while I was working on this project was that I noticed that finding a solution is much harder than I expected (for some reason this happens to me every time). I intentionally left the scope of the project very broad because I had no idea how hard it would be to make this work in Unreal Engine 4. After implementing my first attempt I knew that it would be way too much work to implement everything. Personally I think the most interesting part of this project is drawing the actual desire paths on the terrain so I am limiting the scope to just that.

Pixel painting on landscapes, promising, but not quite it

Today I'm going to explain vertex painting on (landscape) materials using only blueprints. It's quite easy to implement and does the job nicely, but... it is also not working the way I want it to. So it was quite a nice experiment, but I have to look for another solution to my problem. To prevent disappointment for the people that have the same problem as I did (don't worry, I have a solution now!) I'll quickly explain what this approach can and cannot do, before going in dept on how to do it.

So after I implemented this I could paint on the material on a per pixel basis, using only blueprints. The problems I encountered were that it doesn't use a separate layer, meaning that the resulting 'drawing' can't be used as a mask. And the biggest problem was persistence, it seems that Unreal Engine renders the material in a separate thread. I managed to send the player location to this using so called material parameter collections, but I can only use this to send one vector. So if I want to draw a path, ideally stored in something like an array, I have no way to send it all at once. The result was a terrain that flickered as different parts of the array where being drawn and erased in sequence. So take that as a warning before you read on and spend time on implementing this.

Now on to the exciting stuff! As always I had no real plan on how to tackle this issue, so I headed out to google and the Unreal community forums and founds some interesting suggestions. For this particular solution, this one inspired me most. It shows nicely how to change the color of a mesh using only blueprints.

This is a screenshot of the material I made to color the landscape material based on the location of the player.

As usually the problem with blueprints is that they can get quite messy as they get bigger. Maybe I can write a blog about organizing blueprints one day, something that I obviously didn't do this time. But if you look carefully and just follow the lines you must be able to figure this one out.

First thing you have to know about material blueprints is that they are evaluated per pixel, so for every pixel in the material this blueprint is run and determines the color for that pixel. It is very essential to keep this in mind since it gets confusing quickly when you start thinking about the blueprint as setting the whole material at once (trust me, I've been there). Starting from the upper left corner you can see how the blueprint gets the current pixel's location and compares it to the player location. If it is within the affected radius it should be colored with a linear interpolation (Lerp) between green and grey, this gives a nice fall off effect where the green intensity decreases the further the pixel is from the player. If the pixel is outside the affected radius it is simply colored grey.

Now that I have a material that can change it's color based on the player location I have covered the hardest part of this solution. One thing that remains is actually getting the player location in this material, since it can't access it directly. This is where material parameter collections come in. If you want to know how to use them, check the link to the official documentation, I don't have anything to add to that. However I will show how I implemented the setting of these parameters in my solution.

I made this simple function that sets the parameter, I think this speaks for itself. Then I simply call this function every tick and pass the current player location to it:

You can place this function in your level blueprint or to any other class blueprint that you want to use for this.

Now the only thing you have to do is to apply the material we just created to the landscape. Then when you compile and run the game it should look something like this:

As you can see it paints nicely on the landscape, but as I explained before, it doesn't stay. And there is no way to send more data to it then just the vector with a location. So unfortunately I have to find yet another solution.

Vertex painting on landscapes, challenge accepted.

In the previous post I explained how to do pixel painting on meshes. My idea at the time was to also use this technique on landscapes. However, unfortunately Unreal Engine 4 uses a different kind of of object for landscapes, it's not a mesh. So the same functions do not work here. I decided to put this solution in the fridge and try a completely different approach. Maybe I can still use vertex painting later if I want to change the material on meshes as well.

My new attempt will focus on changing properties of the material used by landscapes. I have found some encouraging sources that show how to do it using just Blueprints. I'll update my progress here.

Vertex painting on meshes

My original idea for showing the effects of player movement on the materials in the world was to use vertex painting. With this technique it is possible to set the red, green, blue and alpha channels of a material per vertex of the mesh. If we also count the amount of times a player passes a mesh we can use this to show player movement. While developing I mostly use the RGB-channels, the idea is to later use the alpha channel to create a nice blend between textures.

I make as much use of blueprints as I can, since this is a very simple and intuitive way to program for people with no background in programming, so my code can easily be used and possibly maintained by others in the future. However not everything I want to achieve can be done with just using blueprints so I make some functions in C++. These C++ functions can then be called from within blueprints.

Vertex painting is something that is a standard Unreal Engine 4 feature. It is however something that is usually done in the editor, so before compiling the game. In our case we want to paint the vertices dynamically after compiling, so at runtime. Some searching on the internet taught me that this cannot be achieved in blueprints, as they don't allow you to access material properties. At this moment I'm still very unfamiliar with the structure of Unreal Engine 4 I started looking into examples of how to access these properties in C++. I found a couple that helped me a lot, like this one.

Inspired by what I read I was sure that I could achieve realtime vertex painting so I put together my own test code example. This consisted of a very simple blueprint that called a C++ function on every tick and passed a mesh and a color (that I didn't use yet) as parameters.

One extra thing that needed to be done was to adjust the material to make use of vertex colors, to test this I made an extremely simpel material that only uses vertex colors. A real material would of course be way more complicated. But this shows enough for testing. The material looks like this:

Naturally the C++ function is a little more complex:

int APixelPainter::drawAllPixels(UStaticMeshComponent* StaticMeshComponent, const FLinearColor& FillColor, bool bConvertToSRGB)
{
  if (!StaticMeshComponent || !StaticMeshComponent->StaticMesh)
  {
    return 0;
  }

  const int32 NumMeshLODs = StaticMeshComponent->StaticMesh->GetNumLODs();
  StaticMeshComponent->SetLODDataCount(NumMeshLODs, NumMeshLODs);

  const FColor Color = FillColor.ToFColor(bConvertToSRGB);
  uint32 LODIndex = 0;
  for (FStaticMeshComponentLODInfo& LODInfo : StaticMeshComponent->LODData)
  {
    StaticMeshComponent->RemoveInstanceVertexColorsFromLOD(LODIndex);
    check(LODInfo.OverrideVertexColors == nullptr);

    LODInfo.OverrideVertexColors = new FColorVertexBuffer;
  
    FColor colors[24] = {FColor::Blue, FColor::White,FColor::White, FColor::White, FColor::White, FColor::White,
       FColor::Blue, FColor::White, FColor::White, FColor::White, FColor::White, FColor::White,
       FColor::Blue, FColor::White, FColor::White, FColor::White, FColor::White, FColor::White,
       FColor::Blue, FColor::White, FColor::White, FColor::White, FColor::White, FColor::White };
       
    TArray<FColor> colorArr;
    colorArr.Append(colors, ARRAY_COUNT(colors));
    LODInfo.OverrideVertexColors->InitFromColorArray(colorArr);
    BeginInitResource(LODInfo.OverrideVertexColors);
    
    LODIndex++;
  }

  StaticMeshComponent->CachePaintedDataIfNecessary();
  StaticMeshComponent->MarkRenderStateDirty();
  StaticMeshComponent->bDisallowMeshPaintPerInstance = true;
  return 1;
}

The code is quite self explanatory, I'll go through it quickly. Basically the code overrides the colors of the vertices for each level of detail (LOD). In my example I only have one LOD so I'm not sure if it would work with more, I didn't test that. The new colors are specified per vertex in an array. In a later iteration the new color could be specified on the world position of the vertices. After the override colors are set, the render state of mesh is marked as being dirty, to guarantee that the renderer picks it up to be redrawn.

In the following gif you can see the result. The object is all white before the game runs, no vertex colors are specified. When the game is started the colors are changed in realtime.

I'm quite pleased with this result, of course a lot of additional work is still needed to change this into a good looking desire path. But it shows that I can change the color (or texture for that matter) per vertex in the game at runtime.

Planning for this blog

My idea for this blog is to describe the things I encounter while working on the project that I think are interesting and will help others in dealing with similar issues. In the meantime I will also post about my progress and challenges that I have to face.

I am someone who gets lost in the flow of things that I find interesting, that's how it happened that I am already behind on this blog, before I even properly started. But bear with me I have some interesting things coming up!

Project Proposal

Background

The idea is to create a complex multiplayer environment where people can change the environment by conscious and subconscious actions. One of these subconscious actions that affect the environment is simply the effect of human presence. A clear example of this is that when people frequently travel the same path across a field, the grass will wear down and a visible path dirt track will appear instead. These paths are also known as desire paths. A similar thing happens with kids touching corners of buildings when running around them, the grease and dirt on their fingers will leave stains. Various similar effects can be thought of.

Goal

Show frequently traveled paths in a virtual environment by affecting terrain and assets.

Potential issues

• How to handle height differences
• How to make it look natural
• How to change the appearance based on ground material
• Saving the data
Saving all player moves will require a lot of storage, potentially crippling the game at some point. Especially in persistent a persistent online game.

Implementation idea

This project will be implemented in Unreal Engine 4. The general idea for implementation is to have a grid (heatmap) overlaying the terrain and simply counting the number of times a player passes a grid square. Then the terrain will be adjusted according to this number. The grid can be easily saved and the size won’t increase over time. The visual changes will initially be displayed using vertex painting, with this technique colors and textures of materials can be changed at runtime, without actually changing the material.

Additional ideas

• These ideas will probably not be implemented for this project, but might be interested in the future.
• Effects disappear over time
• Terrain deformation (so not only the material, or an overlay but the actual terrain shapes will change)
• Spawning or destruction of assets based on player movements.