Golfer AI Sketching: Memory

Golfer AI Sketching: Memory

Hello folks!

As promised, today I’m going to talk about the Memory component of our AI golfers!

So far, we’ve taken a look at how the behavior is controlled via State Machines; we’ve looked at how we’ll use Utility to choose the best course of action; and we’ve also taken a look at how the AI will execute Actions through a system of intentions.

But we want our AI to be smart; this means that we want the AI to be able to learn from its past mistakes in order to improve its score and rating. And we do that through memory!

Memory will be separated by holes. The AI will record the intention of the shots it took, the location of each shot, rate the outcome, and the overall performance for any given hole. Next time the AI plays the same hole, it will generate the actions as usual, but it will also compare with the previous taken in similar circumstances, and compare the utility value of that action against the outcome, and weigh it against current circumstances.

A simple example to illustrate the system is driving off the hole below:

memory_shot

In the image above, we have three shooting options that the AI came up with.

  • The top orange option is the shot that the AI took the last time it went around the course
  • The bottom orange option is the shot that the AI calculated as the best for this time around
  • The purple shot is what actually happened

So in the situation above, we have an AI that remembered that the last time around this hole, it took an awesome shot through the left side of the fairway, stopping quite near the edge of the water but within awesome range of the green.

For the other options that the AI came up with, among them the bottom orange one, through the right side of the fairway, was the one that the AI felt most promising at this time.

The outcome of the left-side action was stronger than the predicted outcome for the right-side action. The AI evaluates this. It looks into its memory for the previous outcome, and compares with what it expects of the other outcomes. In this case, it is pretty clear that it is the superior action.

But, alas, just like us humans, the AI will also remember the past in a different light.

The last time it played that particular hole, it had a super drive right of the tee, exceeding its expectations and being regarded as a “star memory”, meaning it values it far higher than what it should. So memories aren’t just a 1 to 1 remembrance of the past: they also have their fair share of rose-tinted glasses. If an AI does better than it expects on a shot, its memory will also be colored by that event. Granted, this will depend on mental attributes for the AI, but it will be a likely occurrence.

And more than that, the accuracy of the memory also decays with time. Since the last time the AI played this hole was months before the current moment, the outcome will be slightly mistaken, as will the parameters the AI used to take the shot. In this case for this particular AI character, he will overestimate his strength,  he will mistaken his position, and he will end up landing the ball square on the bunker trap. This will now take the place as his most recent memory of the driving stroke of this hole. He won’t immediately forget his previous memory; it’ll just have a weaker connection since the next attempt at that resulted in catastrophe.

Next time he plays around, he’ll likely try something new, or he’ll try the previous previous memory (IE: the top orange stroke) but will be more careful this time around.

The memory system will also work to help with the knowledge representation of the hole. The AI will remember each hole in a tile/type/landscape system, so that it can always compare the hole to the memory it has of the hole. So if the hole has changed, either through your own landscaping efforts or otherwise, the AI will know, will disregard the memories of past strokes a bit more, and will focus on trying something new.

This should end with an AI that is always trying to familiarize itself with your course, and has past performance to base future performances off of. It’ll also help in making the AI decide its feelings towards a particular hole (phrasing!), so that it can react to the changes you make to your course.

So I hope you enjoyed this overview! It’s very high level, but it should give you some clarity as to how the AI will work. Now we have the 4 main systems sketched out, I can continue implementing them. Will be done with the new terrain this weekend, then I can focus on finishing the first pass of the AI. For this month, the goal is to have AI golfers playing with your holes (phrasing!) so that I can test out some basics of the systems.

That’s all for now, folks! Stay tuned for more!

Terrain Sculpting

Terrain Sculpting

Hello folks!

As I’ve said previously, while coming up with the concept and the systems for AI, I’ve also decided to rework a bit of the terrain features in order to make it smarter and improve performance.

So I’m almost done plugging in the new terrain system into the game. This time around, I went a bit smarter with my separation of concerns. The current system in the game features a breakdown of the mesh into tiles, quads, triangles, vertices, all controlled by a single TerrainController script which manages the actions on each tile.

It works ok, but it’s a bit overcomplicated in certain areas, and too simple in others. The point that really sticks out is when it has to update the mesh after any alteration is done to it.

Basically, it is currently rebuilding the vertice/triangle/UV list every time you do an alteration. This isn’t ideal, as it contains loops inside loops querying data that hasn’t been changed to find the ones that have been. Plus the current system is also less conducive to having multiple chunks and altering between them. It would be fine if we just stuck to one chunk and made it massive, but that’s not ideal specially if we want really massive levels and easier to manage data.

So, enter the new system! This time around, I have a SmartMesh class who holds the reference to the Mesh component itself, as well as a TerrainChunk component that holds it all together, managed by a TerrainBase class that can decide how many chunks will be spawned and the dimensions for each chunk. Reworking the input management as well means cleaner code and easier to expand. Most important is that all vertex data is held by TerrainVertice objects, which can be directly accessed either y the TerrainChunk or the SmartMesh. The SmartMesh also generates a mesh on startup of the position/index of each TerrainVertice. So what dos this mean?

It means that when a TerrainVertice is changed, it changes its position, and it tells the SmartMesh which index of the vertex list needs to be updated. This is done, and the new vertex list is assigned to the Mesh component. Hey presto; no more nasty loops within loops within loops!

Similar concept will be applied to UV and other aspects of the terrain. So the end result is a cleaner system that’s more manageable. I’ve also picked up more knowledge as I’ve progressed with development, and it has been ideal to implement it now.

Oh, and you’ll also be able to “sculpt” the terrain better, and I’m also reworking the height limitations of the course so you can have nicer features. Check out the simple demo below:

terrain_painting

You’ll be able to “paint” the heights with the new tool. (Also keeping the old options too for those who prefer them). And since the vertices are easier to manage, I’m also updating the height limitations to give you more freedom to build taller things, and smoothing out the process: if you move a vertice too far above a threshold, the neighbouring vertices will also move up, creating a more organic flow.

I should be done with this work by the weekend. At the same time, I’m also implementing the AI system, so stay tuned for that too!

Tomorrow I’ll make a post about the memory system!

Have a nice day, folks!

Golfer AI Sketching: Actions

Golfer AI Sketching: Actions

Hello folks!

So far, we’ve described the state machine used for keeping track and modifying the behavior of the AI, and the utility function that is used to select what course of action the AI should take.

And now it’s time to talk about the other piece of the AI puzzle: Actions!

Put simply, these are the functions that the AI executes to perform an action in game; things like moving, idling, shooting etc. All of those actions are self contained in their own methods, take a parameter or three, and execute until the action is successful or fails.

Of special interest to Boss Golf is the shooting action. In golf, there are countless ways that you can hit your golf ball. You can add some backspin, you can give it some fade, you can try to go straight, you can go for a higher angle, you can go full force etc. Your success with that action, however, depends on myriad factors like your skill level, your confidence, your performance, the weather…

Therefore, for Boss Golf, when it comes to the shooting action, I’ll implement another subsystem that is based on intention, that is, the AI will tell the gameplay system what sort of shooting action it plans on executing, and the gameplay system will take all of the aforementioned factors and calculate the outcome for the AI. This will give more “human” results to the actions of the AI, as they would never be able to hit the same shot exactly the same way twice. There will always be deviations.

Breaking the system down:

  • The AI would come up with a handful of potential shooting actions for a given situation; it would think of a route by hammering it over the pond, another route by cutting through the trees, and another route by just hitting the fairway.
  • It would then parse those actions through the utility system, to see what action would be the one with the best potential reward/utility for the AI
  • After it found the best course of action, it would communicate the intention to the gameplay system, telling it it wants to:
    • Hit ball with club X
    • Hit ball with strength Y
    • Aim shot at square Z
    • Add X amount of backspin
  • The gameplay system would then modify these attributes based on the environmental factors surrounding the AI
    • If the AI is lacking confidence, add some uncertainty to the aiming
    • If the AI is behind in the championship, add some extra strength due to nerves
    • If the AI doesn’t have great skill, add way more backspin than the AI intended
    • If the AI has a lucky modifier, alter the shot slightly so that it lands in a slightly better position
  • The action is executed, the AI hits the ball with the modified intention, and analyzes the result

Of course, this is a higher level breakdown of what is going on behind the scenes, to better illustrate to you the capability of the AI. There will be a myriad other factors that can potential add a modifier to the AI’s decision. Our friend Jumper made a very nice write-up on our discord channel,  giving his take as a sports psychologist on the mind of the players and the many factors that can affect their performance on the pitch.

Now we know how to manage behavior, how to make decisions, and how to act with intention. There’s one more piece of the puzzle missing: Memory! As golfers go around the course, we want them to remember the way they tackled a hole before, so that they can try and improve on it or follow their old steps. I’ll break this feature down on my next post!

That’s it for today’s update, folks!

As an addendum, I’ve been hard at work implementing the AI, and also redoing part of the terrain system as it’s grown to be too bloated and not smart enough for the AI to work with. I’ll be done with that by the next weekend, and hopefully have some AI gameplay to show as well. Such is the nature of game development: the best ideas always come after you’ve implemented a system. This new structure will make it easy for the AI to evaluate the shots, as well as to place decorations and other structures on the course.

So stay tuned for it!

Golfer AI Sketching: Utility

Golfer AI Sketching: Utility

Hello folks!

Today I’ll be talking about another part of the AI of Boss Golf: Utility

Simply put, Utility is a value that determines how useful an action is for the AI to take at a specific moment in time. For example:

Take an enemy AI that has a health value and an ammo value. And it has only three possible actions:

 

  • Use Healthpack
  • Reload
  • Shoot

At the start of a conflict with a player, if the AI is at full health and full ammo, then the utility value of the first two action is pretty much zero: Shooting would be a much more useful action in that specific moment.

As the AI engages in combat with player, things get more murky. If the ammo is approaching zero, then the utility of the reload action increases; the act of reloading becomes more important than shooting as you need bullets in order to shoot. At the same time, as the health of the AI decreases, the utility factor of the use healthpack action increases, as the AI needs health otherwise it will die.

The way this calculation is done depends a lot on the factors and data that the AI has access to. If it knows that the player is at low health, then reloading first instead of healing might be a better option. If it’s a particularly coward enemy, it will give priority to heal itself instead.

And this is just with three possible actions; add in more actions, such as running, finding cover, dodging etc, and you get much more variety of behavior.

The same thing will happen in Boss Golf, specifically when it comes to deciding the type of shot to take along a hole. Let’s use a sample hole to illustrate how it would work:

tricky_hole
A tricky Par 3.

In the Par 3 above, hitting the green from the start seems like a solid choice. The hole is well within the 150 yard range, which is achievable by most golfers that take to the course. But even pros make mistake; and there’s a couple of factors that can affect the success of a stroke:

  • Wind
  • Confidence
  • Performance so far

Wind can drastically affect a stroke, making your ball go lower than expected, slower than expected, veer laterally.

Confidence can affect the quality of your stroke. Lower confidence can lead to slice/hook, lack of power, reduce your accuracy, among other things.

Performance so far. If the golfer has been having a great day, he’ll be more likely to take risks; If he’s playing his ultimate best, he might be more risk-averse in order to keep his good handicap; if he’s having a bad day, he’ll take even less chances in order to try and recover.

All of those attributes (and more) would affect the calculation for utility of the shot. For Boss Golf, the golfers would always calculate a specific number of shot (based on their attributes), calculate the utility for them, and make the choice.

Below I’ve simulated some potential shots and their results, to give you an idea of what could happen:

tricky_hole_results

Some golfers would try and keep to the straight brown shot, getting very close to the hole. Others might go the blue route since they’re lacking the confidence they’d be able to hit the ball across the lake in one go. Some might overshoot it into the green following the pink line because they believe they needed more power. The guy in the green line may have messed up his first shot due to lack of confidence. And the poor fella on the white line just sent it straight into the water hazard.

After taking the shot, the result would also affect his current attributes, affecting his next shooting decisions as a result. So if the shot landed where the golfer wanted or better, his confidence will increase, so he’ll be more prone to taking a risky shot next; by the same token, if the shot ended up going badly, his confidence could decrease, leading him to be more risk averse on his next shot.

And that’s about it. Now we know how the AI will be controlled (via State Machines) and we know how he’ll decide which action to take (via Utility).

But we’re still missing one key piece of the puzzle: the Actions themselves.

Boss Golf features an action system that limits the possibility/quality of actions by the attributes/ability of the golfer. A pro-level AI golfer will have more tools (read: shots) at his disposal than a first-timer out in the tee. He’ll know when to pull, when to push, when to chip, when to approach, and be able to use these actions when out in the green, whereas a first-timer will be limited to less imaginative and simpler shots. As they grow in skill, they’ll be able to add more shots to their arsenal.

All of that will have to be factored in by the player when designing the course. If it’s catered mainly to beginners, it makes little sense to have very difficult courses; if it’s catered to professionals, it would make little sense to have too many easy holes. This balance will be vital in the development of your golf course.

I’ll make a write-up about the actions on my next post this week. Writing about the systems like this has been very helpful in aiding me with visualizing how the system works and building it!

So expect AI integration by the end of this month, bringing life to your course!

That’s all for today folks!

Golfer AI Sketching: State Machines

Golfer AI Sketching: State Machines

Hello folks!

To break the silence today, I’ll talk about the sketching of the AI of the golfers in Boss Golf, something that I’ve been working on over the past week!

I have a great interest in AI, being an AI programmer professionally, so for Boss Golf I want to create some very interesting things with AI, to reinforce the living feeling of the NPCs, as well as give rise to the unpredictable events in the game.

To start off, I’ll describe the main backbone of the AI in Boss Golf: State Machines.

State Machines are basically a set of nodes that determine the current state of the AI (sleeping, idling etc), and how they transition into each other based on set instructions. How/when these transitions happen are decided by the programmer.

For Boss Golf, since gameplay is very segmented, state machines will be used to determine the basic state that the AI is in. Some of these states include, but are not limited to; teeing, putting, approaching, idling, waiting, interacting, resting, moving. The more expanded the state machine is, the more different and interesting behaviors we can get out of the AI.

For example: why separate the act of playing golf into three separate states? (Teeing, Approaching, Puttting) They could easily be condensed into one “playing” state since the actions that the AI will perform are very similar. (moving and hitting the ball)

The answer is that by adding this degree of separation to the behavior, we can have the AI interact and respond differently to events that happen at different points of play, in a more neat or organized way. If rain interrupts play during teeing off, it’ll have less impact on a golfer’s mentality than if he was putting; the stakes are different. Instead of having to keep track of how many strokes he has and how close he is to the hole, we can simply query the current state, and make the AI react accordingly.

And that’s just on one particular possible event. When you combine with other events, and add layers of actions and behaviors, you end up with a more interesting AI rather than one that simple goes around hitting the ball.

We can even add some variety to the transitions based on randomness or the attributes of the AI character. A golfer with poor health/lower stamina may have a bigger chance of switching to the rest state after a hole than a golfer in better health; a golfer with more aggressiveness maybhave a bigger chance of switching to a raging state after a failed easy shot than a calmer golfer.

And that’s the basics of it. I’ve sketched out some basic states that need to be in the game, and am in the process of implementing them. With state machines, we can also add new states at any point in time since they are self contained, so expanding the AI can be done quite easily.

Next time I’ll talk about another aspect of the AI that requires more difficult work to get it working correctly: Utility Functions.

That’s all for today folks!

Golf Gameplay: Character Automatic Movement

Golf Gameplay: Character Automatic Movement

Hello folks!

For today’s golf gameplay-related dev update, I’ve added automatic movement to the character, for reaching the ball and sort of positioning himself to take the shot!

It was easier than expected, thanks to some foresight of past Gus when coding the game. I simply moved the function to shoot the ball away from instantaneous execution after releasing the mouse button over the desired location, and put it in another action executed by the golfer character once it has reached the calculated point of shooting.

This calculation is a bit dirty now, as it only tries to get position slightly away from the ball and perpendicular to the direction from the ball to the desired target. For the current prototyping purposes it works.

The golfer also slightly delay the shot when he arrives in position, in order to “think” about how to hit the ball. I’ll add some animations and fluff to this step later.

You can see the execution of a hole with the new system below:

Golfer walks to the tee and drives the ball.
Golfer teeing off!

 

Chip shot on the edge of the bunker
Chipping it in closer to the green!

 

Little golfer chips it home for a par.
Sweet chip in, Payne!

And that’s it for today’s update on golf gameplay! Still need to automate the moving to the ball more, so that the doesn’t wait for your to finish deciding on the stroke before taking position. And the shot visualizer thing is still broken, giving out NaN results depending on the slope. I’ll make a better fix for that later!

That’s all, folks! Keep on teeing! And may your strokes never slice!

(Also, starting from today, I’ll drop the numbered update from the format and put in a more relevant title off the bat. This will give you, readers, a quicker idea of what the feature is relevant to straight off the title. Categories like Golf Gameplay, Tycoon Gameplay, Management Gameplay etc will make it easier for navigation!)

Dev Update #39: Golfer is in the house!

Dev Update #39: Golfer is in the house!

Hello folks!

Today’s update is about making the course less lonely!

payne_legacy

I’ve added a prototype character to start implementing character controls and AI.

This will affect the shooting in the following way:

-You will decide the shot as you do now, from where the ball currently is and the desired destination.

-Unlike before, now your shot won’t be done instantly: the player character will first have to move to position, and then swing the club as per your “instruction”.

-Once the ball comes to a stop, you’ll be able to decide the shot again, while your character makes his way to the ball. Then once he reaches it, if you’ve decided the shot, he will strike it; if not, he’ll wait for you to do so.

This indirect manipulation of the shots will be vital in implementing the AI. In fact, with your player character, you’ll also be able to let the AI control him if you’d like, so that he can play some rounds of golf and improve while you’re busy off making a new hole. The AI will make use of the systems I’m implementing to create the shots and try to get a good score.

I’ll also test out an attribute system so that your shots have a bit of imperfection based on your character’s strengths and weaknesses. This will refine the golf gameplay further, increasing the challenge and the planning.

You can see a mock up of how the golf gameplay will play out here:

payne_shot

That’s all for today folks!

(Bonus points if you can guess the golfer I’ve based the character on!)