I defined the action space using actual joystick and button inputs, then designed reward and penalty signals around damage dealt, match results, successful punishment, and other outcomes to teach the agents Tekken 3.
Put simply, I rewarded them for dealing damage or winning, and penalized them for taking damage or losing, repeating the process millions to tens of millions of times.
I chose Tekken 3 because it is lightweight, can run at increased speed, and has relatively simple systems. It was also the first Tekken to introduce sidestepping, which seemed like a manageable starting point for building the training environment.
At the very beginning, both agents wiggle the stick and mash random buttons, regardless of distance, frame situation, or remaining health.
It looks pretty much like our first time playing Tekken at the arcade.
As training continued, Player 2 Jin settled on repeatedly using low kicks. It took around a million iterations before the first successful low block.
But the defending Jin still had no idea what to do afterward. He tried his usual throw, only for it to whiff because he was too far away.
The match after 2,103,296 iterations.
Throws are starting to get broken, though it might just be the button mashing. Player 2 Jin still relies on low kicks and crouching jabs, either out of habit or because they keep working.
At a real arcade, that would have earned him a few invitations to the parking lot.
There is even a successful parry in there, accidental or otherwise.
After a few million iterations.
He finally starts throwing electrics.
The execution is not consistent yet, but a basic combo structure is starting to emerge.
Electric, jab, dash electric. Seriously?
Is this really the same button-mashing Player 1 Jin? I am so proud...
The crouching-jab-and-low-kick meta gives way to jabs and electrics. At longer range, the agents start placing a high kick with an animation very similar to the kick in modern Asuka’s staple combo.
It reminds me of Lee throwing out Silver Heel to control space.
But wait, did I just see that?
It did not accomplish much yet, but Player 1 Jin just used a sidestep, the system first introduced in Tekken 3.
That little hint of potential turns into something incredible: he sidesteps counterclockwise around a df+1 and launches with the launching uppercut.
Maybe he loved how cool it looked, or maybe he decided it was effective. From this point on, Player 1 Jin starts throwing out counterclockwise sidestep into the launching uppercut as naturally as breathing...
Now he is bold enough to throw out the launching uppercut right at the start of the round.
But Player 2 Jin is learning too. Over the repeated matches, he starts evading the launching uppercut with his own counterclockwise sidestep and punishing it.
What interests me here is that the two agents developed different styles despite starting with the same character under the same conditions.
Player 1 Jin became a sidestep-into-uppercut machine, while Player 2 developed a habit of using keep-out high kicks and parries.
Their choices on wake-up and their combo routes also developed differently.
“Cut it out, you bastard.”
The accelerated training makes it hard to see, but there is a clean clockwise sidestep followed by a punish.
After his sidestep-into-attack attempts keep getting evaded or blocked, he seems to change his approach: sidestep counterclockwise around a low kick, dash in, then use a jumping launcher.
The dash is particularly impressive. It looks as though he recognizes the whiff but also realizes his attack would not reach without closing the gap first.
Another interesting change: as the match count rises, the aggressive, Tekken 8-style button-pressing meta gradually fades, and blocking becomes more frequent.
Punishment is still basic, but the agents are clearly trying to punish blocked the launching uppercut attempts, even if it is only with a jab.
This is also the first time I start seeing the idea of “sidestep guard.”
Here, a clockwise sidestep into guard neatly blocks Player 2 Jin’s signature high kick.
There has not been nearly enough training to reach the Mishima mind games I wanted to see: wavedashing into dash-crouch or clockwise sidestep-crouch. Even wavedashing itself has not appeared yet. Still, these early stages have produced plenty of interesting moments.
For example, Player 1 Jin uses an axe kick after a throw as a basic wake-up catch, and Player 2 eventually learns to beat it with a quick recovery.
The catch is that I barely know Tekken 3 myself. If any veterans have ideas for making this experiment more interesting, I would love to hear them.
They seem to have decided that attacking first puts them at a disadvantage. They barely commit to attacks anymore, instead using the same fundamental movement you see in modern Tekken: sidestep guard, sidestep crouch, dash crouch, dash guard, backdash guard, and wavedash guard.
It is fascinating to see these behaviors emerge without ever demonstrating the inputs to them.
One whiff gets launched by an electric. After that, backdashing for the rest of the round is usually enough to secure the win, so their strategy seems to be evolving toward movement >>>>> attacking.
When they do attack, it is mostly an electric or a crouching jab.
Training has reached a stalemate: they either land one hit and run out the round, or keep backdashing until a draw. I am stopping here and planning to try another character or another game in the series.
Personally, I would really love to see how this plays out in Tekken 8.
Source: Tekken 8 community on DCInside [View original]