- Oct 30, 2019 The results were stunning: AlphaStar had become among the most sophisticated Starcraft II players on the planet, but remarkably still not quite superhuman. There are roughly 0.2 percent of players.
- AlphaStar is a Benchmark in Aircraft Management and Private Aviation Business. Provide best-in-class flight services to our clients.
- AlphaStar is a Benchmark in Aircraft Management and Private Aviation Business. Provide best-in-class flight services to our clients. Maintain the highest safety standards and adapt best industry practices.
In late January, DeepMind broadcasted a demonstration of their StarCraft II agentAlphaStar. In Protoss v Protoss mirrors on a map used in pro play(Catalyst LE), it successfullybeat two pro players from TeamLiquid, TLO (a Zerg player) and MaNa (a Protoss player).
Maquet Alphastar Features. The Maquet AlphaStar surgical table, with electrically controlled hydraulic drive, provides essential table functions from tabletop positioning and auto-drive to reverse mode.AlphaStar can be modified to suit the needs of any surgical discipline in just a few quick steps. Ideal for smooth workflow in the operating room. Welcome to Amphastar. Amphastar Pharmaceuticals, Inc. Is a specialty pharmaceutical company that is engaged in the development, manufacture and marketing of generic and proprietary injectable, intranasal and inhalation products.
This made waves in both the StarCraft and machine learning communities. I’mmostly an ML person, but I played a lot of casual Brood War growing up and usedto follow the Brood War and SC2 pro scene.
As such, this is a two-part post. The first is a high-level overview of myreactions to the AlphaStar match and other people’s reactions to the match.The second part, linkedhere, is a more detaileddiscussion of what AlphaStar means for machine learning.
In other words, if you’re interested in deep dives into AlphaStar’s StarCraftstrategy, you may want to read something else. I recommend this analysis video by Artosisand this VOD of MaNa’s livestream about AlphaStar.
The DeepMind blog postfor AlphaStar is pretty extensive, and I’ll be assuming you’ve read thatalready, since I’ll be referring to sections of it throughout the post.
The Initial Impact
It was never a secret that DeepMind was working on StarCraft II. One of the firstthings mentioned after the AlphaGo vs Lee Sedol match was that DeepMind wasplanning to look at StarCraft. They’ve given talks at BlizzCon, helped developthe SC2 learning environment, and published a few papers about training agentsin StarCraft II minigames. It was always a matter of time.
For this reason, it hasn’t made as big an impact on me as OpenAI’s1v1 DotA 2 bot.The key difference isn’t how impressive the results were, it was how surprisingit was to hear about them. No one knew OpenAI waslooking at DotA 2, until they announced they had beaten a top player in 1v1(with conditions).Even for AlphaGo, DeepMind published a paper on Go evaluationover a year before the AlphaGo Nature paper (Maddison et al, ICLR 2015).It was on the horizon if you saw the right signs (see my post on AlphaGo if curious).
StarCraft II has had a steady stream of progress reports, and that’s lessened theshock of the initial impact. When you know a team has been working on StarCraftfor several years, and Demis Hassabis tweetsthat the SC2 demonstration will be worth watching…well, it’s hard not to expect something to happen.
In his post-match livestream, MaNa relayed a story about his DeepMind visit.In retrospect, given how many hints there were, it’s funny to hear how far they went to conceal how strongAlphaStar was in the days up to the event.
Me [MaNa] and TLO are going to be representing TeamLiquid, right? They wanted to makesure there wasn’t any kind of leak about the event, or what kind of show theywere putting on. Around the office, we had to cover ourselves with DeepMindhoodies, because me and TLO are representing TeamLiquid, with the TeamLiquidhoodie and TeamLiquid T-shirt. We walk in day oneand the projectmanagers are like, “NOOOO, don’t do that, don’t spoil it, people will see!Here are some DeepMind hoodies, do you have a normal T-shirt?”, and me and TLOare walking in with TeamLiquid gear. We didn’t know they wanted to keep it that spoiler-free.
To be fair, the question was never about whether DeepMind had positive results. It wasabout how strong their results were. On that front, they successfully hid theirprogress, and I was surprised at how strong the agent was.
How Did AlphaStar Win?
Live wallpaper app for mac. Here is an incredibly oversimplified explanation of StarCraft II.
- Each player starts with some workers and a home base. Workers can collectresources, and the home base can spend resources to build more workers.
- Workers can spend resources to build other buildings that produce stronger units, upgradeyour existing units, or provide static defenses.
- The goal is to destroy all your opponent’s buildings.
Within this is a large pool of potential strategy. For example, one thingworkers can do is build new bases. This is called expanding, and it gives youmore economy long run, but the earlier you expand, the more open you are toaggression.
AlphaStar’s style, so to speak, seems to trend in these directions.
- Never stop building workers, even when it delays building your firstexpansion.
- Build lots of Stalkers and micro them to flank and harass the enemy army untilit’s weak enough to lose to an all-in engagement. Stalkers are one of the firstunits you can build, and can hit both ground and air units from range. Theyalso have a Blink ability that lets them quickly jump in and out of battle.
- Support those Stalkers with a few other units.
From the minimal research I’ve done, none of these strategies are entirely new,but AlphaStar pushed the limits of these strategies to new places. Players have massedworkers in the past, but they’ll often stop before hitting peak mining capacity,due to marginal returns on workers. Building workers all the way to the mining cap delays your first expansion, but it also provides redundancy against worker harass, soit’s not an unreasonable strategy.
Similarly, Stalkers have always been a core Protoss unit, but they eventually getcountered by Immortals. AlphaStar seems to play around this counterby using exceptional Stalker micro to force early wins through a timing push.
It’s a bit early to tell whether humans should be copying these strategies.The heavy Stalker build may only be viable with superhuman micro(more on this later). Still, it’s exciting that it’s debatable in the first place.
Below is a diagram from the blog post, visualizing the number of each unit thelearned agents create as a function of training time. We see that Stalkers andZealots dominate the curve. This isn’t surprising, since Stalkers and Zealotsare the first attacking units you can build, and even if you’re planning to useother units, you still need some Stalkers or Zealots for defense.
I believe this is the first StarCraft II agent that learns unit compositions.The previous leading agent was one developed by Tencent (Sun et al, 2018),which followed human-designed unit compositions.
The StarCraft AI Effect
One of the running themes in machine learning is that whenever somebody gets an AI to do something new, others immediately find a reason to say it’s not a big deal.This is done either by claiming that the task solved doesn’t requireintelligence, or by homing in on some inhuman aspect of how the AI works. Forexample, the first chess AIs won thanks to large game tree searches and lots ofhuman-provided knowledge. So you can discount chess AIs by claiming that largetree searches don’t count as intelligence.
The same thing has happened with AlphaStar.Thanks to the wonders of livestreaming and Reddit, I was able to see this live,and boy was that a sight to behold. It reminded me of the routine“Everything is Amazing, and Nobody’s Happy”.(I understand that Louis C.K. has a lot of baggage these days, but I haven’t foundanother clip that expresses the right sentiment, so I’m using it anyways.)
I do think some of the criticisms are fair. The criticisms revolved around two points: the global camera, and AlphaStar’sAPM.
I’m deferring details of AlphaStar’s architecture topart 2, but the short version isthat AlphaStar is allowed to observe everything within vision of units itcontrols. By contrast, humans can only observe the minimap and the units ontheir screen, and must move the camera around to see other things.
![Alphastar Alphastar](/uploads/1/3/7/2/137227434/168377127.png)
Alphastar
There’s one match where MaNa tried building Dark Templars, and the instant theywalked into AlphaStar’s range, it immediately started building Observers tocounter them. A human wouldn’t be able to react to Dark Templars that quickly.This is further complicated by AlphaStar receiving raw game state instead of the visualrender. Getting raw game state likely makes it easierto precisely focus-fire units without overkill, and also heavily nerfs cloakin general. The way cloaking works in StarCraft is that cloaked units areuntargetable, but you can spot faint shimmers wherever there’s a cloaked unit.With proper vigilance, you can spot cloaked units, but it’s easy to miss themwith everything else you need to focus on.AlphaStar doesn’t have to spot the on-screen shimmer of cloak, since the rawgame state simply says “Dark Templar, cloaked, at position (x,y).”
The raw game state seems like an almost unfixable problem (unless you want to godown the computer vision rabbit hole), but it’s not that bad compared to theglobal camera. For what it’s worth, DeepMind trained a new agent without theglobal camera for the final showmatch, and I assume the global camera will notbe used in any future pro matches.
The more significant controversy is around AlphaStar’s APM. On average,AlphaStar acts at 280 actions per minute, less than pro play, but this isn’t thefull picture.According to the Reddit AMA, the limitation is at most 600 APM every 5 seconds,400 APM every 15 seconds, and 300 APM every 60 seconds. This was done to modelboth average pro APM and burst APM, since humans can often reach high peak APMin micro-intensive situations. During the match itself, viewers spotted thatAlphaStar’s burst APM sometimes reached 900 or even 1500 APM, far above whatwe’ve seen from any human.
These stats are backed up by the APM chart:AlphaStar’s average APM is smaller than MaNa’s, but has a longer tail.
Note that TLO’s APM numbers are inflated because the key bindings he uses leads to lots of phantom actionsthat don’t do anything. MaNa’s numbers are more reflective of pro human APM,
I mentioned earlier that AlphaStar really likes Stalkers. At times, it felt likeAlphaStar was building Stalkers in pure defiance of common sense, and it workedanyways because it had such effective blink micro. This was most on display ingame 4, where AlphaStar used Stalkers to whittle down MaNa’s Immortals,eventually destroying all of them in a game-ending victory. (Starts at 1:37:46.)
I saw a bunch of people complaining about the superhuman micro of AlphaStar, andhow it wasn’t fair. And yes, it isn’t.But it’sworth noting that before AlphaStar, it was still an open question whether botscould beat pro players at all, with no restrictions on APM. What, isthe defeat of a pro player in any capacity at all not cool enough? Did Stalkerblink micro stop being fun to watch? Are you not entertained?Why is this such a big deal?
What’s Up With APM?
After thinking about the question, I have a few theories for why people careabout APM so much.
First, StarCraft is notorious for its high APM at the professional level.This started back in Brood War, where people shared absurd demonstrations of howfast Korean pro players were with their execution.
It’s accepted wisdom that if you’re a StarCraft pro, you have to have high APM.This is to the point where many outsiders are scared by StarCraft because theythink you have to have high APM to have any fun playing StarCraft at all.Without the APM to make your units do what you want them to do, you won’t havetime to think about any of the strategy that makes StarCraft interesting.
This is wrong, and the best argument against it is the one Day[9] gave on theeve of the release of StarCraft: Brood War Remastered (starts at 4:30).
There is this illusion that in Brood War, you need to be excellent at yourmechanics before you get to be able to do the strategy. There is this ideathat if you practice for three months, you’ll have your mechanics down andthen get to play the strategy portion. This is totally false. […] If youwatch any pro play, stuff is going wrong all the time. They’re losingtrack of drop ships and missing macro back at home and they have a geyser with1 dude in it and they forget to expand. Stuff’s going wrong all the time,because it’s hard to be a commander.
This execution difficulty is an important human element of gameplay. You can only go so fast, andcan’t do everything at once, so you have to choose where to focus your efforts.
But a computer can do everything at once. I assume a lot of pros would findit unsatisfying if supreme micro was the only way computers could compete withpros at StarCraft.
Second, micro is the flashiest and most visible StarCraft skill.Any StarCraft highlight reel will have a moment where oneplayer’s ridiculous micro lets them barely win a fight they should have lost.For many people, micro is what makes StarCraft a good competitive game,because it’s a way for the better player to leverage their skill to win. Andfrom a spectator perspective, these micro fights are the most exciting parts ofthe game.
The fact that micro is so obvious matters for the third and final theory:DeepMind started by saying their agent acted within human parameters for APM,and then broke the implicit contract.
Everything DeepMind said was true. AlphaStar’s average APM is under pro averageAPM. They did consult with pros to decide what APM limits to use. When this isall mentioned to the viewer, it comes with a bunch of implications. Among themis the assumption that the fight will be fair, and that AlphaStar will not dothings that humans can’t do. AlphaStar willplay in ways that look like a very good pro.
Then, AlphaStar does something superhuman with its micro. Now, the fact thatthis is within APM limits that pros thought were reasonable doesn’t matter.What matters is that the implied contract was broken, and that’s where peoplegot mad. And because micro is soobvious to the viewer, it’s very easy to see why people were mad.I claim that if AlphaStar had usedthousands of APM at all times, people wouldn’t have been upset, because DeepMindnever would have claimed AlphaStar’s APM was within human limits, and everyonewould have accepted AlphaStar’s behavior as the way things were.
We saw a similar thing play out in the OpenAI Five showcase. The DotA team saidthat OpenAI Five had 250ms reaction times, within human limits. One of the humanspicked Axe, aiming for Blink-Call engages. OpenAI Five would insta-Hex Axe everytime they blinked into range, completely negating that strategy.We would never expect humans to do this consistently, and questions about reaction timewere among the first questions asked in the Q&A section.
I feel people are missing the wider picture: we can now train ML models thatcan play StarCraft II at Grandmaster level. It is entirely natural to ask formore restrictions, now that we’ve seen what AlphaStar can do, but I’d ask peoplenot to look down on what AlphaStar has already done. StarCraft II is a hardenough problem that any success should be celebrated, even if the end goal is tobuild an agent more human-like in its behavior.
APM does matter. Assuming all other skills are equal,the player with higher APM is going to win, because they can execute things withmore speed and precision. But APM is nothing without a strategy behind it.This should be obvious if you look at existing StarCraft bots,that use thousands of APM and yet are nowhere near pro level. Turns outlearning StarCraft strategy is hard!
If anything, I find it very impressive that AlphaStar is actually making good decisions withthe APM it has. “Micro” involves a lot of rapid, small-scale decisions aboutwhether to engage or disengage, based off context about what units are around,who has the better position and composition, and guesses on where the rest ofyour opponent’s army is. It’s hard.
For this reason, I didn’t find AlphaStar’s micro that upsetting. Theunderstanding displayed of when to advance and when to retreat was impressiveenough to me, and watchingAlphaStar micro three groups of Stalkers to simultaneouslydo hit-and-runs on MaNa’s army was incredibly entertaining.
At the same time, I could see it getting old. When fighting micro of that caliber,it’s hard to see how MaNa has a chance.
Still, it seems like an easy fix: tighten some of the APM bounds, maybe includelimitations at smaller granularity (say 1 second) to limit burst APM, and seewhat happens. If Stalker micro really is a crutch that prevents itfrom learning stronger strategies, tighter limits should force AlphaStar tolearn something new. (And if AlphaStar doesn’t have to do this, then that wouldbe good to know too.)
What’s Next?
DeepMind is free to do what they want with AlphaStar. I suspect they’ll try toaddress the concerns people have brought up, and won’t stop until they’veremoved any doubt over ML’s ability to beat pro StarCraft II players withreasonable conditions.
There are times where people in game communities worry that big companies arebuilding game AIs purely as a PR stunt, and that they don’t appreciate thebeauty in competitive play. I’ve found this is almost always false, and the sameis true here.
Let me put it thisway: one of the faces of the project is Oriol Vinyals. Based on a 35 Under 35 segmentin the MIT Technology Review, Oriol used to be the best Brood War player inSpain. Then, he worked on a StarCraft AI at UC Berkeley. Eventually, he joinedDeepMind and started working on AlphaStar.
So yeah, I don’t think the AlphaStar team is looking at StarCraft as just another game toconquer. I think they genuinely love the game and won’t stop until AlphaStar isboth better than everyone and able to teach us something new about StarCraft II.
Since blizzcon guests are able to play vs alphastar from the blizzard arcade this year, I decided to give it my best shot. For those of you who don't know me, I am a mid-GM protoss player on the NA server (best finish is rank 50). There were two 'difficulties' of alphastar that were playing; basic and advanced. Advanced being the full power AlphaStar agent that was being tested on the EU ladder. J. Allen Brack said 'if you try this, you will not win'.
Game 1 (Vs AlphaStar Advanced Zerg):
For my first game against AlphaStar I decided to do an off-meta build that I used to use a lot, a glaive adept timing followed by a DT drop. AlphaStar took a bit of damage from my build, but managed to significantly delay my third base with a pack of zerglings. The follow up roach ravager bust completely ran me over, showcasing AlphaStar's excellent mechanics and macro.
Game 2 (Vs AlphaStar Advanced Zerg):
The second game I decided to do a printf style cannon rush on new repugnancy with a double robo immortal follow up. I am not particularly experienced with this build and was hoping AlphaStar would self desctruct. AlphaStar instead expanded all over the map and managed to build a large ravager ling queen army to defeat my immortals by the time i lost my main.
Game 3 (Vs AlphaStar Advanced Protoss):
![Alphastar Alphastar](/uploads/1/3/7/2/137227434/674910934.jpg)
AlphaStar did what at first looked like a normal pvp opener, but made 2 zealots out of its first two gates. I decided to attempt to punish this with some stalker pressure, only to discover that alphastar had used the gas it saved by making zealots to rush a large amount of phoenixes. Without proper tools to deal with phoenix harass my economy was slowly picked apart and I died to a the first big attack.
Game 4: (Vs AlphaStar Advanced Protoss)
AlphaStar opened with a low ground gateway with a fast expand (in pvp, what?) and I decided to punish it with 3 gate robo all in. AlphaStar rushed an immortal and a warp prism and displayed absolutely god-tier micro to hold my push.
Game 5: (Vs AlphaStar Advanced Protoss:
I attempted to do a mostly standard build, I opened with stalker sentry and went into blink. AlphaStar did more or less the same build as me and we both expanded. My blink finished and I decided to go pressure. AlphaStar then moved out with blink and a warp prism, and went I tried to fight it did BOTH BLINK MICRO AND PRISM MICRO AT THE SAME TIME. I was completely unable to fight its army and was quickly defeated.
Game 6: (Vs AlphaStar Advanced Protoss):
Finally, I threw in the towel, this bot needed taste defeat. So I decided to cannon rush again (this time in a matchup that I am a bit more familiar with cannon rushing in). AlphaStar saw my cannon rush and at first did a pretty solid defense, pulling probes to prevent cannons from going up and to block me from walling in a cannon. Eventually I got a cannon up, but the huge amount of probes that alphastar pulled to the low ground would eventually kill it. This is when AlphaStar made its first and last mistake, it had pulled WAY too many probes into the enclosed area by my pylons, and I quickly placed two pylons to trap nearly all of its probes. With no probes mining, AlphaStar died to a few zealots. Sweet victory, take that J. Allen Brack!
Bonus: At the end of game 6, AlphaStar had a single probe left and a pylon. When I found the pylon with my units AlphaStar did the GG spray before being eliminated. The deepmind devs said they didn't tell AlphaStar to do it. Here's a video of AlphaStar's final moments where you can see the GG spray:
Source: Original link
- All AlphaStar final PvZ games summarizedGeneral opening and game plan: Standard Stargate opening with probescout and 2 adepts Probescout doesn’t check for 3rd or where lings are going 2 adepts shade in to deal damage 4min 3rd after 2nd gate in wall Build oracles for harassment and holding 3rd Robo continuously harass with adepts shading into mineral lines ~5 oracles…
- How to Beat AlphaStar: From the Perspective of a Grandmaster Player and Data ScientistTo preface this discussion, I want to state that despite the title of this post, I still think AlphaStar is going to beat most, if not all, of us. The people at Deepmind are brilliant and the demonstration in Janurary was nothing short of remarkable. Yet, perhaps you, the high gold/low diamond zerg player, will…
- 'Toxic' draft turned into one of the most fun games I've ever played :)Map: Sky Temple our team first 3 picks Butcher Nova Nazeebo In the beginning, there was some toxicity the leader tried to ask about who to ban so the first pick could choose whatever they wanted, but I think they were AFK. ( it happens to us all the queues are long lol )…
© Post 'I played (and beat) AlphaStar Advanced At Blizzcon, here is how the games went.' for game StarCraft.
Top 10 Most Anticipated Video Games of 2020
Alphastardx
2020 will have something to satisfy classic and modern gamers alike. To be eligible for the list, the game must be confirmed for 2020, or there should be good reason to expect its release in that year. Therefore, upcoming games with a mere announcement and no discernible release date will not be included.
Top 15 NEW Games of 2020 [FIRST HALF]
2020 has a ton to look forward to..in the video gaming world. Here are fifteen games we're looking forward to in the first half of 2020.