Combat : Vulture vs Zealot
This scenario is a battle situation between Terran’s Vulture (agent) and Protoss’s Zealot (enemy). Terran vulture is one of the most important units in the Terran race and is used in almost every game. Therefore, the good vulture control is key component of winning for Terran race. An expected example of the movement of properly trained agent is to use ‘Patrol’ and ‘Move’ command in proper sequence. It is a movement called ‘P-Control’ which is possible to attack and move simultaneously. This control also can be seen frequently in the game among professional gamers.
Action Space
Vulture needs two types of action to face the zealot.
Move and Patrol. If you use the patrol command in the game, you can automatically attack the nearby enemy.
That’s why the set of actions in this scenario doesn’t need to include the attack command.
Moreover, as described in Combat : Vulture vs Zealot, the patrol is more efficient that attack command, so it is good enough to replace the attack command.
It is also an important question where to move.
In fact, the movement of the unit in StarCraft is possible throughout the entire area of the map.
(except for non walkable tiles), however, it does not matter if you limit the range of movement of the unit to deal with the enemy nearby.
Therefore, the users have to make two decisions. The first is ‘move angle’ which means the direction to go from centered itself and ‘move dist’ which means how much to move to the direction.
For example, if ‘move angle’ is 30 (degree), the number of available movement direction is 12 because it is 360 / 30 = 12, and if ‘move distance’ is 2, it will always move by 2 tiles (1Tile = 32 Pixels) at each step.
It also provides continuous type action. In this case, it is able to move every direction literally.
| Action Type | Type | Output | Description |
|---|---|---|---|
| 0 | Discrete | one integer between 0 ~ action size | Output number means the moving direction. 0 means three o’clock. The total number of action spaces is defined as 360 / (move-dist). ● The last number of action means patrol |
| 1 | Continuous | x, y, action number | ● X, Y : 2D coordinates ● the action number 0 : move, 1 : Patrol |
| 2 | Continuous | theta, radian, action number | ● theta : the angle started from three 0’clock ● radian : distance to move toward the theta ● action number 0 : move, 1 : Patrol |
Observation
It needs a lot of information for vulture to beat zealot. The most important information is the position of vulture and zealot. Depending on the enemy’s and agent’s position, the agent decides whether to flee or attack. Also, all of StarCraft’s attackable units have the attribute of cooldown. You can only attack if cooldown is 0. Detailed observation information is as follows :
- Base Information of unit type
| Observation name | Description | Type |
|---|---|---|
| acceleration | The unit’s accelation amount | int32 |
| armor | The amount of armor the unit type has | int32 |
| cooldown_max | The amount of base cooldown applied to the unit after an attack | int32 |
| damage_amount | The base amount of damage that this weapon can deal per attack | int32 |
| damage_factor | The intended number of missiles/attacks that are used. This is used to multiply with the damage amount to obtain the full amount of damage for an attack. | int32 |
| energy_max | The maximum amount of energy for this unit type | int32 |
| hp_max | The maximum amount of hit points for this unit type | int32 |
| seek_range | The range at which this unit type will start targeting enemy units | int32 |
| shield_max | The maximum amount of shield points for this unit type | int32 |
| sight_range | The sight range of this unit type | int32 |
| top_speed | The unit type’s top movement speed with no upgrades | double |
| weapon_range | The maximum attack range of the weapon, measured in pixels | int32 |
- Current Informaiton of all units in game
| Observation name | Description | Type |
|---|---|---|
| unit_type | The type of unit | string |
| hp | The current amount of hit points for this unit | int32 |
| shield | The current amount of shield points for this unit | int32 |
| energy | The current amount of energy points for this unit | int32 |
| cooldown | The current amount of cooldown for this unit | int32 |
| pos_x | The X coordinates for current position of this unit. The position is roughly the center if the unit | int32 |
| pos_y | The Y coordinates for current position of this unit. The position is roughly the center if the unit | int32 |
| velocity_x | The X component of the unit’s velocity, measured in pixels per frame | double |
| velocity_y | The Y component of the unit’s velocity, measured in pixels per frame | double |
| angle | The unit’s facing direction in radians | double |
| accelerating | True, if the current unit is accelerating. | bool |
| braking | True, if the current unit is slowing down to come to a stop | bool |
| attacking | True, if this unit is currently attacking something | bool |
| is_attack_frame | True, if this unit is currently playing an attack animation | bool |
| invalid_action | Indicates whether last action was valid or not | bool |
- Terrain information (Picture need to be added)
The terrain information includes the following : the distance from un-walkable tiles in specific direction(if there is no un-walkable tile in 10 tiles distance, it will be 320px = 10 Tiles)
Terrain information is very important for the agent’s movement. A movement command to an un-walkable terrain can cause an unexpected outcome (movement to an unwanted place or stop moving) However, depending on the situation, you can solve the problem with only the close terrain information from the agent rather than knowing about all maps.
For this reason, in this scenario, the agent only can get the terrain information according to the action space. If the number of action space is 12, then the agent can get a terrain information about 12 each directions. As described above, the action space number is determined according to ‘move angle’ set by the user. If ‘move angle’ is 10, the action space number is 360 / 10 = 36. In this case, terrain information includes the distance from un-walkable tile of 36 directions.
Reward
The reward is very clear. 1 for win, -1 for lose. So, to improve learning speed or performance, it is essential for user to properly reshape the reward using the observation information.
- +1 : win, zealot is dead
- -1 : lose, vulture is dead
Version
This scenario provides various versions of the map to support multi-agent environment and various difficulty. If there are terrain such as wall, river, hill, sculpture, the agent fights the enemy in a much more difficult environment.
| Version | Description | Terrain | Units |
|---|---|---|---|
| 0 | 1 vulture versus 1 zealot, no terrain | N | 1 Vulture, 1 Zealot |
| 1 | 1 vulture versus 2 zealot, no terrain | N | 1 Vulture, 2 Zealot |
| 2 | 1 vulture versus 1 zealot, with terrain | Y | 1 Vulture, 1 Zealot |
| 3 | 1 vulture versus 2 zealot, with terrain | Y | 1 Vulture, 2 Zealot |
Environment configuration
You should know those parameters when you create the environment.
from saida_gym.starcraft.vultureVsZealot import VultureVsZealot
env = VultureVsZealot(version=0, action_type=0, frames_per_step=6, move_angle=30, move_dist=4, verbose=0)
- version (integer): version of the map. The difficulty is various according to the version. (default = 0)
- action_type (integer) : 0 is the discrete action space, 1 and 2 are the continuous action space.
- frames_per_step (integer): determine how many frames will be skipped in one step of action.
- move_angle (integer) : in case of discrete action type, this value determines the total number of action spaces. 360 degrees divided by ‘move_angle’ will be the number of action spaces.
For example, if ‘move_angle’ is 30, 360/30 = 12 will be the number of action spaces. That means this value should be divisor of 360. - move_dist (integer): determine how far the agent will move in one step. unit = tile size (1 tile = 32px)
- verbose (integer): log level
- bot_runner (bool): None only if you don’t want to execute starcraft bot automatically.
- no_gui (bool): True if you want to render the game.(default : False)
- local_speed (integer): the play speed of starcraft in game (default : 0)
- auto_kill (bool): True if you want to kill ongoing starcraft process. (default : True)
- random_seed (integer): random seed value which determines randomness of starcraft