Evaluate the chances of the forces of light winning in aDota 2 match.
The game Dota 2 involves two sides: teams of the forces of Light (The Radiant) and the forces of Darkness (The Dire). Each team consists of 5 players - characters with their own unique abilities. Before the start of the match, each player chooses one hero out of 100 possible, and all selected team heroes must be different.
The goal of the game is to destroy the throne located in the enemy base.
Players can receive gold and experience for killing other people's heroes or other units. The accumulated experience affects the hero's level and allows him to improve his abilities.
With the accumulated gold, players buy items that improve the characteristics of their heroes or give them new abilities. During the game, teams develop their heroes, defend their part of the field and attack the enemy.
The match is stopped at some (arbitrary) point in time. You are given information about everything that happened before the stop, including the condition of all the heroes. It is necessary to estimate the probability of the Radiant team winning.
Pay attention — this time there is no “guessing” who will win. It is important for us to evaluate the Radiant team's chances of winning.
This problem asks you to use the area under the ROC curve to evaluate the quality of models.
Recall that the area under the curve is also calledROC-AUC (от Area Under Curve). It takes values in the range from 0 to 1, it is believed that the higher the ROC-AUC, the better the quality of the model. A detailed description of ROC-AUC can be found at the end of the document.
A new data set was specially collected for the participants, the archive of which contains the following files with information:
- train_matches.jsonl, test_matches.jsonl - full descriptions of matches (“raw” data);
- train_features.csv, test_features.csv - tables with features from the organizers;
- train_targets.csv - a table with the results of matches from the training set, in particular the winner;
- sample_submission.csv - is an example file with a solution.
Your task is to build an algorithm that evaluates the chances of winning the game for the Radiant team radiant_win. In the previous task, we predicted whether Team Radiant would win or lose a fight by answering 0 or 1, and the quality of the predictions was measured using the Accuracy metric.
Now the task is to estimate the chance (a number in the range [0, 1]) that the Radiant team will win the match. In practice, this means that the answer now indicates not just 1, if, according to our estimates, the Radiant team will win the match, but some number close to 1, for example: 0.9742.
To make the task easier, we have prepared a detailed description of the task, game mechanics, collected data, and ROC-AUC metrics. Participants are also provided with a training tutorialwith a complete solution to the problem, in the Ipython notebook format already familiar to you.
