{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Bradley Terry example notebook\n", "What you will find in this notebook examples of using skpref:\n", "\n", "* for setting up the modelling task based framework\n", "* to fit a classifier that's being read in from scikit-learn on the same problem which in the background uses reduction and aggregation methods.\n", "* to fit a Bradley-Terry model with and without covariates on the pairwise comparison data of basketball matches.\n", "* for applying the GridSearch technique for model selection" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# Optionally change the theme of the notebook to dark\n", "# from jupyterthemes.stylefx import set_nb_theme\n", "# set_nb_theme('chesterish')" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Import skpref modules\n", "import sys\n", "sys.path.insert(0, \"../..\")\n", "from skpref.random_utility import BradleyTerry\n", "from skpref.task import PairwiseComparisonTask\n", "from skpref.base import ClassificationReducer\n", "from skpref.model_selection import GridSearchCV\n", "from skpref.utils import nice_print_results\n", "\n", "# Import scikit-learn packages to be used in tandem with skpref architecture\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.metrics import f1_score\n", "\n", "# Import other useful packages\n", "import pandas as pd\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Reading in the data\n", "The example dataset will be matches played by NBA teams, we will use the 2016 season's matches to predict the results of the 2017 matches. The dataset contains:\n", "\n", "- a column for `team1` and `team2` indicating the two teams that have played each other\n", "- `season_start`, which indicates which season the match belongs to\n", "- `team1_wins` takes the value of 1 if the team in column `team1` win the match, 0 if they lost (there are no ties in basketball)\n", "- `team_1_home` takes the value of 1 if `team1` was playing in their home court 0 if they were paying away (no neutral courts in the NBA)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1team2season_startteam1_winsteam_1_home
0Atlanta HawksToronto Raptors201400
1Atlanta HawksIndiana Pacers201411
2Atlanta HawksSan Antonio Spurs201400
3Atlanta HawksCharlotte Hornets201400
4Atlanta HawksNew York Knicks201411
\n", "
" ], "text/plain": [ " team1 team2 season_start team1_wins team_1_home\n", "0 Atlanta Hawks Toronto Raptors 2014 0 0\n", "1 Atlanta Hawks Indiana Pacers 2014 1 1\n", "2 Atlanta Hawks San Antonio Spurs 2014 0 0\n", "3 Atlanta Hawks Charlotte Hornets 2014 0 0\n", "4 Atlanta Hawks New York Knicks 2014 1 1" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "NBA_results = pd.read_csv('data/NBA_matches.csv')\n", "NBA_results.head()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1team2season_startteam1_winsteam_1_home
9835Washington WizardsHouston Rockets201700
9836Washington WizardsCleveland Cavaliers201700
9837Washington WizardsAtlanta Hawks201701
9838Washington WizardsBoston Celtics201711
9839Washington WizardsOrlando Magic201700
\n", "
" ], "text/plain": [ " team1 team2 season_start team1_wins \\\n", "9835 Washington Wizards Houston Rockets 2017 0 \n", "9836 Washington Wizards Cleveland Cavaliers 2017 0 \n", "9837 Washington Wizards Atlanta Hawks 2017 0 \n", "9838 Washington Wizards Boston Celtics 2017 1 \n", "9839 Washington Wizards Orlando Magic 2017 0 \n", "\n", " team_1_home \n", "9835 0 \n", "9836 0 \n", "9837 1 \n", "9838 1 \n", "9839 0 " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "NBA_results.tail()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "season_split = 2016\n", "train_data = NBA_results[NBA_results.season_start == season_split].copy()\n", "test_data = NBA_results[NBA_results.season_start == season_split+1].copy()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will also use team salary data as covariates in the model later, with the idea being that a team that has more money to pay to their athletes has an advantage over other teams, by having a better chance to attract the top talent in the league." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
teamseason_startsalary
0Atlanta Hawks201458337671
1Atlanta Hawks201571378126
2Atlanta Hawks201695957250
3Atlanta Hawks201799375302
4Boston Celtics201459418142
\n", "
" ], "text/plain": [ " team season_start salary\n", "0 Atlanta Hawks 2014 58337671\n", "1 Atlanta Hawks 2015 71378126\n", "2 Atlanta Hawks 2016 95957250\n", "3 Atlanta Hawks 2017 99375302\n", "4 Boston Celtics 2014 59418142" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "NBA_team_salary_budget = pd.read_csv('data/team_salary_budgets.csv')\n", "NBA_team_salary_budget.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Setting up the tasks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We set up the preference learning task by using the `PairwiseComparisonTask` object in `skpref`. This is the only extra step which might be a completely new concept to seasoned scikit-learn users. Once the task is specified, say in this case a pairwise comparison task, for any models applied in skpref, whether that is a reduction via scikit-learn or even a model that is not a pairwise comparison model, the package will know that the problem itself is a pairwise comparison problem and can perform reduction and aggregation adequately in the background when needed.\n", "\n", "In this example the `PairwiseComparisonTask` has the following components:\n", "\n", "- `primary_table`: the table that contains the observed preferences\n", "- `primary_table_alternatives_names`: the column or columns that contain the alternatives, in this case both columns team1 and team2 contain alternatives\n", "- `primary_table_target_name`: the column that indicates the result of the pairwise comparison\n", "- `target_column_correspondence`: in the case of pairwise comparisons, when the alternatives are split across two columns, the column indicating the result usually takes the form 1/0 to show whether one of the columns, in our case team1 or team2 has been preferred. So in this column the user indicates that when the team1_wins column takes the value 1 that means that the alternative in the column team1 has won.\n", "- `features_to_use`: indicates which columns to use as covariates" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "NBA_results_task_train_LR = PairwiseComparisonTask(\n", " primary_table=train_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=['team_1_home']\n", ")\n", "\n", "# For the test task, it's possible to make a copy of the training task and\n", "# update the primary table\n", "NBA_results_task_predict_LR = PairwiseComparisonTask(\n", " primary_table=test_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=['team_1_home']\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Fitting a Logistic Regression\n", "The only covariate we will use in this for now will be the `team_1_home` column, which should return a method that only learns what the home team advantage was on average, which is the equivalent to fitting a logistic regression where whether team1 is playing home or not is the only covariate.\n", "\n", "$P(\\texttt{team1}\\_\\texttt{wins}=1) = logit(\\alpha + \\beta_1 \\texttt{team}\\_\\texttt{1}\\_\\texttt{home})$" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "my_log_red = ClassificationReducer(LogisticRegression(solver='lbfgs'))\n", "my_log_red.fit_task(NBA_results_task_train_LR)\n", "preds = my_log_red.predict_task(NBA_results_task_predict_LR)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array(['Dallas Mavericks', 'Charlotte Hornets', 'Brooklyn Nets', ...,\n", " 'Washington Wizards', 'Washington Wizards', 'Orlando Magic'],\n", " dtype=object),\n", " array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,\n", " 'Atlanta Hawks', 'Boston Celtics', 'Washington Wizards'],\n", " dtype=object))" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# predict_task returns a SubsetPosetVector which has the attributes\n", "# top_input_data and boot_input_data corresponding to chosen and not chosen \n", "# alternatives.\n", "preds.top_input_data, preds.boot_input_data" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1team2season_startteam1_winsteam_1_home
7380Atlanta HawksDallas Mavericks201710
7381Atlanta HawksCharlotte Hornets201700
7382Atlanta HawksBrooklyn Nets201700
7383Atlanta HawksMiami Heat201700
7384Atlanta HawksChicago Bulls201700
\n", "
" ], "text/plain": [ " team1 team2 season_start team1_wins team_1_home\n", "7380 Atlanta Hawks Dallas Mavericks 2017 1 0\n", "7381 Atlanta Hawks Charlotte Hornets 2017 0 0\n", "7382 Atlanta Hawks Brooklyn Nets 2017 0 0\n", "7383 Atlanta Hawks Miami Heat 2017 0 0\n", "7384 Atlanta Hawks Chicago Bulls 2017 0 0" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "NBA_results_task_predict_LR.primary_table.head()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1team2season_startteam1_winsteam_1_home
9835Washington WizardsHouston Rockets201700
9836Washington WizardsCleveland Cavaliers201700
9837Washington WizardsAtlanta Hawks201701
9838Washington WizardsBoston Celtics201711
9839Washington WizardsOrlando Magic201700
\n", "
" ], "text/plain": [ " team1 team2 season_start team1_wins \\\n", "9835 Washington Wizards Houston Rockets 2017 0 \n", "9836 Washington Wizards Cleveland Cavaliers 2017 0 \n", "9837 Washington Wizards Atlanta Hawks 2017 0 \n", "9838 Washington Wizards Boston Celtics 2017 1 \n", "9839 Washington Wizards Orlando Magic 2017 0 \n", "\n", " team_1_home \n", "9835 0 \n", "9836 0 \n", "9837 1 \n", "9838 1 \n", "9839 0 " ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "NBA_results_task_predict_LR.primary_table.tail()" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dallas Mavericks [0.58 0. 0. ... 0. 0. 0. ]\n", "Atlanta Hawks [0.42 0.42 0.42 ... 0.42 0. 0. ]\n" ] } ], "source": [ "# All this learns so far is the home team advantage, since its the only \n", "# covariate in the test_data table\n", "nice_print_results(\n", " my_log_red.predict_proba_task(NBA_results_task_predict_LR,\n", " outcome=['Dallas Mavericks', 'Atlanta Hawks']))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.42 0.42 0.42 ... 0.58 0.58 0.42]\n", "team2 is preferred [0.58 0.58 0.58 ... 0.42 0.42 0.58]\n" ] } ], "source": [ "nice_print_results(\n", " my_log_red.predict_proba_task(NBA_results_task_predict_LR,\n", " column=['team1', 'team2'])\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting a Bradley Terry model\n", "As we can see in the example above the logistic regression approach does not learn different probabilities for a team winning or losing based on which other team they are playing. The Dallas Mavericks could be playing against the strongest or weakest team in the league and their estimated probability of winning would be the same. The difference between the Bradley-Terry model and logistic regression is that Bradley-Terry learns a function that can estimate whether each team will win or lose given the other team they are playing.\n", "\n", "The task we will use for Bradley-Terry will be defined in a slightly different way, because in the first demo we won't use any covariates, therefore we define `features_to_use=None`\n", "\n", "In the Bradley-Terry model each team gets a latent strength parameter $\\lambda_{\\text{team}}$, for example $\\lambda_{\\text{Atlanta Hawks}}$.\n", "\n", "The Bradley-Terry model learns these strength parameters to maximise the likelihood according to the following formulation for observation $i$:\n", "$$P(\\texttt{team1}\\_\\texttt{wins}=1)_i= \\frac{e^{\\lambda_{\\texttt{team1}_i}}}{e^{\\lambda_{\\texttt{team1}_i}} + e^{\\lambda_{\\texttt{team2}_i}}}$$" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "NBA_results_task_train_BT = PairwiseComparisonTask(\n", " primary_table=train_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=None\n", ")\n", "\n", "NBA_results_task_predict_BT = PairwiseComparisonTask(\n", " primary_table=test_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=None\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Fitting Bradley Terry model\n", "mybt = BradleyTerry(method='BFGS', alpha=1e-5)\n", "mybt.fit_task(NBA_results_task_train_BT)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
entitylearned_strength
0Atlanta Hawks0.047522
1Boston Celtics0.580896
2Brooklyn Nets-1.178393
3Charlotte Hornets-0.278154
4Chicago Bulls-0.037967
5Cleveland Cavaliers0.489737
6Dallas Mavericks-0.386261
7Denver Nuggets-0.040408
8Detroit Pistons-0.225709
9Golden State Warriors1.538386
10Houston Rockets0.765613
11Indiana Pacers-0.005751
12Los Angeles Clippers0.550265
13Los Angeles Lakers-0.773690
14Memphis Grizzlies0.153646
15Miami Heat-0.022175
16Milwaukee Bucks0.018291
17Minnesota Timberwolves-0.470415
18New Orleans Pelicans-0.328205
19New York Knicks-0.548175
20Oklahoma City Thunder0.344454
21Orlando Magic-0.655354
22Philadelphia 76ers-0.716305
23Phoenix Suns-0.888314
24Portland Trail Blazers0.019229
25Sacramento Kings-0.426973
26San Antonio Spurs1.115135
27Toronto Raptors0.462682
28Utah Jazz0.535025
29Washington Wizards0.361368
\n", "
" ], "text/plain": [ " entity learned_strength\n", "0 Atlanta Hawks 0.047522\n", "1 Boston Celtics 0.580896\n", "2 Brooklyn Nets -1.178393\n", "3 Charlotte Hornets -0.278154\n", "4 Chicago Bulls -0.037967\n", "5 Cleveland Cavaliers 0.489737\n", "6 Dallas Mavericks -0.386261\n", "7 Denver Nuggets -0.040408\n", "8 Detroit Pistons -0.225709\n", "9 Golden State Warriors 1.538386\n", "10 Houston Rockets 0.765613\n", "11 Indiana Pacers -0.005751\n", "12 Los Angeles Clippers 0.550265\n", "13 Los Angeles Lakers -0.773690\n", "14 Memphis Grizzlies 0.153646\n", "15 Miami Heat -0.022175\n", "16 Milwaukee Bucks 0.018291\n", "17 Minnesota Timberwolves -0.470415\n", "18 New Orleans Pelicans -0.328205\n", "19 New York Knicks -0.548175\n", "20 Oklahoma City Thunder 0.344454\n", "21 Orlando Magic -0.655354\n", "22 Philadelphia 76ers -0.716305\n", "23 Phoenix Suns -0.888314\n", "24 Portland Trail Blazers 0.019229\n", "25 Sacramento Kings -0.426973\n", "26 San Antonio Spurs 1.115135\n", "27 Toronto Raptors 0.462682\n", "28 Utah Jazz 0.535025\n", "29 Washington Wizards 0.361368" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.params_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use the latent alternative strength parameters that Bradley-Terry models learn to rank the teams, either by sorting the `mybt.params_ DataFrame` by the `learned_strength` parameter, or by running the `rank_entities` function" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Golden State Warriors',\n", " 'San Antonio Spurs',\n", " 'Houston Rockets',\n", " 'Boston Celtics',\n", " 'Los Angeles Clippers',\n", " 'Utah Jazz',\n", " 'Cleveland Cavaliers',\n", " 'Toronto Raptors',\n", " 'Washington Wizards',\n", " 'Oklahoma City Thunder',\n", " 'Memphis Grizzlies',\n", " 'Atlanta Hawks',\n", " 'Portland Trail Blazers',\n", " 'Milwaukee Bucks',\n", " 'Indiana Pacers',\n", " 'Miami Heat',\n", " 'Chicago Bulls',\n", " 'Denver Nuggets',\n", " 'Detroit Pistons',\n", " 'Charlotte Hornets',\n", " 'New Orleans Pelicans',\n", " 'Dallas Mavericks',\n", " 'Sacramento Kings',\n", " 'Minnesota Timberwolves',\n", " 'New York Knicks',\n", " 'Orlando Magic',\n", " 'Philadelphia 76ers',\n", " 'Los Angeles Lakers',\n", " 'Phoenix Suns',\n", " 'Brooklyn Nets']" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.rank_entities(ascending=False)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Atlanta Hawks [0.61 0.58 0.77 ... 0.42 0. 0. ]\n", "Washington Wizards [0. 0. 0. ... 0.58 0.45 0.73]\n" ] } ], "source": [ "# we can create the probability for each team winning in a specific observaion,\n", "nice_print_results(\n", " mybt.predict_proba_task(NBA_results_task_predict_BT,\n", " outcome=['Atlanta Hawks', 'Washington Wizards'])\n", ")" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.61 0.58 0.77 ... 0.58 0.45 0.73]\n", "team2 is preferred [0.39 0.42 0.23 ... 0.42 0.55 0.27]\n" ] } ], "source": [ "nice_print_results(\n", " mybt.predict_proba_task(NBA_results_task_predict_BT,\n", " column=['team1', 'team2'])\n", ")" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],\n", " dtype=object)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.predict_choice_task(NBA_results_task_predict_BT)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "preds = mybt.predict_task(NBA_results_task_predict_BT)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],\n", " dtype=object),\n", " array(['Dallas Mavericks', 'Charlotte Hornets', 'Brooklyn Nets', ...,\n", " 'Atlanta Hawks', 'Washington Wizards', 'Orlando Magic'],\n", " dtype=object))" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds.top_input_data, preds.boot_input_data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Augmenting the models with covariates\n", "In this section we will start introducing more covariates in the models above, we will introduce one additional covariate which is the team salary budget. We can also see how we can define a single task which we can use to run different models in skpref." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "NBA_results_task_train = PairwiseComparisonTask(\n", " primary_table=train_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=['salary', 'team1_home'],\n", " secondary_table=NBA_team_salary_budget,\n", " secondary_to_primary_link={\n", " 'team': ['team1', 'team2'],\n", " 'season_start': 'season_start'\n", " })\n", "\n", "NBA_results_task_predict = PairwiseComparisonTask(\n", " primary_table=test_data,\n", " primary_table_alternatives_names=['team1', 'team2'],\n", " primary_table_target_name ='team1_wins',\n", " target_column_correspondence='team1',\n", " features_to_use=['salary', 'team1_home'],\n", " secondary_table=NBA_team_salary_budget,\n", " secondary_to_primary_link={\n", " 'team': ['team1', 'team2'],\n", " 'season_start': 'season_start'\n", " })" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Reduction to logistic regression with covariates\n", "Here we fit a logistic regression on three covariates, whether `team1` is playing home or not, `team1`'s salary budget and `team2`'s salary budget.\n", "$P(\\texttt{team1}\\_\\texttt{wins}=1) = logit(\\alpha + \\beta_1 \\texttt{team}\\_\\texttt{1}\\_\\texttt{home} + \\beta_2 \\texttt{team1}\\_\\texttt{salary} + \\beta_3 \\texttt{team2}\\_\\texttt{salary})$" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "my_log_red = ClassificationReducer(LogisticRegression(solver='lbfgs'))\n", "my_log_red.fit_task(NBA_results_task_train)\n", "preds = my_log_red.predict_task(NBA_results_task_predict)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1_winsteam_1_homesalary_team1salary_team2
0109937530285753772
10099375302117228164
2009937530295964560
30099375302129458084
4009937530289524016
50199375302107015203
60199375302115375243
\n", "
" ], "text/plain": [ " team1_wins team_1_home salary_team1 salary_team2\n", "0 1 0 99375302 85753772\n", "1 0 0 99375302 117228164\n", "2 0 0 99375302 95964560\n", "3 0 0 99375302 129458084\n", "4 0 0 99375302 89524016\n", "5 0 1 99375302 107015203\n", "6 0 1 99375302 115375243" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# We can investigate the internal table that was fed into LogisticRegression.fit()\n", "my_log_red.model_input.head(7)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 5.35210228e-15, 1.54775613e-08, -1.54775613e-08]])" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# We can also investigate the coefficients which were learned\n", "my_log_red.model.coef_ " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that the coefficients learned for $\\beta_2$ and $\\beta_3$ are very similar to each other, just opposite signs. ClassificationReducer allows users the option to take the difference in features directly rather than split them out, effectively learning the following model:\n", "$P(\\texttt{team1}\\_\\texttt{wins}=1) = logit(\\alpha + \\beta_1 \\texttt{team}\\_\\texttt{1}\\_\\texttt{home} + \\beta_2 (\\texttt{team1}\\_\\texttt{salary} - \\texttt{team2}\\_\\texttt{salary}))$" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "my_log_red = ClassificationReducer(\n", " LogisticRegression(solver='lbfgs'),\n", " take_feature_diff_for_pairwise_comparison=True\n", ")\n", "my_log_red.fit_task(NBA_results_task_train)\n", "preds = my_log_red.predict_task(NBA_results_task_predict)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
team1_winsteam_1_homesalary_diff
01013621530
100-17852862
2003410742
300-30082782
4009851286
501-7639901
601-15999941
\n", "
" ], "text/plain": [ " team1_wins team_1_home salary_diff\n", "0 1 0 13621530\n", "1 0 0 -17852862\n", "2 0 0 3410742\n", "3 0 0 -30082782\n", "4 0 0 9851286\n", "5 0 1 -7639901\n", "6 0 1 -15999941" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_log_red.model_input.head(7)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[2.67602286e-15, 1.54775613e-08]])" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_log_red.model.coef_" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Washington Wizards', 'Washington Wizards'],\n", " dtype=object),\n", " array(['Dallas Mavericks', 'Atlanta Hawks', 'Brooklyn Nets', ...,\n", " 'Atlanta Hawks', 'Boston Celtics', 'Orlando Magic'], dtype=object))" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds.top_input_data, preds.boot_input_data" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.55 0.43 0.51 ... 0.59 0.53 0.61]\n" ] } ], "source": [ "# All this learns so far is the home team advantage, since its the only \n", "# covariate in the test_data table\n", "nice_print_results(\n", " my_log_red.predict_proba_task(NBA_results_task_predict,\n", " column='team1')\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Bradley Terry model with salary covariate\n", "\n", "Here we augment the initial Bradley-Terry model to learn the following relationship:\n", "\n", "$$P(\\texttt{team1}\\_\\texttt{wins}=1)_i= \\frac{e^{(\\lambda_{\\texttt{team1}_i} + \\beta_1 \\texttt{team1}\\_\\texttt{salary}_i)}}{e^{(\\lambda_{\\texttt{team1}_i} + \\beta_1 \\texttt{team1}\\_\\texttt{salary}_i)} + e^{(\\lambda_{\\texttt{team2}_i}+ \\beta_1 \\texttt{team2}\\_\\texttt{salary}_i)}}$$" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Golden State Warriors', 'San Antonio Spurs', 'Houston Rockets',\n", " 'Utah Jazz', 'Boston Celtics', 'Oklahoma City Thunder',\n", " 'Washington Wizards', 'Toronto Raptors', 'Los Angeles Clippers',\n", " 'Denver Nuggets', 'Atlanta Hawks', 'Indiana Pacers',\n", " 'Chicago Bulls', 'Cleveland Cavaliers', 'Memphis Grizzlies',\n", " 'Miami Heat', 'Milwaukee Bucks', 'Charlotte Hornets',\n", " 'Minnesota Timberwolves', 'Portland Trail Blazers',\n", " 'New Orleans Pelicans', 'Sacramento Kings', 'Detroit Pistons',\n", " 'Dallas Mavericks', 'Philadelphia 76ers', 'New York Knicks',\n", " 'Phoenix Suns', 'Los Angeles Lakers', 'Orlando Magic',\n", " 'Brooklyn Nets'], dtype=object)" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt = BradleyTerry(method='BFGS', alpha=1e-5)\n", "mybt.fit_task(NBA_results_task_train)\n", "mybt.rank_entities(ascending=False)" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.69 0.48 0.75 ... 0.65 0.43 0.82]\n", "team2 is preferred [0.31 0.52 0.25 ... 0.35 0.57 0.18]\n" ] } ], "source": [ "nice_print_results(mybt.predict_proba_task(NBA_results_task_predict, column=['team1', 'team2']))" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],\n", " dtype=object)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.predict_choice_task(NBA_results_task_predict)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],\n", " dtype=object)" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.predict_task(NBA_results_task_predict).top_input_data" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Multinomial Logit Model Regression Results
Dep. Variable: CHOICE No. Observations: 2,460
Model: Multinomial Logit Model Df Residuals: 2,429
Method: MLE Df Model: 31
Date: Wed, 01 Mar 2023 Pseudo R-squ.: 0.107
Time: 16:10:40 Pseudo R-bar-squ.: 0.089
AIC: 3,107.966 Log-Likelihood: -1,522.983
BIC: 3,288.012 LL-Null: -1,705.142
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
coef std err z P>|z| [0.025 0.975]
salary 1.717e-08 3.65e-06 0.005 0.996 -7.13e-06 7.17e-06
Atlanta Hawks 0.0810 41.439 0.002 0.998 -81.138 81.300
Boston Celtics 0.7344 52.254 0.014 0.989 -101.682 103.151
Brooklyn Nets -0.9380 65.383 -0.014 0.989 -129.086 127.210
Charlotte Hornets -0.1415 50.101 -0.003 0.998 -98.339 98.056
Chicago Bulls 0.0621 46.030 0.001 0.999 -90.156 90.280
Cleveland Cavaliers -0.0049 112.740 -4.33e-05 1.000 -220.970 220.960
Dallas Mavericks -0.4891 46.300 -0.011 0.992 -91.236 90.258
Denver Nuggets 0.2237 69.393 0.003 0.997 -135.784 136.232
Detroit Pistons -0.4001 55.137 -0.007 0.994 -108.468 107.667
Golden State Warriors 1.4940 41.902 0.036 0.972 -80.632 83.620
Houston Rockets 0.9021 50.079 0.018 0.986 -97.252 99.056
Indiana Pacers 0.0727 44.101 0.002 0.999 -86.363 86.508
Los Angeles Clippers 0.2355 78.355 0.003 0.998 -153.337 153.808
Los Angeles Lakers -0.7116 42.901 -0.017 0.987 -84.796 83.373
Memphis Grizzlies -0.0434 58.483 -0.001 0.999 -114.668 114.581
Miami Heat -0.0820 42.759 -0.002 0.998 -83.888 83.724
Milwaukee Bucks -0.1289 51.425 -0.003 0.998 -100.920 100.663
Minnesota Timberwolves -0.1561 78.274 -0.002 0.998 -153.569 153.257
New Orleans Pelicans -0.3903 42.901 -0.009 0.993 -84.476 83.695
New York Knicks -0.6348 44.785 -0.014 0.989 -88.412 87.143
Oklahoma City Thunder 0.4593 47.565 0.010 0.992 -92.766 93.685
Orlando Magic -0.7402 44.632 -0.017 0.987 -88.217 86.736
Philadelphia 76ers -0.5043 60.789 -0.008 0.993 -119.649 118.640
Phoenix Suns -0.6594 63.503 -0.010 0.992 -125.123 123.804
Portland Trail Blazers -0.2206 65.293 -0.003 0.997 -128.192 127.751
Sacramento Kings -0.3933 41.448 -0.009 0.992 -81.630 80.843
San Antonio Spurs 0.9525 53.485 0.018 0.986 -103.876 105.781
Toronto Raptors 0.2806 56.240 0.005 0.996 -109.949 110.510
Utah Jazz 0.8459 77.655 0.011 0.991 -151.355 153.047
Washington Wizards 0.2946 43.216 0.007 0.995 -84.407 84.997
" ], "text/plain": [ "\n", "\"\"\"\n", " Multinomial Logit Model Regression Results \n", "===================================================================================\n", "Dep. Variable: CHOICE No. Observations: 2,460\n", "Model: Multinomial Logit Model Df Residuals: 2,429\n", "Method: MLE Df Model: 31\n", "Date: Wed, 01 Mar 2023 Pseudo R-squ.: 0.107\n", "Time: 16:10:40 Pseudo R-bar-squ.: 0.089\n", "AIC: 3,107.966 Log-Likelihood: -1,522.983\n", "BIC: 3,288.012 LL-Null: -1,705.142\n", "==========================================================================================\n", " coef std err z P>|z| [0.025 0.975]\n", "------------------------------------------------------------------------------------------\n", "salary 1.717e-08 3.65e-06 0.005 0.996 -7.13e-06 7.17e-06\n", "Atlanta Hawks 0.0810 41.439 0.002 0.998 -81.138 81.300\n", "Boston Celtics 0.7344 52.254 0.014 0.989 -101.682 103.151\n", "Brooklyn Nets -0.9380 65.383 -0.014 0.989 -129.086 127.210\n", "Charlotte Hornets -0.1415 50.101 -0.003 0.998 -98.339 98.056\n", "Chicago Bulls 0.0621 46.030 0.001 0.999 -90.156 90.280\n", "Cleveland Cavaliers -0.0049 112.740 -4.33e-05 1.000 -220.970 220.960\n", "Dallas Mavericks -0.4891 46.300 -0.011 0.992 -91.236 90.258\n", "Denver Nuggets 0.2237 69.393 0.003 0.997 -135.784 136.232\n", "Detroit Pistons -0.4001 55.137 -0.007 0.994 -108.468 107.667\n", "Golden State Warriors 1.4940 41.902 0.036 0.972 -80.632 83.620\n", "Houston Rockets 0.9021 50.079 0.018 0.986 -97.252 99.056\n", "Indiana Pacers 0.0727 44.101 0.002 0.999 -86.363 86.508\n", "Los Angeles Clippers 0.2355 78.355 0.003 0.998 -153.337 153.808\n", "Los Angeles Lakers -0.7116 42.901 -0.017 0.987 -84.796 83.373\n", "Memphis Grizzlies -0.0434 58.483 -0.001 0.999 -114.668 114.581\n", "Miami Heat -0.0820 42.759 -0.002 0.998 -83.888 83.724\n", "Milwaukee Bucks -0.1289 51.425 -0.003 0.998 -100.920 100.663\n", "Minnesota Timberwolves -0.1561 78.274 -0.002 0.998 -153.569 153.257\n", "New Orleans Pelicans -0.3903 42.901 -0.009 0.993 -84.476 83.695\n", "New York Knicks -0.6348 44.785 -0.014 0.989 -88.412 87.143\n", "Oklahoma City Thunder 0.4593 47.565 0.010 0.992 -92.766 93.685\n", "Orlando Magic -0.7402 44.632 -0.017 0.987 -88.217 86.736\n", "Philadelphia 76ers -0.5043 60.789 -0.008 0.993 -119.649 118.640\n", "Phoenix Suns -0.6594 63.503 -0.010 0.992 -125.123 123.804\n", "Portland Trail Blazers -0.2206 65.293 -0.003 0.997 -128.192 127.751\n", "Sacramento Kings -0.3933 41.448 -0.009 0.992 -81.630 80.843\n", "San Antonio Spurs 0.9525 53.485 0.018 0.986 -103.876 105.781\n", "Toronto Raptors 0.2806 56.240 0.005 0.996 -109.949 110.510\n", "Utah Jazz 0.8459 77.655 0.011 0.991 -151.355 153.047\n", "Washington Wizards 0.2946 43.216 0.007 0.995 -84.407 84.997\n", "==========================================================================================\n", "\"\"\"" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mybt.bt_with_feats.get_statsmodels_summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Example using GridSearchCV()\n", "The models we have fitted above also have hyperparameters, such as the method of gradient descent or regularisation. To optimise the hyperparameter selection, we can use `GridSearchCV()`. `GridSearchCV()` tries out a series of hyperparameter combinations and runs a k-fold cross-validation on an accuracy metric determined by the user to check which ones have performed best." ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The model with the best parameters was:\n", "BradleyTerry(alpha=2, method='BFGS')\n", "With a score of -0.6265008194657992\n", "All the trials results summarised in descending score\n", " alpha method mean_test_score\n", "1 2 BFGS -0.626501\n", "0 1 BFGS -0.626742\n", "2 4 BFGS -0.628853\n" ] } ], "source": [ "to_tune = {'alpha': [1, 2, 4], 'method': ['BFGS']}\n", "gs_bt = GridSearchCV(BradleyTerry(), to_tune, cv=3, scoring='neg_log_loss')\n", "gs_bt.fit_task(NBA_results_task_train)\n", "gs_bt.inspect_results()" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The model with the best parameters was:\n", "BradleyTerry(alpha=4, method='BFGS')\n", "With a score of 0.6337744652191032\n", "All the trials results summarised in descending score\n", " alpha method mean_test_score\n", "2 4 BFGS 0.633774\n", "1 2 BFGS 0.631136\n", "0 1 BFGS 0.630085\n" ] } ], "source": [ "# Showing that sklearn.metrics works also\n", "to_tune = {'alpha': [1, 2, 4], 'method': ['BFGS']}\n", "gs_bt = GridSearchCV(BradleyTerry(), to_tune, cv=3, scoring=f1_score)\n", "gs_bt.fit_task(NBA_results_task_train)\n", "gs_bt.inspect_results()" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The model with the best parameters was:\n", "ClassificationReducer(model=LogisticRegression(C=0.5, penalty='l1',\n", " solver='saga'))\n", "With a score of -0.6865126660183437\n", "All the trials results summarised in descending score\n", " model__C model__fit_intercept model__penalty model__solver \\\n", "0 0.5 True l1 saga \n", "13 4.0 True l2 saga \n", "6 1.0 False l1 saga \n", "2 0.5 False l1 saga \n", "5 1.0 True l2 saga \n", "16 8.0 True l1 saga \n", "10 2.0 False l1 saga \n", "19 8.0 False l2 saga \n", "9 2.0 True l2 saga \n", "14 4.0 False l1 saga \n", "18 8.0 False l1 saga \n", "4 1.0 True l1 saga \n", "11 2.0 False l2 saga \n", "1 0.5 True l2 saga \n", "17 8.0 True l2 saga \n", "15 4.0 False l2 saga \n", "7 1.0 False l2 saga \n", "8 2.0 True l1 saga \n", "3 0.5 False l2 saga \n", "12 4.0 True l1 saga \n", "\n", " mean_test_score \n", "0 -0.686513 \n", "13 -0.686516 \n", "6 -0.686516 \n", "2 -0.686517 \n", "5 -0.686517 \n", "16 -0.686517 \n", "10 -0.686518 \n", "19 -0.686518 \n", "9 -0.686518 \n", "14 -0.686518 \n", "18 -0.686518 \n", "4 -0.686518 \n", "11 -0.686519 \n", "1 -0.686519 \n", "17 -0.686519 \n", "15 -0.686520 \n", "7 -0.686520 \n", "8 -0.686520 \n", "3 -0.686521 \n", "12 -0.686522 \n" ] } ], "source": [ "to_tune = {'C': [0.5, 1, 2, 4, 8], 'solver': ['saga'], 'penalty': ['l1','l2'],\n", " 'fit_intercept': [True, False]}\n", "gs_lr = GridSearchCV(ClassificationReducer(LogisticRegression()), to_tune,\n", " cv=3, scoring='neg_log_loss')\n", "gs_lr.fit_task(NBA_results_task_train)\n", "gs_lr.inspect_results()" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,\n", " 'Washington Wizards', 'Washington Wizards', 'Washington Wizards'],\n", " dtype=object)" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gs_lr.predict_task(NBA_results_task_predict).top_input_data" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.55 0.43 0.51 ... 0.59 0.53 0.61]\n" ] } ], "source": [ "nice_print_results(gs_lr.predict_proba_task(NBA_results_task_predict, column='team1'))" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "team1 is preferred [0.67 0.47 0.7 ... 0.64 0.45 0.79]\n" ] } ], "source": [ "nice_print_results(gs_bt.predict_proba_task(NBA_results_task_predict, column='team1'))" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['Golden State Warriors', 'San Antonio Spurs', 'Houston Rockets',\n", " 'Utah Jazz', 'Boston Celtics', 'Oklahoma City Thunder',\n", " 'Washington Wizards', 'Toronto Raptors', 'Los Angeles Clippers',\n", " 'Denver Nuggets', 'Atlanta Hawks', 'Indiana Pacers',\n", " 'Chicago Bulls', 'Cleveland Cavaliers', 'Memphis Grizzlies',\n", " 'Miami Heat', 'Milwaukee Bucks', 'Charlotte Hornets',\n", " 'Minnesota Timberwolves', 'Portland Trail Blazers',\n", " 'Detroit Pistons', 'New Orleans Pelicans', 'Sacramento Kings',\n", " 'Philadelphia 76ers', 'Dallas Mavericks', 'New York Knicks',\n", " 'Phoenix Suns', 'Los Angeles Lakers', 'Orlando Magic',\n", " 'Brooklyn Nets'], dtype=object)" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gs_bt.rank_entities(ascending=False)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.11" }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }