Bradley Terry example notebook

What you will find in this notebook examples of using skpref:

  • for setting up the modelling task based framework

  • to fit a classifier that’s being read in from scikit-learn on the same problem which in the background uses reduction and aggregation methods.

  • to fit a Bradley-Terry model with and without covariates on the pairwise comparison data of basketball matches.

  • for applying the GridSearch technique for model selection

[1]:

# Optionally change the theme of the notebook to dark
# from jupyterthemes.stylefx import set_nb_theme
# set_nb_theme('chesterish')

[2]:

# Import skpref modules
import sys
sys.path.insert(0, "../..")
from skpref.random_utility import BradleyTerry
from skpref.task import PairwiseComparisonTask
from skpref.base import ClassificationReducer
from skpref.model_selection import GridSearchCV
from skpref.utils import nice_print_results

# Import scikit-learn packages to be used in tandem with skpref architecture
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score

# Import other useful packages
import pandas as pd
import numpy as np

Reading in the data

The example dataset will be matches played by NBA teams, we will use the 2016 season’s matches to predict the results of the 2017 matches. The dataset contains:

  • a column for team1 and team2 indicating the two teams that have played each other

  • season_start, which indicates which season the match belongs to

  • team1_wins takes the value of 1 if the team in column team1 win the match, 0 if they lost (there are no ties in basketball)

  • team_1_home takes the value of 1 if team1 was playing in their home court 0 if they were paying away (no neutral courts in the NBA)

[3]:

NBA_results = pd.read_csv('data/NBA_matches.csv')
NBA_results.head()

[3]:
team1 team2 season_start team1_wins team_1_home
0 Atlanta Hawks Toronto Raptors 2014 0 0
1 Atlanta Hawks Indiana Pacers 2014 1 1
2 Atlanta Hawks San Antonio Spurs 2014 0 0
3 Atlanta Hawks Charlotte Hornets 2014 0 0
4 Atlanta Hawks New York Knicks 2014 1 1 
[4]:

NBA_results.tail()

[4]:
team1 team2 season_start team1_wins team_1_home
9835 Washington Wizards Houston Rockets 2017 0 0
9836 Washington Wizards Cleveland Cavaliers 2017 0 0
9837 Washington Wizards Atlanta Hawks 2017 0 1
9838 Washington Wizards Boston Celtics 2017 1 1
9839 Washington Wizards Orlando Magic 2017 0 0 
[5]:

season_split = 2016
train_data = NBA_results[NBA_results.season_start == season_split].copy()
test_data = NBA_results[NBA_results.season_start == season_split+1].copy()

We will also use team salary data as covariates in the model later, with the idea being that a team that has more money to pay to their athletes has an advantage over other teams, by having a better chance to attract the top talent in the league.

[6]:

NBA_team_salary_budget = pd.read_csv('data/team_salary_budgets.csv')
NBA_team_salary_budget.head()

[6]:
team season_start salary
0 Atlanta Hawks 2014 58337671
1 Atlanta Hawks 2015 71378126
2 Atlanta Hawks 2016 95957250
3 Atlanta Hawks 2017 99375302
4 Boston Celtics 2014 59418142

Setting up the tasks

We set up the preference learning task by using the PairwiseComparisonTask object in skpref. This is the only extra step which might be a completely new concept to seasoned scikit-learn users. Once the task is specified, say in this case a pairwise comparison task, for any models applied in skpref, whether that is a reduction via scikit-learn or even a model that is not a pairwise comparison model, the package will know that the problem itself is a pairwise comparison problem and can perform reduction and aggregation adequately in the background when needed.

In this example the PairwiseComparisonTask has the following components:

  • primary_table: the table that contains the observed preferences

  • primary_table_alternatives_names: the column or columns that contain the alternatives, in this case both columns team1 and team2 contain alternatives

  • primary_table_target_name: the column that indicates the result of the pairwise comparison

  • target_column_correspondence: in the case of pairwise comparisons, when the alternatives are split across two columns, the column indicating the result usually takes the form 1/0 to show whether one of the columns, in our case team1 or team2 has been preferred. So in this column the user indicates that when the team1_wins column takes the value 1 that means that the alternative in the column team1 has won.

  • features_to_use: indicates which columns to use as covariates

[7]:

NBA_results_task_train_LR = PairwiseComparisonTask(
    primary_table=train_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=['team_1_home']
)

# For the test task, it's possible to make a copy of the training task and
# update the primary table
NBA_results_task_predict_LR = PairwiseComparisonTask(
    primary_table=test_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=['team_1_home']
)

Fitting a Logistic Regression

The only covariate we will use in this for now will be the team_1_home column, which should return a method that only learns what the home team advantage was on average, which is the equivalent to fitting a logistic regression where whether team1 is playing home or not is the only covariate.

\(P(\texttt{team1}\_\texttt{wins}=1) = logit(\alpha + \beta_1 \texttt{team}\_\texttt{1}\_\texttt{home})\)

[8]:

my_log_red = ClassificationReducer(LogisticRegression(solver='lbfgs'))
my_log_red.fit_task(NBA_results_task_train_LR)
preds = my_log_red.predict_task(NBA_results_task_predict_LR)

[9]:

# predict_task returns a SubsetPosetVector which has the attributes
# top_input_data and boot_input_data corresponding to chosen and not chosen
# alternatives.
preds.top_input_data, preds.boot_input_data

[9]:

(array(['Dallas Mavericks', 'Charlotte Hornets', 'Brooklyn Nets', ...,
        'Washington Wizards', 'Washington Wizards', 'Orlando Magic'],
       dtype=object),
 array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,
        'Atlanta Hawks', 'Boston Celtics', 'Washington Wizards'],
       dtype=object))

[10]:

NBA_results_task_predict_LR.primary_table.head()

[10]:
team1 team2 season_start team1_wins team_1_home
7380 Atlanta Hawks Dallas Mavericks 2017 1 0
7381 Atlanta Hawks Charlotte Hornets 2017 0 0
7382 Atlanta Hawks Brooklyn Nets 2017 0 0
7383 Atlanta Hawks Miami Heat 2017 0 0
7384 Atlanta Hawks Chicago Bulls 2017 0 0 
[11]:

NBA_results_task_predict_LR.primary_table.tail()

[11]:
team1 team2 season_start team1_wins team_1_home
9835 Washington Wizards Houston Rockets 2017 0 0
9836 Washington Wizards Cleveland Cavaliers 2017 0 0
9837 Washington Wizards Atlanta Hawks 2017 0 1
9838 Washington Wizards Boston Celtics 2017 1 1
9839 Washington Wizards Orlando Magic 2017 0 0 
[12]:

# All this learns so far is the home team advantage, since its the only
# covariate in the test_data table
nice_print_results(
    my_log_red.predict_proba_task(NBA_results_task_predict_LR,
                                  outcome=['Dallas Mavericks', 'Atlanta Hawks']))

Dallas Mavericks  [0.58 0.   0.   ... 0.   0.   0.  ]
Atlanta Hawks     [0.42 0.42 0.42 ... 0.42 0.   0.  ]

[13]:

nice_print_results(
    my_log_red.predict_proba_task(NBA_results_task_predict_LR,
                                  column=['team1', 'team2'])
)

team1 is preferred  [0.42 0.42 0.42 ... 0.58 0.58 0.42]
team2 is preferred  [0.58 0.58 0.58 ... 0.42 0.42 0.58]

Fitting a Bradley Terry model

As we can see in the example above the logistic regression approach does not learn different probabilities for a team winning or losing based on which other team they are playing. The Dallas Mavericks could be playing against the strongest or weakest team in the league and their estimated probability of winning would be the same. The difference between the Bradley-Terry model and logistic regression is that Bradley-Terry learns a function that can estimate whether each team will win or lose given the other team they are playing.

The task we will use for Bradley-Terry will be defined in a slightly different way, because in the first demo we won’t use any covariates, therefore we define features_to_use=None

In the Bradley-Terry model each team gets a latent strength parameter \(\lambda_{\text{team}}\), for example \(\lambda_{\text{Atlanta Hawks}}\).

The Bradley-Terry model learns these strength parameters to maximise the likelihood according to the following formulation for observation \(i\):

\[P(\texttt{team1}\_\texttt{wins}=1)_i= \frac{e^{\lambda_{\texttt{team1}_i}}}{e^{\lambda_{\texttt{team1}_i}} + e^{\lambda_{\texttt{team2}_i}}}\]
[14]:

NBA_results_task_train_BT = PairwiseComparisonTask(
    primary_table=train_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=None
)

NBA_results_task_predict_BT = PairwiseComparisonTask(
    primary_table=test_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=None
)

[15]:

# Fitting Bradley Terry model
mybt = BradleyTerry(method='BFGS', alpha=1e-5)
mybt.fit_task(NBA_results_task_train_BT)

[16]:

mybt.params_

[16]:
entity learned_strength
0 Atlanta Hawks 0.047522
1 Boston Celtics 0.580896
2 Brooklyn Nets -1.178393
3 Charlotte Hornets -0.278154
4 Chicago Bulls -0.037967
5 Cleveland Cavaliers 0.489737
6 Dallas Mavericks -0.386261
7 Denver Nuggets -0.040408
8 Detroit Pistons -0.225709
9 Golden State Warriors 1.538386
10 Houston Rockets 0.765613
11 Indiana Pacers -0.005751
12 Los Angeles Clippers 0.550265
13 Los Angeles Lakers -0.773690
14 Memphis Grizzlies 0.153646
15 Miami Heat -0.022175
16 Milwaukee Bucks 0.018291
17 Minnesota Timberwolves -0.470415
18 New Orleans Pelicans -0.328205
19 New York Knicks -0.548175
20 Oklahoma City Thunder 0.344454
21 Orlando Magic -0.655354
22 Philadelphia 76ers -0.716305
23 Phoenix Suns -0.888314
24 Portland Trail Blazers 0.019229
25 Sacramento Kings -0.426973
26 San Antonio Spurs 1.115135
27 Toronto Raptors 0.462682
28 Utah Jazz 0.535025
29 Washington Wizards 0.361368

We can use the latent alternative strength parameters that Bradley-Terry models learn to rank the teams, either by sorting the mybt.params_ DataFrame by the learned_strength parameter, or by running the rank_entities function

[17]:

mybt.rank_entities(ascending=False)

[17]:

['Golden State Warriors',
 'San Antonio Spurs',
 'Houston Rockets',
 'Boston Celtics',
 'Los Angeles Clippers',
 'Utah Jazz',
 'Cleveland Cavaliers',
 'Toronto Raptors',
 'Washington Wizards',
 'Oklahoma City Thunder',
 'Memphis Grizzlies',
 'Atlanta Hawks',
 'Portland Trail Blazers',
 'Milwaukee Bucks',
 'Indiana Pacers',
 'Miami Heat',
 'Chicago Bulls',
 'Denver Nuggets',
 'Detroit Pistons',
 'Charlotte Hornets',
 'New Orleans Pelicans',
 'Dallas Mavericks',
 'Sacramento Kings',
 'Minnesota Timberwolves',
 'New York Knicks',
 'Orlando Magic',
 'Philadelphia 76ers',
 'Los Angeles Lakers',
 'Phoenix Suns',
 'Brooklyn Nets']

[18]:

# we can create the probability for each team winning in a specific observaion,
nice_print_results(
    mybt.predict_proba_task(NBA_results_task_predict_BT,
                            outcome=['Atlanta Hawks', 'Washington Wizards'])
)

Atlanta Hawks       [0.61 0.58 0.77 ... 0.42 0.   0.  ]
Washington Wizards  [0.   0.   0.   ... 0.58 0.45 0.73]

[19]:

nice_print_results(
    mybt.predict_proba_task(NBA_results_task_predict_BT,
                            column=['team1', 'team2'])
)

team1 is preferred  [0.61 0.58 0.77 ... 0.58 0.45 0.73]
team2 is preferred  [0.39 0.42 0.23 ... 0.42 0.55 0.27]

[20]:

mybt.predict_choice_task(NBA_results_task_predict_BT)

[20]:

array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,
       'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],
      dtype=object)

[21]:

preds = mybt.predict_task(NBA_results_task_predict_BT)

[22]:

preds.top_input_data, preds.boot_input_data

[22]:

(array(['Atlanta Hawks', 'Atlanta Hawks', 'Atlanta Hawks', ...,
        'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],
       dtype=object),
 array(['Dallas Mavericks', 'Charlotte Hornets', 'Brooklyn Nets', ...,
        'Atlanta Hawks', 'Washington Wizards', 'Orlando Magic'],
       dtype=object))

Augmenting the models with covariates

In this section we will start introducing more covariates in the models above, we will introduce one additional covariate which is the team salary budget. We can also see how we can define a single task which we can use to run different models in skpref.

[23]:

NBA_results_task_train = PairwiseComparisonTask(
    primary_table=train_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=['salary', 'team1_home'],
    secondary_table=NBA_team_salary_budget,
    secondary_to_primary_link={
        'team': ['team1', 'team2'],
        'season_start': 'season_start'
    })

NBA_results_task_predict = PairwiseComparisonTask(
    primary_table=test_data,
    primary_table_alternatives_names=['team1', 'team2'],
    primary_table_target_name ='team1_wins',
    target_column_correspondence='team1',
    features_to_use=['salary', 'team1_home'],
    secondary_table=NBA_team_salary_budget,
    secondary_to_primary_link={
        'team': ['team1', 'team2'],
        'season_start': 'season_start'
    })

Reduction to logistic regression with covariates

Here we fit a logistic regression on three covariates, whether team1 is playing home or not, team1’s salary budget and team2’s salary budget. \(P(\texttt{team1}\_\texttt{wins}=1) = logit(\alpha + \beta_1 \texttt{team}\_\texttt{1}\_\texttt{home} + \beta_2 \texttt{team1}\_\texttt{salary} + \beta_3 \texttt{team2}\_\texttt{salary})\)

[24]:

my_log_red = ClassificationReducer(LogisticRegression(solver='lbfgs'))
my_log_red.fit_task(NBA_results_task_train)
preds = my_log_red.predict_task(NBA_results_task_predict)

[25]:

# We can investigate the internal table that was fed into LogisticRegression.fit()
my_log_red.model_input.head(7)

[25]:
team1_wins team_1_home salary_team1 salary_team2
0 1 0 99375302 85753772
1 0 0 99375302 117228164
2 0 0 99375302 95964560
3 0 0 99375302 129458084
4 0 0 99375302 89524016
5 0 1 99375302 107015203
6 0 1 99375302 115375243 
[26]:

# We can also investigate the coefficients which were learned
my_log_red.model.coef_

[26]:

array([[ 5.35210228e-15,  1.54775613e-08, -1.54775613e-08]])

We can see that the coefficients learned for \(\beta_2\) and \(\beta_3\) are very similar to each other, just opposite signs. ClassificationReducer allows users the option to take the difference in features directly rather than split them out, effectively learning the following model: \(P(\texttt{team1}\_\texttt{wins}=1) = logit(\alpha + \beta_1 \texttt{team}\_\texttt{1}\_\texttt{home} + \beta_2 (\texttt{team1}\_\texttt{salary} - \texttt{team2}\_\texttt{salary}))\)

[27]:

my_log_red = ClassificationReducer(
    LogisticRegression(solver='lbfgs'),
    take_feature_diff_for_pairwise_comparison=True
)
my_log_red.fit_task(NBA_results_task_train)
preds = my_log_red.predict_task(NBA_results_task_predict)

[28]:

my_log_red.model_input.head(7)

[28]:
team1_wins team_1_home salary_diff
0 1 0 13621530
1 0 0 -17852862
2 0 0 3410742
3 0 0 -30082782
4 0 0 9851286
5 0 1 -7639901
6 0 1 -15999941 
[29]:

my_log_red.model.coef_

[29]:

array([[2.67602286e-15, 1.54775613e-08]])

[30]:

preds.top_input_data, preds.boot_input_data

[30]:

(array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,
        'Washington Wizards', 'Washington Wizards', 'Washington Wizards'],
       dtype=object),
 array(['Dallas Mavericks', 'Atlanta Hawks', 'Brooklyn Nets', ...,
        'Atlanta Hawks', 'Boston Celtics', 'Orlando Magic'], dtype=object))

[31]:

# All this learns so far is the home team advantage, since its the only
# covariate in the test_data table
nice_print_results(
    my_log_red.predict_proba_task(NBA_results_task_predict,
                                  column='team1')
)

team1 is preferred  [0.55 0.43 0.51 ... 0.59 0.53 0.61]

Bradley Terry model with salary covariate

Here we augment the initial Bradley-Terry model to learn the following relationship:

\[P(\texttt{team1}\_\texttt{wins}=1)_i= \frac{e^{(\lambda_{\texttt{team1}_i} + \beta_1 \texttt{team1}\_\texttt{salary}_i)}}{e^{(\lambda_{\texttt{team1}_i} + \beta_1 \texttt{team1}\_\texttt{salary}_i)} + e^{(\lambda_{\texttt{team2}_i}+ \beta_1 \texttt{team2}\_\texttt{salary}_i)}}\]
[32]:

mybt = BradleyTerry(method='BFGS', alpha=1e-5)
mybt.fit_task(NBA_results_task_train)
mybt.rank_entities(ascending=False)

[32]:

array(['Golden State Warriors', 'San Antonio Spurs', 'Houston Rockets',
       'Utah Jazz', 'Boston Celtics', 'Oklahoma City Thunder',
       'Washington Wizards', 'Toronto Raptors', 'Los Angeles Clippers',
       'Denver Nuggets', 'Atlanta Hawks', 'Indiana Pacers',
       'Chicago Bulls', 'Cleveland Cavaliers', 'Memphis Grizzlies',
       'Miami Heat', 'Milwaukee Bucks', 'Charlotte Hornets',
       'Minnesota Timberwolves', 'Portland Trail Blazers',
       'New Orleans Pelicans', 'Sacramento Kings', 'Detroit Pistons',
       'Dallas Mavericks', 'Philadelphia 76ers', 'New York Knicks',
       'Phoenix Suns', 'Los Angeles Lakers', 'Orlando Magic',
       'Brooklyn Nets'], dtype=object)

[33]:

nice_print_results(mybt.predict_proba_task(NBA_results_task_predict, column=['team1', 'team2']))

team1 is preferred  [0.69 0.48 0.75 ... 0.65 0.43 0.82]
team2 is preferred  [0.31 0.52 0.25 ... 0.35 0.57 0.18]

[34]:

mybt.predict_choice_task(NBA_results_task_predict)

[34]:

array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,
       'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],
      dtype=object)

[35]:

mybt.predict_task(NBA_results_task_predict).top_input_data

[35]:

array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,
       'Washington Wizards', 'Boston Celtics', 'Washington Wizards'],
      dtype=object)

[36]:

mybt.bt_with_feats.get_statsmodels_summary()

[36]:
Multinomial Logit Model Regression Results
Dep. Variable: CHOICE No. Observations: 2,460
Model: Multinomial Logit Model Df Residuals: 2,429
Method: MLE Df Model: 31
Date: Wed, 01 Mar 2023 Pseudo R-squ.: 0.107
Time: 16:10:40 Pseudo R-bar-squ.: 0.089
AIC: 3,107.966 Log-Likelihood: -1,522.983
BIC: 3,288.012 LL-Null: -1,705.142
coef std err z P>|z| [0.025 0.975]
salary 1.717e-08 3.65e-06 0.005 0.996 -7.13e-06 7.17e-06
Atlanta Hawks 0.0810 41.439 0.002 0.998 -81.138 81.300
Boston Celtics 0.7344 52.254 0.014 0.989 -101.682 103.151
Brooklyn Nets -0.9380 65.383 -0.014 0.989 -129.086 127.210
Charlotte Hornets -0.1415 50.101 -0.003 0.998 -98.339 98.056
Chicago Bulls 0.0621 46.030 0.001 0.999 -90.156 90.280
Cleveland Cavaliers -0.0049 112.740 -4.33e-05 1.000 -220.970 220.960
Dallas Mavericks -0.4891 46.300 -0.011 0.992 -91.236 90.258
Denver Nuggets 0.2237 69.393 0.003 0.997 -135.784 136.232
Detroit Pistons -0.4001 55.137 -0.007 0.994 -108.468 107.667
Golden State Warriors 1.4940 41.902 0.036 0.972 -80.632 83.620
Houston Rockets 0.9021 50.079 0.018 0.986 -97.252 99.056
Indiana Pacers 0.0727 44.101 0.002 0.999 -86.363 86.508
Los Angeles Clippers 0.2355 78.355 0.003 0.998 -153.337 153.808
Los Angeles Lakers -0.7116 42.901 -0.017 0.987 -84.796 83.373
Memphis Grizzlies -0.0434 58.483 -0.001 0.999 -114.668 114.581
Miami Heat -0.0820 42.759 -0.002 0.998 -83.888 83.724
Milwaukee Bucks -0.1289 51.425 -0.003 0.998 -100.920 100.663
Minnesota Timberwolves -0.1561 78.274 -0.002 0.998 -153.569 153.257
New Orleans Pelicans -0.3903 42.901 -0.009 0.993 -84.476 83.695
New York Knicks -0.6348 44.785 -0.014 0.989 -88.412 87.143
Oklahoma City Thunder 0.4593 47.565 0.010 0.992 -92.766 93.685
Orlando Magic -0.7402 44.632 -0.017 0.987 -88.217 86.736
Philadelphia 76ers -0.5043 60.789 -0.008 0.993 -119.649 118.640
Phoenix Suns -0.6594 63.503 -0.010 0.992 -125.123 123.804
Portland Trail Blazers -0.2206 65.293 -0.003 0.997 -128.192 127.751
Sacramento Kings -0.3933 41.448 -0.009 0.992 -81.630 80.843
San Antonio Spurs 0.9525 53.485 0.018 0.986 -103.876 105.781
Toronto Raptors 0.2806 56.240 0.005 0.996 -109.949 110.510
Utah Jazz 0.8459 77.655 0.011 0.991 -151.355 153.047
Washington Wizards 0.2946 43.216 0.007 0.995 -84.407 84.997

Example using GridSearchCV()

The models we have fitted above also have hyperparameters, such as the method of gradient descent or regularisation. To optimise the hyperparameter selection, we can use GridSearchCV(). GridSearchCV() tries out a series of hyperparameter combinations and runs a k-fold cross-validation on an accuracy metric determined by the user to check which ones have performed best.

[37]:

to_tune = {'alpha': [1, 2, 4], 'method': ['BFGS']}
gs_bt = GridSearchCV(BradleyTerry(), to_tune,  cv=3, scoring='neg_log_loss')
gs_bt.fit_task(NBA_results_task_train)
gs_bt.inspect_results()

The model with the best parameters was:
BradleyTerry(alpha=2, method='BFGS')
With a score of -0.6265008194657992
All the trials results summarised in descending score
   alpha method  mean_test_score
1      2   BFGS        -0.626501
0      1   BFGS        -0.626742
2      4   BFGS        -0.628853

[38]:

# Showing that sklearn.metrics works also
to_tune = {'alpha': [1, 2, 4], 'method': ['BFGS']}
gs_bt = GridSearchCV(BradleyTerry(), to_tune,  cv=3, scoring=f1_score)
gs_bt.fit_task(NBA_results_task_train)
gs_bt.inspect_results()

The model with the best parameters was:
BradleyTerry(alpha=4, method='BFGS')
With a score of 0.6337744652191032
All the trials results summarised in descending score
   alpha method  mean_test_score
2      4   BFGS         0.633774
1      2   BFGS         0.631136
0      1   BFGS         0.630085

[39]:

to_tune = {'C': [0.5, 1, 2, 4, 8], 'solver': ['saga'], 'penalty': ['l1','l2'],
           'fit_intercept': [True, False]}
gs_lr = GridSearchCV(ClassificationReducer(LogisticRegression()), to_tune,
                     cv=3, scoring='neg_log_loss')
gs_lr.fit_task(NBA_results_task_train)
gs_lr.inspect_results()

The model with the best parameters was:
ClassificationReducer(model=LogisticRegression(C=0.5, penalty='l1',
                                               solver='saga'))
With a score of -0.6865126660183437
All the trials results summarised in descending score
    model__C  model__fit_intercept model__penalty model__solver  \
0        0.5                  True             l1          saga
13       4.0                  True             l2          saga
6        1.0                 False             l1          saga
2        0.5                 False             l1          saga
5        1.0                  True             l2          saga
16       8.0                  True             l1          saga
10       2.0                 False             l1          saga
19       8.0                 False             l2          saga
9        2.0                  True             l2          saga
14       4.0                 False             l1          saga
18       8.0                 False             l1          saga
4        1.0                  True             l1          saga
11       2.0                 False             l2          saga
1        0.5                  True             l2          saga
17       8.0                  True             l2          saga
15       4.0                 False             l2          saga
7        1.0                 False             l2          saga
8        2.0                  True             l1          saga
3        0.5                 False             l2          saga
12       4.0                  True             l1          saga

    mean_test_score
0         -0.686513
13        -0.686516
6         -0.686516
2         -0.686517
5         -0.686517
16        -0.686517
10        -0.686518
19        -0.686518
9         -0.686518
14        -0.686518
18        -0.686518
4         -0.686518
11        -0.686519
1         -0.686519
17        -0.686519
15        -0.686520
7         -0.686520
8         -0.686520
3         -0.686521
12        -0.686522

[40]:

gs_lr.predict_task(NBA_results_task_predict).top_input_data

[40]:

array(['Atlanta Hawks', 'Charlotte Hornets', 'Atlanta Hawks', ...,
       'Washington Wizards', 'Washington Wizards', 'Washington Wizards'],
      dtype=object)

[41]:

nice_print_results(gs_lr.predict_proba_task(NBA_results_task_predict, column='team1'))

team1 is preferred  [0.55 0.43 0.51 ... 0.59 0.53 0.61]

[42]:

nice_print_results(gs_bt.predict_proba_task(NBA_results_task_predict, column='team1'))

team1 is preferred  [0.67 0.47 0.7  ... 0.64 0.45 0.79]

[43]:

gs_bt.rank_entities(ascending=False)

[43]:

array(['Golden State Warriors', 'San Antonio Spurs', 'Houston Rockets',
       'Utah Jazz', 'Boston Celtics', 'Oklahoma City Thunder',
       'Washington Wizards', 'Toronto Raptors', 'Los Angeles Clippers',
       'Denver Nuggets', 'Atlanta Hawks', 'Indiana Pacers',
       'Chicago Bulls', 'Cleveland Cavaliers', 'Memphis Grizzlies',
       'Miami Heat', 'Milwaukee Bucks', 'Charlotte Hornets',
       'Minnesota Timberwolves', 'Portland Trail Blazers',
       'Detroit Pistons', 'New Orleans Pelicans', 'Sacramento Kings',
       'Philadelphia 76ers', 'Dallas Mavericks', 'New York Knicks',
       'Phoenix Suns', 'Los Angeles Lakers', 'Orlando Magic',
       'Brooklyn Nets'], dtype=object)