Final Project - CMSC320 - Spring 2025

1) Header with Contributions

Title

Spring 2025 Data Science Project

Aryan Malhotra

Contributions:

A: Project idea - Aryan

B: Dataset Curation and Preprocessing - Aryan

C: Data Exploration and Summary Statistics - Aryan

D: ML Algorithm Design/Development - Aryan

E: ML Algorithm Training and Test Data Analysis - Aryan

F: Visualization, Result Analysis, Conclusion - Aryan

G: Final Tutorial Report Creation - Aryan

2) Introduction

GitHub: https://github.com/dank0i/CMSC320-FinalProject

Topic:

My topic is on European Football, specfically matches and players, to analyze trends in soccer like player stats to goals scored, or influential events during a match that may have decided the victor, and answer questions like which side matters the most, if we can predict the number of goals in a game, etc. The biggest question I wanted to answer was, who would win the UEFA Champions League, since we'd know the finalists before the deadline. I know this will probably be a huge task, so my aim is to get it above 50% accurate at least.

Why:

I’m a huge soccer fan. I grew up watching the sport, but sadly could never pick a club to support. I would watch so many matches regardless of who was winning, and it became a family tradition that on nights where the team that my dad and brother supported played, we would make bets on the score to see who would pick dinner. I also think it's a great dataset to make analyses on referee calls, whether skill/talent matters more than strategy in games, and other conclusions.

3) Data Curation

Dataset: https://www.kaggle.com/datasets/hugomathien/soccer

The sources:

http://football-data.mx-api.enetscores.com/ - Scores, events, lineups for matches between 2008-2016

http://www.football-data.co.uk/ - Market values, etc.

http://sofifa.com/ - Additional data from Soccer video games about player stats and attributes like stamina, agility, etc.

Import code block

import pandas as pd
import sqlite3
import matplotlib.pyplot as plt
import kaggle
import scipy.stats as sc

Importing the data

kaggle.api.authenticate()
kaggle.api.dataset_download_files('hugomathien/soccer', path='dataset', unzip=True)

Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can run 'chmod 600 /root/.config/kaggle/kaggle.json'
Dataset URL: https://www.kaggle.com/datasets/hugomathien/soccer

Setting up a database and converting each to a dataframe

conn = sqlite3.connect("dataset/database.sqlite")
crsr = conn.cursor()

query = "SELECT name FROM sqlite_master WHERE type='table';"
tables = crsr.execute(query).fetchall()
print(tables)

playeratt_db = pd.read_sql("SELECT * FROM Player_Attributes", conn)
player_db = pd.read_sql("SELECT * FROM Player", conn)
match_db = pd.read_sql("SELECT * FROM Match", conn)
league_db = pd.read_sql("SELECT * FROM League", conn)
country_db = pd.read_sql("SELECT * FROM Country", conn)
team_db = pd.read_sql("SELECT * FROM Team", conn)
teamatt_db = pd.read_sql("SELECT * FROM Team_Attributes", conn)

[('sqlite_sequence',), ('Player_Attributes',), ('Player',), ('Match',), ('League',), ('Country',), ('Team',), ('Team_Attributes',)]

4) Exploratory Data Analysis

The main characteristics of this database are the matches table with almost 26 thousand entires and 115 columns, and the player table with 11 thousand entries.

The player attributes table is huge with lots of data, however most of it is from the game FIFA, and has a large amount of null entires. However, this may be useful in determining if a certain statistic boosts a player's potential.

For features that are over-represented, since this dataset was made to calculate betting odds for soccer matches, as a large amount of the columns for the matches table have betting odds for game results related to the team side. Still, this is useful for seeing whether a certain side is favored, but I will try not to use it.

i) I want to explore whether there's a correlation between team side (home, away) and goals scored by the team

Ho = The side of the team does not effect goals scored

Ha = The side of the time does affect goals scored

plt.hist(match_db.groupby('home_team_api_id').mean('home_team_goal')[['home_team_goal']], alpha = 0.6, label = 'Goals scored by teams when at home')
plt.hist(match_db.groupby('home_team_api_id').mean('away_team_goal')[['away_team_goal']], alpha = 0.6, label = 'Goals scored by teams when at away')
plt.title("Mean of goals scored when at home vs away")
plt.legend()
plt.xlabel("Mean goals")
plt.ylabel("Density")
plt.show()

res = match_db.groupby('home_team_api_id')[['home_team_goal','away_team_goal']].mean()
test = sc.ttest_rel(res['home_team_goal'], res['away_team_goal'])
test

TtestResult(statistic=np.float64(4.350588355363938), pvalue=np.float64(1.867155302072682e-05), df=np.int64(298))

After usimg a TTest, the p-value is 1.86e^5, which is much less than 0.05, so we can reject the null hypothesis, showing that there is a correlation between team side and goals, likely that being at home means you score more.

ii) I want to see if there is a correlation between winning the game and the team side

Ho = There is no correlation between the team side and the winning team

Ha = There is a correlation between the team side and the winning team

To do this, I'll have to make a new column showing the result of the game.

def gameres(n):
  if (n == 0):
    return 'Draw'
  elif (n > 0):
    return 'Win'
  else:
    return 'Loss'

match_db['gameresult'] = match_db['home_team_goal'] - match_db['away_team_goal'] # with respect to home team
match_db['gameresult'] = match_db['gameresult'].apply(lambda x: gameres(x))
res = match_db['gameresult'].value_counts()
res.plot(kind='bar', ylabel='Density', xlabel = 'Result of match', title = 'Result of games for the home team')

<Axes: title={'center': 'Result of games for the home team'}, xlabel='Result of match', ylabel='Density'>

winres = match_db.groupby(['home_team_api_id', 'gameresult'])['gameresult'].count()
drawarray = []
lossarray = []
winarray = []
for ((id, result), count) in winres.items():
  if (result == 'Loss'):
    lossarray.append(count)
  elif (result == 'Draw'):
    drawarray.append(count)
  else:
    winarray.append(count)
anova = sc.f_oneway(lossarray, drawarray, winarray)
anova.pvalue

np.float64(5.078239586675978e-27)

Using ANOVA, we find that our pvalue is 5.07e^-27, which is less than 0.05, so we can reject the null hypothesis, showing that there is a correlation between teamside and winning the game.

To see which side, I wil use a post-hoc test.

adhoc = sc.tukey_hsd(lossarray, drawarray, winarray)
print(adhoc)

Tukey's HSD Pairwise Group Comparisons (95.0% Confidence Interval)
Comparison  Statistic  p-value  Lower CI  Upper CI
 (0 - 1)      2.836     0.212    -1.118     6.789
 (0 - 2)    -15.020     0.000   -18.974   -11.067
 (1 - 0)     -2.836     0.212    -6.789     1.118
 (1 - 2)    -17.856     0.000   -21.812   -13.899
 (2 - 0)     15.020     0.000    11.067    18.974
 (2 - 1)     17.856     0.000    13.899    21.812

Thus, group 2, showing that the home side has a different chance (higher) of winning.

iii) Since the results of a soccer game heavily depend on whether your strikers/attackers can score, I wanted to see if the potential statistic of a playerhad a signficant correlation with their current finishing ability

Ho = There is no correlation between potential and finishing ability

Ha = There is a correlation between potential and finishing ability

plt.hist(playeratt_db['potential'].values, alpha = 0.6, label = 'Potential Stat')
plt.hist(playeratt_db['finishing'].values, alpha = 0.6, label = 'Finishing Stat')
plt.legend()
plt.title("Mean of potential vs mean of finishing")
plt.xlabel("Mean values")
plt.ylabel("Density")
plt.show()

potentialvalues = playeratt_db['potential'].dropna().values
finishingvalues = playeratt_db['finishing'].dropna().values
pearson = sc.pearsonr(potentialvalues, finishingvalues)
pearson

PearsonRResult(statistic=np.float64(0.2866840527575014), pvalue=np.float64(0.0))

Since our Pearson's coefficient value is 0.28, it suggests a very weak linear relationship, so players with better finishing may not have a higher potential.

5) Primary Analysis

There are 3 main questions I wanted to answer:

• Which side had the best odds of winning the game, and by how much? (home or away)
• Which team would win the 2025 UEFA Champions League?
• Will this be a high scoring game>

The first question was answered earlier, it's the home team by a significant margin (about 46% chance to win a game at home).

The second question will be answered by making a model based on the data, using classification to predict the winners of the UCL, between Liverpool FC and Paris Saint-Germain. I will also use the previous matches of the UCL to test the data.

5.1) Predicting which team would win the 2025 UCL

Import block

from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
import seaborn as sns
import numpy as np

Visualization functions

def heatmap_report(confmatrix):
  plt.figure(figsize=(6, 5))
  sns.heatmap(confmatrix, annot=True, fmt='d', cmap='Blues')
  plt.title('Confusion Matrix')
  plt.xlabel('Predicted')
  plt.ylabel('Actual')
  plt.show()

def class_report(classrep):
  df = pd.DataFrame(classrep)
  df.iloc[:3, :3].plot(kind='bar')
  plt.title('Classification Report')
  plt.xlabel('Statistics')
  plt.ylabel('Value')
  plt.legend(['Loss', 'Draw','Win'])
  plt.show()

def plot_stackedbar(confmatrix):
  true = np.sum(confmatrix, axis=1)
  correct = np.diag(confmatrix)
  incorrect = true - correct
  x = np.arange(3)
  plt.bar(x, correct, label='Correct', color='green')
  plt.bar(x, incorrect, bottom=correct, label='Incorrect', color='red')
  plt.xticks(x, ['Loss', 'Draw','Win'])
  plt.ylabel('Number of Samples')
  plt.xlabel('Result')
  plt.title('Correct vs Incorrect Predictions')
  plt.legend()
  plt.show()

I will be using Random Forest for the classification, so I need to change the game results column.

def gamereschange(n):
  if (n == 'Win'):
    return 2
  elif (n == 'Draw'):
    return 1
  else:
    return 0

match_db['gameresult'] = match_db['gameresult'].apply(lambda x: gamereschange(x))

Since the team 'names' are just API IDs here, I need to one-hot encode them so they're not treated as statistical values, for X

X = pd.get_dummies(match_db[['home_team_api_id', 'away_team_api_id']], columns=['home_team_api_id', 'away_team_api_id'])
y = match_db['gameresult']

Splitting the data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Training the model, which will predict the outcome for the home team

model = RandomForestClassifier()
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=model, X=X_train, y=y_train, cv=skf)

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

classreport = classification_report(y_test, y_pred)
acc_score = accuracy_score(y_test, y_pred)

print("Classification Report:")
print(classreport)
print("Accuracy Score:", acc_score)
print("Score: ", score)

random_cf = confusion_matrix(y_test, y_pred) # saving for later comparison

Classification Report:
              precision    recall  f1-score   support

           0       0.40      0.39      0.39      1470
           1       0.27      0.19      0.22      1317
           2       0.55      0.65      0.59      2409

    accuracy                           0.46      5196
   macro avg       0.40      0.41      0.40      5196
weighted avg       0.43      0.46      0.44      5196

Accuracy Score: 0.45688991531947654
Score:  [0.43348569 0.43805629 0.44382968 0.45428296 0.43551492]

Visualization

heatmap_report(confusion_matrix(y_test, y_pred))

class_report(classification_report(y_test, y_pred, output_dict=True))

Since the accuracy is low with just using home and away historical data (matches the 46% base home chance), so I will be adding attributes like a home side bias, along with betting odds to boost the accuracy.

match_db['home_advantage'] = 1 # since the table is ordered by home teams, i can just make this 1

Adding the betting odds in

Since they are in odds and not probabilities, I have to go through each and override the features to win, draw, and loss probabilities.

betters = [
    ('B365', 'B365H', 'B365D', 'B365A'),
    ('BWH', 'BWH', 'BWD', 'BWA'),
    ('IWH', 'IWH', 'IWD', 'IWA'),
    ('LBH', 'LBH', 'LBD', 'LBA'),
    ('PSH', 'PSH', 'PSD', 'PSA'),
    ('WHH', 'WHH', 'WHD', 'WHA'),
    ('SJH', 'SJH', 'SJD', 'SJA'),
    ('VCH', 'VCH', 'VCD', 'VCA'),
    ('GBH', 'GBH', 'GBD', 'GBA'),
    ('BSH', 'BSH', 'BSD', 'BSA')
]

for prefix, home, draw, away in betters:
    match_db[home] = match_db.apply(lambda row: (1/row[home])/(1/row[home]+1/row[draw]+1/row[away]), axis=1)
    match_db[draw] = match_db.apply(lambda row: (1/row[draw])/(1/row[home]+1/row[draw]+1/row[away]), axis=1)
    match_db[away] = match_db.apply(lambda row: (1/row[away])/(1/row[home]+1/row[draw]+1/row[away]), axis=1)

Filling NAs with mean, splitting the data and training

features = ['home_advantage','home_team_api_id', 'away_team_api_id', 'B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA', 'IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD', 'WHA', 'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA']
X = pd.get_dummies(match_db[features], columns=['home_team_api_id', 'away_team_api_id'], drop_first=True).fillna(match_db[features].mean())
y = match_db['gameresult']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = RandomForestClassifier(n_estimators=200, class_weight='balanced', random_state=42)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=model, X=X_train, y=y_train, cv=skf)

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

random_bet_cf = confusion_matrix(y_test, y_pred) # saving for later comparison

Classification Report:
              precision    recall  f1-score   support

           0       0.46      0.44      0.45      1470
           1       0.31      0.13      0.18      1317
           2       0.56      0.75      0.64      2409

    accuracy                           0.51      5196
   macro avg       0.44      0.44      0.42      5196
weighted avg       0.47      0.51      0.47      5196

Accuracy Score: 0.506158583525789
Score:  [0.50084195 0.49747414 0.50613423 0.5045717  0.49470645]

Hmmm, still not above the ~50% threshold, but close.

Visualization

heatmap_report(confusion_matrix(y_test, y_pred))

class_report(classification_report(y_test, y_pred, output_dict=True))

A popular machine learning model for sports datasets is XGBoost, so I wanted to see if it helps after splitting the data again.

import xgboost as xgb

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = xgb.XGBClassifier(n_estimators=100, learning_rate=0.1, max_depth=6, random_state=42)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=model, X=X_train, y=y_train, cv=skf)

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

xgb_cf = confusion_matrix(y_test, y_pred) # saving for later comparison

Classification Report:
              precision    recall  f1-score   support

           0       0.48      0.43      0.45      1470
           1       0.25      0.02      0.04      1317
           2       0.54      0.85      0.66      2409

    accuracy                           0.52      5196
   macro avg       0.42      0.43      0.38      5196
weighted avg       0.45      0.52      0.44      5196

Accuracy Score: 0.51905311778291
Score:  [0.51166707 0.51647823 0.52658167 0.51106833 0.51323388]

Visualization

heatmap_report(confusion_matrix(y_test, y_pred))

class_report(classification_report(y_test, y_pred, output_dict=True))

Slightly higher accuracy and over the threshold, but much worse results for predicting draws...

Maybe an ensemble of both will get us over 50% while keeping our recall for group 1 (draws) good?

from sklearn.ensemble import VotingClassifier

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

rf = RandomForestClassifier(n_estimators=200, class_weight='balanced', random_state=42)
xgbc = xgb.XGBClassifier(n_estimators=100, learning_rate=0.1, max_depth=6, random_state=42)

ensemble = VotingClassifier(estimators=[('rf', rf), ('xgb', xgbc)])

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=ensemble, X=X_train, y=y_train, cv=skf)

ensemble.fit(X_train, y_train)
y_pred = ensemble.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

ensemble_cf = confusion_matrix(y_test, y_pred) # saving for later comparison

Classification Report:
              precision    recall  f1-score   support

           0       0.45      0.51      0.48      1470
           1       0.31      0.10      0.16      1317
           2       0.57      0.73      0.64      2409

    accuracy                           0.51      5196
   macro avg       0.44      0.45      0.43      5196
weighted avg       0.47      0.51      0.47      5196

Accuracy Score: 0.5103926096997691
Score:  [0.49627135 0.49747414 0.50541256 0.5060154  0.49831569]

Visualization

heatmap_report(confusion_matrix(y_test, y_pred))

class_report(classification_report(y_test, y_pred, output_dict=True))

Slightly better results for group 1 now! Overall, I wish I could get more than 50% on the accuracy, but the recall and F1-score are good for wins, so I'm happy with that, especially since it's better than just assuming the 46% from home wins and the 33% from blind guessing.

Comparison

Which model is the best?

print("RandomForestClassifier, with no betting odds data")
plot_stackedbar(random_cf)

print("RandomForestClassifier, with betting odds data")
plot_stackedbar(random_bet_cf)

print("XGBoost")
plot_stackedbar(xgb_cf)

print("Ensemble of both")
plot_stackedbar(ensemble_cf)

It's not as simple as picking out the highest accuracy model since statistics like precision and recall are very important here.

Predicting draw results still stays very weak throughout, but since we're predicting tournament results where draws are not possible, we can disregard those and train the model based on wins and losses only.

features = ['home_advantage','home_team_api_id', 'away_team_api_id', 'B365H', 'B365A', 'BWH', 'BWA', 'IWH', 'IWA', 'LBH', 'LBA', 'PSH', 'PSA', 'WHH', 'WHA', 'SJH', 'SJA', 'VCH', 'VCA', 'GBH', 'GBA', 'BSH', 'BSA']
no_draws = match_db[match_db['gameresult'] != 1]

X = pd.get_dummies(no_draws[features], columns=['home_team_api_id', 'away_team_api_id'], drop_first=True).fillna(no_draws[features].mean())
y = no_draws['gameresult'].apply(lambda x: 1 if x == 2 else 0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model_final = xgb.XGBClassifier(n_estimators=100, learning_rate=0.1, max_depth=6, random_state=42) # for later predictions
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=model, X=X_train, y=y_train, cv=skf)

model_final.fit(X_train, y_train)
y_pred = model_final.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

Classification Report:
              precision    recall  f1-score   support

           0       0.65      0.48      0.55      1485
           1       0.72      0.84      0.78      2392

    accuracy                           0.70      3877
   macro avg       0.69      0.66      0.67      3877
weighted avg       0.70      0.70      0.69      3877

Accuracy Score: 0.7036368326025277
Score:  [0.71050935 0.68590777 0.70428894 0.6923573  0.70493389]

Visualization

true = np.sum(ensemble_nodraws_cf, axis=1)
correct = np.diag(ensemble_nodraws_cf)
incorrect = true - correct
x = np.arange(2)
plt.bar(x, correct, label='Correct', color='green')
plt.bar(x, incorrect, bottom=correct, label='Incorrect', color='red')
plt.xticks(x, ['Loss','Win'])
plt.ylabel('Number of Samples')
plt.xlabel('Result')
plt.title('Correct vs Incorrect Predictions')
plt.legend()
plt.show()

70% accuracy! Not bad! Definitely a huge improvement and a great model to predict the winner.

5.2) Are there going to be lots of goals? Are we in for a rollercoaster of emotions?

Creating the dataset using a filter for more than 2 goals, and betting ods, and target set

match_db['high_scoring'] = (match_db['home_team_goal'] + match_db['away_team_goal']) > 2

X = match_db[['B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA', 'IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD', 'WHA', 'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA']]
y = match_db['high_scoring']

Time for more splitting and training!

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=ensemble, X=X_train, y=y_train, cv=skf)

ensemble.fit(X_train, y_train)
y_pred = ensemble.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

goals_cf = confusion_matrix(y_test, y_pred)

Classification Report:
              precision    recall  f1-score   support

       False       0.53      0.69      0.60      2602
        True       0.55      0.38      0.45      2594

    accuracy                           0.53      5196
   macro avg       0.54      0.53      0.52      5196
weighted avg       0.54      0.53      0.52      5196

Accuracy Score: 0.5342571208622017
Score:  [0.52586    0.53476064 0.53403897 0.5392204  0.530077  ]

Visualization

df = pd.DataFrame(classification_report(y_test, y_pred, output_dict=True))
df.iloc[:3, :3].plot(kind='bar')
plt.title('Classification Report')
plt.xlabel('Statistics')
plt.ylabel('Value')
plt.legend(['False','True'])
plt.show()

true = np.sum(goals_cf, axis=1)
correct = np.diag(goals_cf)
incorrect = true - correct
x = np.arange(2)
plt.bar(x, correct, label='Correct', color='green')
plt.bar(x, incorrect, bottom=correct, label='Incorrect', color='red')
plt.xticks(x, ['False','True'])
plt.ylabel('Number of Samples')
plt.xlabel('Result')
plt.title('Correct vs Incorrect Predictions')
plt.legend()
plt.show()

Decent statistics across the board. I wish they were better, but for a noisy, unpredictable game like soccer, I'm happy with this.

5.3) How often do the home team just fall apart?

After seeing the results of the high scoring games and seeing how the home team does tend to have an advantage, I wanted to see if I could predict how often the home team just doesn't score any goals.

This includes both draws and losses, and is meant to show that even with home advantage, soccer is just a beautifully unpredictable game.

Creating the dataset and target set

match_db['home_scored'] = (match_db['home_team_goal'] == 0)

X = match_db[['B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA', 'IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD', 'WHA', 'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA']]
y = match_db['home_scored']

Splitting and training, as usual

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=ensemble, X=X_train, y=y_train, cv=skf)

ensemble.fit(X_train, y_train)
y_pred = ensemble.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

nohome_cf = confusion_matrix(y_test, y_pred)

Classification Report:
              precision    recall  f1-score   support

       False       0.77      0.99      0.87      4010
        True       0.42      0.02      0.04      1186

    accuracy                           0.77      5196
   macro avg       0.60      0.51      0.45      5196
weighted avg       0.69      0.77      0.68      5196

Accuracy Score: 0.7702078521939953
Score:  [0.77219148 0.77579986 0.77195093 0.77237729 0.77189605]

Visualization

df = pd.DataFrame(classification_report(y_test, y_pred, output_dict=True))
df.iloc[:3, :3].plot(kind='bar')
plt.title('Classification Report')
plt.xlabel('Statistics')
plt.ylabel('Value')
plt.legend(['False','True'])
plt.show()

true = np.sum(nohome_cf, axis=1)
correct = np.diag(nohome_cf)
incorrect = true - correct
x = np.arange(2)
plt.bar(x, correct, label='Correct', color='green')
plt.bar(x, incorrect, bottom=correct, label='Incorrect', color='red')
plt.xticks(x, ['False','True'])
plt.ylabel('Number of Samples')
plt.xlabel('Result')
plt.title('Correct vs Incorrect Predictions')
plt.legend()
plt.show()

While the accuracy and the 'true' group reports are really good for a game like soccer, the 'false' group is failing.

You can't get much better than this in my testing.

5.4) Is this game going to be a snoozefest?

What about the opposite? How nice would it be if you could predict whether a match was going to have 0 goals, and just not watch it then?

Creating a filter and target set, then training

match_db['goals'] = (match_db['home_team_goal'] + match_db['away_team_goal']) > 0

X = match_db[['B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA', 'IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD', 'WHA', 'SJH', 'SJD', 'SJA', 'VCH', 'VCD', 'VCA', 'GBH', 'GBD', 'GBA', 'BSH', 'BSD', 'BSA']]
y = match_db['goals']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
score = cross_val_score(estimator=model, X=X_train, y=y_train, cv=skf)

ensemble.fit(X_train, y_train)
y_pred = ensemble.predict(X_test)

print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Accuracy Score:", accuracy_score(y_test, y_pred))
print("Score: ", score)

nogoals_cf = confusion_matrix(y_test, y_pred)

Classification Report:
              precision    recall  f1-score   support

       False       0.09      0.15      0.11       391
        True       0.93      0.88      0.90      4805

    accuracy                           0.82      5196
   macro avg       0.51      0.51      0.51      5196
weighted avg       0.86      0.82      0.84      5196

Accuracy Score: 0.8225558121632025
Score:  [0.92350253 0.92302141 0.92350253 0.92372474 0.92348412]

Visualization

df = pd.DataFrame(classification_report(y_test, y_pred, output_dict=True))
df.iloc[:3, :3].plot(kind='bar')
plt.title('Classification Report')
plt.xlabel('Statistics')
plt.ylabel('Value')
plt.legend(['False','True'])
plt.show()

true = np.sum(nogoals_cf, axis=1)
correct = np.diag(nogoals_cf)
incorrect = true - correct
x = np.arange(2)
plt.bar(x, incorrect, bottom=correct, label='Incorrect', color='red')
plt.bar(x, correct, label='Correct', color='green')
plt.xticks(x, ['True','False'])
plt.ylabel('Number of Samples')
plt.xlabel('Result')
plt.title('Correct vs Incorrect Predictions')
plt.legend()
plt.show()

Again, we see the same pattern of true being really good and accurate, but false failing.

6) Insight and Conclusions

6.1) Predicting the Winner

When it comes to predicting the outcome of the match, the models struggle heavily at predicting draws, while being decent at losses and great at predicting wins.

A lot of this comes down to the class imbalance in dataset itself, where there are much larger number of wins, than compared to draws or even losses. Not to mention just how beautifully unpredictable this sport is. No one ever can be certain of who the winner is going to be, and is why machine learning models to predict the outcome are still fairly inaccurate, and either need to be restricted to national or local leagues, or need sacrifices like dropping draws.

This is why we couldn't just take the highest accuracy model (XGBoost) and call it a day, even if it is the best at predicting wins, because the data is skewed towards wins in such a way that predicting a win for everything would still give a usable accuracy (around 40-45%).

By incorporating the ensemble with RandomForest, dropping draws, and using the betting odds (which are a combination of market predictions and expert analyses), we're able to get a very good model of about 70% accuracy, which is perfect for predicting tournament winners.

This means that we're able to predict the winner with a very good F1-score, meaning our predictions should be accurate, especially if our model predicts a win result.

This is what I'll use in the final section to answer the major question I posed in my intro.

6.2) Lots of Goals? No Home Goals? No Goals at All?

Our predictions for the number of goals falls short for the same reasons, class imbalance.

Football is heavily skewed towards matches being decided by 1 goal, either by the home team or away team, especially between 2008 to 2016, which is what this dataset is based of. This period was more relient on defense and goalkeeping talent, and made it exceptionally hard for players to score. Rather, games were decided on strategy and tactics, especially for bigger clubs like Real Madrid, where tactical fouls were a huge thing.

This skewness is immediately visible in the visualizations: 0 goals / 0 home goals skew HEAVILY towards false, especially with the metrics being so low.

Still, we're able to see that should our model predict that there's going to be goals, we can be sure it's either 1 or 2 goals, but less certain it's more than 2.

Finale

Who wins the UCL 2025 Final, between Inter and PSG?

Let's start by predicting the semi-final of the UCL, between PSG and Arsenal FC.

The input frame

input = pd.DataFrame([{
    'home_advantage': 1,
    'B365H': 2.2, 'B365A': 3.2,
    'BWH': 2.3, 'BWA': 3.1,
    'IWH': 2.25, 'IWA': 3.05,
    'LBH': 2.25, 'LBA': 3.1,
    'PSH': 2.32, 'PSA': 3.08,
    'WHH': 2.2, 'WHA': 3.2,
    'SJH': 2.2, 'SJA': 3.2,
    'VCH': 2.3, 'VCA': 3.19,
    'GBH': 2.25, 'GBA': 3.0,
    'BSH': np.nan, 'BSA': np.nan  # this betting site shut down
}])

Since the encoding has already been done, I need an (ugly) way of setting everything else to 0

home_team_ids = match_db['home_team_api_id'].unique()
away_team_ids = match_db['away_team_api_id'].unique()

for idx in home_team_ids:
  input[f'home_team_api_id_{idx}'] = 0
for idx in away_team_ids:
  input[f'away_team_api_id_{idx}'] = 0

input['home_team_api_id_9847'] = 1,  # PSG
input['away_team_api_id_9825'] = 1,  # Arsenal

model_feature_columns = model_final.feature_names_in_
input = input.reindex(columns=model_final.feature_names_in_, fill_value=0)

input = input.fillna(input.mean())

prediction = model_final.predict(input)
print("Predicted Result:", ('Win' if prediction[0] == 1 else 'Loss'))

Predicted Result: Win

This matches what we saw in the match! Unfortunately, we are unable to use this to predict whether there was more than 2 goals, since the result was 2-1, since betting odds for draws don't exist for tournaments.

Finally, lets predict the UCL 2025 Final, with Inter vs PSG.

final_input = pd.DataFrame([{
    'B365H': 2.20, 'B365A': 3.25,
    'BWH': 2.20, 'BWA': 3.25,
    'IWH': 2.25, 'IWA': 3.20,
    'LBH': 2.25, 'LBA': 3.10,
    'PSH': 2.27, 'PSA': 3.30,
    'WHH': 2.25, 'WHA': 3.10,
    'SJH': np.nan, 'SJA': np.nan,  # no odds for the final yet
    'VCH': 2.22, 'VCA': 3.17,
    'GBH': 2.20, 'GBA': 3.25,
    'BSH': np.nan, 'BSA': np.nan,  # this betting site shut down
    'home_advantage': 0 # since the final takes place at a neutral venue, no home advantage
}])

for idx in home_team_ids:
  final_input[f'home_team_api_id_{idx}'] = 0
for idx in away_team_ids:
  final_input[f'away_team_api_id_{idx}'] = 0

final_input['home_team_api_id_9847'] = 1,  # PSG
final_input['away_team_api_id_8636'] = 1,  # Inter

model_feature_columns = model_final.feature_names_in_
final_input = final_input.reindex(columns=model_final.feature_names_in_, fill_value=0)

Finally, the prediction.

final_input = final_input.fillna(input.mean())

prediction = model_final.predict(final_input)
print("Predicted Result:", ('Win' if prediction[0] == 1 else 'Loss'))

Predicted Result: Win

Let's see if I'm right! My model predicts PSG will win!

This project was a lot of fun, and I defnitely bit off more than I can chew with such an unpredictable sport, but the research, different models, and troubleshooting, was all a very good experience! Again, I wish I could make the model more accurate towards the minority classes, but I hope you enjoyed it!