Model Bias

Measuring inherent bias in machine learning models

Model bias refers to the systematic deviation in predictions generated by a machine learning model that can be attributed to factors beyond the user's preferences. Biases can arise from various sources, such as historical data, algorithmic limitations, or user behavior. For example, a recommender system may give more weight to popular items, even if they are not the best fit for a particular user. This bias is referred to as the popularity bias.

In the shopping cart recommendation example, we define the following config to measure popularity bias

check_popularity_bias = {
    'type': uptrain.Anomaly.POPULARITY_BIAS,
    'algorithm': uptrain.BiasAlgo.POPULARITY_BIAS,
    'sessions': x_train_sku,   
}

With popularity bias added in the check, we obtain the following histogram of the frequency of predicted items versus their popularity. Most predicted items have low popularity, implying low popularity bias.

A histogram of the frequency of predicted items versus their popularity. Most predicted items have low popularity, implying low popularity bias.

UpTrain provides several tools to monitor and detect model bias. One of the most common methods is to compare the predictions with the actual user feedback or ground truth. The accuracy of the model's predictions can be measured using standard metrics such as precision, recall, and F1 score. The metrics can help detect bias in predictions, such as if the system is recommending only popular items.

Another way to detect bias is to analyze the distribution of predictions across different user groups. For example, if the recommender system is biased towards a particular demographic or geography, it may lead to under-representation or over-representation of certain items. This can be checked by monitoring the distribution of recommendations and comparing it with the ground truth.

UpTrain also allows users to define custom monitors to detect and handle model bias specific to their use case. In our shopping cart recommendation example, we added a custom check to measure the cosine distance between the embeddings of the recommended items and the items that were actually bought. The config check was defined as below

custom_monitor = {
    'type': uptrain.Anomaly.CUSTOM_MONITOR,
    'initialize_func': cosine_dist_init,
    'check_func': cosine_distance_check,
    'need_gt': True,
    'dashboard_name': 'cosine_distance'
}

where cosine_dist_init and cosine_distance_check were custom-defined by the user (checkout the example to see their definitions).

In the obtained dashboard, we observe that a lot of them have zero cosine distance (implying that the recommendations were spot on). Also, we observe that the predictions are concentrated around the low cosine distance (< 0.4) space.

Overall, UpTrain provides a range of tools to monitor and detect recommendation bias in a recommender system. These tools can help improve the fairness and accuracy of the system, and ultimately enhance the user experience.

Check out UpTrain's blog on recommendation bias to learn about fairness and bias issues with recommender systems.