Methods that make machine learning model predictions more accurate overall can reduce accuracy for subgroups not shown. A new approach may be helpful.
For workers who use machine learning models to help them make decisions, knowing when to trust the model’s predictions is not always easy, especially because these models are often so complex that their inner workings remain a mystery.
Users sometimes use a technique known as selective regression, where the model estimates its confidence level for each prediction and will reject predictions when its confidence is too low. Humans can then examine those cases, gather additional information, and make decisions about each case manually.
But while selective regression has been shown to improve the overall performance of a model, researchers at MIT and the MIT-IBM Watson AI Lab have found that the technique can have an impact. vice versa for groups of people that are underrepresented in the data set. As the reliability of the model increases with selective regression, its chances of making the correct prediction also increase, but this is not always the case for all subgroups.
For example, a loan approval recommendation model might make fewer errors than average, but it could actually make more false predictions for Black or female applicants. One reason this might happen is that the confidence measure of the model is trained using over-represented groups and may not be accurate for these under-represented groups.
Once they identified this problem, the MIT researchers developed two algorithms to overcome the problem. Using real-world datasets, they show that algorithms that reduce performance disparities affect marginalized subgroups.
“Ultimately, this is about getting smarter about the patterns you give humans to process. Instead of just minimizing some broad error rates for the model, we wanted to make sure the error rates between groups are intelligently taken into account,” said MIT senior author Greg Wornell, Sumitomo Professor of Engineering at the Department of Electrical Engineering and Computer Science. (EECS) who leads the Signals, Information, and Algorithms Laboratory in the Electronics Research Laboratory (RLE) and is a member of the MIT-IBM Watson AI Lab.
Join Wornell on paper are co-authors Abhin Shah, an EECS graduate, and Yuheng Bu, a postdoc in RLE; as well as Joshua Ka-Wing Lee SM ’17, ScD ’21 and Subhro Das, Rameswar Panda, and Prasanna Sattigeri, research staff at the MIT-IBM Watson AI Lab. The paper will be presented at the International Conference on Machine Learning.
To predict or not to predict
Regression is a technique for estimating the relationship between a dependent variable and independent variables. In machine learning, regression analysis is often used for predictive tasks, such as predicting the price of a house based on its features (number of bedrooms, square footage, etc.) Selectively, the machine learning model can make one of two choices for each input – it can either make a prediction or skip the prediction if it is not confident enough in its decision.
When the model abstracts, it reduces the portion of the sample it makes predictions, called the coverage. By making predictions only on inputs for which it is highly reliable, the overall performance of the model is improved. But this can also amplify biases that exist in the data set, which occur when the model does not have enough data from certain subgroups. This can lead to errors or bad predictions for poorly represented individuals.
The MIT researchers aimed to ensure that, as the model’s overall error rate improved with selective regression, the performance for every subgroup also improved. They call this mono-selective risk.
“It is difficult to come up with a proper concept of fairness for this particular issue. But by enforcing this criterion, which risks monotonous selection, we can ensure that the model’s performance is actually getting better across all subgroups as you decrease the fit,” Shah said. .
Focus on fairness
The team developed two neural network algorithms that impose this criterion of fairness to solve the problem.
An algorithm that ensures that the features the model uses to make predictions contain all information about sensitive attributes in the data set, such as race and gender, that are related to the item variable. attention target. Sensitive attributes are features that may not be used for decisions, usually because of law or organizational policy. The second algorithm uses a calibration technique to ensure that the model makes the same prediction for an input, regardless of whether any sensitive attributes are added to that input.
The researchers tested these algorithms by applying them to real-world datasets that could be used in high-stakes decision making. One, the insurance dataset, is used to predict the total annual medical costs charged to the patient using demographic statistics; Another, a crime dataset, is used to predict the number of violent crimes in communities using socioeconomic information. Both datasets contain sensitive attributes for each individual.
When they implemented their algorithms on the standard machine learning approach for selective regression, they reduced the disparity by achieving lower error rates for minority subgroups in each data set. Furthermore, this was done without significantly affecting the overall error rate.
“We found that if we don’t impose certain constraints, in the case of a really confident model, it can actually make more errors, which can be very expensive in some applications. , such as healthcare. So if we reverse the trend and make it more intuitive, we will come across a lot of these errors. The main goal of this work is to avoid bugs that go undetected,” says Sattigeri.
The researchers plan to apply their solutions to other applications, such as predicting house prices, student GPA or loan interest rates, to see if the algorithms need to be calibrated. for those tasks or not, Shah said. They also want to explore techniques for using less sensitive information during model training to avoid privacy issues.
And they hope to improve confidence estimates in selective regression to prevent situations where the model’s confidence is low, but its prediction is correct. This can reduce the workload on humans and further streamline the decision-making process, says Sattigeri.
Written by Adam Zewe
Source: Massachusetts Institute of Technology
