Xuhang Fan (Shawn)INFO
Forecasters do not like use algorithms and rely more on their own judgements, which results in unwanted errors. This papre researched the reason of algorithm aversion and how to promote usage. They found that providing a chance to slightly modify the algorithm can increase the adoption. The mechanism behind the adjustment is that people feel more assured when they got the chance to eliminate potential errors.
Note: this paper closely relates to AI agents and fine-tuning models.
Algorithm Aversion or not
whether to use algorithm of use own judgments
several survey studies indicated that algorithms are at least not sufficiently used.
distaste for “imperfect“ algorithm
- Dietvorst et al. (2015): people with no information about the algorithm’s performance and did not realize that the algorithm is imperfect
Overcoming Algorithm Aversion
many papers argued that aversion stems from an intolerance of inevitable error
Experiment
Task: estimate the percentiles of 20 real high school seniors on a standardized math test using some demographic signals
Manipulation: people can use own judgements only; algorithm only; adjust algorithm results up to 10%
Dynamic form: in study 3, people are allowed to adjust their strategies after having some experience with the performance of algorithms
- people are less satisfied with model performance, other two are similar
Results: people tend to use model partially, and this approach ensures good prediction performance
Key results
Usage
Satisfaction
Choose after stage-1 experience
