MAIO

• “BCG consultants using AI completed 12.2% more tasks while doing it 25.1% faster. They also
produced over 40% higher quality results compared to those not using AI. That seems quite
incredible.”
• BUT
• “Despite these productivity gains, the study found a significant increase in errors when tasks
fell outside the AI enhancement. Consultants using AI were 19 percentage points more likely to
produce incorrect solutions in such scenarios.”

• “Consistent with prior work, our results show that access to GPT-4 significantly improves performance
(48% improvement for GPT Base and 127% for GPT Tutor). However, we additionally find that when
access is subsequently taken away, students actually perform worse than those who never had access
(17% reduction for GPT Base). ”
• “ Our results suggest that students attempt to use GPT-4 as a "crutch" during practice problem
sessions, and when successful, perform worse on their own.

Let’s say human can choose how much weight to put on AI decision
With full autonomy (people can choose any weight from 0 to 100% on AI)
• People put Naïve weight – close to 50% and does not change it even if the performance goes down

• People put better weights and improve performance of the system
• Due to bounded autonomy humans are more sensitive to AI accuracy, put more weight to
“accurate” agent

image

Automation