In April 2023, researchers from the Nielsen Norman Group published a new study on generative AI that brings valuable insight into how this technology improves productivity, work quality, brainstorming sessions, and reduces time dedicated to task management.
Their premise is that users shouldn’t solemnly rely on ChatGPT results without fact-checking them because, that way, the output may be faulty and therefore unreliable.
That’s why, the study aims to find out whether it’s worth the human effort if people use ChatGPT and check its output, edit it and correct it.
What segments does AI improve?
The researchers came to, what they call, confident conclusions, based on the following parameters:
- The group that relied on ChatGPT delivered both documents in 17 minutes, whereas the other group took about 27 minutes.
In other words, the group that wrote the documents by themselves could optimally produce around 17.7 documents in an 8-hour workday, whereas that number could increase up to 28.3 if they used ChatGPT.
Or, to put it more blandly, AI tools could bring up to 59% of productivity improvement in a workplace.
However, it wouldn’t be much of a help if the number of outputs increased but they remained low quality.
Commentators complain that one can trick the AI into saying nasty things and that the output is often misleading. As to the first, I say: so what. You can trick Excel to calculate things wrong by using erroneous formulas. The measure of a professional tool is not whether it does something wrong when you deliberately abuse it. The question is whether results are good when the tool is used as intended.
Jacob Nielsen
Another conclusion regarding this study was that documents written with the help of AI received better grades: 4.5 (with AI) versus 3.8 (without AI) on a scale of 1-7.
Again, these statistics were recorded among participants who mostly didn’t have any prior experience with AI tools. Thus, it’s only expected that the numbers will be even higher as the user gets more experienced.
Long-term improvements will likely be much bigger, as users discover better ways of using the tool and adapt their work styles accordingly. Something called the task–artifact cycle, where the largest benefits from a new tool come from adapting the way you work to the new capabilities offered by the tool. This is in contrast to automating existing business processes without change, which is often suboptimal.
Jacob Nielsen
Why does AI improve these segments?
As the researchers in this study pointed out, it’s much more interesting to understand the “why” than the “what”.
They admit they have only partially come close to answering this question, but these are the most important takeaways:
- The use of ChatGPT reduced skill inequalities;
- Using ChatGPT changed the way users had spent their time.
In other words, this study shows that the “rough draft” step usually lasts twice as long as the editing time; that’s the primary factor that impacts the reduction in time. Moreover, investing additional time in editing can result in delivering high-quality content, and better one than you would normally do without the help of AI.
Or, to quote Jacob Nielsen himself, in AI-assisted documents, there’s “less time spent on cranking out initial draft text and more time spent polishing the final result.”
Considering that time is now perceived as the highest luxury, it’s understandable why ChatGPT “saving you time” is a game-changing quality.
Conclusion
Although the study showed that using AI tools can have a certain impact on content quality, it is much more valuable when it comes to improving productivity.
And finally, it’s safe to say that the more you use it, the better your results will be. Because, as with any other language, you only get more fluent in it the more you practice it for your specific purposes and environment.
