The machines take over: a comparison of various supervised learning approaches for automated scoring of divergent thinking tasks
dc.contributor.author | Buczak, Philip | |
dc.contributor.author | Huang, He | |
dc.contributor.author | Forthmann, Boris | |
dc.contributor.author | Doebler, Philipp | |
dc.date.accessioned | 2024-02-15T10:40:12Z | |
dc.date.available | 2024-02-15T10:40:12Z | |
dc.date.issued | 2022-08-08 | |
dc.description.abstract | Traditionally, researchers employ human raters for scoring responses to creative thinking tasks. Apart from the associated costs this approach entails two potential risks. First, human raters can be subjective in their scoring behavior (inter-rater-variance). Second, individual raters are prone to inconsistent scoring patterns (intra-rater-variance). In light of these issues, we present an approach for automated scoring of Divergent Thinking (DT) Tasks. We implemented a pipeline aiming to generate accurate rating predictions for DT responses using text mining and machine learning methods. Based on two existing data sets from two different laboratories, we constructed several prediction models incorporating features representing meta information of the response or features engineered from the response’s word embeddings that were obtained using pre-trained GloVe and Word2Vec word vector spaces. Out of these features, word embeddings and features derived from them proved to be particularly effective. Overall, longer responses tended to achieve higher ratings as well as responses that were semantically distant from the stimulus object. In our comparison of three state-of-the-art machine learning algorithms, Random Forest and XGBoost tended to slightly outperform the Support Vector Regression. | en |
dc.description.abstract | Correction for this article: https://doi.org/10.1002/jocb.627 | en |
dc.identifier.uri | http://hdl.handle.net/2003/42333 | |
dc.identifier.uri | http://dx.doi.org/10.17877/DE290R-24170 | |
dc.language.iso | en | de |
dc.relation.ispartofseries | The journal of creative behavior;57(1) | |
dc.rights.uri | https://creativecommons.org/licenses/by-nc/4.0/ | de |
dc.subject | Divergent thinking | en |
dc.subject | Creative quality | en |
dc.subject | Human ratings | en |
dc.subject | Supervised learning | en |
dc.subject | Random Forest | en |
dc.subject | Gradient boosting | en |
dc.subject | Support Vector Regression | en |
dc.subject.ddc | 310 | |
dc.title | The machines take over: a comparison of various supervised learning approaches for automated scoring of divergent thinking tasks | en |
dc.type | Text | de |
dc.type.publicationtype | Article | de |
dcterms.accessRights | open access | |
eldorado.secondarypublication | true | de |
eldorado.secondarypublication.primarycitation | Buczak, P., Huang, H., Forthmann, B. and Doebler, P. (2023), The Machines Take Over: A Comparison of Various Supervised Learning Approaches for Automated Scoring of Divergent Thinking Tasks. J Creat Behav, 57: 17-36. https://doi.org/10.1002/jocb.559 | de |
eldorado.secondarypublication.primaryidentifier | https://doi.org/10.1002/jocb.559 | de |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Journal of Creative Behavior - 2022 - Buczak - The Machines Take Over A Comparison of Various Supervised Learning.pdf
- Size:
- 2.59 MB
- Format:
- Adobe Portable Document Format
- Description:
- DNB
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 4.85 KB
- Format:
- Item-specific license agreed upon to submission
- Description: