What did you learn about CMDA from your readings?
I went down a rabbit hole of how CMDA can be applied to conversations surrounding ChatGPT and other AI chatbots.
The article Data-driven artificial intelligence to automate researcher assessment (Weber & Duarte, 2021) presents a data-driven AI approach to automate researcher assessment by classifying candidate researchers as fit or unfit for a specific job placement. The approach adopts case-based reasoning, which allows adaptation, machine learning, and explainability. The methodology considers career trajectories and provides explanations, making it more accurate than purpose-independent classifiers. “…human decisions may be neither transparent nor reproducible. The approach in this article describes how to use AI methods to, from a job description, select the best fit candidates, while being transparent and reproducible.” The proposed approach addresses the limitations of human decision-making and ensures transparency, reproducibility, and accuracy in the assessment process.
I am familiar with AI technology being used to traverse a candidate’s resume and identifying keywords that align with the job description to help recruiters sift through large amounts of candidates to select those who might get an interview; however, what if the AI could also evaluate the interview conversation and further evaluate the potential candidates based on the live responses? While this article does not directly connect to my future research, I do find it extremely interesting that this potential now exists.
/
My search for “artificial intelligence to automate “qualitative” methods” also returned multiple software technologies that can be useful for researchers when evaluating qualitative research results:
- Speak – https://speakai.co/
- Schlesinger Group – https://www.schlesingergroup.com/
- Remesh – https://www.remesh.ai/
- Insights – https://www.insights.us/
- Qualsights – https://www.qualsights.com/
- Delvinia – https://www.delvinia.com/
- Quester – https://www.quester.com/
I’m curious if anyone in the group has experience with any of these tools and learn how they were used for accdemic research.
I also found it interesting that the skill of qualitative research can be used in industry as well as education. Customer survey feedback responses will require significant qualitative skills to evaluate the potentially large amount of incoming respones data. In this case, an automated AI tool would be needed to be able to evaluate the data more quickly than what can be evaluated bya a human. However, I still believe thathuman-oversight will be necessary, as there could be errors within the survey coding or perhaps a unique circumstance arrose that affected the survey responses; an AI would not be aware of the abnormalities.
What appears to be useful?
This got me thinking about how we might be able to use qualitative research methods to evaluate conversations using AI tools and chatbots. I am intereted to see what types of questions are being asked of ChatGPT, and the depth of responses that are being provided. In my experience, the responses have been thorough, but lack the feel of humanism. While ChatGPT will directly tell you that it is not human, nor does it have human charastics, I feel that with a little bit of massaging we (society) could get more increased instances of human-like responses from ChatGPT. In this case I feel we can use the CMDA framework evaluate those conversations.
What may be challenging? Why?
In the aforementioned scenario of using the CMDA framework for ChatGPT conversations, the challenging component would be How do we get access to these ChatGPT conversations? Is OpenAI willing to share the data they’re gathering from their ChatGPT conversations? If so, there will likely be so much data that it will be near-impossible for a human to conduct the qualitative CMDA research, and we’d likly need an AI to conduct the research. It’s not lost on me that an AI would be conducting research related to humanistic conversational traights on another AI…seems ironic.
Are there specific settings in your own life as a researcher/practitioner where this may be the right method to answer your questions? Why or why not?
I’m not sure yet. I am engrossed with the firestorm of ChatGPT useage and OpenAI advancements, as well as it’s competitators. I am following this technology advancement closely, but there is so much coming all at once I am not quite sure where to start from a research perspective.
