Choosing AI: Research Designs

I have attached a PDF about the presentation we are doing. The slides are complete. Please add speaker notes to Part 1 and 2 (Slides 3 – 8)

Here is a reference on the discussion topic, questions, and some of my answers below:

Discussion Topic and Questions:

A Discussion topic around research design and choosing the right pattern for future AI research.

When studying new AI technology, how do we make sure our research is actually meaningful and not just chasing a passing trend? Since AI moves so fast and research can take years to complete, choosing the right “blueprint” for a study is one of the biggest challenges researchers face today.

Here are a few key points to consider for our discussion:

• Avoiding Fads: How can researchers ensure their work stays relevant if the specific AI tool they are studying becomes obsolete before the study is even published?
• The Goal of the Study: Is it better to focus on theory testing (using experiments and surveys to prove a point) or theory building (using deep observation and case studies to understand “how” and “why” people use AI)?
• Choosing the Focus: Does it make more sense to study the AI technology itself, the individual person using it, or the organization that implemented it?
• Quality vs. Quantity: We often hear that “more data is better,” but how do we balance massive data collection with the need for clear, specific research questions?
• The Validity Trade-off: In AI research, is it more important to have a controlled lab environment (high control) or a real-world setting (high realism), knowing it is difficult to have both?

Which approach do you think is most effective for understanding the long-term impact of AI on society?

Also, what type of study would fit well within the zone of validity, and meets the best tradeoff between theory and application? Internal vs external validity?

Discussion Answers: (My Findings)

The domain of artificial intelligence (AI) is a fast-paced field where researchers need to focus on designs that are not based on short-lived trends, but focus on fundamental theoretical foundations and versatile approaches instead of tools that become outdated soon (Ofosu-Ampong, 2024). To prevent fads, it is advisable that studies concentrate on the eternal questions of AI and its moral and social ramifications so that they are not forgotten once the technologies change, allowing one to think about what can be revised according to the new advances. In terms of the objective of the study, the measures of theory building using deep observations and case studies are the best at the discovery of how and why AI adoption mechanisms, but theory testing via experiments and surveys confirms the hypothesis and strengthens evidence (Abbasi et al., 2024). A mixed methodology is the best, which implies certain inductive exploration and deductive validation to develop broad perceptions of the dynamic nature of AI. This plan counters threats of irrelevance because research is based on long-term principles.

The choice of the research focus must consider several levels. This includes the AI technology itself to be technically effective, individual users to be responsive to it, or organizations to be dynamic in terms of its implementation, and a multi-level examination to help in providing the richest understanding (Ofosu-Ampong, 2024). Reliable data collection necessitates specific and clear questions to influence effective data collection because too much data may conceal meaningful trends without adding value to it. The tradeoff between validity is between controlled lab settings with high internal validity due to the isolation of variables and real-life settings with high external validity due to authentic settings, which is difficult to balance (Abbasi et al., 2024). To ensure that the results are reliable and applicable, researchers ought to adopt the compromising approach by adopting such designs as field experiments. This cautious balance helps to avoid methodology fallacies of AI research.

In the case of determining the long-term effects of AI on society, a mixed-method approach that combines the construction of theory in real-world settings with tests to validate the theory is the best, as it would capture subtle and long-term impacts (Ofosu-Ampong, 2024). Longitudinal studies would be appropriate in the zone of validity, providing a middle ground of tradeoff between internal and external validity by maintaining internal validity by imposing consistent controls and increasing external validity through long and naturalistic observations. This type is good as it is the one that bridges between theory and practice, allowing implementation of abstract ideas into practical directions of societal integration (Abbasi et al., 2024). Further, emphasis on external validity of broad relevance at the expense of complete internal rigor is the best way to maximize research contribution over time. This kind of design makes sure that AI questions bring about significant progress.

References

Abbasi, A., Parsons, J., Pant, G., Liu, O. R., & Sarker, S. (2024). Pathways for Design Research on Artificial Intelligence. Information Systems Research, 35(2).

Ofosu-Ampong, K. (2024). Artificial Intelligence Research: A Review on Dominant Themes, Methods, Frameworks, and Future Research Directions.

Telematics and Informatics Reports, 14, 100127100127.