Artificial Social Agents: Empirical Evaluation, Machine Behaviour, and Rapid PrototypingModule description #51204 / Version 1
- 6 CP
- Portfolio examination
- Ungraded
- Winter and summer semester
- English
| Module / Version | #51204 / #1 |
|---|---|
| Validity | Since Winter semester 2026/27 |
| Available language(s) | English, German |
| Credit points | 6 |
| Person responsible for the module | Wiese, Eva |
| Grading | Ungraded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät V |
|---|---|
| Institute | Institut für Psychologie und Arbeitswissenschaft (IPA) |
| Academic chair | 35321100 FG Kognitionspsychologie und Kognitive Ergonomie (KKE) |
Contact
| Office | MAR 3-2 |
|---|---|
| Contact person | Not specified |
| E-mail address | Not specified |
| Website | Not specified |
Learning outcomes
Students in cognitive science, human factors, and HCI are trained to study human behaviour empirically. This course asks them to apply that discipline to artificial agents: large language models, social robots, and interactive AI systems.
The course is practical and project-based: in small teams, students design and build a reproducible empirical evaluation of an artificial agent using AI-assisted rapid prototyping, and document it as a portfolio.
After completing the module, students can:
formulate an empirically testable question about artificial agent behaviour, or about human interaction with artificial agents;
operationalize a construct such as trust, mind perception, calibration, social attention, or cognitive offloading;
design controlled tasks, stimuli, prompts, or interaction scenarios for evaluating LLMs, robots, or AI agents;
implement a small prototype, eval harness, simulation, or research tool using AI-assisted rapid prototyping;
exercise AI stewardship: critically delegate, verify, override, and refuse AI output, so that speed does not compromise correctness, fairness, or safety;
preserve reproducibility through code, prompts, model/settings logs, stimuli, and run instructions;
analyze failure modes and capability boundaries, not only aggregate performance;
communicate an empirical claim with appropriate limits and without anthropomorphic overclaiming.
The course is taught in English. All work is also submitted in English.
The course is practical and project-based: in small teams, students design and build a reproducible empirical evaluation of an artificial agent using AI-assisted rapid prototyping, and document it as a portfolio.
After completing the module, students can:
formulate an empirically testable question about artificial agent behaviour, or about human interaction with artificial agents;
operationalize a construct such as trust, mind perception, calibration, social attention, or cognitive offloading;
design controlled tasks, stimuli, prompts, or interaction scenarios for evaluating LLMs, robots, or AI agents;
implement a small prototype, eval harness, simulation, or research tool using AI-assisted rapid prototyping;
exercise AI stewardship: critically delegate, verify, override, and refuse AI output, so that speed does not compromise correctness, fairness, or safety;
preserve reproducibility through code, prompts, model/settings logs, stimuli, and run instructions;
analyze failure modes and capability boundaries, not only aggregate performance;
communicate an empirical claim with appropriate limits and without anthropomorphic overclaiming.
The course is taught in English. All work is also submitted in English.
Content
Core content (ex-cathedra)
Introduction to Artificial Social Agents
AI-assisted rapid prototyping with explicit verification: delegation, review, documentation, and reproducibility.
AI stewardship: delegate / verify / override / refuse, and the fluency, proxy, single-perspective, and over-automation traps.
Ethics and validity: human-subjects boundaries, GDPR, bias, transparency, and the risk of overclaiming model capabilities.
Project-based explorative content (hands-on learning)
Machine behaviour: studying AI systems as behaving systems with empirical methods.
LLMs as experimental participants: prompt-as-stimulus design, repeated trials, model/seed controls, comparison to published patterns.
Artificial social agents: mind perception, trust, intentionality, gaze/joint attention, and human-machine teamwork.
Introduction to Artificial Social Agents
AI-assisted rapid prototyping with explicit verification: delegation, review, documentation, and reproducibility.
AI stewardship: delegate / verify / override / refuse, and the fluency, proxy, single-perspective, and over-automation traps.
Ethics and validity: human-subjects boundaries, GDPR, bias, transparency, and the risk of overclaiming model capabilities.
Project-based explorative content (hands-on learning)
Machine behaviour: studying AI systems as behaving systems with empirical methods.
LLMs as experimental participants: prompt-as-stimulus design, repeated trials, model/seed controls, comparison to published patterns.
Artificial social agents: mind perception, trust, intentionality, gaze/joint attention, and human-machine teamwork.
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Artificial Social Agents: Empirical Evaluation, Machine Behaviour, and Rapid Prototyping | Project | Winter and summer semester | [en] | 4 | WS 2026/27 (1) ISIS |
Courses in the course catalog
Courses created from the teaching units of this module.
Winter semester 2026/27 (1)
Artificial Social Agents: Empirical Evaluation, Machine Behaviour, and Rapid Prototyping (Project) 1
- Artificial Social Agents: Empirical Evaluation, Machine Behaviour, and Rapid Prototyping (Projekt) Blockveranstaltung Fr./Sa. 6./7.11.2026, 8./9.101.2027, 5./6.02.2027, 10-18 Uhr
Workload and credit points
Artificial Social Agents: Empirical Evaluation, Machine Behaviour, and Rapid Prototyping (Project)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 6 | 6 h | 36 h |
| project (independent team project work) | 1 | 120 h | 120 h |
| Pre/post processing | 6 | 1 h | 6 h |
| 162 h | |||
The workload of the module adds up to 162 hours. The module therefore comprises 6 credit points.
Description of teaching and learning methods
The course is an integrated course (IV) taught in a block format across three weekend blocks: a kickoff for finding and scoping research questions and forming teams; an intensive supervised hackathon build; and a final block for presentations, demos, and an ethics/validity reflection. The majority of the workload is independent team project work between blocks, with mentored check-ins.
A no-penalty drop deadline is set for 2 pm on the first day, before team formation is complete.
A no-penalty drop deadline is set for 2 pm on the first day, before team formation is complete.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
- basic empirical-methods knowledge;
- willingness to work with code, prompts, data, and prototypes;
- basic programming helpful but not required if teams are mixed.
- willingness to work with code, prompts, data, and prototypes;
- basic programming helpful but not required if teams are mixed.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Ungraded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| Challenge framing | 15 | practical | Challenge description (500 words) + oral presentation (5’) + Q&A (5’) |
| Empirical report | 15 | written | 1500-2000 words fully referred to with minimum 20 scientific citations. Format : markdown or pdf (uploaded to Github repository |
| Individual contribution | 20 | written | Markdown file in Github repository (min 500 words) A clear statement endorsed by all team members on respective individual contributions |
| Reproducibility package | 10 | practical | Markdown file in Github repository (min. 500 words) |
| Final presentation | 10 | oral | presentation (10’) + Q&A (5’) |
| Hackathon participation + Team artefact (open source) | 30 | practical | GitHub-Repository mit Code, Lizenz und README |
Exam description
- basic empirical-methods knowledge;
- willingness to work with code, prompts, data, and prototypes;
- basic programming helpful but not required if teams are mixed.
- willingness to work with code, prompts, data, and prototypes;
- basic programming helpful but not required if teams are mixed.
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
The maximum capacity of participants is 40.
Registration procedures
see ISIS course
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
- Accompanying course materials on AI-assisted rapid prototyping and AI stewardship (provided).
- Binz, M., Akata, E., … Schulz, E. (2025). A foundation model to predict and capture human cognition. Nature, 644, 1002-1009.
- Rahwan, I., et al. (2019). Machine behaviour. Nature, 568, 477-486.
- Rahwan, I., Shariff, A., & Bonnefon, J.-F. (2025). The science fiction science method. Nature, 644, 51-58.
- Wiese, E., et al. - selected work on mind perception, gaze cueing, and joint attention with artificial agents (Making eyes with robots; Humans can't resist robot eyes).
Assigned programs
This module is not used in any program.
Miscellaneous
Not specified