tl;dr

Interactive and engaging Workshop on Simulations for Information Access at SIGIR 2024 in Washington, D.C. (onsite, limited hybrid participation) - Submit your lightning talks / extended abstracts to EasyChair until May 9, 2024 (AoE)

Updates

About

Simulations in various forms have been used to evaluate information access systems, like search engines, recommender systems, or conversational agents. In the form of the Cranfield paradigm, a simulation setup is well-known in the IR community, but user simulations have recently gained interest. While user simulations help to reduce the complexity of evaluation experiments and help with reproducibility, they can also contribute to a better understanding of users. Building on recent developments in methods and toolkits, the SIGIR 2024 Workshop on Simulations for Information Access (Sim4IA @ SIGIR 2024) aims to bring together researchers and practitioners to form an interactive and engaging forum for discussions on the future perspectives of the field. An additional aim is to plan an upcoming TREC/CLEF campaign.

The Sim4IA workshop will be co-located with the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval on July 18, 2024 in Washington D.C., USA.

Objective

This workshop aims to bring together experts and enthusiasts from academia and industry to delve into the challenges, methodologies, and applications of simulating user interactions with information access systems. Simulating user behavior is crucial for evaluating the effectiveness and efficiency of retrieval algorithms, user interfaces, and personalized recommendation systems.

We anticipate a highly interactive and engaging, full-day workshop to foster collaborations and bring the community together. We are keen to hear different views and opinions on the current and future directions for simulations in information access systems. Therefore, we are planning to offer two keynotes, a panel discussion with selected participants, lightning talks, and guided breakout sessions:

  • Keynotes. We intend to have two keynotes representing viewpoints from industry and academia.
  • Lightning talks. We allocate short slots (3-5 minute talks) for accepted opinion pieces and encore talks from a selection of different viewpoints for onsite presentation.
  • Panel discussion. We will moderate a panel with invited panelists from a diverse set of backgrounds championing different subfields and views.
  • Breakout group discussions. The core of this workshop is to initiate a discussion among the participants. We will guide the breakout discussions.

To enable interaction with a broader set of participants, we offer limited hybrid participation in addition to onsite attendance via Zoom.

Tentative Schedule

Time Agenda
9:00-9:30 Welcome
9:30-10:00 Keynote 1
10:00-10:30 Lightning talks
11:00-12:00 Panel discussion
12:00-12:30 Lightning talks
13:30-14:00 Keynote 2
14:00-15:30 Breakout group discussions
16:00-17:00 Reports of the group discussions and closing

Call for Participation

We welcome contributions of original perspectives or already published work to support exchange as lightning talks (extended abstracts) given by in-person attendees. A lightning talk might present an unusual perspective, rough sketch, fresh idea, research vision, or understudied problem. It can explore controversial topics or discuss exploratory solutions but generally sparks a discussion at the workshop. Works that are already published are highly welcome in this format.

Topics of Interest

We invite submissions on a wide range of topics related to simulating users for information access systems, including but not limited to:

Methods for simulating user actions/behavior

  • Synthetic queries (and their relations to real queries)
  • Simulating browsing and clicking behavior
  • Simulating stopping behavior
  • Simulating rich search interactions
  • Interactions with conversational agents

Modeling information needs, knowledge, and cognitive states

  • Connecting information needs and the information hole (ASK)
  • Modeling the connection between knowledge and information need
  • Knowledge states beyond a vocabulary

Modeling contexts and personas

  • Users’ contexts - what to model?
  • Personas and simulation
  • First-time usage vs. learning processes

Building user simulators

  • Toolkits and resources
  • NLP methods and LLMs

Validation of user simulators

  • Requirements for conversational simulators

Submission Details

For a lightning talk, an extended abstract of up to two pages (incl. references, ACM double column) should be submitted, which will be editorially reviewed. Presented talks will be featured in a SIGIR Forum publication for which presenters will be invited as co-authors.

The review process is single-blind. Submissions should be submitted electronically via EasyChair: https://easychair.org/conferences/?conf=sim4ia-sigir2024

Dates

  • Extended abstract submission: May 9, 2024 (AoE)
  • Extended abstract acceptance notification: May 23, 2024
  • Workshop day: July 18, 2024

Registration

The workshop will be in-person. We offer limited hybrid options, but every participant must register via the official SIGIR registration portal (additional venue information and SIGIR author information available)

Please register via the official SIGIR Registration site.

The workshop day fee is 200 USD (early registration, please check for later registration fees).

  • Early registration deadline: May 15, 2024.
  • Standard registration deadline: June 15, 2024.

We look forward to meeting you in D.C.!

Organizers

  • Philipp Schaer, TH Köln – University of Applied Sciences, Germany
  • Christin Kreutz, TH Köln – University of Applied Sciences, Germany
  • Krisztian Balog, University of Stavanger, Norway
  • Timo Breuer, TH Köln – University of Applied Sciences, Germany
  • Norbert Fuhr, University of Duisburg-Essen, Germany