AI Innovation Institute

Abstract: To be helpful assistants, AI agents must be aware of their own capabilitiesand limitations. This includes knowing when to answer from parametricknowledge versus using tools, when to trust tool outputs, and when toabstain or hedge. Such capabilities are hard to teach through supervisedfine-tuning because they require constructing examples that reflect theagent's specific capabilities. We therefore propose a radically new approachto teaching agents what they know: collaborative self-play. We constructmulti-agent collaborations in which the group is rewarded for collectivelyarriving at correct answers. The desired meta-knowledge emerges fromthe incentives built into the structure of the interaction. We focus on smallsocieties of agents that have access to heterogeneous tools (corpus-specificretrieval), and therefore must collaborate to maximize their success withminimal effort. Experiments show that group-level rewards for multiagent communities can induce policies that transfer to improve tool use andselective prediction in single-agent scenarios. Speaker: Syed Mostofa Monsur Location: Old Computer Science Building - CS2311