Abstract: Reward hacking, where a reasoning model exploits loopholes in a reward function to achieve high rewards without solving the intended task, poses a significant threat. This behavior may be explicit, i.e. verbalized in the model's chain-of-thought (CoT), or implicit, where the CoT appears benign thus bypasses CoT monitors. To detect implicit reward hacking, we propose TRACE (Truncated Reasoning AUC Evaluation). Our key observation is that hacking occurs when exploiting the loophole is easier than solving the actual task. This means that the model is using less effort than required to achieve high reward. TRACE quantifies effort by measuring how early a model's reasoning becomes sufficient to obtain the reward. We progressively truncate a model's CoT at various lengths, force the model to answer, and estimate the expected reward at each cut-off. A hacking model, which takes a shortcut, will achieve a high expected reward with only a small fraction of its CoT, yielding a large area under the reward-vs-length curve. TRACE achieves over 65% gains over our strongest 72B CoT monitor in math reasoning, and over 30% gains over a 32B monitor in coding. We further show that TRACE can discover unknown loopholes during training. Overall, TRACE offers a scalable unsupervised approach for oversight where current monitoring methods prove ineffective.
Speaker: Dikshya
Location: Old Computer Science Building - CS2311
Speaker: Dikshya
Location: Old Computer Science Building - CS2311
AI is everywhere and so are the privacy concerns that come with it. At its core, the most common forms of AI we use today are online digital services and thus inherit the usual privacy risks. We'll take a look at indirect prompt injection- a technique that can trick AI tools into revealing or extracting private information as well as techniques being used in academic contexts to manipulate systems and even mislead researchers.
Register here for the online session.
Register here for the online session.
You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes three short talks on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.
Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.
Tuesday, December 10, 2024, 12:00 pm -- CDS, Bldg. 725, Training Room
Speakers
Esther Tsai, CFN
Yugang Zhang, CFN
Sanket Jantre, CDS
Join Zoom Meeting
https://bnl.zoomgov.com/j/
Meeting ID: 161 176 4217
Passcode: 855752
The Office for Research and Innovation at Stony Brook University invites you to attend the inaugural Wolf Den, an evening designed to bring together members of the regional innovation and entrepreneurial ecosystem.
Meet investors, researchers, startup founders, and business leaders to exchange ideas, foster collaboration, and strengthen connections that drive technology development and economic growth across Long Island.
Agenda
4:30 - 5:00 PM | Grab some cheer & mingle
5:00 - 5:40 PM | Welcome remarks and AI Panel
5:40 - 6:00PM | Featured lightning pitches
6:00 - 7:00 PM | Food, drinks and great conversations!
Attendees will have the opportunity to learn more about Stony Brook's entrepreneurship ecosystem, hear company pitches from emerging startups, and engage in meaningful networking with innovators, investors and community partners.
Refreshments will be served. Registration is required.
In partnership with Accelerate Long Island.
https://www.stonybrook.edu/commcms/innovation/_events/wolfden.php
As part of a grant project funded by the AI3 Institute, a group of instructors participated in a faculty development program, Fostering Writing-to-Learn Skills with Critical AI Literacy: A Faculty Development and Student Support Program. This program was developed to support instructors across campus with navigating/integrating AI in their courses specifically around writing intensive/involved assignments. We would like to invite anyone interested to the culmination of this program, a mini-symposium, where the participants will share practical changes they made or are making around writing intensive/involved assignments and AI.
Location: Wang 201
A light lunch will be served. Please register by Friday, November 7th.
AI on Campus: Your Thoughts, Your Future
Join the Conversation: Share Your Thoughts about Learning, Academics, and AI
The world of college is changing fast, and Artificial Intelligence (AI) is at the center of it. We are part of the Institute on AI, Pedagogy, and the Curriculum with AAC&U, and we need to hear from the people AI affects most: you!
This is an open discussion for all students to share their honest experiences, their top concerns, and their best ideas about AI in our academic environment. We'll be diving into these key questions:
How can AI actually make learning better or easier? What opportunities do you see for using AI tools to enhance your assignments, research, or skills?
What are your biggest worries about AI? Is it about cheating, being graded fairly, or preparing for the job market? How is AI impacting your workload or stress levels?
What specific tools, workshops, or policies would help you use AI responsibly and successfully? (Think training, software, or clear rules.)
Dates/Times:
Wednesday, 2/4 at 2pm
Thursday, 2/5 at 12pm
Please register in advance for the Zoom link.
Can't Make It? Share Your Feedback!
Don't worry if you can't attend! You can still share your thoughts via video in our AI Zoom Room or via email: rose.tirotta-esposito@stonybrook.edu.
Videos will not be shared publicly and comments will only be shared in aggregate.
Your voice matters. Come tell us how AI is affecting your studies, your stress, and your success!
Dr. Rose Tirotta-Esposito (Assistant Provost; Director of CELT)
Dr. Elizabeth Hewitt (Associate Professor in the Department of Technology and Society (DTS) in the College of Engineering and Applied Sciences)
Chris Kretz (Associate Librarian and Head of Academic Engagement at SBU Libraries)
Prof. Rajiv Lajmi (Assistant Professor in the School of Health Professions and Chair of Applied Health Informatics)
Dr. Matthew Salzano (Assistant Professor in the Department of Communication in the School of Communication and Journalism)
The Provost's Spotlight Talks feature eminent visitors to the university as well as Stony Brook faculty members who have recently been recognized for outstanding contributions in their field.
Transmedia artist Stephanie Dinkins, Kusama endowed chair in art in the College of Arts and Sciences at Stony Brook University, brings her expertise in AI to the next Spotlight Talk with The Stories We Encode: AI, Love and the Future of Algorithmic Care on Tuesday, October 22, at 3:30 pm in the Charles B. Wang Center Theatre.
Working at the intersection of emerging technologies and social collaboration, Dinkins was named a 2023 TIME 100 Most Influential People in AI. She was recognized for her work with Not the Only One, an ongoing project in which she trained an AI on three generations of Black women to give it cultural roots, a deep history, and a perspective that existing systems do not offer.
The event is free and open to the public, and the discussion will be followed by a reception in the Wang Theatre lobby, hosted by the College of Arts and Sciences for new and promoted faculty.
About the Talk
AI's impact on society necessitates addressing longstanding human rights issues and prejudices. To ensure AI benefits humanity, we must confront institutional biases, rethink our relationship with other beings and emerging technologies, and reconcile ideals with actual power structures. This involves recognizing systemic inequalities, redefining human identity, and equitably distributing resources. AI, if developed and used ethically, offers an opportunity to reimagine a more equitable world for all inhabitants.
You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.
Abstract: Two-dimensional (2D) materials such as graphene, hBN, and TMDs offer atomically sharp interfaces and unprecedented tunability when vertically assembled into van der Waals heterostructures. These stacks have enabled discoveries ranging from moiré superconductivity and correlated insulators to quantum emitters and next-generation nanoelectronic devices. Yet constructing high-quality heterostructures remains largely artisanal: researchers manually identify exfoliated flakes, align a polymer stamp by eye, and finely adjust temperature and contact geometry through tacit skill. This manual workflow is difficult to reproduce, scales poorly, and prevents systematic exploration of the enormous combinatorial space of materials, twist angles, and interfacial conditions. AutoLab is an autonomous platform that translates this tacit human expertise into programmable, feedback-driven control. Instead of pressing flakes with predefined trajectories, AutoLab uses machine vision to detect polymer-wafer contact, dynamically regulates contact evolution through closed-loop actuation and temperature control, and captures high-quality flakes with the cleanliness and precision of expert manual fabrication. The system integrates perception, decision making, and motion planning into a single robotic framework, enabling reproducible stacking, wafer-level coverage, and accelerated discovery. Beyond 2D materials, AutoLab illustrates a broader paradigm for AI-native scientific automation: codifying human experimental reasoning into algorithms that interrogate data in real time, adaptively adjust instrumentation, and generate scalable, high-fidelity datasets. Such platforms could generalize to diverse research domains--quantum device fabrication, optical alignment, surface science, autonomous microscopy, and other workflows where expert intuition currently limits throughput and reproducibility. By bridging artisanal manipulation and robotic autonomy, AutoLab points toward a future where scientific discovery is accelerated by machines that not only execute instructions, but learn, respond, and collaborate with human scientists.
Biography: Dr. Yutao Li is a research associate from Department of Condensed Matter Physics and Material Science, Brookhaven National Laboratory. He has 8 years of experience in 2D material sample fabrication, and investigation in their electronic transport, optical and mechanical properties.
Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.
Location: CDS, Bldg. 725, Training Room
Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1604383624?pwd=ffQ5cUPNxTI7nzClKQO6cnsNbhF9Vf.1
Meeting ID: 160 438 3624
Passcode: 558449