The Challenges of Machine Learning in Adversarial Settings by Patrick McDaniel, Pennsylvania State University
Abstract: Advances in AI and machine learning have enabled new applications and services to interpret and process inputs in previously unthinkable complex environments. Autonomous cars, data analytics, adaptive communication and self-aware software systems are now revolutionizing markets by achieving or exceeding human performance. In this talk, I consider the evolving use of machine learning in security-sensitive contexts and explore why many systems are vulnerable to nonobvious and potentially dangerous manipulation. Here, we examine sensitivity in any application whose misuse might lead to harm--for instance, forcing adaptive network in an unstable state, crashing an autonomous vehicle or bypassing an adult content filter. I explore the use of machine learning in this area particularly in light of recent discoveries in the creation of adversarial samples and defenses against them and posit on future attacks on machine learning. The talk is concluded with a discussion of the technological and societal challenges we face as a result of current and future advances in intelligent computing.
Bio: Patrick McDaniel is the William L. Weiss Professor of Information and Communications Technology and Director of the Institute for Networking and Security Research in the School of Electrical Engineering and Computer Science at the Pennsylvania State University. Professor McDaniel is also a Fellow of the IEEE and ACM and the director of the NSF Frontier Center for Trustworthy Machine Learning. He also served as the program manager and lead scientist for the Army Research Laboratory's Cyber-Security Collaborative Research Alliance from 2013 to 2018. Patrick's research centrally focuses on a wide range of topics in computer and network security and technical public policy. Prior to joining Penn State in 2004, he was a senior research staff member at AT&T Labs-Research.
Abstract: Advances in AI and machine learning have enabled new applications and services to interpret and process inputs in previously unthinkable complex environments. Autonomous cars, data analytics, adaptive communication and self-aware software systems are now revolutionizing markets by achieving or exceeding human performance. In this talk, I consider the evolving use of machine learning in security-sensitive contexts and explore why many systems are vulnerable to nonobvious and potentially dangerous manipulation. Here, we examine sensitivity in any application whose misuse might lead to harm--for instance, forcing adaptive network in an unstable state, crashing an autonomous vehicle or bypassing an adult content filter. I explore the use of machine learning in this area particularly in light of recent discoveries in the creation of adversarial samples and defenses against them and posit on future attacks on machine learning. The talk is concluded with a discussion of the technological and societal challenges we face as a result of current and future advances in intelligent computing.
Bio: Patrick McDaniel is the William L. Weiss Professor of Information and Communications Technology and Director of the Institute for Networking and Security Research in the School of Electrical Engineering and Computer Science at the Pennsylvania State University. Professor McDaniel is also a Fellow of the IEEE and ACM and the director of the NSF Frontier Center for Trustworthy Machine Learning. He also served as the program manager and lead scientist for the Army Research Laboratory's Cyber-Security Collaborative Research Alliance from 2013 to 2018. Patrick's research centrally focuses on a wide range of topics in computer and network security and technical public policy. Prior to joining Penn State in 2004, he was a senior research staff member at AT&T Labs-Research.
Title: Cultural Biases, World Languages, and User Privacy in Large Language Models
Abstract: In this talk, I will highlight three key aspects of large language models: (1) cultural bias in LLMs and pre-training data, (2) decoding algorithm for low-resource languages, and (3) human-centered design for real-world applications.
The first part focuses on systematically assessing LLMs' favoritism towards Western culture. We take an entity-centric approach to measure the cultural biases among LLMs (e.g., GPT-4, Aya, and mT5) through natural prompts, story generation, sentiment analysis, and named entity tasks. One interesting finding is that a potential cause of cultural biases in LLMs is the extensive use and upsampling of Wikipedia data during the pre-training of almost all LLMs. The second part will introduce a constrained decoding algorithm that can facilitate the generation of high-quality synthetic training data for fine-grained prediction tasks (e.g., named entity recognition, event extraction). This approach outperforms GPT-4 on many non-English languages, particularly low-resource African languages. Lastly, I will showcase an LLM-powered privacy preservation tool designed to safeguard users against the disclosure of personal information. I will share findings from an HCI user study that involves real Reddit users utilizing our tool, which in turn informs our ongoing efforts to improve the design of AI models.
Bio:
Wei Xu is an Associate Professor in the College of Computing and Machine Learning Center at the Georgia Institute of Technology, where she is the director of the NLP X Lab. Her research interests are in natural language processing and machine learning, with a focus on Generative AI, robustness and fairness of large language models, multilingual LLMs, as well as AI for science, education, accessibility, and privacy research. She is a recipient of the NSF CAREER Award, Google Academic Research Award, CrowdFlower AI for Everyone Award, Best Paper Awards and Honorable Mentions at COLING'18, ACL'23, ACL'24. She also received research funds from DARPA and IARPA. She is currently an executive board member of NAACL. Join Zoom Meeting https://stonybrook.zoom.us/j/98855994362?pwd=F2qnpwL85fhCBHAEW9ZBpXihfwGHsj.1 (ID: 98855994362, passcode: 172797) Join by phone (US) +1 646-876-9923 (passcode: 172797) Joining instructions: https://www.google.com/url?q=https://applications.zoom.us/addon/invitation/detail?meetingUuid%3DuDJcUTvyQueZkCaUSAwFlg%253D%253D%26signature%3Da3d49e0f7f2e74e7130f7308c74bd85ba7b99587b98ba2e34238bb657ca51a09%26v%3D1&sa=D&source=calendar&usg=AOvVaw2jTn5cjfRG8vXU8KHHlU2Y Meeting host: H.Andrew.Schwartz@stonybrook.edu
Join Zoom Meeting:
https://stonybrook.zoom.us/j/98855994362?pwd=F2qnpwL85fhCBHAEW9ZBpXihfwGHsj.1
Abstract: In this talk, I will highlight three key aspects of large language models: (1) cultural bias in LLMs and pre-training data, (2) decoding algorithm for low-resource languages, and (3) human-centered design for real-world applications.
The first part focuses on systematically assessing LLMs' favoritism towards Western culture. We take an entity-centric approach to measure the cultural biases among LLMs (e.g., GPT-4, Aya, and mT5) through natural prompts, story generation, sentiment analysis, and named entity tasks. One interesting finding is that a potential cause of cultural biases in LLMs is the extensive use and upsampling of Wikipedia data during the pre-training of almost all LLMs. The second part will introduce a constrained decoding algorithm that can facilitate the generation of high-quality synthetic training data for fine-grained prediction tasks (e.g., named entity recognition, event extraction). This approach outperforms GPT-4 on many non-English languages, particularly low-resource African languages. Lastly, I will showcase an LLM-powered privacy preservation tool designed to safeguard users against the disclosure of personal information. I will share findings from an HCI user study that involves real Reddit users utilizing our tool, which in turn informs our ongoing efforts to improve the design of AI models.
Bio:
Wei Xu is an Associate Professor in the College of Computing and Machine Learning Center at the Georgia Institute of Technology, where she is the director of the NLP X Lab. Her research interests are in natural language processing and machine learning, with a focus on Generative AI, robustness and fairness of large language models, multilingual LLMs, as well as AI for science, education, accessibility, and privacy research. She is a recipient of the NSF CAREER Award, Google Academic Research Award, CrowdFlower AI for Everyone Award, Best Paper Awards and Honorable Mentions at COLING'18, ACL'23, ACL'24. She also received research funds from DARPA and IARPA. She is currently an executive board member of NAACL. Join Zoom Meeting https://stonybrook.zoom.us/j/98855994362?pwd=F2qnpwL85fhCBHAEW9ZBpXihfwGHsj.1 (ID: 98855994362, passcode: 172797) Join by phone (US) +1 646-876-9923 (passcode: 172797) Joining instructions: https://www.google.com/url?q=https://applications.zoom.us/addon/invitation/detail?meetingUuid%3DuDJcUTvyQueZkCaUSAwFlg%253D%253D%26signature%3Da3d49e0f7f2e74e7130f7308c74bd85ba7b99587b98ba2e34238bb657ca51a09%26v%3D1&sa=D&source=calendar&usg=AOvVaw2jTn5cjfRG8vXU8KHHlU2Y Meeting host: H.Andrew.Schwartz@stonybrook.edu
Join Zoom Meeting:
https://stonybrook.zoom.us/j/98855994362?pwd=F2qnpwL85fhCBHAEW9ZBpXihfwGHsj.1
Donghee Yvette Wohn, Ph.D.
Inaugural chair of the Department of Technology, AI, and Society, Stony Brook University
Short bio: Dr. Wohn (she/her) is a Professor at the State University of New York at Stony Brook (Stony Brook University), where she is the inaugural chair of the Department of Technology, AI, and Society and director of the Social Interaction Lab (socialinteractionlab.com). Her research is in the area of Human-Computer Interaction (HCI), where she studies the characteristics and consequences of social interactions in online environments. Funded by the National Science Foundation, Mozilla Foundation, and Yahoo, her main projects examine 1) content moderation, online harassment and the creation/maintenance of online safe spaces, 2) social exchange in digital economies & digital patronage (creator-supporter dynamics), and 3) identity and online self-presentation.
Amin Shirangi, PhD candidate in the Department of Technology, AI and Society (DTAS), moderates the discussion.
Location: NCS 120
Zoom: https://stonybrook.zoom.us/j/92841206016?pwd=lj6NYMgwG1OCawgiVGwIalSf1bGWbb.1&jst=2
Hosted by the College of Business faculty and staff, this virtual session will explore how our MS in Business Analytics & Intelligence program can enable you to become a future leader with AI expertise. This session will also cover the program's benefits, curriculum, cost, and admission requirements.
Register here.
Register here.
You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.
Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.
At our Oct 7 Mixer, BNL's newly minted interim director, John Hill will be present to give opening remarks and kick us off on a new year of impactful scientific AI collaborations.
Abstract: Weather extremes and strong seasonal-to- subseasonal variability pose growing challenges to urban populations, infrastructure, and energy systems. Yet, most cities remain data deserts: routine weather observations are sparse, with stations concentrated at airports rather than within the urban core. This lack of coverage limits our ability to monitor and predict fine-scale urban weather patterns precisely where they matter most. We present a new AI-driven framework for optimal sensor placement and urban weather monitoring. Unlike traditional approaches, our method leverages physics- based simulations together with Bayesian experimental design principles, but does so using a computationally efficient variational inference strategy that makes large-scale optimization tractable. This allows us to guide sensor networks in a way that minimizes information loss while capturing spatiotemporal variability at city scales. Applied to Phoenix, Arizona, our framework outperforms random sensor placement strategies, especially when only a limited number of sensors can be deployed. Importantly, the same AI models that guide sensor placement also function as a real-time nowcasting tool, providing urban weather information over the entire domain, beyond sensor locations. Together, these capabilities offer a scalable pathway to reduce urban data deserts, enhance monitoring of weather extremes, and improve resilience planning for energy, transportation, and public health systems.
Biography: Dr. Katia Lamer is an atmospheric scientist and the Director of the Center for Multiscale Applied Sensing at Brookhaven National Laboratory. Originally from Canada, she earned her B.S. and M.S. in Atmospheric and Oceanic Sciences from McGill University and a Ph.D. in Meteorology from Penn State University. Her research focuses on atmospheric boundary layer processes and remote sensing technologies, with a strong emphasis on data science. At Brookhaven, she is known for her work with the CMAS mobile observatories and its facility that connect fundamental atmospheric science to real-world applications, improving weather prediction, environmental monitoring, and urban climate resilience. Her work has been featured in public outlets such as New Scientist and Wired. Dr. Lamer also serves as an invited member of the World Meteorological Organization's Data Assimilation and Observing Systems Working Group, and the American Meteorological Society's Boundary Layer and Turbulence Committee. puting, communications and sensing, all enabled by AI.
Location: CDS, Bldg. 725, Training Room
Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1604383624?pwd=ffQ5cUPNxTI7nzClKQO6cnsNbhF9Vf.1
Meeting ID: 160 438 3624 | Passcode: 558449
Fall 2026, Wednesdays 2 to 3:20 pm, NCS 220 and Zoom link to be announced soon.
The seminar will be jointly taught by Prof. Dimitris Samaras (samaras@cs.stonybrook.edu).
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.
To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome!
The seminar will be jointly taught by Prof. Dimitris Samaras (samaras@cs.stonybrook.edu).
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.
To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome!
Fall 2026, Wednesdays 2 to 3:20 pm, NCS 220 and Zoom link to be announced soon.
The seminar will be jointly taught by Prof. Dimitris Samaras (samaras@cs.stonybrook.edu).
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.
To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome!
The seminar will be jointly taught by Prof. Dimitris Samaras (samaras@cs.stonybrook.edu).
The overall purpose of this seminar is to bring together people with interests in Computer Vision theory and techniques and to examine current research issues. This course will be appropriate for people who already took a Computer Vision graduate course or already had research experience in Computer Vision.
To enroll in this course, you must either: (1) be in the Ph.D. program or (2) receive permission from the instructors.
Each seminar will consist of multiple short talks (around 15 minutes) by multiple students. Students can register for 1 credit for CSE656. Registered students must attend and present a minimum of 2 talks. Registered students must attend in person. Up to 3 absences will be excused. Everyone else is welcome!
AI3 Seminar
Meir Feder
Professor, School of Electrical Engineering
Jokel Chair in Information Theory, School of Electrical and Computer Engineering
Tel-Aviv University
Information-Theoretic Framework for Understanding Modern Machine-Learning
Abstract:
Information Theory views learning as universal prediction under log loss, characterized through regret bounds. Unlike the classical results that considered ``small'' model classes and provided uniform regret, the proposed framework provides non-uniform, model dependent bounds utilizing an effective notion of architecture-based model complexity. This complexity is defined by the probability mass or volume of the set of all models in the vicinity of the target model \theta_0, in an informational distance. This volume might be hard to evaluate, yet by local analysis it is related to spectral properties of the expected Hessian or the Fisher Information Matrix at \theta_0, leading to tractable approximations. We argue that successful architectures possess a broad complexity range, enabling learning in highly over-parameterized model classes. The framework sheds light on the role of inductive biases, the effectiveness of stochastic gradient descent (SGD) algorithm, and phenomena such as flat minima. It unifies online, batch, supervised, and generative settings, and applies across the stochastic-realizable and agnostic regimes. Moreover, it provides insights into the success of modern machine-learning architectures, such as deep neural networks and transformers, suggesting that their broad complexity range naturally arises from their layered structure. These insights open the door to the design of alternative architectures with potentially comparable or even superior performance.
Biography:
Meir Feder received the Sc.D. degree in Electrical Engineering and Ocean Engineering in 1987 from the Massachusetts Institute of Technology (MIT) and the Woods Hole Oceanographic Institution (WHOI). After being a Research Associate and a Lecturer at MIT, he joined the School of Electrical Engineering, Tel-Aviv University in 1990, where he is the Jokel Chaired Professor and the former founding head of Tel-Aviv university center for Artificial intelligence and Data science (TAD). Parallel to his academic career, he is closely involved with the high-tech industry: he founded 5 companies, among them Peach Networks (Acq: MSFT) and Amimon (Acq:LON.VTC). Recently, with his renewed interest in machine learning and AI, he co-founded Run:ai (Acq:NVDA), a virtualization, orchestration, and acceleration platform for AI infrastructure, acquired by Nvidia to support its GPU cloud operation.
Prof. Feder received several academic and professional awards including the IEEE Information Theory Society best paper award, the Padovani lectureship, the creative thinking award of the Israeli Defense Forces, and the Research Prize of the Israeli Electronic Industry, awarded by the President of Israel. For the technology he developed in Amimon, he received the 2020 Scientific and Engineering Award of the Academy of Motion Picture Arts and Sciences (OSCAR) and was announced the principal inventor of the technology that attained the 73rd Engineering Emmy Award of the Television Academy.
Location: NCS120