Face Editing with Machine Learning presented by Zhixin Shu

ABSTRACT: The face is the most informative feature of humans and has been a long-standing research topic in Computer Vision and Graphics. Images of faces are also ubiquitous in photography and social media, and people have devoted significant resources to capturing and editing face images. Face editing can be broadly viewed as the encoding, manipulation and the decoding of some representations for face images. The challenges are that we want to manipulate an image in a controllable way and generate results that are both desirable and as realistic as possible. This thesis explores different Machine Learning-based face-editing approaches. I discuss the role of machine learning for achieving desirable edits by learning both the physical aspects as well as the statistical manifold of human faces. In my work for eye-editing, I discuss the importance of understanding multiple physical elements of a face image, such as shape, illumination, pose, etc. In a deep-learning-based approach, I introduce image formation domain knowledge to the construction and training of a neural network. This network provides transparent access to the disentangled representations of the aforementioned physical properties. With this network, we can achieve various face editing tasks in forms of representation manipulation. After that, I introduce Deforming Autoencoders, a network that learns to disentangle shape and appearance in an unsupervised manner. This disentanglement is beneficial for the learning of some other factors of variations, such as illumination and facial expression. In an extension of Deforming Autoencoders, we incorporate non-rigid structure-from-motion to learn a 3D morphable model for faces that only requires an image set for training. At last, I describe an image-to-image network for 3D face reconstruction, which also utilizes structure-from-motion in deep learning. With real face images in training, this network not only reconstructs 3D faces more accurately than prior art but also has better generalization ability in real-life testing cases.


New York Scientific Data Summit (NYSDS) is a premier annual conference that brings together researchers and thought leaders from academia, national labs and industry to exchange ideas and foster collaboration focused on data-driven science and technology. Co-hosted by Brookhaven National Laboratory and the Institute for Advanced Computational Science (IACS) at Stony Brook University, NYSDS 2025 will take place on September 11-12, 2025, in the SUNY Global Center in New York City.

NYSDS 2025 will spotlight artificial intelligence (AI), machine learning (ML) and robotics - fields currently at a pivotal point with transformative impacts on science and technology. From accelerating computationally demanding simulations to discerning signals from noisy data, AI/ML has become an integral part of the scientific workflows. Despite many advances, challenges remain to ensure that AI/ML applications are reliable, explainable and trustworthy.

Robotics, a growing field that couples AI with physically actuated mechanical bodies, has seen increased interest in areas spanning science, technology and manufacturing. The need for real-time decision-making and control, along with the intricate morphology of robots, makes robotics an intriguing application of AI, advanced computing and optimization.


This NYSDS 2025 is open to the public. To be eligible to attend, all participants must register online by August 30, 2025. For questions or assistance with registering, please contact the Summit Coordinator.

Register here.

Artificial Intelligence is rapidly reshaping research, education, and industry--but its growth carries important environmental implications. From the energy demands of large-scale computing to AI's potential to advance climate modeling, conservation, and sustainable design, the relationship between AI and the environment is both challenging and promising. This interdisciplinary panel explores AI's ecological footprint, its role in environmental solutions, and how universities can pursue innovation while upholding sustainability commitments.

Panelists:
Dana Golden -- PhD student in Economics, Stony Brook University.
Dr. Sharon Pochron -- Associate Professor in Sustainability Studies Program, School of Marine and Atmospheric Sciences, Stony Brook University.
Dr. Jordanna Sprayberry -- Associate Professor, Ecology & Evolution, Director of Undergraduate Biology, Stony Brook University.
Dr. Lav Varshney -- Director of the Artificial Intelligence Innovation Institute (AI3) and inaugural Della Pietra Infinity Chair, Stony Brook University.

Register here.
The talk will be exclusively on zoom https://stonybrook.zoom.us/j/7851507944 Speaker: Sooyeon Lee, Rochester Institute of Technology Title: Design and Evaluation of Accessible AI Technologies for Users with Disabilities Abstract: Over one billion people in the world live with some type of disability. Many of them experience barriers in accessing information or using technologies, which can limit social interactions in both physical and digital spaces. In my research, I focus on investigating and designing nonvisual interaction for the community of blind users and non-audio and non-speech interaction for the community of deaf and hard of hearing users. In this talk, I will first present my research investigating nonvisual interaction prototypes for supporting shopping activities for blind users, with an exploration of one-way instructional and two-way conversational interactions and with a variety of form factors and communication modalities through the use of human-computer interaction research methodologies. I will also discuss incorporation of AI technology and its impact on the nonvisual guidance experiences, and further meanings of independence and new ways for designing independence for people with visual impairments. This collaborative work included AI researchers, the community of the blind, and an industry research partner. Additionally, I will discuss my findings and further exciting research opportunities. Secondly, I will overview research projects investigating AI-based applications and tools that support deaf and hard of hearing people's equitable information access and societal participation. This work addresses engagement in online social media spaces, workplace communication, participation in gig work, and interaction with mainstream technology through American Sign Language (ASL) interaction. I will focus on a recent project on users' experiences with AI deep-fake face-transformation technologies to support anonymous participation of deaf and hard of hearing signers in online social media. Lastly, I will discuss my future research directions informed and inspired by this prior and current research. Bio: Sooyeon Lee is a postdoctoral research associate in the Golisano College of Computing and Information Sciences at Rochester Institute of Technology. She received her Ph.D., advised by Dr. John M. Carroll, in Information Sciences and Technology from the College of Information Sciences and Technology at The Pennsylvania State University, and she also conducted design research at Google and Uber. Her research is in the fields of Human-Computer Interaction and Human-AI Interaction with focus on accessibility. She designs, builds, and evaluates new systems and applications that address accessibility barriers. Her work investigates the diversity of users, explores and leverages emerging technologies, and adopts human-centered design and inclusive design approaches in an interdisciplinary research framework. She has multiple publications in top-tier human-computer interaction and computing accessibility journals and conferences, including ACM CHI, CSCW, ASSETS, and TACCESS, and she has received a Best Paper Award Nomination at ASSETS 2021. She has served on Associate Chair for the ACM CHI conference and will serve on Program Committee for ASSETS 2022.

You are cordially invited to attend the biweekly Brookhaven AI Mixer (BAM). BAM includes one short talk on AI research happening at BNL, followed by an open mixer over coffee and snacks for everyone to network and discuss all things AI. The first half hour will consist of presentations that will be available via ZOOM, and the second half hour will be for in person only networking.

Join us every other Tuesday at noon in CDSD's Training Room (building 725, 2nd floor) to learn about interesting AI methods and applications, engage with potential collaborators, prepare for pending FASST funding calls, and build a community of AI for Science at BNL.

AI for Neutrino Oscillation Fits

Abstract: Neutrino oscillation experiments face the problem of performing likelihood fits in a very highdimensional space to extract the oscillation parameters from measured spectra. The current strategy for this is to fix all but a few parameters, reducing the dimensionality of the fit to a manageable number, but this risks missing correlations between the parameters, which can impact the systematics of the measurement. This is an area where artificial intelligence and machine learning could make great improvements. I will discuss the problem, explain how it is currently dealt with, and sketch one possible way of implementing AI to solve it, using a sampling method combining Smolyak's algorithm, for efficient sampling using sparse grids, with an adaptive grid refinement to increase sampling in regions that are more likely to contain the global minimum.

Speaker: Steven Linden is a physicist in the Instrumentation Department at BNL working on neutrino and dark matter experiments. He got his PhD from Yale in 2010 doing analysis on the MiniBooNE experiment and then worked on various dark matter detectors (MiniCLEAN, Pico, SENSEI) at SNOLAB in Canada for nearly ten years before moving to BNL.

Location: CDS, Bldg. 725, Training Room

Join ZoomGov Meeting: https://bnl.zoomgov.com/j/1614473319?pwd=e4QSSgFHqDzHx870ixJpwuG3yqBere.1

Meeting ID: 161 447 3319
Passcode: 733283

Launching a University-Wide AI Innovation Institute:

Last spring, the Office of the Provost led a group of over 30 faculty, staff, and administrators to consider how we can expand and leverage our strengths in AI research and discovery. The resulting recommendation was to launch a university-wide AI Innovation Institute (AI3), which would expand the Institute for AI-driven Discovery and Innovation established in 2018 from a department-level institute within the College of Engineering and Applied Science (CEAS) to the university-wide AI Innovation Institute reporting to the provost.

As a university-wide enterprise, the AI Innovation Institute (AI3) is intended to accelerate, coordinate, and organize AI innovation and education across Stony Brook. The institute will serve to empower the entire university community and beyond, catalyzing core AI research, curriculum innovation, and societal change in the ever-evolving landscape of knowledge work.

The AI Town Hall, led by AI3 Interim Director Skiena, is an open house event that will provide an overview of the major AI initiatives on campus, including the new AI Seed Grant program and Stony Brook's role in New York State's Empire AI program. The session will include time for questions and discussion about the future of AI at Stony Brook.

AI3, SBU Libraries and IACS present
at International Love Data Week
sponsored by The Office of the Provost and
Educational and Institutional Effectiveness (EIE)

Special Talk and Panel Discussion

How I Learned to Stop Worrying and Love AI (For Now)

with Paul Fain from The Job and Work Shift

A reporter's take on what we know--and what we don't know--about AI's emerging impacts on the labor market. The discussion will include the latest research from economists and the AI labs themselves about how workers are using AI, and current thinking among experts on how the tech's rapid deployment will play out across job roles, industries, and regions.

Panel discussion to follow with:

  • Lav Varshney, Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute
  • Nicholas Johnson, Director of AI, SBU Libraries
  • Marianna Savoca, Associate Vice President for Career Readiness and Experiential Education
Paul Fain is co-founder of Work Shift, editor of the must-read newsletter, The Job, and host of The Cusp podcast. A veteran higher education reporter, Paul is perhaps the nation's top journalist focused on connections between education and work. He started Work Shift after a decade as a senior reporter and then news editor at Inside Higher Ed, where he led the outlet's coverage of low-income and first-generation students, college completion, community colleges, federal policy, and emerging models of higher education. He also was the founding host of the successful podcast, The Key with Inside Higher Ed, and has contributed chapters for books on innovation in higher education, published by the Harvard University Press and the Stanford University Press. Earlier in his career, Paul was a senior reporter at The Chronicle of Higher Education.

Limited Seats!

Registration is required.
The Center of Excellence in Wireless and Information Technology (CEWIT) will host the 16th International Conference on Emerging Technologies for a Smarter World (CEWIT2020) virtually on November 5, 2020. The conference will center on the four major fields which are penetrating our business and personal lives: Machine Learning, Artificial Intelligence, Blockchain and Computational Medicine. For more info visit: https://www.cewit.org/.