Part six of our AI Researcher Profile series invites Assistant Professor Yifan Sun of the Department of Computer Science, to speak of her ongoing collaborative research in optimization and machine learning.
AI Institute: Tell me about your position as an Assistant Professor of the Department of Computer Science.
Professor Sun: I am currently working on several interesting projects in continuous optimization, as well as some collaborations with other CS faculty, in particular in natural language processing. I am also teaching the current Machine Learning class (CSE 512). So far it’s been great working with the students and faculty at Stony Brook!
AI: How did you find yourself at Stony Brook University?
PS: I graduated with my PhD in Electrical Engineering from the University of California, Los Angeles in 2015. While I enjoyed my time there, I didn’t really know if I wanted to do research full-time, and wanted to explore my options a bit, so I went on to work at Technicolor. There, I was working in a fantastic research environment where I was leading several interesting projects, yet simultaneously surrounded by other researchers and PhD interns in data mining, deep learning, HCI, etc. It was great to see how theory work fits into the grand scale - when people working in applications actually use it, and how to ensure that in this kind of collaboration, everybody gets something out of it. I also learned the very important skill of contextualizing the business benefit of my projects to the non-technical big-wigs, which had the side effect of making me constantly question the impact of my efforts. Near the end, however, I decided I wanted to be a full-time theoretician, so I schemed to rejoin academia. After a year-long job search, that searching brought me to sign on with Stony Brook University, where I saw an ideal blend of strong individual research and rich collaborations.
AI: Please tell us about your work as a postdoc in Canada and France.
PS: Before joining the university, I did two postdocs, one in Vancouver, Canada at the University of British Columbia (UBC), and the other in Paris, France, at The French National Institute for Research in Digital Science and Technology (INRIA). In addition to the research experience, the culmination of both experiences gave me a great overview over the different ways that optimization-theory-based labs can be run, which helped inspire my own vision as to how to run a group at SBU--whether it be through reading groups, collaborations, or online seminars. For example, at INRIA, we had an entire floor of students, postdocs, and faculty all working on very related fields of optimization, reinforcement learning, deep learning, computer vision, etc, and I saw how that lead to, if not collaborations, then better contextualization just by always being aware of each others’ research. In general, working in these diverse environments helped me contextualize how my research can fit and enhance the broader optimization community as well, across a global scale. After France, I found my way to Stony Brook.
AI: What has your experience been like as a new faculty member amidst the COVID-era?
PS: The negative side is that everyone to me is a little black box on a computer scene. Somehow the natural flow of research that comes with two people standing together in front of a whiteboard is diminished through a computer. But, I believe we are still lucky to have such great electronic tools at our disposal. Additionally, faculty at SBU have been very supportive, including me in many activities both professional and recreational, so despite starting post quarantine, I very much feel a part of the community. However, to me, the biggest difference is teaching. While for me as a professor, it’s not that different (I create more online content, and spend time trolling Piazza, but otherwise the overhead is limited), it seems that the learning process for the students is significantly hindered when they can’t help each other with the more difficult lectures. I guess across the board, networking has become a little bit harder, and requires much more initiative than just “dropping in an office” like before.
AI: What is one of your more rewarding experiences in regards to your research?
PS: I do not have one experience, but in general I really enjoy the human connection. Whether it is two people working on the same project, and seeing a bunch of wishful thinking turn into a product over months of prodding and revising, or going to a giant conference and connecting with people who are after the same questions as you, or watching people working on entirely different things present their work and recognizing the same sweat and tears in their eyes, to me it’s the communal effort that makes it rewarding.
AI: What machine learning projects are you working on?
PS: My main effort thus far is to work with some NLP faculty in the CS department on improving training of large NLP models. These problems are not neat and ideal as the cute math problems we usually work on in optimization theory, but require a lot of “domain knowledge” to know what will and won’t work--in fact the #1 answer to all my suggestions in the first few months is “we / someone already tried that and here’s the paper” or “and it failed miserably.” My hope is that gaining this “insider information” will help me formulate more realistic and important questions to answer theoretically. I am also working with folks in neuroscience and sociology on some very interesting ideas which I hope to develop, and my hope is to keep expanding these efforts in more areas!
AI: How does continuous optimization apply to machine learning?
PS: In all kinds of training, you usually start by posing your goal as a loss function, which you then want to minimize. The most “popular” of these minimizing schemes is stochastic gradient descent (SGD), which is a “Big-Data friendly” continuous optimization method. Many applications in machine learning rely either on this method, or some variation of it. Therefore, understanding how SGD and its variants work in a wide assortment of applications is important in forming training performance guarantees.
AI: What do you see for the future of your work in machine learning and optimization?
PS: The biggest questions that people care about, when using ML tools, are reliability and interpretability. Everyone, in a sense, is after these two “holy grails” in some way or another. In optimization, we are after these goals by presenting training error bounds -- after so many runs of your method, this is how well we think your model will run (or something like that). Now, the job of getting these training performance guarantees are far from over, but even if we could know them, they don’t tell the perfect story--how a model behaves on data it is trained on can be very different than how it behaves in a real-world situation! This is the problem of generalization, and while being heavily studied, it is still the much harder open problem, and I believe it will remain open for a long time, possibly forever. For now, the goal is to, step by step, chip away at the gap between theory and practice!
AI: Do you have any suggestions for somebody looking to get involved with Artificial Intelligence?
PS: There are so many different opportunities right now. Whether it is taking a class in college, or taking a class in graduate school, or simply just cold emailing people to see if they need research assistants. One thing that students often underestimate is how they can benefit from industry experience. As I spoke to earlier, my time in Technicolor really helped shape my research goals, just by seeing it in the context of other related fields. Right now, all the big companies are hiring -- Google, Microsoft, Amazon, Adobe, Baidu, -- but even if those seem daunting, it’s well worth the effort to look into smaller companies, even startups, who are trying to integrate AI and ML into their products. The other appealing factor in industry is that there is high turnover, which is great for someone who is still learning and unsure about their final goal. You don’t need to pick a job for your entire life, just the next few years. Just keep trying new things and working with new people.
AI: Is there anything else you are working on, or that you would like to comment on, in regards to your research?
PS: I’m just getting started. Right now, my goal is to have a solid foundation with the students I am already supervising. At the same time, I’m trying to establish more and more connections with colleagues within the Institute and beyond. Ask me this question this time next year, I hope to have a lot more to say!
AI: Thank you, Professor Sun.