Toward Human-Centered XR: Bridging Cognition and Computation
Qi Sun
New York UniversityAbstract
Virtual and Augmented Reality enables unprecedented possibilities for displaying virtual content, sensing physical surroundings, and tracking human behaviors with high fidelity. However, we still haven't created "superhumans" who can outperform what we are in physical reality, nor a "perfect" XR system that delivers infinite battery life or realistic sensation. In this talk, I will discuss some of our recent research on leveraging eye/muscular sensing and learning to model our perception, reaction, and sensation in virtual environments. Based on the knowledge, we create just-in-time visual content that jointly optimizes human (such as reaction speed to events) and system performance (such as reduced display power consumption) in XR.
About the speaker
Qi Sun is an assistant professor at New York University. Before joining NYU, he was a research scientist at Adobe Research. He received his PhD at Stony Brook University. His research interests lie in perceptual computer graphics, VR/AR, computational cognition, and computational displays. He is a recipient of the IEEE Virtual Reality Best Dissertation Award, with his research recognized as Best Paper Awards in ACM SIGGRAPH and IEEE ISMAR.
Learning with Real and Unreal Data in the Era of Foundation Models
Rogerio Feris
IBM ResearchAbstract
Pre-training large language and vision models on massive-scale datasets has led to major advances in AI, albeit accompanied with shortcomings related to privacy, usage rights, data protection, bias, and ethical issues. A promising direction to address these challenges is to train models using synthetic data. In this talk, I will first discuss our work on optimizing synthetic data generation to maximize transfer learning performance for specific vision tasks. Then, in the same vein, I will cover our recent work on self-specialization of large language models to specific domains based on synthetic data. Finally, I will show an application of these techniques for training a multimodal foundation model to automatically produce commentary for video highlights of the 2023 Wimbledon and US Open tennis tournaments.
About the speaker
Rogerio Feris is principal scientist and manager at the MIT-IBM Watson AI lab. He is broadly interested in teaching machines to see, listen, and read, with little or no supervision, like humans do. In particular, his research work has been centered on data efficiency (learning more from less), model efficiency, and multimodal perception methods that combine vision, sound/speech, and language. He is also passionate about doing fundamental research as well as developing systems that make a real-world impact. His work has not only been published in top AI conferences, but has also been integrated into multiple products, and covered by media outlets such as the New York Times, ABC News, and CBS 60 minutes.
The role of the US Office of Naval Research Global in the promotion of basic science
Kyle Gustafson
ONR GlobalAbstract
Since 1946, the Office of Naval Research Global has provided funding and research collaboration opportunities with partners around the world, with a mission of promoting global security, building awareness of emerging technology, and the promotion of peace and democratic values. Today, the ONR Global Science Director office in São Paulo nears its 10-year anniversary in Brazil. Alongside the Air Force Office of Scientific Research and the US Army International Technology Center, we give grants for basic research projects, conferences and workshops (such as SIBGRAPI), and visiting scientists to the US. ONR headquarters in Washington DC provides $2 billion of scientific research funding to academia and national labs in the US each year.
About the speaker
Dr. Kyle Gustafson is a Navy physicist serving as a Science Director (International Program Officer) at the Office of Naval Research Global, based in Sao Paulo since October 2021. He is responsible for technology awareness and basic research funding for future Naval Applications, scouting projects and conferences to fund mainly in Brazil, Argentina, and Uruguay. Dr. Gustafson has studied and worked in computational plasma physics, systems biology, circadian genomics, nonlinear dynamics, infectious disease modeling, underwater and structural acoustics, Digital Twin, and time series analysis for maritime applications. He has published nearly 30 peer-reviewed papers and presented at dozens of research conferences with a variety of colleagues in data-driven cyber-physical machine learning models, high-performance computing, genomic data analysis, computational biology, and numerical methods in plasma physics.
Perspectives on Quantum Machine Learning
João Paulo Papa
UNESPAbstract
Quantum computing emerged as a promising approach to cope with NP-hard problems. However, as the number of qubits increases, their fault tolerance decreases exponentially, characterizing the so-called Noisy Intermediate-Scale Quantum (NISQ) era. Moreover, the number of qubits available in quantum computers is far from the ideal. This talk reviews fundamental concepts about quantum machine learning and two applications: question-answering in natural language processing and pattern classification using the Optimum-Path Forest.
About the speaker
João Paulo Papa holds a Bachelor's degree in Information Systems from UNESP (2002), a Master's degree in Computer Science from UFSCAR (2005), a PhD in Computer Science from UNICAMP (2008), and has postdoc in Computer Science from both State University of Campinas (2009) and Harvard University (2015). Currently, he is Associate Professor in the Department of Computing at Universidade Estadual Paulista Júlio de Mesquita Filho (UNESP), and a collaborating researcher at the Department of Computing at the Federal University of São Carlos. He is currently a senior editor of the journal IEEE Signal Processing Letters, as well as a member of the editorial board of the journals Computer Methods in Biomechanics and Biomedical Engineering, Journal of Next Generation Information Technology, and International Journal of Bio-Inspired Computation. He is co-editor of a book in the area of meta-heuristic optimization, as well as has collaborations with several international institutions. He has been a guest professor at the Ostbayerische Technische Hochschule (Germany), Coordinator of the Postgraduate Program in Computer Science at UNESP, as well as a Fellow of the Alexander von Humboldt Foundation supported by CAPES, Brazilian representative at the International Association for Pattern Recognition (ICPR) and senior IEEE member. He was a full member of the Special Commission for Computer Graphics and Image Processing (CEGRAPI) in the period 2015-2018. He has experience in the area of Computer Science, with an emphasis on Computing Systems, working mainly on the following topics: image processing, machine learning, and pattern recognition. He also participates in projects in the area of Petroleum Engineering with the Center for Geosciences Applied to Petroleum (UNESPETRO), UNESP at Rio Claro. He is currently chair of the IEEE Task Force on Business Intelligence and Knowledge Management.
