Nov. 12 - 15 2024: MISTI MIT Media Lab x Smart Living Lab Workshop - Part 2

Media Lab Introduction

Prof. Joe Paradiso will give an introduction to the MIT Media Lab.

Joe Paradiso is the Alexander W. Dreyfoos (1954) Professor at MIT's Program in Media Arts and Sciences. He directs the MIT Media Lab's Responsive Environments Group, which explores how sensor networks augment and mediate human experience, interaction and perception. He received a B.S. in electrical engineering and physics summa cum laude from Tufts University, and a Ph.D. in physics from MIT with Prof. Ulrich Becker in the Nobel Prize-winning group headed by Prof. Samuel C.C. Ting at the MIT Laboratory for Nuclear Science.

Paradiso's research focuses include ubiquitous computing, embedded systems, sensor networks, wearable and body area networks, energy harvesting and power management for embedded sensors, and interactive media. He also designed and built one of the world's largest modular synthesizers, and has designed MIDI systems for the musicians Pat Metheny and Lyle Mays. The synthesizer currently streams live-generated audio over the internet.

Website: http://www.media.mit.edu/~joep/

Human-Building Interaction and Augmented Intelligence

This talk will begin with a brief discussion of the research questions that have shaped Denis Lalanne's career. The Human-IST Institute will then be introduced, including the context of its founding, its multidisciplinary approach to developing human-centered interactive technologies, and its ongoing contributions to research and teaching. Examples of past and current projects in multimodal interaction and human-building interaction within the Smart Living Lab will be explored, with a particular focus on technologies designed to enhance occupants' interactions and comfort within the built environment. Finally, an overview of the evolution of SIGCHI in Switzerland over the past four decades will be provided, encouraging reflection on the future of the Swiss HCI landscape, with a particular focus on augmented intelligence.

Denis Lalanne is a full professor in the Department of Informatics at the University of Fribourg, where he serves as director of the Human-IST Institute, scientific head of the Swiss Center for Augmented Intelligence, lead of the 'Human-Building Interaction' group within the Smart Living Lab, and Swiss representative at IFIP TC13. He completed a PhD in Interactive Intelligence at the Swiss Federal Institute of Technology (EPFL), followed by a postdoctoral fellowship in the USER group at IBM Almaden Research Center, a year as lecturer at the University of Avignon, and a valuable role as a usability engineer in a Swiss start-up. He then joined the University of Fribourg, where he established the Human-IST Institute. His research focuses on enhancing human-machine interactions through multimodal interaction, information visualization, accessibility, human-building interaction, and human-AI collaboration.

Website: https://diuf.unifr.ch/people/lalanned/

Sensing Across Species

This talk presents innovative technological solutions for environmental and ecological monitoring aimed at enhancing human health, wildlife conservation, and biodiversity protection. It focuses on three projects: AirSpecs, which integrates environmental sensors into eyewear to assess human comfort in changing environments; BuzzCam, which uses acoustic sensors to monitor endangered bumblebee species; and CollarID, which tracks wildlife behavior and environmental conditions to aid conservation efforts. These projects demonstrate how advanced sensing technologies can provide critical data for conservation and biodiversity protection.

Patrick Chwalek is a PhD candidate in the Responsive Environments Group, where he develops and deploys sensors for real-world applications. He explores how environments influence people’s physical and mental states, and how comfort varies with engagement. He also creates devices to study wildlife in their natural habitats, such as Arctic birds, Patagonian bees, and Botswanan African Wild Dogs.

Website: https://patrickchwalek.com/

Data-Driven Insights on Walkability: Modeling mobility patterns, urban forms and social implications to inform interventions toward 15-min city

How can we evaluate what drives walkability and walking behavior in a data-driven way? By leveraging data on pedestrian counts, mobility patterns, and social cohesion from surveys, spatial demographics, and urban form features, we are developing a model to explore empirical understanding among these variables. The goal is then to identify interventions that promote sustainable mobility and align with the 15min city concept.

Kanaha Shoji enjoys traveling, exploring new art and architecture, and learning the differences in various cities. That curiosity brought her to ETHOS at Smart Living Lab, where she explores the relationships between humans and the built environment. Her primary focus is walkability in cities.

Website: https://www.linkedin.com/in/shojik

Enhancing Client Engagement and Therapist Feedback in Low-Vision Training through Collaborative Augmented Reality

Blind and low-vision (BLV) individuals are often encouraged to undergo low-vision training to adjust to vision loss and develop adaptive strategies using their residual vision. However, these sessions can become repetitive and unengaging for clients, reducing the effectiveness of the training. Additionally, BLV specialists may face difficulties in understanding their clients' visual needs and search strategies, often relying on task outcomes to provide feedback. In my thesis, I propose a collaborative augmented reality environment, designed in collaboration with occupational therapists, to enhance client engagement during low-vision training by adapting tasks to their needs and enabling clients to train within familiar surroundings. Moreover, by offering therapists real-time visual cues of clients' visual search strategies, the system aims to facilitate more effective and tailored feedback.

Yong-Joon Thoo is a PhD Candidate at the Human-IST Institute of the University of Fribourg. His research aims to improve the quality of life of blind and low-vision people by providing them with an collaborative augmented reality based platform to train their residual vision and develop adaptive strategies in an engaging manner.

Website: https://yjthoo.human-ist.ch/

Towards Personalized AI for Health

I will be presenting my published work PhysioLLM: Supporting Personalized Health Insights with Wearables and Large Language Models and some ongoing efforts on leveraging multimodal data and LLM for physical and mental health.

Cathy M. Fang is a PhD Student at MIT Media Lab's Fluid Interfaces Group. Her research interests are multi-modal input and interactions for mixed reality, spatial and context-aware computing, wearable interoceptive intervention, computational cognition and perception. Previously, she has worked at Microsoft, Apple, IDEO, and Magic Leap. She holds a bachelor's degree with honors from Carnegie Mellon University in Mechanical Engineering and Human-Computer Interaction.

Website: https://cathy-fang.com/about.html

An Ecological Approach to Perceptual Augmentation

This talk explores the intersection of perceptual augmentation and implicit communication in AI systems, proposing novel approaches to enhance human-AI interaction and monitoring. Drawing inspiration from the diverse sensory abilities found in nature and the implicit signals provided by mechanical systems, we challenge traditional approaches to sensory enhancement and AI system monitoring. Our research adopts an ecological approach grounded in Gibsonian psychology, emphasizing the importance of perception over mere sensory input and the development of meaningful augmentation methods that can impact daily life.

Sam Chin is a graduate student in the Responsive Environments Group at the MIT Media Lab. Her research explores how technology can be used to expand the boundaries of human perception and improve the way we experience the world around us. On one hand, she builds hardware and interfaces for sensory augmentation. On the other, she examines the neuroscience of how we learn and integrate senses.

Website: https://www.samch.in/

Statistical data analysis in human-centric buildings research

In a broader sense, my research seeks to bridge the gap between current practices in statistical analysis and building research to enable better inferences of complex human-building interactions. I am particularly interested in how synthetic data simulation can enhance our understanding of a model’s behaviour, thereby fostering a deeper insight into statistical inference techniques. In this talk, I will discuss the prevailing challenges in statistical analysis within human-centric buildings research and explore how synthetic data simulation may offer potential solutions to these issues.

Matteo Favero is a Postdoctoral Researcher at the École Polytechnique Fédérale de Lausanne (EPFL), Switzerland. He is a member of the ETHOS Lab: Engineering and Technology for Human-Oriented Sustainability. The ETHOS Lab is an interdisciplinary research group focused on using data-driven and computational tools to create interventions in the built environment that advance our social and environmental sustainability goals. He completed his PhD at the Norwegian University of Science and Technology (NTNU) in October 2022, Civil and Environmental Engineering Department. Before enrolling in the PhD program, Matteo received an MSc and BSc in Building Engineering from Politecnico di Milano, Italy.

Matteo’s research interests and expertise include (dynamic) thermal comfort, occupant-centric models for thermal comfort in buildings and statistical thinking.

Website: https://people.epfl.ch/matteo.favero

Accessibility Remediation of Digital Educational Documents

Providing access to education for all requires accessible digital documents. However, accessibility principles have yet to be integrated into the practices of publishers and teachers who create such documents. Moreover, the remediation of educational documents remains time-consuming when it is done by specialists (e.g., accessibility centers). As a solution, this Ph.D. research focuses on the development of a computer system that relies on artificial intelligence to provide accessible documents to low-vision (LV) learners. Findings highlight the rich ecosystem of actors providing (e.g., specialists, teachers) and using educational documents, the system requirements specified by LV learners and accessibility remediation specialists as well as key considerations about the accessibility and usability of such systems.

Maximiliano Jeanneret Medina is a PhD candidate in the Human-IST Institute of the University of Fribourg. His research focuses on the design, development, and evaluation of interactive computer systems using approaches and methods grounded in the disciplines of Human-Computer Interaction and Information Systems. His current research looks at developing and evaluating accessible and usable interfaces for people with diverse visual abilities.

Website: https://human-ist.unifr.ch/en/institute/team/maximiliano-jeanneret-medina.html

Detecting individual behavior changes and their impacts in terms of energy consumption in buildings

Buildings are a great contributor in global greenhouse gas emission, often exceeding their expected energy consumption in their occupancy phase. It is acknowledged that occupant behavior plays a critical role in this excess consumption, with the energy consumption gap often attributed to the oversimplification of human behavior in simulation models. Based on the analysis of heterogeneous data (surveys, mobility data, infrastructure data,...) this thesis will focus on better understanding individual behavior changes, their multiple dimensions and their magnitude.

Marion Schoenenweid is a PhD candidate in the Human-IST research group at the University of Fribourg and the Energy Institute at HEIA Fribourg. Sensitive to ecological issues, she is particularly interested in the interactions between human behavior and buildings. She believes in the positive impact of placing humans at the center to help solve technical challenges.

Website:https://human-ist.unifr.ch/en/institute/team/marion-schoenenweid.html

Roadmap for Human-Centric Sensing and Environmental Control to Enhance Occupant Comfort and Health

Shifting to a paradigm of personalized sensing and control is expected to surpass the limitations of traditional room-level systems, further enhancing occupant comfort, health, and energy-saving potential. However, significant challenges remain in the areas of human-centric sensing, preference learning, and environmental control. This talk will introduce a roadmap addressing these three key aspects.

Tianqi Liu is a PhD candidate in the Human-Oriented Built Environment Lab (HOBEL) at EPFL. His research focuses on human-centric sensing and environmental control, with an emphasis on thermal comfort and indoor air quality in office environments.

Website: https://people.epfl.ch/tianqi.liu?lang=en

How Can a Single Human Embody a Multi-Agent System?

Pablo Palle is a graduate student at EPFL, currently pursuing his Master's thesis with the MIT Media Lab's Responsive Environments group. After working on the Birdly project at the Laboratory of Intelligent Systems (LIS) at EPFL, he developed a strong interest in swarm systems.

Website: