Wednesday, January 21, 2015

Title: "Reputation Inflation"
Speaker: John Horton (Stern School of Business, New York University)
Date: Tuesday, February 3rd
Time: 12-1pm
Room: NSH 1507

Abstract:
We document reputation "inflation"---a strong, positive trend in average feedback levels---in a large online labor market. Exploiting the matched employer-employee nature of the data, we rule out the possibility that inflation is driven solely by market compositional effects. We argue that this inflation is caused by two factors: (1) negative feedback is more costly to give than positive feedback, and (2) the cost to the rating employer from giving bad feedback is increasing in the market penalty of bad feedback to the rated worker. In an adverse selection model with these assumptions, we show that there is a stable equilibrium in which all rating employers give good feedback regardless of worker performance. We describe a marketplace intervention designed to address reputation inflation: employers were asked to give private feedback about workers in addition to the status quo public feedback. This private feedback differed substantially from the public feedback and was more predictive of future worker performance than public feedback. After this private feedback was collected for a year, aggregated measures of this feedback were experimentally revealed to new employers who were screening applicants. We find that when buyers can condition their screening and hiring choices on this aggregated private feedback, they do so, suggesting they view it as a valuable source of information about workers.

Bio:
John Horton joined New York University Stern School of Business in September 2013 as an Assistant Professor of Information, Operations and Management Sciences. Professor Horton's research focuses on the intersection of labor economics, market design and information systems. He is particularly interested in improving the efficiency and equity of matching markets. After completing his Ph.D. and prior to joining Stern, Professor Horton served for two years as the staff economist for oDesk, an online labor market. Professor Horton received a B.S. in Mathematics from the United States Military Academy at West Point and a Ph.D. in Public Policy from Harvard University.
































Monday, December 1, 2014

Title: Centering the Humans in Human Computation
Speaker: Lilly Irani (Communication & Science Studies, University of California, San Diego)
Date: Tuesday, December 2
Time: 12-1pm
Room: NSH 3305

Abstract:
As HCI researchers explore the possibilities of human computation, their focus has been on the humans enabled by HComp rather than the human workers who power it. Based on five years of design and ethnographic engagement, this talk analyzes the labor inequalities designed into Amazon Mechanical Turk, one popular human computation system. With Six Silberman (UC Irvine) and thousands of Turk workers, I have designed and maintained a system called Turkopticon that both addresses and draws attention to those inequalities. Turkopticon is an activist system that allows workers to publicize and evaluate their relationships with employers. As a common infrastructure, Turkopticon also enables workers to engage in mutual aid. I'll discuss lessons learned from designing and running the system, offer implications for how activists approach design interventions, and conclude with other projects that attempt to center humans in human computation.

Bio:
Lilly Irani is an Assistant Professor of Communication & Science Studies at University of California, San Diego. Her work examines and intervenes in high-technology work practices in situ to understand their relationships with broader cultural, political, and social processes. Her work has appeared at CSCW, CHI, New Media & Society, Science, Technology & Human Values and other venues. Her work has also been covered in The Nation, The Huffington Post, and NPR. Previously, she spent four years as a User Experience Designer at Google. She has a B.S. and M.S. in Computer Science, both from Stanford University and a Ph.D. in Informatics from UC Irvine.































Thursday, November 6, 2014

Title: Emergent Crowdwork during Disaster Events
Speaker: Kate Starbird (Department of Human Centered Design & Engineering, University of Washington)
Date: Tuesday, November 11
Time: 12-1pm
Room: NSH 3305

Abstract:
Crisis events in the physical world are now precipitating mass convergence events online, where thousands and in some cases millions of people turn to social media to seek and share information. This activity includes a new form of spontaneous volunteerism—digital volunteerism—where individuals and organizations come together in online spaces to provide assistance, both to those affected and to emergency responders. Often this takes the form of informational assistance, as volunteers help to process, filter, categorize, map and route information. My research has focused on ways in which remote volunteers contribute to these efforts. In this talk, I will cover some of this previous research, and discuss as well more recent studies examining how members of affected communities, including emergency responders and volunteers, come together with remote volunteers to participate in "emergent crowdwork" after disaster events.

Bio:
Kate Starbird is an Assistant Professor at the Department of Human Centered Design & Engineering (HCDE) at the University of Washington. Dr. Starbird examines online interaction and collaboration in the context of crisis events, specifically looking at how people appropriate social media to help inform others and coordinate response efforts during natural disasters and other large-scale crises. One focal area of her research is digital volunteerism, where remote individuals attempt to provide assistance to those affected. In a related project, she is investigating the flow of misinformation with particular focus on the "work" of the crowd to challenge and correct false rumors. Dr. Starbird's research, which incorporates theory and methods from social science and computer science, is situated within the fields of human-computer interaction (HCI) and computer supported cooperative work (CSCW) as well as the emerging research areas of crowdsourcing and crisis informatics.
































Thursday, October 23, 2014

Title: LabintheWild: Conducting Large-Scale, Uncompensated Online Experiments Around the World
Speaker: Katharina Reinecke (School of Information, University of Michigan)
Date: Tuesday, October 28
Time: 12-1pm
Room: NSH 3305

Abstract:
HAn estimated 95% of research findings in psychology, behavioral economics, and related fields are based on studies with student samples from Western and industrialized countries. As a consequence, some of the knowledge derived from these usually small and locally-recruited student sample populations has been found to be non-generalizable — a finding that led researchers to call these participant samples ``WEIRD'', an acronym for Western, Educated, Industrialized, Rich, and Democratic (Henrich et al. 2010). In this talk, I will show how web-based experimentation with uncompensated samples has the potential to support the generation, replication, verification, and extension of results with larger and more diverse sample populations than previously seen. I will introduce the experimental online platform LabintheWild, which provides participants with personalized feedback in exchange for participation in behavioral studies. In comparison to conventional in-lab studies, LabintheWild enables the recruitment of participants at larger scale and from more diverse demographic and geographic backgrounds: In the last two years, LabintheWild was visited more than 2.5 million times, and nearly 850,000 visitors from more than 200 countries completed an experiment (an average of about 1,000 participants/day). I will discuss how participants hear about the platform, and why they choose to participate. Presenting the results of several experiments, I will additionally show that LabintheWild is a feasible tool to make science less WEIRD.

Bio:
Katharina Reinecke is an Assistant Professor at the University of Michigan School of Information. She received her Ph.D. in Computer Science from the University of Zurich, Switzerland, in 2010, and spent her postdoctoral years at Harvard School of Engineering and Applied Sciences. Her research focuses on cultural differences in users’ interaction with technology with the goal to create culturally intelligent user interfaces that automatically adapt to people’s preferences and perception. To achieve large-scale comparisons of various cultures, she employs datasets from companies such as Doodle and from her own experimental website LabintheWild.org.
































Wednesday, October 15, 2014

Title: Machine Learning for Personalized Clustering
Speaker: Yisong Yue (Computing and Mathematical Sciences, California Institute of Technology)
Date: Tuesday, October 21
Time: 12-1pm
Room: NSH 3305

Abstract:
How would you browse and organize attractions for a potential trip to Paris? How would you organize research articles while conducting a literature review? Such sensemaking tasks are often facilitated via rich user interfaces that enable users to interact more meaningfully with the data. In this talk, I will focus on the problem of learning personalized user models from rich user interactions, and in particular from clustering interactions (i.e., grouping recommended items into clusters). Clustering interactions enable users to express similarity or redundancy between different items, and are a natural way for users to organize and group information. I will describe a new machine learning setting, called collaborative clustering. Analogous to collaborative filtering, in collaborative clustering the goal is to leverage how existing users cluster or group items in order to predict similarity models for other users’ clustering tasks. I will also describe some (very) preliminary work on modeling the full closed-loop interactive clustering setting.

Bio:
Yisong Yue is an assistant professor in the Computing and Mathematical Sciences Department at the California Institute of Technology. He was previously a research scientist at Disney Research. Before that, he was a postdoctoral researcher in the Machine Learning Department and the iLab at Carnegie Mellon University. He received a Ph.D. from Cornell University and a B.S. from the University of Illinois at Urbana-Champaign. Yisong's research interests lie primarily in the theory and application of statistical machine learning. He is particularly interested in developing novel methods for spatiotemporal reasoning, structured prediction, interactive learning systems, and learning with humans in the loop. In the past, his research has been applied to information retrieval, recommender systems, text classification, learning from rich user interfaces, analyzing implicit human feedback, data-driven animation, sports analytics, policy learning in robotics, and adaptive routing & allocation problems.


































Thursday, September 25, 2014

Title: Crowd-Powered Interactive Systems
Speaker: Walter S. Lasecki (Computer Science, University of Rochester)
Date: Tuesday, October 14
Time: 12-1pm
Room: NSH 3305

Abstract:
I create and deploy interactive systems that use a combination of human and machine intelligence to operate robustly in real-world settings. Unlike prior work in human computation, our “Crowd Agent” model allows crowds of people to support real-time interactive systems. For example, Scribe allows non-experts to caption speech in real-time for deaf and hard of hearing users, where prior approaches were either not accurate enough, or required professionals with years of training; Chorus allows multi-session conversations with a virtual personal assistant; and Apparition allows designers to rapidly prototype new interactive interfaces from sketches in real-time. In this talk, I will describe how computationally-mediated groups of people can solve problems that neither people nor computers can solve alone, and scaffold AI systems using the real-world data they collect.

Bio:
Walter S. Lasecki is a Computer Science Ph.D. candidate at the University of Rochester advised by Jeffrey Bigham (CMU). He creates interactive intelligent systems that are robust enough to be used in real-world settings by combining both human and machine intelligence to exceed the capabilities of either. Mr. Lasecki received a B.S. in Computer Science and Mathematics from Virginia Tech in 2010, and an M.S. from the University of Rochester in 2011. He was named a Microsoft Research Ph.D. Fellow in 2013, has held visiting positions at Stanford and Google[x], and is currently a visiting Ph.D. Student at CMU. 

































Friday, March 28, 2014

Title: Designing Social Computing Systems Around Relationships
Speaker: Eric Gilbert (School of Interactive Computing, Georgia Tech)
Date: Wednesday, April 2
Time: 12-1pm
Room: GHC 6501

Abstract:
Relationships are the heart of social media: they make it *social*. In this talk, I will present two social computing systems that place relationships and networks (i.e., multiple relationships) at the center of their design. First, I will present our work on modeling tie strength (i.e., how strong a relationship is), and how it can act as a tool for both design and analysis. Specifically, I'll present We Meddle, a Twitter app that builds categories of friends by inferring tie strength. Next, I will present a second Twitter app called Link Different that is inspired by the structure of a single social network triad. Link Different lets its users know how many of their followers already a saw link via someone else they follow. Hundreds of thousands of people used these two systems many millions of times. Grounding my argument in them, I'll conclude the talk by suggesting that taking this kind sociological approach to social computing suggests many new open problems for design.

Bio:
Eric Gilbert is an Assistant Professor in the School of Interactive Computing at Georgia Tech. He joined the Georgia Tech faculty in 2011 after finishing a Ph.D. in CS at Illinois. Dr. Gilbert leads the comp.social lab, a research group that focuses on building and studying social media. His work is supported by grants from Yahoo!, Google, the NSF and DARPA. Dr. Gilbert has also founded several social media sites, and has received four best paper awards and two nominations from ACM's SIGCHI. His research has recently been featured in The Wall Street Journal, The Atlantic, MIT's Technology Review and on CNN and NPR. One of his favorite activities in life is drinking coffee while hanging out on the internet.