Tuesday, December 5, 2017

Extreme Democracy

Title: Extreme Democracy
Speaker: Ariel Procaccia, Associate Professor in Computer Science Department, Carnegie Mellon University
Time: December 12, 2017; 12:00-1:00
Room: Gates-Hillman Center 6501


Abstract:

I will present several forms of democratic decision making that go far beyond your run-of-the-mill election. Specifically, I will discuss (i) liquid democracy, which allows voters to transitively delegate their votes; (ii) participatory budgeting, in which residents of a city or country vote on how its budget should be divided; and (iii) virtual democracy, which automates ethical decisions by holding elections among models of real voters. I will focus on the computational challenges that these new paradigms give rise to. 

Bio:

Ariel Procaccia is an Associate Professor in the Computer Science Department at Carnegie Mellon University. He usually works on problems at the interface of computer science and economics. His distinctions include the IJCAI Computers and Thought Award (2015), the Sloan Research Fellowship (2015), the NSF Faculty Early Career Development Award (2014), and the IFAAMAS Victor Lesser Distinguished Dissertation Award (2009); as well as half a dozen paper awards, including Best Paper (2016) and Best Student Paper (2014) at the ACM Conference on Economics and Computation (EC). He is co-editor of the Handbook of Computational Social Choice (Cambridge University Press, 2016). 


Thursday, October 12, 2017

Towards a Universal Knowledge Accelerator

Title: Towards a Universal Knowledge Accelerator
Speaker: Aniket Kittur, Associate Professor in Human-Computer Interaction Institute, Carnegie Mellon University
Time: 12:00-1:00
Room: Gates-Hillman Center 6501

Abstract:
The human mind remains an unparalleled engine of innovation, with its unique ability to make sense of complex information and find deep analogical connections driving progress in science and technology over the past millennia. The recent explosion of online information available in virtually every domain should present an opportunity for accelerating this engine; instead, it threatens to slow it as the information processing limits of individual minds are reached. 

In this talk I discuss our efforts towards building a universal knowledge accelerator: a system in which the sensemaking people engage in online is captured and made useful for others, leading to virtuous cycles of constantly improving information sources that in turn help people more effectively synthesize and innovate. Approximately 70 billion hours per year in the U.S. alone are spent on complex online sensemaking in domains ranging from scientific literature to health; capturing even a fraction of this could provide significant benefits. We discuss three integrated levels of research that are needed to realize this vision: at the individual level in understanding and capturing higher order cognition; at the computational level in developing new interaction systems and AI partners for human cognition; and at the social level in developing complex and creative crowdsourcing and social computing systems.


Bio:
Aniket Kittur is an Associate Professor and holds the Cooper-Siegel Chair in the Human-Computer Interaction Institute at Carnegie Mellon University. His research looks at how we can augment the human intellect using crowds and computation. He has authored and co-authored more than 70 peer-reviewed papers, 14 of which have received best paper awards or honorable mentions. Dr. Kittur is a Kavli fellow, has received an NSF CAREER award, the Allen Newell Award for Research Excellence, major research grants from NSF, NIH, Google, and Microsoft, and his work has been reported in venues including Nature News, The Economist, The Wall Street Journal, NPR, Slashdot, and the Chronicle of Higher Education. He received a BA in Psychology and Computer Science at Princeton, and a PhD in Cognitive Psychology from UCLA.


Wednesday, October 4, 2017

Learning from People

Title: Learning from People
Speaker: Nihar B. Shah, Assistant Professor in Machine Learning and Computer Science, Carnegie Mellon University

Abstract:
Learning from people represents a new and expanding frontier for data science. Two critical challenges in this domain are of developing algorithms for robust learning and designing incentive mechanisms for eliciting high-quality data. In this talk, I describe progress on these challenges in the context of two canonical settings, namely those of ranking and classification. In addressing the first challenge, I introduce a class of "permutation-based" models that are considerably richer than classical models, and present algorithms for estimation that are both statistically optimal and significantly more robust than prior state-of-the-art methods. I also discuss how these algorithms automatically adapt and are simultaneously also optimal over the classical models, thereby enjoying a surprising a win-win. As for the second challenge, I present a class of "multiplicative" incentive mechanisms, and show that they are the unique mechanisms that can guarantee honest responses. Extensive experiments on a popular crowdsourcing platform reveal that the theoretical guarantees of robustness and efficiency indeed translate to practice, yielding several-fold improvements over prior art.

Bio:
Nihar B. Shah is an Assistant Professor in the Machine Learning and Computer Science departments at CMU. He is a recipient of the the 2017 David J. Sakrison memorial prize from EECS Berkeley for a "truly outstanding and innovative PhD thesis", the Microsoft Research PhD Fellowship 2014-16, the Berkeley Fellowship 2011-13, the IEEE Data Storage Best Paper and Best Student Paper Awards for the years 2011/2012, and the SVC Aiya Medal 2010. His research interests include statistics, machine learning, and game theory, with a current focus on applications to learning from people.


Saturday, April 22, 2017

Solving Photo Mysteries with Expert-Led Crowdsourcing

Title: Solving Photo Mysteries with Expert-Led Crowdsourcing
Speaker: Kurt Luther, Department of Computer Science, Virginia Tech
Time: 12:30 - 1:30pm
Room: Gates-Hillman Complex 6501

Abstract:

Despite the old adage that a picture is worth a thousand words, images often need context to be meaningful to their viewers. In this talk, I show how expert-led crowdsourcing, a novel approach that combines the relative strengths of experts and amateur crowds, can be used to solve photo mysteries. In one example, I conducted a qualitative study of image verification experts in journalism, national security, and human rights organizations to understand how they perform geolocation, the process of mapping the precise location where a photo or video was taken. This research informed the design of GroundTruth, a system where experts collaborate with crowds to geolocate unknown images. In another example, I partnered with a historical photography magazine to develop Civil War Photo Sleuth, a system that leverages crowdsourcing and computer vision techniques to help experts identify unknown soldier portraits from the 19th century. I also discuss broader challenges and opportunities in crowdsourced investigations, open-source intelligence, and collaborative sensemaking illustrated by these examples.

Bio:

Kurt Luther is an assistant professor of computer science at Virginia Tech, where he is also affiliated with the Center for Human-Computer Interaction, the Department of History, and the Hume Center for National Security and Technology. He directs the Crowd Intelligence Lab (http://crowd.cs.vt.edu), an interdisciplinary research group exploring how crowdsourcing systems can support creativity and discovery. He is principal investigator for over $1.5M in sponsored research, including an NSF CAREER Award. Previously, Dr. Luther was a postdoctoral fellow in the HCI Institute at Carnegie Mellon University. He received his Ph.D. in human-centered computing from Georgia Tech, where he was a Foley Scholar, and his B.S. in computer graphics technology from Purdue University. He has also worked at IBM Research, Microsoft Research, and YouTube/Google.


Saturday, April 15, 2017

Supporting Collective Ideation at Scale

Abstract:
A growing number of online collective ideation platforms, such as OpenIDEO or Quirky, have demonstrated the potential of large-scale collaborative innovation in various domains. However, these platforms also introduce new challenges. People have to wade through a sea of possibly mundane and redundant ideas before encountering genuinely inspiring ones. Further, once all ideas are collected, the communities have to spend a lot of time and effort to synthesize the ideas into a few solutions. Alternatively, an intelligent system can select and present ideas for its users instead of leaving them to look for inspirations in a haphazard way.


In this talk, I will show how a system can decide which ideas to present to the users and when to do so. I will introduce a computational model of an idea space, two crowdsourcing methods to generate this model and the model's application for creativity-enhancing interventions. I will also present an empirical study on the effects of timing of example delivery on people's idea generation.


Bio:
Pao is a Ph.D. candidate in Computer Science focusing on Human-Computer Interaction (HCI) research at Harvard University. She works with Prof. Krzysztof Gajos in the Intelligent Interactive Systems Group. Her research explores how we can apply intelligent technologies and crowdsourcing to enable novel ways for people to come up with creative ideas together. Pao received her B.S. in Electrical Engineering and M.S. in Computer Science from Stanford University where she worked in Stanford HCI group.



Tuesday, April 4, 2017

The Collaboration and Communication Networks within the Crowd

Title: The Collaboration and Communication Networks within the Crowd
Speaker: Siddharth Suri, Microsoft Research, New York City
Time: 12:30-1:30pm
Room: Newell-Simon Hall 1507

Title: The Collaboration and Communication Networks within the Crowd
Abstract: 
Since its inception, crowdsourcing has been considered a black-box approach to solicit labor from a crowd of workers. Furthermore, the crowd has been viewed as a group of independent workers dispersed all over the world. One goal of this work is to show that crowdworkers collaborate to fulfill technical and social needs left by the platform they work on. That is, crowdworkers are not the independent, autonomous workers they are often assumed to be, but instead work within a social network of other crowdworkers. Crowdworkers collaborate with members of their networks to 1) manage the administrative overhead associated with crowdwork, 2) find lucrative tasks and reputable employers and 3) recreate the social connections and support often associated with brick and mortar-work environments. We also build on and extend these discoveries by mapping the entire communication network of workers on Amazon Mechanical Turk, a leading crowdsourcing platform. We execute a task in which over 10,000 workers from across the globe self-report their communication links to other workers, thereby mapping the communication network among workers. Our results suggest that while a large percentage of workers indeed appear to be independent, there is a rich network topology over the rest of the population. That is, there is a substantial communication network within the crowd. The existence of these networks could have implications for the burgeoning literature that involves conducting behavioral experiments and research on crowdsourcing sites. Overall, our evidence combines ethnography, interviews, survey data and larger scale data analysis from four crowdsourcing platforms. This paper draws from an ongoing, longitudinal study of crowdwork that uses a mixed methods approach to understand the cultural meaning, political implications, and ethical demands of crowdsourcing. 

Bio:
Siddharth “Sid” Suri is a computational social scientist.  His research lies at the intersection of computer science, behavioral economics and crowdsourcing.  Sid is currently writing a book with Mary Gray titled “On-Demand: Crowds, Platform Economies, and the Future of Work in Precarious Times” that combines ethnography and computer science to understand the future of work.

Sid earned his Ph.D. in computer and information science from the University of Pennsylvania in 2007 under the supervision of Michael Kearns. After that he was a postdoctoral associate working with Jon Kleinberg in the computer science department at Cornell University.  Then he moved to the Human & Social Dynamics group at Yahoo! Research led by Duncan Watts.  Currently, Sid is one of the founding members of Microsoft Research, New York City.



Wednesday, March 22, 2017

Title: Cognitive modeling explorations with crowdsourced predictions and opinions
Speaker: Michael Lee, Professor of Cognitive Sciences, University of California Irvine
Time: 12:30-1:30
Room: NSH 1507

Abstract:
The analysis of crowdsourced data can be treated a cognitive modeling problem, with the goal of accounting for how any why people produced the behaviors that were observed. We explore this cognitive approach in a series of examples, involving Thurstonian models of ranking, calibration models of probability estimation, and attention and similarity models of category learning. Many of the demonstrations use crowd-sourced data from ranker.com. Some involve "wisdom of the crowd" predictions, while others aim to describe and explain the structure of people's opinions. Throughout the talk, we emphasize the tight interplay between theory and application, highlighting not just when existing cognitive theories and models can help address crowd-sourcing problems, but also when real-world applications demand solutions to new basic research challenges in the cognitive sciences.

Bio:
Michael Lee is a Professor of Cognitive Sciences at the University of California Irvine. His research focuses on modeling cognitive processes, especially of decision making, and the Bayesian implementation, evaluation, and application of those models. He has published over 150 journal and conference papers, and is the co-author of the graduate textbook "Bayesian cognitive modeling: A practical course". He is a former President of the Society for Mathematical Psychology, a winner of the William K. Estes award of that society, and a winner of the best applied paper from the Cognitive Science Society. Before moving the U.S., he worked as a senior research scientist for the Australian Defence Science and Technology Organization, and has consulted for the Australian and US DoD, as well as various universities and companies, including the crowd-sourcing platform Ranker.


Friday, March 17, 2017

Title: Human-in-the-loop Analytics
Speaker: Michael Franklin, Liew Family Chair of Computer Science, University of Chicago
Time: 2:30 -3:30pm
Room: NSH 1507

Abstract:
The “P“ in AMPLab stands for "People" and an important research thrust in the lab was on integrating human processing into analytics pipelines. Starting with the CrowdDB project on human-powered query answering and continuing into the more recent SampleClean and AMPCrowd/Clamshell projects, we have been investigating ways to maximize the benefit that can be obtained through involving people in data collection, data cleaning, and query answering.  In this talk I will present an overview of these projects and discuss some future directions for hybrid cloud/crowd data-intensive applications and systems.

Bio:
Michael J. Franklin is the Liew Family Chair of Computer Science and Sr. Advisor to the Provost for Computation and Data at the University of Chicago where his research focuses on database systems, data analytics, data management and distributed computing systems.  Franklin previously was the Thomas M. Siebel Professor and chair of the Computer Science Division of the EECS Department at the University of California, Berkeley.   He co-founded and directed Berkeley’s Algorithms, Machines and People Laboratory (AMPLab), which created industry-changing open source Big Data software such as Apache Spark and BDAS, the Berkeley Data Analytics Stack.   At Berkeley he also served as an executive committee member for the Berkeley Institute for Data Science.  He currently serves as a Board Member of the Computing Research Association and on the NSF CISE Advisory Committee.  Franklin is an ACM Fellow and a two-time recipient of the ACM SIGMOD “Test of Time” award. His other honors include the Outstanding Advisor award from Berkeley’s Computer Science Graduate Student Association, and the “Best Gong Show Talk” personally awarded by Andy Pavlo at this year’s CIDR conference.

For more information about Dr. Franklin, visit https://cs.uchicago.edu/directory/michael-franklin and https://people.eecs.berkeley.edu/~franklin/


Friday, March 3, 2017

Constructing Visual Metaphors: Using the Design Process to Crowdsource a Creative Task

Title: Constructing Visual Metaphors: Using the Design Process to Crowdsource a Creative Task
Speaker: Lydia Chilton, Stanford University (Columbia University starting in Fall 2017)
Date: Tues, March 7
Time: 12:30-1:30pm
Room: NSH 1507

Abstract:
Visual Metaphors are a communication tool used to draw users' attention in print media, ads, public service announcements and art. They involve blending two symbols together visually to convey a new meaning. This is a creative problem with many solutions, but some solutions have more impact and meaning to readers than others.

I will introduce the problem of visual metaphors, and describe our early stages in crowdsourcing this problem. I will discuss how we had to adapt the design process to apply to microtasks and the lessons we have learned so far about designing media that speaks directly to reader’s low-level perceptual processing.

Bio:
Lydia Chilton is an assistant professor in the Computer Science Department of Columbia University in the City of New York. Actually, she won't technically start that position until July. She is currently a post-doc working with Maneesh Agrawala at Stanford University at the intersection of graphics, HCI and crowdsourcing. She has been doing crowdsourcing for ten years and is excited to see how the original goals of crowdsourcing are being realized by a large community of talented researchers. 




Monday, February 20, 2017

Real-Time Crowdsourcing for Complex Tasks

Title: Real-Time Crowdsourcing for Complex Tasks
Speaker: Walter Lasecki, Assistant Professor, University of Michigan
Date: Tues, Feb 21
Time: 12:30-1:30
Room: NSH 1507
Abstract:
Creating robust intelligent systems that can operate in real-world settings at super-human performance levels requires a combination of human and machine contributions. Crowdsourcing has allowed us to scale the ubiquity of these human computation systems, but the challenges in mixing human and machine effort remain a limiting factor of these systems. My lab’s work on modeling crowds as collective agents  has helped alleviate some of these challenges at a system level, but how we can create cohesive ecosystems of crowd-powered tools that together solve more complex and diverse needs remains an open question. In this talk, I will discuss some initial and ongoing work that aims to create complex crowdsourcing systems for applications that cannot be solved using only a single tool. 

Bio:
Walter S. Lasecki is an Assistant Professor of Computer Science and Engineering at the University of Michigan, Ann Arbor, where he directs the Crowds+Machines (CROMA) Lab. He and his students create interactive intelligent systems that are robust enough to be used in real-world settings by combining both human and machine intelligence to exceed the capabilities of either. These systems let people be more productive, and improve access to the world for people with disabilities. Dr. Lasecki received his Ph.D and M.S. from the University of Rochester in 2015 and a B.S. in Computer Science and Mathematics from Virginia Tech in 2010. He has previously held visiting research positions at CMU, Stanford, Microsoft Research, and Google[x].​


Tuesday, February 7, 2017

Perpetually enhancing user experience through collaborative, dynamic, personalized experimentation

Title: Perpetually enhancing user experience through collaborative, dynamic, personalized experimentation
Speaker: Joseph Jay Williams, Harvard University
Date: Tues, Feb 14
Time: 12:30-1:30pm
Room: NSH 3305 (NOTE: DIFFERENT ROOM THAN OUR USUAL ONE!)


Abstract:
There is a proliferation of websites and mobile apps for helping people learn new concepts (e.g. online courses), and learn how to change health habits and behavior (e.g. websites for reducing depression, apps for quitting smoking). How can we use data from real-world users to rapidly enhance and personalize these technologies? I show how we can build self-improving systems by reimagining randomized A/B experimentation as an engine for collaboration, dynamic enhancement, and personalization. I present a novel system that enhanced learning from math problems, through crowdsourcing explanations and automatically experimenting to discover the best. My second application boosted responses to an email campaign, by experimentally discovering how to personalize motivational messages to a user's activity level. These self-improving systems use experiments as a bridge between designers, social-behavioral scientists and researchers in statistical machine learning.


Bio:
Joseph Jay Williams is a Research Fellow at Harvard's Office of the Vice Provost for Advances in Learning, and a member of the Intelligent Interactive Systems Group in Computer Science. He completed a postdoc at Stanford University in the Graduate School of Education in Summer 2014, working with the Office of the Vice Provost for Online Learning and the Open Learning Initiative. He received his PhD from UC Berkeley in Computational Cognitive Science, where he applied Bayesian statistics and machine learning to model how people learn and reason. He received his B.Sc. from University of Toronto in Cognitive Science, Artificial Intelligence and Mathematics, and is originally from Trinidad and Tobago. More information about his research and papers is at www.josephjaywilliams.com.





Sunday, February 5, 2017

Crowdbotics: Writing Software with Crowds

Title: Crowdbotics: Writing Software with Crowds
Speaker: Anand Kulkarni, Founder of Crowdbotics, previous Chief Scientist of LeadGenius
Date: Tues, Feb 7
Time: 12:30 - 1:30pm
Room: NSH 1507

Abstract:
Crowd computing systems are now capable of doing more sophisticated work than ever before, accelerating the pace of creative work such as creating movies, carrying out research, and writing stories.  How can crowds be used to automate software creation? This is a complex problem, requiring good methods for expert selection, program synthesis and collaboration.  We’ll discuss strategies for accelerating software creation using crowds of geographically-diverse communities of software engineers, as well as the cultural and technical challenges that emerge.

We'll also discuss several open problems in crowd computing, including efforts by our team and others to create a crowdsourcing compiler, a hypothetical system that can optimally divide an open-ended task between artificial intelligence systems and teams of humans.  These systems blur the line between crowd-powered software and real-world organizations, raising important questions about the future of work.

Bio:
Anand is founder of Crowdbotics, a startup using crowdsourcing and machine intelligence to accelerate the process of software development. Crowdbotics brings developers from new-to-coding communities worldwide into a global community of software engineers that collaborate 1-1 with technology companies.  Prior to that, Anand was Chief Scientist of LeadGenius, a Y Combinator, Sierra Ventures, and Lumia Capital-backed startup using human computation and deep learning to automate account-based marketing (ABM). LeadGenius has raised over $20M in venture funding and developed best-in-class marketing automation technology used by Fortune 500 customers like Google, eBay, and Box. In conjunction with nonprofits like the World Bank, LeadGenius generates fairly-paid digital employment for over 500 individuals in 40 countries.  Anand has been named as one of Forbes Magazine's "30 under 30" Top 30 Entrepreneurs Under 30. Anand has published over a dozen papers in ACM, AAAI and IEEE magazines, journals, and conferences. Anand previously held a National Science Foundation graduate research fellowship in mathematics. He holds degrees in Industrial Engineering and Operations Research, Mathematics, and Physics from UC Berkeley.




Sunday, January 29, 2017

Collective Creativity Systems: Design families and remix landscapes

Title: Collective Creativity Systems: Design families and remix landscapes
Speaker: Jeffrey Nickerson, Professor, School of Business, Stevens Institute of Technology
Date: Tues, Jan 31
Time: 12:30-1:30pm
Room: NSH 1507

Abstract:
Collective creativity systems often include modification and recombination mechanisms, forms of remixing. In addition, metasystems can accelerate the exploration of design space. For example, customizers create design families by defining parameter ranges. These customizers can in turn be modified. The effect of this reuse for customization is analyzed in the 3D printing community Thingiverse. The study focuses on the design artifacts and their inheritance history, as well as their shape and semantic distance from each other. Some ways of using coordination and computation to catalyze design space exploration are discussed, with examples from several online communities.

Bio:
Jeffrey Nickerson is a professor in the School of Business at Stevens Institute of Technology. His current research focuses on collective creativity: through observation and experiment, he seeks to understand how systems composed of humans and machines can explore design space. He has a Ph.D. from New York University in Computer Science, as well as an M.F.A. in Graphic Design from Rhode Island School of Design. He is now working on two NSF-funded projects related to online community-based design and problem solving.

Tuesday, January 17, 2017

Speaker: Daniela Braga (CEO of DefinedCrowd)
Date: Tuesday, Jan 24
Time: 12:30-1:30pm
Room: NSH 1507

Abstract:
DefinedCrowd was born out of the founder’s personal frustration around data quality and slowness when dealing with data providers. Artificial Intelligence is next big technology milestone which replaces humans in menial tasks, giving us more time to enjoy more human-human interactions and focus on the really creative tasks. But to achieve this technology leap, two things are required: high quality training data and state-of-the-art machine learning algorithms. In this talk, I will walk through the reasons behind the creation of DefinedCrowd and how we are solving the data quality problem faster and at scale, by combining the latest methodologies on crowdsourcing with machine learning. DefinedCrowd is a global company headquartered in Seattle, WA, with an R&D center in Lisbon, Portugal, graduated from the Microsoft Seattle Accelerator in Machine Learning (with less than 1%  of acceptance rate and recognized by Microsoft as one of the fastest growing companies in the space) and which raised $1.1 M seed round in September 2016 from Amazon, Sony and Portugal Ventures. DefinedCrowd was also featured in http://www.inc.com/lisa-calhoun/7-artificial-intelligence-startups-to-watch-in-2017.html as one of the 7 AI companies to watch in 2017.

Bio: 
Founder and CEO of DefinedCrowd, one of the fastest growing startups in the AI space. With seventeen years working in Speech Technology both in academia and industry in Portugal, Spain, China and the US, Daniela Braga has deep expertise in Speech Science and is one the world leaders of Crowdsourcing adoption in large enterprises. Previously at Microsoft worked in pretty much all stacks of Speech Technology and shipped 26 languages for Exchange 14, 10 TTS voices in Windows 8 and was involved in Cortana. At Voicebox, created the Data Science and Crowdsourcing team, where she introduced Crowdsourcing for big data solutions and re-structured the Engineering infrastructure around data collection, processing, ingestion, instrumentation, storage, browsing and discoverability. Her effort has resulted in reducing data collection and processing costs by 80%; her approach has been adopted in multiple organizations. Dr. Braga is oftentimes guest lecturer in the University of Washington, USA, is the author of more than 90 scientific papers and several patents.