| Welcome | News | Research | CV | Personal | Blog | Contact |
I am developing new concepts for how computing technology can empower the different stages of the collective intelligence lifecycle. Click on a project name (below) to learn more about it:
Funders
Collaborators
The Deliberatorium
Challenge
Current social media fail badly when we try to engage large crowds in deliberating
about how to solve complex problems, typically generating huge volumes
of highly redundant disorganized content of very mixed quality, making it prohibitively expensive to find the "good stuff", as well as difficult
to measure and improve how well the crowd meets the customer's needs. This problem plagues a broad swath of institutions, including news media,
business, government, and NGOs.
Approach
The Deliberatorium is a web-based system that combines ideas from argumentation theory and social computing
to address this critical challenge. Under development since 2007, it has been used by thousands of individuals in such
institutions as Intel, the Federal Bureau of Land Management, and the Italian Democratic Party.
Access
Contact me (m_klein@mit.edu) to get a login.
You can also download the open source version of the deliberatorium by clicking here.
It does not have many of the features of the latest research code, but it's a great way to get started.
Videos
The Deliberative Survey (17")
Introduction to MIT Deliberatorium (10")
Introduction to the Deliberatorium (20" research talk at the
2012 Yale Epistemic Democracy Workshop)
Selected Publications
Popular Press
PlaceSpeak Blog,
How Science Can Help Us Disagree,
Sloan Management Review,
Nature,
New York Times,
MIT Technology Insider (page 11),
Information Week,
MIT Tech Talk,
The Independent (UK),
Human Technology Journal,
Next Generation Democracy (Chapter 4),
Business Innovation in the Cloud (Chapter 9),
finalist for 2011 Management Innovation Challenge
Intel, National Science Foundation, European Union FP7 program, John Templeton Foundation
Nicholas Adams,
Jared Bataillon,
Abraham Bernstein,
Nancy Bordier,
Marco Cioffi,
Jeffrey Conklin,
Gregorio Convertino,
Anna de Liddo,
Cristina Garcia,
Ralf Groetker,
Anatoliy Gruzd,
Ali Gurkan,
Luca Iandoli,
Catholijn Jonker,
James Lannigan,
Chencan Xu & Yawen Li,
Yiftach Nagar,
Corey Nunez,
Ivana Quinto,
Hajo Reijers,
Carlo Savoretti,
Paolo Spada,
Simon Buckingham Shum,
Alex Smallwood,
Catherine Spence,
Mark Tovey,
Cyril Velikanov,
Michael Winikoff
The Deliberative Survey
The Deliberative Survey is a radically re-envisioned, next-generation version of the Deliberatorium that is designed to address two
key challenges faced by our earlier R&D efforts:
The Deliberative Survey combines innovation in the UX experience as well as in the use of sophisticated AI technology such as Large Language Models, to address these weaknesses, with the
ultimate goal of providing a high-throughput, fully scaleable, zero-training, and highly cost-effective solution for harvesting the crowd's wisdom on complex contentious problems.
Complex Negotiation
Challenge
Current negotiation mechanisms fare well for the simple contexts they were
designed for (e.g. a small number of parties, a small number of independent issues such as price) but
are ill-suited to complex negotiations which include many parties, as well as many interdependent issues.
Approach
We are defining novel algorithms that help agents negotiate complex contracts with many interdependent issues, integrating
ideas from nonlinear optimization as well as game theory. We have, further, been using machine learning techniques to determined which algorithms work
best in which kinds of negotiation scenarios, by running simulated negotiations with thousands of systematicaly-generated negotiation scenarios.
Videos
An Introduction to Nonlinear Negotiation (9")
Selected Publications
Funders
National Science Foundation
Collaborators
Reyhan Aydogan,
Yaneer Bar-Yam,
Miguel Angel Lopez Carmona,
Peyman Faratin,
Katsuhide Fujita,
Hiromitsu Hattori,
Takayuki Ito,
Catholijn Jonker,
Ivan Marsa Maestre,
Hiroki Sayama,
Shelley Zhang
Idea Filtering with the Bag of Lemons
Challenge
Open innovation platforms (web sites where crowds post ideas in a shared space)
enable us to elicit huge volumes of potentially valuable solutions for problems we care about, but identifying the best ideas
in these collections can be prohibitively expensive and time-consuming.
Approach
Our approach, called the "bag of lemons",
enables crowd to filter ideas with accuracy superior to conventional (Likert scale) rating approaches, but in only a fraction of the time.
The key insight behind this approach is that crowds are much better at eliminating bad ideas than at identifying good ones.
Selected Publications
Popular Press
Collaborators
Cris Garcia
Artifact-Centric Knowledge Sharing
Challenge
How can we increase the efficiency and completeness of knowledge-sharing around highly complex technical artifacts
such as airplanes and oil refineries? The sheer scale of the knowledge to share overwhelms conventional (keyword-based) search schemes, resulting in
low retrieval precision and accuracy, and thus poor knowledge sharing, missed opportunities for
improvement, and even unaddressed safety problems.
Approach
This project explores how the knowledge about a complex artifact can be indexed as semi-formal
models attached to digital descriptions of the artifact itself. Documents with knowledge about a turbine blade can,
for example, be hotlinked to the part of the CAD model that represents that blade. Or they can be hotlinked to elements in the process model which
describes how the turbine works. This project has developed semi-formal rationale capture languages,
intuitive user interfaces to enter the knowledge, and search tools to help people find relevant knowledge.
Selected Publications
Funders
Siemens, British Petroleum, US Army Research Lab, Boeing
Collaborators
Prof. Abraham Bernstein
Conflict Management in Collaborative Design
Challenge
Complex artifacts, ranging from physical artifacts like airplanes to virtual artifacts like software or process models,
are created by many, sometimes thousands, of designers working on different but interdependent parts or concerns. If conflicts between
these activities are detected late, or resolved imperfectly, this can have a huge negative impact on the timeliness, quality, and cost of the finished product.
Existing conflict management approaches (e.g. change memos, multi-functional product review meetings, design-build teams) are however slow, expensive, and error-prone.
Approach
This project explores how AI technologies such as constraint management and diagnosis can greatly increase the speed
and effectiveness of conflict detection and resolution in large-scale collaborative design settings.
Selected Publications
Funders
National Science Foundation, Boeing
Designing Robust Systems
Challenge
Complex systems are typically designed by describing the normative flow of events
in an ideal world, and later augmenting it in an ad hoc way to include steps for handling (anticipating and avoiding, or detecting and resolving) any
exceptions that may occur. This unsystematic process can be seriously deficient in terms of addressing all important possible
exceptions, and in using best practices to deal with them. Such errors in process design have brought companies to bankruptcy (cf Barings Bank).
Approach
This project has been developing knowledge-based tools that help designers analyze normative process models,
systematically enumerate possible exceptions, and suggest best practice techniques for handling these exceptions.
The approach is based on a growing taxonomy base of widely-usable best-practice strategies for detecting, diagnosing and resolving exceptions.
Selected Publications
Funders
NSF, Defense Logistics Agency, Neptune Technologies, British Telecom, Hewlett-Packard, Boeing, University of Lecce (Italy)
Collaborators
Chrysanthos Dellarocas
Emergent Dysfunctions in Socio-Technical Systems
Challenge
Large-scale socio-technical systems, made up of many interacting human and/or machine components, now operate at unprecedented levels of scale,
speed and interdependency, and thus can be prone to highly dysfunctional emergent dysfunctions (e.g. failure cascades, thrashing) that are new to our experience
and thus difficult to anticipate and avoid, or detect and resolve.
Approach
This project has been cataloging the ways complex socio-technical systems (e.g. for resource sharing or collaborative decision making)
can get mired in emergent dysfunctions, as well as developing novel techniques for avoiding or resolving such problems.
Selected Publications
Funders
DARPA, NSF
Collaborators
Yaneer Bar-Yam,
Chrysanthos Dellarocas,
Juan Antonio Rodriguez-Aguilar,
Hiroki Sayama
Deliberation Analytics
Challenge
How can we monitor and guide open crowds, when they are deliberating in peer-to-peer contexts,
so that their aggregate efforts progress efficiently towards good, well-evaluated solutions?
Approach
We are developing powerful analytics that datamine the digital traces
of crowdsourced deliberation engagements to assess how well the deliberation is going, understand the skills and styles of the
crowd members, and guide the crowd's attention to better achieve the goals of the deliberation customer.
This project builds upon a semi-automated, crowdsourced approach to tagging the deliberative content of
social media interactions, and is using the resulting coding to derive powerful new types of social network analysis (e.g. to assess
the degree of balkanization, or bias, in a community).
Videos
Deliberation Metrics
Selected Publications
Funders
EU FP7 program
Collaborators
Anna de Liddo,
Anatoliy Gruzd,
Luca Iandoli,
James Lannigan,
Hajo Reijers,
Simon Buckingham Shum
Pareto-Centric Collective Decision-Making
This project is exploring how we can integrate advanced techniques for collective innovation, idea filtering, and complex negotiation to engage large
crowds in finding pareto-front solutions to complex and contentious problems. This direction
promises high potential impact in such domains as participatory democracy and collaborative design. At a technical level, this work will
require innovations across the collective problem-solving lifecycle, including crowdsourced moderation, crowd analytics, adaptive incentive mechanisms,
idea filtering, complex negotiation, and algorithmic report generation.
Selected Publications
