Contribute

We’d love to learn about the predictive questions that impact your business the most and show you how Endor can help. We can get you up & running fast.

Thank you!

Your submission has been received

Oops! Something went wrong while submitting the form.

HI.

We’d love to learn about your predictive questions and show you how Endor can help.

Thank you!

Your submission has been received

Oops! Something went wrong while submitting the form.

Social Physics

Social Physics (Wikipedia) is a revolutionary new science which uses big data analysis and the mathematical laws of biology to understand the behavior of human crowds, enabling Endor to overcome traditional Machine Learning limitations. This new science originated at MIT through research by Prof. Alex “Sandy” Pentland and Dr. Yaniv Altshuler. It was further developed by Endor using proprietary technology, resulting in a powerful prediction engine that is able to explain and predict any sort of human behavior, even when the behavior is rapidly changing and evolving.

Simply put, Social Physics is based on the premise that every event-data representing human activity (e.g. phone call records, credit card purchases, taxi rides, web activity) is guaranteed to contain a special set of human activity patterns that are embedded within that data. These mathematical invariances, which are common to all human data-types across all demographics, can then serve as a filter for detecting emerging behavioral patterns before they can be observed by any other technique.

“Social Physics is about behavioral analysis in big data, but it takes it to a completely new level. We were very fortunate to find Endor and work with it”

Dr. Alan Boehme, CTO, the Coca Cola Company

Dr. Yaniv Altshuler, Co Founder & CEO
Rethinking Predictive Analytics, Data Driven, NY

Illustrating the power of Social Physics

The marketing department of a large bank constantly calls customers who are potentially in need of a loan. The revenues of the department are directly derived from the portion of the customers who respond positively to the offer. As the direct-marketing costs involved in this ongoing campaign are significant, it is crucial to contact the right customers, at the right time: Too late, and they may have already taken a loan from another source. Too soon – and the need has yet to become material. Two tools can be used to predict who these customers are: A Machine Learning model developed in-house by the bank’s data science team; and Endor’s engine. Here is a simplified representation of what each tool concluded:

Who will take a short term loan next week?

ASK

Machine Learning model

A segment of 5,650 customers who:

  • Are married
  • Have a household income of >$100k
  • Have a credit score > 650
  • Have been customers for at least two years
  • Have at least one bank credit card

Among these customers, the vast majority of those who will indeed seek a mortgage will also be identified by Endor.

Social Physics

954

Couples who are about to get married

235

Customers who are going through a divorce

17

Families who moved from France because of a recent terror attack, and already had an account at the bank

42

Startup employees who just sold their company to Facebook

51

Customers who recently graduated from a local real estate entrepreneurship course

Accuracy and performance

The customer group that was detected by the Machine Learning model comprised of both customers who will indeed respond positively to a marketing offer by the bank (“True Positives”), and those who will not (“False Positives”).

By way of illustration, let us assume that the True Positives comprise 10% of the model’s results. Extensive experiments reveal that we can expect the vast majority of those 10% to also be detected by Endor’s Social Physics engine, with two main differences: (a) many of the False Positives of the Machine Learning model will not be reported by the Endor engine; and (b) the Endor results list will contain many additional True Positives, which are not detected by the traditional model. The implication of this is a significant improvement in sales efforts (thanks to the better precision / recall trade-off).

Example: In a recent test 15 million Tweets’ meta-data were provided to Endor as raw data for analysis. In addition, the customer revealed the identity of 50 Twitter accounts known to be ISIS activists that were contained in the input data, and tested Endor’s ability to detect an additional 74 accounts that were hidden within the data. Endor’s engine completed the task on a single laptop in only 24 minutes (measured from the time the raw data was introduced into the system until the final results were available), identifying 80 Twitter accounts as “lookalikes” to the provided example, 45 of which (56%) turned out to be part of the list of the 74 hidden accounts. Importantly, this provided an extremely low false alarm rate (35 False Positive results), so that the customer could easily afford to have human experts investigate the identified targets.

We are coming to realize that human behavior is determined as much by the patterns of our culture as by rational, individual thinking. These patterns can be described mathematically, and used to make accurte predictions

DR. NURIA OLIVER
SCIENTIFIC DIRECTOR, TELEFONICA

Why?

Human reality is composed of many small temporary events and changes.

Endor, grounded in Social Physics, incorporates the underlying dynamics of human behavior and is therefore better equipped to uncover small groups in the population who are likely to behave in a certain way due to recent, unexpected events.


Endor is therefore uniquely capable of identifying dynamic signals in human behavior data that no other method can sense. This is because traditional Machine Learning and Deep Learning methods are not able to distinguish between these signals and noise. Without Social Physics, these signals lack any sort of statistical significance.

Prof. Alex “Sandy” Pentland, MIT. Co Founder
Sustainable digital ecology through trusted networks

Prof. Alex “Sandy” Pentland, MIT. Co Founder
How good ideas spread_Social Physics new science

Prof. Alex “Sandy” Pentland, MIT. Co Founder
How social networks make us smarter

Dr. Yaniv Altshuler, Co Founder & CEO
Rethinking predictive analytics

Prof. Alex “Sandy” Pentland, MIT. Co Founder
From Ideas to Actions

Dr. Yaniv Altshuler, Co Founder & CEO
Social Physics

Dr. Yaniv Altshuler

Co Founder & CEO

Prof. Alex "Sandy" Pentland

Co Founder, Director at MIT Media Lab

Machine Learning and Deep Learning Vs. Social Physics 

Which one is better for which purpose?

In solving a business query using data science and big data analytics tools, both Machine Learning and Social Physics are viable options. In the table below we try to help you identify the appropriate tool, based on its attributes.

Machine Learning is better for mechanical / physical-driven data

For example: monitoring an oil drill pump’s control data to predict malfunction; face recognition.

Social Physics is better for human behavior data

For example: analyzing financial transactions to predict who will purchase a premium service.

Why?

Human behavior is erratic, unpredictable, noisy, complex, and dynamic. Mathematically speaking, in contrast to ”static problems” (such as face recognition), human behavior is dominated by a huge number of “temporal” signals, each affecting a small group of individuals. Hence, it is very hard to “learn” human data and produce consistent, stable models representing it.

Endor uses Social Physics to detect such temporal signals, and therefore is specifically tailored to human-based data.

Why Social Physics?

Here are the primary advantages of using Social Physics as a tool for predictive analytics of human data

1

Small data sets

Traditional Machine Learning

Able to analyze small data sets, but requires expert data scientists and is a time-consuming process.

Deep Learning (without Social Physics)

Requires large amounts of data for every question.

Social Physics

Requires very little data to answer any question related to human behavior.
The results are generated automatically (no need for data scientists to be involved).

Why

Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.

2

Features vs. raw data

Traditional Machine Learning

Requires a skilled data scientist and / or a domain expert in order to define and select the right features representation of the raw data.

Deep Learning (without Social Physics)

Does not require features and can process raw data, but is limited to   extremely narrow types of problems.

Social Physics

Does not require features and can process raw data for any type of predictive problem (for human behavior).

Why

Machine Learning requires a long, often manual, process of transforming raw data into meaningful features. This is typically done from scratch for every problem, and for any new type of data.

Although Deep Learning deals with feature crafting automatically, it still requires large amounts of data, and data requirements increase in line with the complexity of the problem. Therefore, it is limited to “simple behaviors.”

In addition, Deep Learning is usually also confined to “static problems,” as Deep Learning dynamics require a vast amount of data that is usually unavailable at the typical company.

Social Physics automatically transforms any raw human behavior data into a canonical form of human behavioral clusters. Using this canonical representation, Endor is able to contend with all data types and all questions, regardless of data size, and to generate a unified human-behavior data set which then uses the power of Deep Learning to answer any predictive question.

3

Users and expertise needed

Traditional Machine Learning

Machine Learning experts, usually with the assistance of domain experts, who help craft semantic features.

Deep Learning (without Social Physics)

Deep Learning experts.

Social Physics

Business users.
All you need to do is provide an example of “people you want to find more of”.

Why

Machine Learning requires “learning” the underlying normal behavior of a large data set, or leveraging prior domain expertise. Endor already incorporates the underlying dynamic of human behavior data.

4

Pace of data change

Traditional Machine Learning

Limited to slow-changing data.

Changes in the data requires continuous intervention of domain experts, in order to tweak the features.

Deep Learning (without Social Physics)

While it is able to deal with dynamics, it is limited to slow-changing data (a harsh limitation when it comes to human behavior data!).

Social Physics

Can easily analyze fast-changing data sources, and does so automatically (no need for domain experts).

Why

Endor’s engine is specifically tailored to human behavior data,  and therefore inherently works on data of a dynamic nature.
As Social Physics is a set of mathematical invariances that are embedded in any human dataset, it can even detect signals representing extremely short time segments.
In other words, it is able to identify emerging changes before they are observable by other techniques.

5

Scope of analysis

Traditional Machine Learning

Specific / Limited.

Deep Learning (without Social Physics)

Social Physics

Broad / Any question regarding human behavior.

Why

For Machine Learning, the learning process must be repeated for each dataset and question, since the automatically-selected model features need to be re-learned. Social Physics is based on underlying human behavior principles which are not question-specific.

6

Data cleaning

Traditional Machine Learning

Machine Learning is highly susceptible to noises and gaps in the data and requires a long and expensive data cleaning process.

Deep Learning (without Social Physics)

Deep Learning often requires a careful process of transforming the data into a format acceptable by the Deep Learning tool.

Social Physics

No data cleaning required.

Why

Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.

Exploring further

Social Physics: How Social Networks Can Make Us Smarter

From one of the world’s leading data scientists, a landmark tour of the new scienice of idea flow offering revolutionary insights into the mysteries of collective intelligence and social influence.

Security and Privacy in Social Networks

Security and Privacy in Social Networks brings to the forefront innovative approaches for analyzing and enhancing the security and privacy dimensions in online social networks, and is the first comprehensive attempt dedicated entirely to this field.

Beyond the Echo Chamber

Social explorers spend enormous amounts of time searching for new people and ideas—but not necessarily the best people or ideas. Instead, they seek to form connections with many different kinds of people and to gain exposure to a broad variety of thinking.

Social Physics in the MIT Media Lab

How can we create organizations and governments that are cooperative, productive, and creative? These are the questions of Social Physics, and they are especially important right now, because of global competition, environmental challenges, and government failure.

New Solutions for Cybersecurity

Ongoing cyberattacks, hacks, data breaches, and privacy concerns demonstrate vividly the inadequacy of existing methods of cybersecurity and the need to develop new and better ones. This book brings together experts from across MIT to explore recent advances in cybersecurity from management, technical, and sociological perspectives.

Tuning Social Networks to Gain the Wisdom of the Crowd

As we engage more and more with social networking sites, there is always the danger of a “group think” mentality–when people follow a group consensus rather than critically evaluate information.

Swarms and Network Intelligence in Search

Presents recent research on swarms and network intelligence in search
systems. Applies swarm intelligence methods to search technology.

Frontiers of Financial Technology: Expeditions in future commerce, from blockchain and digital banking to prediction markets and beyond

Financial technology innovation has exploded in the popular consciousness, and promises a radical transformation of the global financial services industry. Over $20 billion is expected to be invested in fintech projects in 2016.