Contribute
Thank you!
Your submission has been received
Oops! Something went wrong while submitting the form.
“We are coming to realize that human behaviour is determined as much by the patterns of our culture as by rational, individual thinking. These patterns can be described mathematically, and used to make accurate predictions”
Prof. Alex “Sandy”Pentland, MIT, Co Founder.
One of "The World's 7 Most Powerful Data Scientists” - Forbes
Social Physics (Wikipedia) is a revolutionary new science which uses big data analysis and the mathematical laws of biology to understand the behavior of human crowds, enabling Endor to overcome traditional Machine Learning limitations. This new science originated at MIT through research by Prof. Alex “Sandy” Pentland and Dr. Yaniv Altshuler. It was further developed by Endor using proprietary technology, resulting in a powerful prediction engine that is able to explain and predict any sort of human behavior, even when the behavior is rapidly changing and evolving.
Simply put, Social Physics is based on the premise that every event-data representing human activity (e.g. phone call records, credit card purchases, taxi rides, web activity) is guaranteed to contain a special set of human activity patterns that are embedded within that data. These mathematical invariances, which are common to all human data-types across all demographics, can then serve as a filter for detecting emerging behavioral patterns before they can be observed by any other technique.
Prof. Alex "Sandy" Pentland, MIT, Co Founder
The marketing department of a large bank constantly calls customers who are potentially in need of a loan. The revenues of the department are directly derived from the portion of the customers who respond positively to the offer. As the direct-marketing costs involved in this ongoing campaign are significant, it is crucial to contact the right customers, at the right time: Too late, and they may have already taken a loan from another source. Too soon – and the need has yet to become material. Two tools can be used to predict who these customers are: A Machine Learning model developed in-house by the bank’s data science team; and Endor’s engine. Here is a simplified representation of what each tool concluded:
The customer group that was detected by the Machine Learning model comprised of both customers who will indeed respond positively to a marketing offer by the bank (“True Positives”), and those who will not (“False Positives”).
By way of illustration, let us assume that the True Positives comprise 10% of the model’s results. Extensive experiments reveal that we can expect the vast majority of those 10% to also be detected by Endor’s Social Physics engine, with two main differences: (a) many of the False Positives of the Machine Learning model will not be reported by the Endor engine; and (b) the Endor results list will contain many additional True Positives, which are not detected by the traditional model. The implication of this is a significant improvement in sales efforts (thanks to the better precision / recall trade-off).
Example: In a recent test 15 million Tweets’ meta-data were provided to Endor as raw data for analysis. In addition, the customer revealed the identity of 50 Twitter accounts known to be ISIS activists that were contained in the input data, and tested Endor’s ability to detect an additional 74 accounts that were hidden within the data. Endor’s engine completed the task on a single laptop in only 24 minutes (measured from the time the raw data was introduced into the system until the final results were available), identifying 80 Twitter accounts as “lookalikes” to the provided example, 45 of which (56%) turned out to be part of the list of the 74 hidden accounts. Importantly, this provided an extremely low false alarm rate (35 False Positive results), so that the customer could easily afford to have human experts investigate the identified targets.
Why?
Human reality is composed of many small temporary events and changes.
Endor, grounded in Social Physics, incorporates the underlying dynamics of human behavior and is therefore better equipped to uncover small groups in the population who are likely to behave in a certain way due to recent, unexpected events.
Endor is therefore uniquely capable of identifying dynamic signals in human behavior data that no other method can sense. This is because traditional Machine Learning and Deep Learning methods are not able to distinguish between these signals and noise. Without Social Physics, these signals lack any sort of statistical significance.
“ WE ARE COMING TO REALIZE THAT HUMAN BEHAVIOR IS DETERMINED AS MUCH BY THE PATTERNS OF OUR CULTURE AS BY RATIONAL, INDIVIDUAL THINKING. THESE PATTERNS CAN BE DESCRIBED MATHEMATICALLY, AND USED TO MAKE ACCURTE PREDICTIONS”
DR. NURIA OLIVER | SCIENTIFIC DIRECTOR, TELEFONICA
Co Founder & CEO
Co Founder, Director at MIT Media Lab
Prof. Alex “Sandy” Pentland, MIT. Co Founder
Sustainable digital ecology through trusted networks
Prof. Alex “Sandy” Pentland, MIT. Co Founder
How good ideas spread_Social Physics new science
Prof. Alex “Sandy” Pentland, MIT. Co Founder
How social networks make us smarter
Dr. Yaniv Altshuler, Co Founder & CEO
Rethinking predictive analytics
Prof. Alex “Sandy” Pentland, MIT. Co Founder
From Ideas to Actions
Dr. Yaniv Altshuler, Co Founder & CEO
Social Physics
Which one is better for which purpose?
In solving a business query using data science and big data analytics tools, both Machine Learning and Social Physics are viable options. In the table below we try to help you identify the appropriate tool, based on its attributes.
For example: monitoring an oil drill pump’s control data to predict malfunction; face recognition.
For example: analyzing financial transactions to predict who will purchase a premium service.
Human behavior is erratic, unpredictable, noisy, complex, and dynamic. Mathematically speaking, in contrast to ”static problems” (such as face recognition), human behavior is dominated by a huge number of “temporal” signals, each affecting a small group of individuals. Hence, it is very hard to “learn” human data and produce consistent, stable models representing it.
Endor uses Social Physics to detect such temporal signals, and therefore is specifically tailored to human-based data.
Here are the primary advantages of using Social Physics as a tool for predictive analytics of human data
Traditional Machine Learning
Able to analyze small data sets, but requires expert data scientists and is a time-consuming process.
Deep Learning (without Social Physics)
Requires large amounts of data for every question.
Social Physics
Requires very little data to answer any question related to human behavior.
The results are generated automatically (no need for data scientists to be involved).
Why
Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.
Traditional Machine Learning
Requires a skilled data scientist and / or a domain expert in order to define and select the right features representation of the raw data.
Deep Learning (without Social Physics)
Does not require features and can process raw data, but is limited to extremely narrow types of problems.
Social Physics
Does not require features and can process raw data for any type of predictive problem (for human behavior).
Why
Machine Learning requires a long, often manual, process of transforming raw data into meaningful features. This is typically done from scratch for every problem, and for any new type of data.
Although Deep Learning deals with feature crafting automatically, it still requires large amounts of data, and data requirements increase in line with the complexity of the problem. Therefore, it is limited to “simple behaviors.”
In addition, Deep Learning is usually also confined to “static problems,” as Deep Learning dynamics require a vast amount of data that is usually unavailable at the typical company.
Social Physics automatically transforms any raw human behavior data into a canonical form of human behavioral clusters. Using this canonical representation, Endor is able to contend with all data types and all questions, regardless of data size, and to generate a unified human-behavior data set which then uses the power of Deep Learning to answer any predictive question.
Traditional Machine Learning
Machine Learning experts, usually with the assistance of domain experts, who help craft semantic features.
Deep Learning (without Social Physics)
Deep Learning experts.
Social Physics
Business users.
All you need to do is provide an example of “people you want to find more of”.
Why
Machine Learning requires “learning” the underlying normal behavior of a large data set, or leveraging prior domain expertise. Endor already incorporates the underlying dynamic of human behavior data.
Traditional Machine Learning
Limited to slow-changing data.
Changes in the data requires continuous intervention of domain experts, in order to tweak the features.
Deep Learning (without Social Physics)
While it is able to deal with dynamics, it is limited to slow-changing data (a harsh limitation when it comes to human behavior data!).
Social Physics
Can easily analyze fast-changing data sources, and does so automatically (no need for domain experts).
Why
Endor’s engine is specifically tailored to human behavior data, and therefore inherently works on data of a dynamic nature.
As Social Physics is a set of mathematical invariances that are embedded in any human dataset, it can even detect signals representing extremely short time segments.
In other words, it is able to identify emerging changes before they are observable by other techniques.
Traditional Machine Learning
Specific / Limited.
Deep Learning (without Social Physics)
Social Physics
Broad / Any question regarding human behavior.
Why
For Machine Learning, the learning process must be repeated for each dataset and question, since the automatically-selected model features need to be re-learned. Social Physics is based on underlying human behavior principles which are not question-specific.
Traditional Machine Learning
Machine Learning is highly susceptible to noises and gaps in the data and requires a long and expensive data cleaning process.
Deep Learning (without Social Physics)
Deep Learning often requires a careful process of transforming the data into a format acceptable by the Deep Learning tool.
Social Physics
No data cleaning required.
Why
Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.
Here are the primary advantages of using Social Physics as a tool for predictive analytics of human data
Traditional Machine Learning
Able to analyze small data sets, but requires expert data scientists and is a time-consuming process.
Deep Learning (without Social Physics)
Requires large amounts of data for every question.
Social Physics
Requires very little data to answer any question related to human behavior.
The results are generated automatically (no need for data scientists to be involved).
Why
Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.
Traditional Machine Learning
Requires a skilled data scientist and / or a domain expert in order to define and select the right features representation of the raw data.
Deep Learning (without Social Physics)
Does not require features and can process raw data, but is limited to extremely narrow types of problems.
Social Physics
Does not require features and can process raw data for any type of predictive problem (for human behavior).
Why
Machine Learning requires a long, often manual, process of transforming raw data into meaningful features. This is typically done from scratch for every problem, and for any new type of data.
Although Deep Learning deals with feature crafting automatically, it still requires large amounts of data, and data requirements increase in line with the complexity of the problem. Therefore, it is limited to “simple behaviors.”
In addition, Deep Learning is usually also confined to “static problems,” as Deep Learning dynamics require a vast amount of data that is usually unavailable at the typical company.
Social Physics automatically transforms any raw human behavior data into a canonical form of human behavioral clusters. Using this canonical representation, Endor is able to contend with all data types and all questions, regardless of data size, and to generate a unified human-behavior data set which then uses the power of Deep Learning to answer any predictive question.
Traditional Machine Learning
Machine Learning experts, usually with the assistance of domain experts, who help craft semantic features.
Deep Learning (without Social Physics)
Deep Learning experts.
Social Physics
Business users.
All you need to do is provide an example of “people you want to find more of”.
Why
Machine Learning requires “learning” the underlying normal behavior of a large data set, or leveraging prior domain expertise. Endor already incorporates the underlying dynamic of human behavior data.
Traditional Machine Learning
Limited to slow-changing data.
Changes in the data requires continuous intervention of domain experts, in order to tweak the features.
Deep Learning (without Social Physics)
While it is able to deal with dynamics, it is limited to slow-changing data (a harsh limitation when it comes to human behavior data!).
Social Physics
Can easily analyze fast-changing data sources, and does so automatically (no need for domain experts).
Why
Endor’s engine is specifically tailored to human behavior data, and therefore inherently works on data of a dynamic nature.
As Social Physics is a set of mathematical invariances that are embedded in any human dataset, it can even detect signals representing extremely short time segments.
In other words, it is able to identify emerging changes before they are observable by other techniques.
Traditional Machine Learning
Specific / Limited.
Deep Learning (without Social Physics)
Social Physics
Broad / Any question regarding human behavior.
Why
For Machine Learning, the learning process must be repeated for each dataset and question, since the automatically-selected model features need to be re-learned. Social Physics is based on underlying human behavior principles which are not question-specific.
Traditional Machine Learning
Machine Learning is highly susceptible to noises and gaps in the data and requires a long and expensive data cleaning process.
Deep Learning (without Social Physics)
Deep Learning often requires a careful process of transforming the data into a format acceptable by the Deep Learning tool.
Social Physics
No data cleaning required.
Why
Endor does not require big data to generate results, since Social Physics already incorporates the underlying dynamics of human behavior driven data. Hence, even with very small data sets it can immediately produce accurate predictions and actionable signals.
