Survey data analysis - take care!
Updated: 4 days ago
Surveys are powerful tools for getting data about things we care about. Designed well, surveys are an important tool for decision makers seeking to improve their organisation. For example, they are often used to better understand current culture or to assess organisational maturity.
Organisational surveys ask for opinions and reflect the workplace experiences of those responding. Each person has their own perspectives and biases, so no single response should be relied upon. However if the survey numbers are statistically significant, it is possible to draw useful conclusions and to see patterns in the responses.
Such surveys have important limits. They can indicate what people are feeling and thinking, and hence tell us quite a lot about morale, culture, behaviours and capabilities. A good survey can tell us whether there is a problem - such as a gap between what is happening and what we want to happen. However they don't directly tell us why this is the case, or indeed what might need to be changed to improve things sustainably. It is a mistake to jump from survey response, to action. Gaps are symptoms not root causes, nor do they indicate what needs to change for better outcomes.
No amount of statistical analysis of survey data can change this. Root causes in human organisations are by nature complex, and the links between causes and effects are generally subtle, indirect and inter-connected. The lesson for leaders and decision makers is to use organisational surveys only for what they are good at - showing the current state of affairs.
As an aside, accessing and anlaysing metadata has the same inherent limits. Having larger amounts of data does not change the character of the data, nor the limits of analysis engines. Even with machine learning and AI, data analysis can only go so far.
To know how to respond to what we have learned from our well-designed survey(s) requires a different kind of work, as implied by the complexity quadrant of the Cynefin Model (Snowdon). This is where RiskIQ works - and where amazing results can be achieved simply by matching the method of inquiry and analysis to the nature of the complex challenges faced.