This week there is reflection on the small data errors that persist with Big Data, and it is suggested that Big Data is ushering in a "new Darwinian moment". Some insight on data driven strategies for the social sector is given while the state of digital civil society in China and Brazil is highlighted. Best practice for crowdfunding that incorporates data is also explained.
Citing inaccuracies found with the use of Google Flu trends, usually identified as a big data success story, this post questions the extent to which decisions made using big data are prone to serious error. He warns that while big data can tremendously help scientists, entrepreneurs and governments, we must not forget basic statistics as a lot of small data problems also occur and worsen when using big data. He says that analysis based on correlation without theory is inevitably fragile and that big data is prone to not just sampling error but sampling bias. The larger the dataset the bigger the problem, however, little attention is paid to this. While having transparency about how many hypotheses were tested and how many contrary results were found can help address this companies don’t want to reveal such information. Better statistical methods are needed to solve such problems and truly grasp the big data opportunity.
In this video Abhishek Mehta, CEO of Treseta explains the challenges of being an entrepreneur, and developing a startup. He also explains some of the trends and the blind spots of big data. He says that data is the fuel for what he calls “a new Darwinian moment.” and that its value lies in a combination of technology, data and science. According to him the days of selling databases is over as it is accepted that open source works. Also CIOs are no longer the buyers and applications are the next generation predictive analytics software, which will help monetize problems later on. He also talks about the new Spark application that he has built - an in-memory database like framework that allows advanced algorithms to run in-memory, at scale.
In this blog post Jonathan Sotsky recaps his experiences with the Knight Foundation’s work on new forms of research and data visualization following the Stanford Social Innovation Review (SSIR) webinar “Data-Driven Strategy in the Social Sector”. He provides advice on how foundations and even small nonprofits with limited resources can leverage data to increase their effectiveness and advance knowledge in their fields. He says that leadership buy in is key to becoming a data driven organisation and that funders can better apply new approaches to research and data to increase their impact and effectiveness.
In her post Lucy Bernholz discusses the role of digital data and digital civil society in Beijing, China and São Paolo, Brazil. She says that the level of registration and oversight for NGOs in China allows for more robust data collection on civil society in comparison with other parts of the world. She says that Brazil has a dispersed, diverse, fragmented economy of civil organizations and that comparable data about the sector there is lacking. Some want to develop a Brazilian "Blueprint" of the sector as the country becomes more digital. She gives more insight by providing links to her presentations on the topic.
In this post Beth Kanter gives input on some useful crowdfunding data points like average gift size and reviews the numbers around crowdfunding campaign that use platforms like Causes, Causevox, FirstGiving, Razoo, and StayClassy. She also summarizes some best practices for nonprofits that use crowd funding platforms.